Root Out Bias at Every Stage of Your AI-Development Process

Executive Summary

Bias mitigation is a fairly technical process, where certain techniques can be deployed depending on the stage in the machine learning pipeline: pre-processing, in-processing and post-processing. Each offers a unique opportunity to reduce underlying bias and create a technology that is honest and fair to all. Leaders must make it a priority to take a closer look at the models and techniques for addressing bias in each of these stages to identify how best to implement the models across their technology. Ultimately, there is no way to completely eliminate AI bias, but it’s the industry’s responsibility to collaborate and help mitigate its presence in future technology. With AI playing an increasing important role in our lives, and with so much promise for future innovation, it is necessary that we acknowledge and address prejudice in our technology, as well as in our society.

LoulouVonGlup/Getty Images

AI has long been enabling innovation, with both big and small impacts. From AI-generated music, to enhancing the remote fan experience at the U.S. Open, to managing coronavirus patients in hospitals, it seems like the future is limitless. But, in the last few months, organizations from all sectors have been met with the realities of both Covid-19 and increasing anxiety over social justice issues, which has led to a reckoning within companies about the areas where more innovation and better processes are required. In the AI industry, specifically, organizations need to embrace their role in ensuring a fairer and less-biased world.

It’s been well-established that machine learning models and AI systems can be inherently biased, some more than others — a result most commonly attributed to the data being used to train and develop them. In fact, researchers have been working on ways to address and mitigate bias for years. And as the industry looks forward, it’s vital to shine a light on the various approaches and techniques that will help create more just and accurate models.

Bias mitigation is a fairly technical process, where certain techniques can be deployed depending on the stage in the machine learning pipeline: pre-processing  (preparing the data before building and training models), in-processing (modifications to algorithms during the training phase), and post-processing (applying techniques after training data has been processed). Each offers a unique opportunity to reduce underlying bias and create a technology that is honest and fair to all. Leaders must make it a priority to take a closer look at the models and techniques for addressing bias in each of these stages to identify how best to implement the models across their technology.


First, we need to address the training data. This data is used to develop machine learning models, and is often where the underlying bias seeps in. Bias can be introduced by the selection or sampling of the training data itself. This may involve unintentionally excluding certain groups, so that when the resulting model gets applied to these groups, the accuracy is inevitably lower than it is for the groups that were included in the training data. Additionally, training data usually requires labels used to “teach” the machine learning model during training. These labels often come from humans, which of course risks the introduction of bias. For label data in particular, it is crucial to ensure that there is a diversity of demographics in the human labelers to ensure that unconscious biases don’t creep in.

Insight Center

Counterfactual fairness is one technique scientists use to ensure that outcomes are the same in both the actual world and in a “counterfactual world,” where individuals belong to a completely different demographic. A great example of where this is of value is in university admissions — let’s say William from Los Angeles, who is white, and Barack from Chicago, who is African American, have similar GPAs and test scores. Does the model process the data the same if demographic information is swapped?

When predicting outcomes or making decisions, such as who gets the final university acceptance letter of the year, the training data and resulting models should be carefully vetted and tested before being fully implemented. It is especially important to assess variance in performance across sensitive factors like race and gender.


When training a machine learning model, in-processing models offer unique opportunities to encourage fairness and use regularization to tackle bias.

Adversarial training techniques can be applied to mitigate bias, where the machine learning model is jointly trained to simultaneously minimize errors in the primary objective (e.g., confirming or rejecting university admissions) while also penalizing the ability of another part of the model to predict some sensitive category (e.g., race).

My company recently conducted research on de-biasing approaches for examining gender bias in speech emotion recognition. Our research found that fairer, more consistent model accuracy can be achieved by applying a simple de-biasing training technique — here we compared a state-of-the-art approach on adversarial training to an approach with no de-biasing. Without any de-biasing, we found that emotional activation model accuracy is consistently lower for females compared to male audio samples. However, by applying a simple modification to the error term during the model training, we were able to effectively mitigate this bias while maintaining good overall model accuracy.


Post-processing is a final safeguard that can be used to protect against bias. One technique, in particular, has gained popularity: Reject Option-Based Classification. This process assumes that discrimination happens when models are least certain of a prediction. The technique exploits the “low confidence region” and rejects those predictions to reduce bias in the end game. This allows you to avoid making potentially problematic predictions. Also, by monitoring the volume of rejected inferences, engineers and scientists can be alerted to changes in the characteristics of the data seen in production and new bias risks.

The Road to Fairer AI

It is imperative that modern machine learning technology is developed in a manner that deliberately mitigates bias. Doing this effectively won’t happen overnight, but raising awareness of the presence of bias, being honest about the issues at hand, and striving for better results will be fundamental to growing the technology. As I wrote a year ago, the causes and solutions of AI bias are not black and white. Even “fairness” itself must be quantified to help mitigate the effects of unwanted bias.

As we navigate the lasting effects of the pandemic and social unrest, mitigating AI bias will continue to become more important. Here are several ways to get your own organization to focus on creating fairer AI:

  • Ensure that training samples include diversity to avoid racial, gender, ethnic, and age discrimination.
  • Whether labeling audio samples or generic data, it is critical to ensure that there are multiple and different human annotations per sample and that those annotators come from diverse backgrounds.
  • Measure accuracy levels separately for different demographic categories to see whether any group is being treated unfairly.
  • Consider collecting more training data from sensitive groups that you are concerned may be at risk of bias — such as different gender variants, racial or ethnic groups, age categories, etc. — and apply de-biasing techniques to penalize errors.
  • Regularly audit (using both automatic and manual techniques) production models for accuracy and fairness, and regularly retrain/refresh those models using newly available data.

Ultimately, there is no way to completely eliminate AI bias, but it’s the industry’s responsibility to collaborate and help mitigate its presence in future technology. With AI playing an increasing important role in our lives, and with so much promise for future innovation, it is necessary that we acknowledge and address prejudice in our technology, as well as in our society.

Are You Ready to Be Coached?

Executive Summary

Before you decide to work with an executive coach, assess your readiness to ensure you’ll actually benefit and grow from the experience. Take a look at yourself in the context of seven characteristics of successful coachees. Are you willing to hold yourself accountable for making progress? Are you open to new behaviors and ways of thinking? Are you ready to exercise the discipline necessary to stick to your coaching goals? Expect that the experience will cause you both excitement and some anxiety, and be ready to have an honest conversation with your coach about which characteristics are challenging for you. You may find that you’re not yet ready to get the most out of executive coaching, or you may gain insight into what it will take for you to meaningfully develop as a leader.

Jay Rew/Getty Images

Executive coaching can help you achieve higher performance and greater personal satisfaction at work. While you may be aware that you need to make changes — in behavior, mindset, or both — to advance your career, you won’t reap the benefits of coaching unless you’re prepared to fully engage in the process. This requires a substantial investment of time and effort, so before you move forward, the most important question you should ask yourself is, “Am I ready to be coached?”

Having discussed challenging client experiences with many accomplished executive coaches, it’s clear that the corresponding question — “Is this leader coachable?” — figures prominently in their evaluation of whether and how to proceed. Drawing on these conversations, I identified seven core characteristics that differentiate leaders who evolve through coaching from those who don’t.

Tolerance for discomfort. Successful coaching requires you to be proactive in embracing new ways of perceiving and acting. In doing so, you will likely experience fear or emotional blocks about new realizations and realities. You must be able to endure these periods of discomfort to realize the rewards of taking new and different approaches.

Openness to experimentation. Trying something new means taking risks, and experiments with new behaviors may not work the first time. Waiting for the perfect timing or perfect performance will stand in the way of progress. If you think you already have the answers and are unwilling to explore new options, you are unlikely to be open or do the necessary reflection to change. You have to try out new ideas and actions, fail, learn, and try again.

Ability to look beyond the rational. Behavior is not rational — it’s driven by emotions like fear, anger, and pride. Just because you “know” what to do doesn’t mean that you’ll act accordingly. You’ll gain a deeper understanding of your own behaviors and relationships if you explore their emotional dimensions.

Willingness to take responsibility. It’s hard to change if you don’t believe you have the power to shape your future. Blaming the organization, the boss, too many responsibilities, and so on will block you from growth. Even if there is some truth in your reasoning, it’s impossible to move forward if you see yourself as a victim. You have to hold yourself accountable for making progress.

Capacity for forgiveness. Even if you feel you’ve been mistreated, it’s essential to make peace with the past and channel your energy into progress. The need to “be right” or “show them” is rarely helpful for you or the people you work with. You must be willing to forgive and move on.

Self-discipline. Somewhat counterintuitively, your development as a leader will likely require you to let go of ways of thinking and behaving that helped make you successful in the past and be prepared to live with the consequences. It may be hard for others to accept changes in your personal or work relationships. For example, you may have succeeded up to this point by saying yes to helping out colleagues and making yourself available. But disciplining yourself to say no and learning to focus on what’s important are essential parts of becoming a more effective leader. Even if those around you bristle at you no longer being available 24/7, you have to stay focused on your coaching goals.

Ability to ask for support. Finally, you must be engaged with other potential supporters, not just your coach, throughout the coaching process. You are accountable for change, but you will develop faster if you make yourself vulnerable to others (judiciously), including your boss, peers, and even direct reports. Share goals, ask for advice, listen with curiosity, and most critically, accept and act on the constructive feedback you receive.

It’s normal to feel both excitement and trepidation when deciding to work with an executive coach. Start by assessing the degree to which you have these seven characteristics, then discuss which are the most challenging for you. You may mutually decide that it’s not the right time to proceed. More likely, it will help you develop a stronger relationship and a deeper awareness of how to meaningfully develop as a leader through coaching.

51% of Americans Left a Job in 2020

iHire’s 2020 Talent Retention report has found that 51.1% of American employees have left their jobs in the past year. The report also notes that job dissatisfaction has risen by 7.4% from 2019. 

The report has noted despite the high unemployment rates, layoffs haven’t been the only drivers for high turnover in 2020. 

iHire Talent Retention Report 2020

Of the 51.1% of employees who left their jobs, only 26.2% left involuntarily either being laid off or terminated while 24.9% had left voluntarily.

 Other findings of the report include:

  • Dissatisfaction with jobs was observed to have climbed year-over-year with 29.6% of those surveyed saying they were either ‘dissatisfied’ or ‘very dissatisfied’ with their current or most recent jobs.
  • Only 18.8% admit to being ‘‘very satisfied’ with their jobs, while 31.9% feel ‘somewhat satisfied’ and 19.8% were lukewarm towards their jobs.
  • Employees still do value traditional benefits and perks in the workplace. Among these include a raise or bonus (50%), a healthier work/life balance (25.7%), clear growth or advancement opportunities (25.4%), and a better benefits package (22.0%). 
  • Interest in changing careers is also on the rise with 61.8% of those surveyed are considering making a major change in the coming year. Among those, 28.9% say that a change is very likely.
  • A further 35.3% pondering career changers say they are reluctant to make a change due to financial risks, while 22.8% are simply unsure what type of career to pursue.

How Small Business Owners Can Retain Employees

Keeping employees for the long-term is important to the success of a business. Having staff remain in the workplace translates into the business saving time and resources needed for training new staff. 

Businesses will need to ease the concerns of employees by working on a strong employee engagement strategy. Especially more so during times of uncertainty and instability, employers should prioritize providing traditional, basic perks such as fair pay, flexible scheduling, medical insurance and workplace safety. They will also need to clearly provide a road map for their employees’ professional growth.


How to Solve Your Most Difficult Leadership and Talent Challenges

Guest post from Stephen Shapiro:

To find better solutions to your most difficult business problems, paradoxically, you don’t want o focus on solutions.

Instead you want to make sure you are asking the right questions. Changing the question changes the range of possible solutions.

Changing Just One Word Can Change Your Solutions

If your challenge is, “How can we hire the right talent?”, you might put your energies into time-consuming campus outreach programs, complicated recruitment campaigns, or expensive technology.

But changing the question to, “How can we retain the right talent?” may shift your focus to internal motivation and performance management strategies.

Simply changing one word yields a completely different set of answers.

You could change it again to, “How can we develop the right talent?” Now we are looking at leadership opportunities that might not have been previously considered.

Before investing in developing solutions and strategies to your problems, first make sure you are moving in the right direction.

Be More Specific to Reduce Waste

When problem-solving, it is common to start off with an opportunity that is too broad. When we ask broad questions, we invite a lot of wasted energy. When asking the question, “How can I improve the business?”, (the default question associated with most suggestion boxes), you could get literally hundreds or thousands of possible answers.

In fact, over 99% of the ideas submitted to most suggestion boxes are low value and are not implemented. This wastes the time of those who submit the ideas and those who have to evaluate the duds.

But if you make the problem statement more specific, you focus people on what matters most. 

Going back to the original statement: “How can we hire the right talent?” What does “right” mean? Maybe the question could be, “How can we hire for unique skills that make our products differentiating?”

This now focuses your efforts on a specific skillset. Of course, it might lead you to ask the question, “What differentiates our products from the competition?” Answering this gives you deeper insights into your business and the people that are required to support it.

Sometimes Being More Abstract Can Increase Creativity

Although we often start with broad problem statements, there are times when we are too specific. Either our focus is so specific that it limits our ability to find solutions. Or in some cases our questions are really just solutions masquerading as questions.

I remember a client who was focused on, “How can we use 360-degree feedback to improve performance?”

Their myopic focus on this one tool limited their ability to “see” other leadership development solutions.

The broader question might be, “How can we improve performance?” 360-degree feedback was too specific.

This then forced them to ask, “What is the performance issue we need to solve?” After some analysis, they determined that the issue was a silo mentality within the company. When they shifted the question to, “How can we break down silos in our organization?”, they found a wider range of solutions. As it turns out, 360-degree feedback was not part of the approach.

Our Best Solutions are Often Invisible

The questions we ask impact the solutions that are visible. Subtle changes to the problem statement can reveal solutions that were previously hidden.

Or to paraphrase a quote that is attributed to Albert Einstein, “If I had an hour to save the world, I would spend 59 minutes defining the problem and one minute finding solutions.” From my experience, most organizations are spending 60 minutes solving problems that are unimportant or irrelevant.

When everyone in your organization learns how to powerfully reframe business problems, you will get better results, faster, with lower risk. It’s the simplest tool you have to find the best solutions that will grow your business.

For over 20 years, Stephen Shapiro has presented his provocative strategies on innovation to audiences in 50 countries. During his 15-year tenure with the consulting firm Accenture, he led a 20,000-person innovation practice. He is the author of six books, including his latest: Invisible Solutions: 25 Lenses that Reframe and Help Solve Difficult Business Problems. His Personality Poker® system has been used around the world to create high-performing innovation teams. In 2015 he was inducted into the Speaker Hall of Fame.

How to Speak to Customers to Build Trust

Not only is consumer confidence and trust rather low these days, we’re also serving customers in a new, distanced way. The usual reliance on facial expressions and body language to make a connection with a customer and build trust is often no longer available. And given the generalized anxiety that comes with a global pandemic, feelings of trust can be especially hard to come by these days.

A growing body of research on language use in service interactions can help. Please join our speakers, authors of “Speaking to Customers in Uncertain Times,” as they show how very specific word choices and language strategies can make all the difference in connecting with customers. They’ll give practical advice on “speaking terms” that lead to customer trust.