Excerpt from: Diversity & Inclusion Technology: The Rise of a Transformative Market

On the surface, technology seems like the magic bullet for helping conscientious companies combat D&I challenges. After all, we recognize that no people are truly free of bias, so why not rely on bias-free technology? That all seems perfect until we reflect on how software is created and chosen by those same biased people. So how should we think about AI in the context of D&I?

In our recent study with Mercer, we examined the emerging market for D&I tech, and we also tackled the question of AI and raised some questions about its inherent benefits and risks.

Here is an excerpt from that report which looks more closely at the role of AI in D&I:

The potential dangers of AI in D&I technologies—and some ways to address them

To debunk some of the myths associated with the use of these technologies, we’ve outlined the potential dangers and limitations of using AI when it comes to decision making. We’ve also suggested some considerations leaders can take into account, below to become better informed consumers of AI products.

Potential Dangers of Artificial Intelligence in D&I Applications:

• Decisions based on AI algorithms have grown in complexity (often lacking a clear logical flow that humans can understand), opaqueness (as of result of limited transparency, regulation, and accountability), ubiquity (AI has become mainstream), and exclusiveness (especially if developed by small and/or homogenous teams).
• There are increased concerns around algorithmic design stemming from inadvertent human biases that can embed unintended discriminatory features into the algorithm. Introducing diversity and inclusion principles into algorithmic outcomes can be difficult if said principles are not practiced before designing the algorithms because machine learning is based on existing data.
• In instances where there is limited human oversight and involvement in the development and/or implementation of AI technology, there may be limited transparency and accountability on how predictive tools reach their decisions. AI systems are like black boxes, making it hard to identify potential bias and analyze decisions reached by predictive tools.
• AI can amplify stereotypes, adversely impacting underrepresented and marginalized populations.
• Widespread use of AI has moved fast, with little scrutiny and oversight from regulatory bodies. The responsibility of upholding ethics is often in the hands of AI developers, which brings the need to establish and sustain a code of AI ethics to the forefront.

Five Key Considerations for Using D&I AI in Organizations:

1. Recognize that AI and algorithms are not neutral because they are created and trained by humans with innate biases. Therefore, make every effort to understand their full benefits and limitations. Ask D&I technology vendors if they conduct algorithmic audits and risk assessments to determine how their predictive tools reach decisions and their potential impact on underrepresented populations.
2. Become well-versed in understanding your D&I technology data by becoming aware of how D&I data is used, stored, and processed within algorithms. Use your internal HR and workforce analytics to monitor and better understand the impact of using D&I technologies on your specific organization.
3. Establish a governance process that maintains a holistic view across sources of data and algorithms.
4. Formalize and communicate a standard organizational framework that establishes a shared understanding of how and when your organization makes decisions stemming from AI technologies to ensure transparency and accountability among stakeholders.
5. Use artificial intelligence information directionally, as one piece of the larger puzzle, rather than as an absolute. Consider the macro-level picture of AI data by incorporating other sources of information to better substantiate your decisions.