Events

One of the main questions we had going into the Learning Tech Ecosystem Study was, “what are learning leaders considering when they create learning tech ecosystems?” While each organization is doing different things, several threads have emerged:

• User experience
• Skills!
• Data integration
• Sustainability
• Ecosystems for everyone

Experience

Number 1 answer? Survey says: Experience! A lot of the hype from the articles we reviewed in our literature review and almost every interview we conducted mentioned making the learning experience and supporting tech a little less Minesweeper and a little more Fortnite.

Employee experience, the ease of access, and usability are front and center in ecosystem decisions. Learning leaders shared the common desire use technology that allows them to make learning available to the employee and make it accessible in the moment of need to ensure a seamless experience.

The main reasons for this? 1) Establishing a clear signal through the noise to help employees understand what is truly important and beneficial to them; and 2) Matching the experience employees get internally with that they can get externally – hoping to engage employees and nudge them in ways that helps them personally and the organization as a whole.

Skills!

L&D leaders also highlighted the importance of shifting their efforts from providing role-based learning to skill and capabilities-based learning. As up-skilling and re-skilling become must-have conversations and development goal for leaders, democratizing learning was a goal that we heard frequently in our conversations.

This fundamentally changes the type of ecosystems that organizations need. They need to be more flexible, more integrated with the work employees are doing, and more adaptable. Instead of completing a curriculum for a role, organizations are encouraging employees to develop skills, and then using technology to help them determine where those skills can help them.

Data Integration

L&D leaders are also beginning to think much more seriously about data. In our interviews, almost every learning leader mentioned the usefulness of data – not just to help them do their own jobs better, but to provide more information to the larger organization and to individual employees by seamlessly integrating data from different functions: talent, staffing, recruiting, career, performance, and experience.

While this was one of the largest challenges that L&D leaders raised, they were also thoughtful in their responses when we asked them how they were addressing it. Many are going beyond the analytics provided by any one tool and instead finding ways to consolidate that information. Answers ranged from using an ‘anchor’ technology as the system of record and integrating data feeds from other tools to using Power BI or simple Tableau dashboards to gather, crunch, and present metrics.

"The current state of the market is requiring CLOs to become CIOs."

Interviewed Learning Leader

Sustainability

Another point that leaders surfaced was this idea of sustainability in two ways: the viability of the individual technologies in the ecosystem long-term, and the completeness and effectiveness of the ecosystem as a whole.

Several leaders were cautious of using newer technologies because of the current consolidation of the learning tech market, and it is difficult to know if they’ll be around in a year. One leader shared a few questions he asks himself when vetting tech:

• How long has the organization been in business?
• Are venture capitalists betting on the technology?
• Do their other customers have similar challenges to ours, and does the vendor partner well?
• Are the companies young and nimble enough to react?

While it’s never guaranteed that vendors of any sort have long-term sustainability, asking the right questions upfront can save a lot of headache down the road.

Many leaders also mentioned their need to have their ecosystems grow and morph along with their organizations and its needs. We found that leaders with this need are often open to more experimentation, but also much more insistent that vendors play nicely together. In fact, one leader is approaching this by conducting joint sessions with several vendors in their ecosystem. This leader brings these vendors in together to help them understand their company needs and asks them to work together on solutions that will meet those needs.

Ecosystems for Everyone

Finally, an unexpected concern of many leaders is designing an ecosystem that equally benefits the deskless employees, those located in rural environments, and the hesitant adopter.

Deskless workers currently make up about 80% of the global worker population,¹ and yet, many learning technologies overlook them: from not addressing them at all (a misstep) to assuming that making something mobile and responsive solves all problems (it doesn’t). Leaders we talked to emphasized the differences in audiences and identifying technologies that will work for all, or bifurcating the ecosystem so that it will serve all.

Really great tech ideas can be derailed over the simplest things: bandwidth for instance. As one of our roundtable attendees put it, “It’s hard trying to make a seamless experience especially for those employees who are not even online.” Leaders are in need of solutions that allow their deskless employees (and those in more rural parts of the world) to access learning on their personal mobile devices on their own time instead of locking it behind a firewall.¹

Additionally, with several generations (sometimes up to 5)² working together in a company, the rate of adopting new technologies can vary, ranging from enthusiasm to reluctance. A learning leader we spoke to is tackling this by instituting a blended approach of technology enabled content being delivered by an instructor in a classroom setting.

"It's hard trying to make a seamless experience especially for employees who are not even online."

Interviewed Learning Leader

What’s happening in the world of D&I since February 2019?

Since we published our report early this year, the space and the market has continued to gain momentum. As we had predicted, existing bias within algorithms are coming front and center in the D&I space.¹ In May, the U.S congress proposed a law that would hold companies accountable for algorithms that result in discrimination and would require them to regularly evaluate their tools for accuracy, fairness, bias and discrimination.²

As we predicted, existing bias within algorithms are coming front and center in the D&I space.

Bias in artificial intelligence (AI) has become a larger issue as more of it enters our lives. In our broader society, one of the most notable examples is that of gender bias in virtual assistants – that most of these assistants have female names and voices (think Alexa, Siri, etc.) and have submissive personalities. In response, UNESCO recently published a report that points to the importance of digital skills among girls and women and the lack of diversity in the technology sector.³ To challenge these biases, Virtue, the creative agency owned by publisher Vice, in collaboration with Copenhagen Pride, Equal AI, Koalition Interactive and thirtysoundsgood, developed a genderless virtual assistant called “Q”.⁴ As part of their #BiasCorrect campaign, Catalyst, along with Burns Group, launched a Slack plug-in that alerts the user when biased language is being used against women.⁵

We heard from a number of new D&I tech vendors after we published our February report.

As technology becomes more advanced and enters into new fields, companies are starting to find unexpected and unforeseen D&I use cases for technology not designed for that particular purpose. One such example is Microsoft, which recently piloted their fingerprint biometrics-enabled cards for social benefits in Mexico’s Sonora state. This technology proved to be extremely popular with the senior population. While the initial objective was to improve safety and security in card related transactions, the elderly population reported they were able to use it with much greater ease. The technology will allow Microsoft to securely, transparently and reliably move funds to the people who might otherwise struggle to access them.⁶

What have we done in the last six months on D&I tech?

On our end, we have had a lot of conversations about diversity and inclusion and technology. For example, we shared our most recent update on D&I technology at UNLEASH 2019 in Las Vegas and in countless other conversations. In addition, we just launched our new one-page D&I tech infographic, to share our findings more succinctly. Further, we spoke about the power of networks and technology for career women at the ASU/GSV summit and launched a related study on Women, Networks, and Technology. You can see both our review of literature and some of the initial findings from our interviews; the final report will be published later this summer.

Technology providers with engagement platforms are also beginning to highlight the role such tools can play in promoting diversity and inclusion programs in companies through recognition, sharing, and feedback.

What new technology have we seen?

As we predicted, we heard from a number of new D&I tech vendors after we published the D&I tech report. We want to provide you with a round-up of the new technology providers and also update our interactive market map. In the sections below, we review the new technology we’ve seen since launching the report.

Acquire

As we stated in the original report, talent acquisition has the largest percentage of D&I technology solutions focused on sourcing and selecting diverse candidates. We identified the following new vendors in this space:

• Incluzion is a talent acquisition marketplace that provides companies with a way to recruit, hire and pay diverse freelance talent.
• Mom Source Network offers virtual networking to connect moms to others facing similar challenges and to connect those who are looking to return to the workforce with women currently working.

Talent acquisition has the largest percentage of D&I technology solutions focused on sourcing and selecting diverse candidates.

Develop & Advance

Our study on women, networks and technology led us to discover several vendors in the mentoring/career management space that are focused on career development for women and minority groups:

• Everwise, a mentoring solution, helps companies with development content and curriculum for women leaders.
• River offers mentoring software and services to organizations, helping them support their employee development initiatives such as D&I and Leadership Development through structured formal mentoring programs.
• InstaViser offers cloud-based software that integrates with existing CRM software and other management platforms, helping corporates, academic institutions and non-profits manage and scale their mentorship programs. Features include smart matching, specific diversity mentoring, scheduling, video and audio conferencing, and more.

Image 1: Screenshot of InstaViser’s technology | Source: InstaViser, 2019.

Engage & Retain

We touched upon the role technology, such as virtual reality, can play in sexual harassment training and in allowing anonymous reporting in our study. Over the course of the past few months, we came across several additional vendors that are developing technologies that help report and address issues of workplace harassment:

• Bravely provides a platform for confidential conversations with coaches, allowing marginalized groups and minorities within the organizations to speak up without the fear of retribution and to talk about issues they face at work.
• Project Callisto provides a platform, called Callisto Expansion, that allows survivors of professional sexual coercion to securely and anonymously store information about their perpetrator. It connects survivors to attorneys who can help them understand their options and available actions.
• tEQuitable helps companies reduce misconduct and proactively prevent bias, discrimination, and harassment. It offers employees a safe place to navigate issues ranging from micro-aggressions/micro-inequities to overt discrimination. The product also provides the company with data on behavioral trends, identifies systemic culture issues, and recommendations for remediation.

Image 2: Screenshot of tEQuitable’s Technology | Source: tEQuitable, 2019.

• Vault allows users to report sexual harassment in a confidential and safe way. These reports can be kept private until the user is ready to share them with the case manager. The platform also allows other employees to weigh in on an incident at the same time in a structured manner and the reports are time-stamped.
• Another vendor, Woices, also launched a mobile app that allows users to report incidents of harassment and share them anonymously. While still in its initial launch phase, the solution will be able to use AI to identify patterns in reporting and make recommendations based on insights.

Image 3: Screenshot of Woices Technology | Source: Woices, 2019.

In our search, we also came across platforms that allow women to communicate and share openly:

• Elpha is a community where women in tech talk candidly online. One of their recent initiatives focuses on identifying top companies for women, and the results based on ratings and feedback submitted by over 1,000 of their members which will be published on their website.

Image 4: Screenshot of Elpha’s technology | Source: Elpha, 2019.

• Similarly, while not a technology solution, Panda provides a platform for women to connect and network across the globe. Once part of their directory, members can network and exchange ideas through it.

Technology providers with engagement platforms are also beginning to highlight the role such tools can play in promoting diversity and inclusion programs in companies through recognition, sharing, and feedback:

• Inspirus, a Sodexo Group company, promotes diversity and inclusion initiatives through engagement, recognition, micro-learning and events. The Inspirus Employee Engagement platform allows employees to participate in D&I training courses and track their progress, recognize inclusive behaviors, and promote awareness of D&I-focused events.

Image 5: Screenshot of Inspirus Technology | Source: Inspirus, 2019.

Analyz

• Affirmity, focused on analytics and pay equity in the diversity and inclusion space, launched its ERG platform early this year to help organizations manage and scale their employee resource groups. It also offers an ERG mobile app to employees.
• Paygaps.com is a cloud based platform that simplifies the gender pay gap reporting and data analysis process.

Image 6: Screenshot of Affirmity’s Technology | Source: Affirmity, 2019.

Organizational Network Analysis (ONA) is emerging as a growing field that organizations are increasingly looking to use to identify opportunities for greater inclusion, understanding gender differences in behavior, identifying high-potentials, and building more effective relationships with underrepresented populations:

• Humanyze measures collaboration in organizations with advanced analytics, ONA, and behavioral science to increase the speed and accuracy of operational decisions.
• Another ONA vendor, Innovisor helps companies break down collaboration barriers by revealing gender issues and biases.
• Finally, Polinode provides a platform designed to collect, analyze and visualize data on relationships within organizations.

Image 7: Screenshots of Polinode’s Technology | Source: Polinode, 2019.

Market Map

We have also updated our interactive D&I technology market map and moved it to the RedThread website. You can access the new interactive map of the vendors here, as well as the original, comprehensive report here, in which we explore in depth:

ONA is emerging as a growing field that organizations are increasingly looking to use to identify opportunities for greater inclusion.

• The history and advent of this inflection point for D&I
• Detailed market observations and analysis on the D&I technology market
• The trajectory, acceleration and shape of the D&I technology market
• Potential risks and benefits of D&I technology
• Specific types of technology solutions
• Detailed vendor landscape and product offerings
• Case studies and customer perspectives
• Predictions for D&I technology over the next 18 months

Conclusion

As is evident from the number of new additions of technologies over the past few months, the D&I technology market continues to grow rapidly. This has been equally matched by the growing concerns over exacerbating existing bias as well the legal risks associated with them. Incidents of data privacy breaches such as those at Facebook and Google have brought issues of data security and ethics to the forefront, making employees and organizations wary of adopting new technologies. The number of lawsuits alleging sexual harassment have been on the rise as well, as stated by the U.S EEOC’s October 2018 report on sexual harassment.⁷ Interestingly, there has been a drop in the number of high-profile accusations, which may reflect a change in organizations approach to handling such cases (potentially responding proactively versus reactively).⁸ As the D&I technology market grows, the human component remains a crucial element. Leading practices, challenges faced, and lessons learned will be crucial sources as organizations look for ways to effectively manage these technologies.

Finally, we will be updating our D&I technology tool on a regular basis as we come across new vendors in this space. If you are a technology vendor in the D&I space and think you should be included in our tool, but are not, reach out to us here.

Appendix A: Table of Referenced Vendors and Capabilities

Six decades and not much change

We have been talking about learning measurement for more than six decades. Ever since Kirkpatrick came up with his 4-level model, learning & development (L&D) functions have been trying to understand the relationship between what they do and business outcomes. Since that time, over 20 have been developed to help L&D functions support their claim to impact.

Lately, we have become really interested in a variation of that question: How does what and how we measure affect the organization? What resulted was a surprise. While we approached it with a “there’s got to be a right way” mindset, what we have found is quite different. In fact, through this study, we have made two large realizations:

Realization 1: There has never been a more important time to get learning impact right.

Let’s start with a short history lesson. At the beginning of the first industrial revolution, we started using assembly lines to create products. These lines were very inefficient and fraught with problems: people learned only on the job, no one had a clear idea of what their roles were, and the workloads were incredibly uneven.

Then along came a guy named Frederick Winslow Taylor. Taylor was a mechanical engineer who applied engineering principles to the lines in order to clean them up and make the more efficient.

One of the first things he did was to establish processes and divide up the work. Then he trained employees to do that work in efficient ways. He, in essence, turned a chaotic malfunctioning system involving both individuals and equipment into one cohesive well-functioning machine.

This made Taylor famous. He wrote a book on these principles called The Principles of Scientific Management and is considered one of the world’s first management consultants.

As industry progressed, these principles were applied to types of work beyond assembly lines. They helped organizations grow and develop by structuring their workforce and ensuring that everyone knew what their role was and how to do it most efficiently.

Today, organizations continue to define roles, add people, train employees the ‘correct’ way, eliminate wasted time and effort, and gain efficiencies using these principles. This focus on efficiency has worked for a really long time. The US, for example, had fairly consistent gains in productivity from when we started measuring in 1947 through 2007.

However, like any system or machine, it’s impossible to make it infinitely efficient. At some point, any system will reach its optimal efficiency and additional effort won’t yield very big gains. We’re seeing that now in the market.

 

Image 1: US Productivity is Falling Off – Possibly Because of a Focus on Efficiency | Source: Data from the US Bureau of Labor Statistics

 

Image 1 shows trend lines from 1947 to 2007 (blue line), trend lines from 2001 through 2007 (orange), and the actual productivity gains from 2007 to 2017. As you can see, our actual productivity growth is the lowest it has been since we started measuring.

So efficiency is not getting us as far as it used to. And we need to start rethinking our reliance on it for business gains. Interestingly, more evolved organizations already are. A few months ago, we stumbled upon some research done by IBM. This study compares business focuses of industry outperformers (shown in orange in Image 2) to all other companies in that industry (shown in gray). The results are telling.

 

Image 2: Significant Outperformers Focus on Different things | Source: IBM Global Study

 

While the rest of the industry is focused on improving operational efficiency, outperformers focus on items that are inherently messy and inefficient, such as developing new products and services, expanding into new markets, and developing new distribution channels. And they need workforces who can think outside of the proverbial black box, take calculated risks, and think critically.

So what does this have to do with L&D and its impact? For starters, we’re dealing with a completely different world. When Taylor basically invented the L&D department, he did so to ensure that people were being trained to do a certain role in the most efficient way possible.

But L&D’s traditional methods and measures are not going to get it done. Most organizations cannot wait (and most likely don’t care) for L&D functions to calculate an ROI on a program or initiative to determine if it was effective and then adjust; it’s too late by then. Organizations need L&D functions to react more quickly and with flexibility – which requires a radically different mindset than the one currently en vogue.

We think that L&D functions everywhere need a chakabuku – a swift, spiritual kick to the head that alters their reality forever.¹ L&D functions desperately need to rethink their measurement strategies and understand the implications that they have – not just for employee development, but for organizational performance as well.

Realization 2: Learning Impact is Hard.

We were hoping that our research efforts would point to definitive things that all organizations should do. What we found was just the opposite. We talked to over 40 really smart leaders doing really innovative things when it came to how they were showing and having impact. But none of them were doing the exact same things.

Some of these leaders identified strongly with the idea of ROI. Some lived and died by Kirkpatrick, Phillips, or some other model. Some focused heavily on drawing a straight line from what they did to business results. But all of them were having at least some modicum of success. As it turns out, how learning impacts an organization is specific to that organization, its goals, and its resources.

Fortunately, patterns exist. While metrics and tactics should be aligned to the organization, business goals, and employee development needs, our literature review, roundtables, and interviews led us to 7 patterns that more evolved organizations follow for moving beyond simply showing impact to actually having impact.

 

Image 3: Learning Impact Patterns Found in More Evolved L&D Functions | Source: RedThread Reserach, 2019

 

Over the next 7 weeks, we’ll be sharing these patterns, examples of what organizations are doing, and questions you can ask yourself about your own learning impact practices.

Next article in series: Learning Impact: Tying to Business Goals.