Events

We'd like to thank the learning leaders who took part in this discussion! We think collaboration makes the research better, and we learned a lot from this roundtable.
In July 2019 we conducted our second Learning Tech Ecosystems Roundtable. The turnout was excellent and we were able to have a great discussion regarding the strategy of thinking through a learning tech ecosystem.

The hour-long web-chat resulted in a really rich discussion, the highlights of which are shown on the mindmap below. Big branches represent the main topics.

Click the graphic to get a bigger view.

Learning Tech Ecosystem Roundtable #2 Mindmap

Source: RedThread Research, 2019

 

One of the main questions we had going into the Learning Tech Ecosystem Study was, “what are learning leaders considering when they create learning tech ecosystems?” While each organization is doing different things, several threads have emerged:

• User experience
• Skills!
• Data integration
• Sustainability
• Ecosystems for everyone

Experience

Number 1 answer? Survey says: Experience! A lot of the hype from the articles we reviewed in our literature review and almost every interview we conducted mentioned making the learning experience and supporting tech a little less Minesweeper and a little more Fortnite.

Employee experience, the ease of access, and usability are front and center in ecosystem decisions. Learning leaders shared the common desire use technology that allows them to make learning available to the employee and make it accessible in the moment of need to ensure a seamless experience.

The main reasons for this? 1) Establishing a clear signal through the noise to help employees understand what is truly important and beneficial to them; and 2) Matching the experience employees get internally with that they can get externally – hoping to engage employees and nudge them in ways that helps them personally and the organization as a whole.

Skills!

L&D leaders also highlighted the importance of shifting their efforts from providing role-based learning to skill and capabilities-based learning. As up-skilling and re-skilling become must-have conversations and development goal for leaders, democratizing learning was a goal that we heard frequently in our conversations.

This fundamentally changes the type of ecosystems that organizations need. They need to be more flexible, more integrated with the work employees are doing, and more adaptable. Instead of completing a curriculum for a role, organizations are encouraging employees to develop skills, and then using technology to help them determine where those skills can help them.

Data Integration

L&D leaders are also beginning to think much more seriously about data. In our interviews, almost every learning leader mentioned the usefulness of data – not just to help them do their own jobs better, but to provide more information to the larger organization and to individual employees by seamlessly integrating data from different functions: talent, staffing, recruiting, career, performance, and experience.

While this was one of the largest challenges that L&D leaders raised, they were also thoughtful in their responses when we asked them how they were addressing it. Many are going beyond the analytics provided by any one tool and instead finding ways to consolidate that information. Answers ranged from using an ‘anchor’ technology as the system of record and integrating data feeds from other tools to using Power BI or simple Tableau dashboards to gather, crunch, and present metrics.

"The current state of the market is requiring CLOs to become CIOs."

Interviewed Learning Leader

Sustainability

Another point that leaders surfaced was this idea of sustainability in two ways: the viability of the individual technologies in the ecosystem long-term, and the completeness and effectiveness of the ecosystem as a whole.

Several leaders were cautious of using newer technologies because of the current consolidation of the learning tech market, and it is difficult to know if they’ll be around in a year. One leader shared a few questions he asks himself when vetting tech:

• How long has the organization been in business?
• Are venture capitalists betting on the technology?
• Do their other customers have similar challenges to ours, and does the vendor partner well?
• Are the companies young and nimble enough to react?

While it’s never guaranteed that vendors of any sort have long-term sustainability, asking the right questions upfront can save a lot of headache down the road.

Many leaders also mentioned their need to have their ecosystems grow and morph along with their organizations and its needs. We found that leaders with this need are often open to more experimentation, but also much more insistent that vendors play nicely together. In fact, one leader is approaching this by conducting joint sessions with several vendors in their ecosystem. This leader brings these vendors in together to help them understand their company needs and asks them to work together on solutions that will meet those needs.

Ecosystems for Everyone

Finally, an unexpected concern of many leaders is designing an ecosystem that equally benefits the deskless employees, those located in rural environments, and the hesitant adopter.

Deskless workers currently make up about 80% of the global worker population,¹ and yet, many learning technologies overlook them: from not addressing them at all (a misstep) to assuming that making something mobile and responsive solves all problems (it doesn’t). Leaders we talked to emphasized the differences in audiences and identifying technologies that will work for all, or bifurcating the ecosystem so that it will serve all.

Really great tech ideas can be derailed over the simplest things: bandwidth for instance. As one of our roundtable attendees put it, “It’s hard trying to make a seamless experience especially for those employees who are not even online.” Leaders are in need of solutions that allow their deskless employees (and those in more rural parts of the world) to access learning on their personal mobile devices on their own time instead of locking it behind a firewall.¹

Additionally, with several generations (sometimes up to 5)² working together in a company, the rate of adopting new technologies can vary, ranging from enthusiasm to reluctance. A learning leader we spoke to is tackling this by instituting a blended approach of technology enabled content being delivered by an instructor in a classroom setting.

"It's hard trying to make a seamless experience especially for employees who are not even online."

Interviewed Learning Leader

What’s happening in the world of D&I since February 2019?

Since we published our report early this year, the space and the market has continued to gain momentum. As we had predicted, existing bias within algorithms are coming front and center in the D&I space.¹ In May, the U.S congress proposed a law that would hold companies accountable for algorithms that result in discrimination and would require them to regularly evaluate their tools for accuracy, fairness, bias and discrimination.²

As we predicted, existing bias within algorithms are coming front and center in the D&I space.

Bias in artificial intelligence (AI) has become a larger issue as more of it enters our lives. In our broader society, one of the most notable examples is that of gender bias in virtual assistants – that most of these assistants have female names and voices (think Alexa, Siri, etc.) and have submissive personalities. In response, UNESCO recently published a report that points to the importance of digital skills among girls and women and the lack of diversity in the technology sector.³ To challenge these biases, Virtue, the creative agency owned by publisher Vice, in collaboration with Copenhagen Pride, Equal AI, Koalition Interactive and thirtysoundsgood, developed a genderless virtual assistant called “Q”.⁴ As part of their #BiasCorrect campaign, Catalyst, along with Burns Group, launched a Slack plug-in that alerts the user when biased language is being used against women.⁵

We heard from a number of new D&I tech vendors after we published our February report.

As technology becomes more advanced and enters into new fields, companies are starting to find unexpected and unforeseen D&I use cases for technology not designed for that particular purpose. One such example is Microsoft, which recently piloted their fingerprint biometrics-enabled cards for social benefits in Mexico’s Sonora state. This technology proved to be extremely popular with the senior population. While the initial objective was to improve safety and security in card related transactions, the elderly population reported they were able to use it with much greater ease. The technology will allow Microsoft to securely, transparently and reliably move funds to the people who might otherwise struggle to access them.⁶

What have we done in the last six months on D&I tech?

On our end, we have had a lot of conversations about diversity and inclusion and technology. For example, we shared our most recent update on D&I technology at UNLEASH 2019 in Las Vegas and in countless other conversations. In addition, we just launched our new one-page D&I tech infographic, to share our findings more succinctly. Further, we spoke about the power of networks and technology for career women at the ASU/GSV summit and launched a related study on Women, Networks, and Technology. You can see both our review of literature and some of the initial findings from our interviews; the final report will be published later this summer.

Technology providers with engagement platforms are also beginning to highlight the role such tools can play in promoting diversity and inclusion programs in companies through recognition, sharing, and feedback.

What new technology have we seen?

As we predicted, we heard from a number of new D&I tech vendors after we published the D&I tech report. We want to provide you with a round-up of the new technology providers and also update our interactive market map. In the sections below, we review the new technology we’ve seen since launching the report.

Acquire

As we stated in the original report, talent acquisition has the largest percentage of D&I technology solutions focused on sourcing and selecting diverse candidates. We identified the following new vendors in this space:

• Incluzion is a talent acquisition marketplace that provides companies with a way to recruit, hire and pay diverse freelance talent.
• Mom Source Network offers virtual networking to connect moms to others facing similar challenges and to connect those who are looking to return to the workforce with women currently working.

Talent acquisition has the largest percentage of D&I technology solutions focused on sourcing and selecting diverse candidates.

Develop & Advance

Our study on women, networks and technology led us to discover several vendors in the mentoring/career management space that are focused on career development for women and minority groups:

• Everwise, a mentoring solution, helps companies with development content and curriculum for women leaders.
• River offers mentoring software and services to organizations, helping them support their employee development initiatives such as D&I and Leadership Development through structured formal mentoring programs.
• InstaViser offers cloud-based software that integrates with existing CRM software and other management platforms, helping corporates, academic institutions and non-profits manage and scale their mentorship programs. Features include smart matching, specific diversity mentoring, scheduling, video and audio conferencing, and more.

Image 1: Screenshot of InstaViser’s technology | Source: InstaViser, 2019.

Engage & Retain

We touched upon the role technology, such as virtual reality, can play in sexual harassment training and in allowing anonymous reporting in our study. Over the course of the past few months, we came across several additional vendors that are developing technologies that help report and address issues of workplace harassment:

• Bravely provides a platform for confidential conversations with coaches, allowing marginalized groups and minorities within the organizations to speak up without the fear of retribution and to talk about issues they face at work.
• Project Callisto provides a platform, called Callisto Expansion, that allows survivors of professional sexual coercion to securely and anonymously store information about their perpetrator. It connects survivors to attorneys who can help them understand their options and available actions.
• tEQuitable helps companies reduce misconduct and proactively prevent bias, discrimination, and harassment. It offers employees a safe place to navigate issues ranging from micro-aggressions/micro-inequities to overt discrimination. The product also provides the company with data on behavioral trends, identifies systemic culture issues, and recommendations for remediation.

Image 2: Screenshot of tEQuitable’s Technology | Source: tEQuitable, 2019.

• Vault allows users to report sexual harassment in a confidential and safe way. These reports can be kept private until the user is ready to share them with the case manager. The platform also allows other employees to weigh in on an incident at the same time in a structured manner and the reports are time-stamped.
• Another vendor, Woices, also launched a mobile app that allows users to report incidents of harassment and share them anonymously. While still in its initial launch phase, the solution will be able to use AI to identify patterns in reporting and make recommendations based on insights.

Image 3: Screenshot of Woices Technology | Source: Woices, 2019.

In our search, we also came across platforms that allow women to communicate and share openly:

• Elpha is a community where women in tech talk candidly online. One of their recent initiatives focuses on identifying top companies for women, and the results based on ratings and feedback submitted by over 1,000 of their members which will be published on their website.

Image 4: Screenshot of Elpha’s technology | Source: Elpha, 2019.

• Similarly, while not a technology solution, Panda provides a platform for women to connect and network across the globe. Once part of their directory, members can network and exchange ideas through it.

Technology providers with engagement platforms are also beginning to highlight the role such tools can play in promoting diversity and inclusion programs in companies through recognition, sharing, and feedback:

• Inspirus, a Sodexo Group company, promotes diversity and inclusion initiatives through engagement, recognition, micro-learning and events. The Inspirus Employee Engagement platform allows employees to participate in D&I training courses and track their progress, recognize inclusive behaviors, and promote awareness of D&I-focused events.

Image 5: Screenshot of Inspirus Technology | Source: Inspirus, 2019.

Analyz

• Affirmity, focused on analytics and pay equity in the diversity and inclusion space, launched its ERG platform early this year to help organizations manage and scale their employee resource groups. It also offers an ERG mobile app to employees.
• Paygaps.com is a cloud based platform that simplifies the gender pay gap reporting and data analysis process.

Image 6: Screenshot of Affirmity’s Technology | Source: Affirmity, 2019.

Organizational Network Analysis (ONA) is emerging as a growing field that organizations are increasingly looking to use to identify opportunities for greater inclusion, understanding gender differences in behavior, identifying high-potentials, and building more effective relationships with underrepresented populations:

• Humanyze measures collaboration in organizations with advanced analytics, ONA, and behavioral science to increase the speed and accuracy of operational decisions.
• Another ONA vendor, Innovisor helps companies break down collaboration barriers by revealing gender issues and biases.
• Finally, Polinode provides a platform designed to collect, analyze and visualize data on relationships within organizations.

Image 7: Screenshots of Polinode’s Technology | Source: Polinode, 2019.

Market Map

We have also updated our interactive D&I technology market map and moved it to the RedThread website. You can access the new interactive map of the vendors here, as well as the original, comprehensive report here, in which we explore in depth:

ONA is emerging as a growing field that organizations are increasingly looking to use to identify opportunities for greater inclusion.

• The history and advent of this inflection point for D&I
• Detailed market observations and analysis on the D&I technology market
• The trajectory, acceleration and shape of the D&I technology market
• Potential risks and benefits of D&I technology
• Specific types of technology solutions
• Detailed vendor landscape and product offerings
• Case studies and customer perspectives
• Predictions for D&I technology over the next 18 months

Conclusion

As is evident from the number of new additions of technologies over the past few months, the D&I technology market continues to grow rapidly. This has been equally matched by the growing concerns over exacerbating existing bias as well the legal risks associated with them. Incidents of data privacy breaches such as those at Facebook and Google have brought issues of data security and ethics to the forefront, making employees and organizations wary of adopting new technologies. The number of lawsuits alleging sexual harassment have been on the rise as well, as stated by the U.S EEOC’s October 2018 report on sexual harassment.⁷ Interestingly, there has been a drop in the number of high-profile accusations, which may reflect a change in organizations approach to handling such cases (potentially responding proactively versus reactively).⁸ As the D&I technology market grows, the human component remains a crucial element. Leading practices, challenges faced, and lessons learned will be crucial sources as organizations look for ways to effectively manage these technologies.

Finally, we will be updating our D&I technology tool on a regular basis as we come across new vendors in this space. If you are a technology vendor in the D&I space and think you should be included in our tool, but are not, reach out to us here.

Appendix A: Table of Referenced Vendors and Capabilities

Six decades and not much change

We have been talking about learning measurement for more than six decades. Ever since Kirkpatrick came up with his 4-level model, learning & development (L&D) functions have been trying to understand the relationship between what they do and business outcomes. Since that time, over 20 have been developed to help L&D functions support their claim to impact.

Lately, we have become really interested in a variation of that question: How does what and how we measure affect the organization? What resulted was a surprise. While we approached it with a “there’s got to be a right way” mindset, what we have found is quite different. In fact, through this study, we have made two large realizations:

Realization 1: There has never been a more important time to get learning impact right.

Let’s start with a short history lesson. At the beginning of the first industrial revolution, we started using assembly lines to create products. These lines were very inefficient and fraught with problems: people learned only on the job, no one had a clear idea of what their roles were, and the workloads were incredibly uneven.

Then along came a guy named Frederick Winslow Taylor. Taylor was a mechanical engineer who applied engineering principles to the lines in order to clean them up and make the more efficient.

One of the first things he did was to establish processes and divide up the work. Then he trained employees to do that work in efficient ways. He, in essence, turned a chaotic malfunctioning system involving both individuals and equipment into one cohesive well-functioning machine.

This made Taylor famous. He wrote a book on these principles called The Principles of Scientific Management and is considered one of the world’s first management consultants.

As industry progressed, these principles were applied to types of work beyond assembly lines. They helped organizations grow and develop by structuring their workforce and ensuring that everyone knew what their role was and how to do it most efficiently.

Today, organizations continue to define roles, add people, train employees the ‘correct’ way, eliminate wasted time and effort, and gain efficiencies using these principles. This focus on efficiency has worked for a really long time. The US, for example, had fairly consistent gains in productivity from when we started measuring in 1947 through 2007.

However, like any system or machine, it’s impossible to make it infinitely efficient. At some point, any system will reach its optimal efficiency and additional effort won’t yield very big gains. We’re seeing that now in the market.

 

Image 1: US Productivity is Falling Off – Possibly Because of a Focus on Efficiency | Source: Data from the US Bureau of Labor Statistics

 

Image 1 shows trend lines from 1947 to 2007 (blue line), trend lines from 2001 through 2007 (orange), and the actual productivity gains from 2007 to 2017. As you can see, our actual productivity growth is the lowest it has been since we started measuring.

So efficiency is not getting us as far as it used to. And we need to start rethinking our reliance on it for business gains. Interestingly, more evolved organizations already are. A few months ago, we stumbled upon some research done by IBM. This study compares business focuses of industry outperformers (shown in orange in Image 2) to all other companies in that industry (shown in gray). The results are telling.

 

Image 2: Significant Outperformers Focus on Different things | Source: IBM Global Study

 

While the rest of the industry is focused on improving operational efficiency, outperformers focus on items that are inherently messy and inefficient, such as developing new products and services, expanding into new markets, and developing new distribution channels. And they need workforces who can think outside of the proverbial black box, take calculated risks, and think critically.

So what does this have to do with L&D and its impact? For starters, we’re dealing with a completely different world. When Taylor basically invented the L&D department, he did so to ensure that people were being trained to do a certain role in the most efficient way possible.

But L&D’s traditional methods and measures are not going to get it done. Most organizations cannot wait (and most likely don’t care) for L&D functions to calculate an ROI on a program or initiative to determine if it was effective and then adjust; it’s too late by then. Organizations need L&D functions to react more quickly and with flexibility – which requires a radically different mindset than the one currently en vogue.

We think that L&D functions everywhere need a chakabuku – a swift, spiritual kick to the head that alters their reality forever.¹ L&D functions desperately need to rethink their measurement strategies and understand the implications that they have – not just for employee development, but for organizational performance as well.

Realization 2: Learning Impact is Hard.

We were hoping that our research efforts would point to definitive things that all organizations should do. What we found was just the opposite. We talked to over 40 really smart leaders doing really innovative things when it came to how they were showing and having impact. But none of them were doing the exact same things.

Some of these leaders identified strongly with the idea of ROI. Some lived and died by Kirkpatrick, Phillips, or some other model. Some focused heavily on drawing a straight line from what they did to business results. But all of them were having at least some modicum of success. As it turns out, how learning impacts an organization is specific to that organization, its goals, and its resources.

Fortunately, patterns exist. While metrics and tactics should be aligned to the organization, business goals, and employee development needs, our literature review, roundtables, and interviews led us to 7 patterns that more evolved organizations follow for moving beyond simply showing impact to actually having impact.

 

Image 3: Learning Impact Patterns Found in More Evolved L&D Functions | Source: RedThread Reserach, 2019

 

Over the next 7 weeks, we’ll be sharing these patterns, examples of what organizations are doing, and questions you can ask yourself about your own learning impact practices.

Next article in series: Learning Impact: Tying to Business Goals.

Introduction

In the spring of 2019, when summer still felt like a distant dream, we conducted an online poll and more than 20 interviews with people analytics practitioners and solution providers to understand their current challenges and what they most wanted to know about the people analytics technology market.

We did this so we could better understand what our overall research – including our online survey and literature review – on people analytics technology should focus on. We figured you’re likely curious to know about what we learned, so we’ve summarized our findings in this article, specifically focusing on

• Poll results
• Interviews: what we heard
• Overall impressions
• Our next steps

Poll results

Our poll revealed some interesting insights into what is on top of minds when folks think of people analytics technology. We asked respondents (vendors, practitioners, and others with an interest in the field) to tell us what they think we should 1) cover in our research, 2) not cover in our research, and 3) any other feedback or ideas they wanted to share. The findings reaffirmed our suspicions: the top issue is better understanding the solution categories and capabilities for people analytics technologies.

Interviews: What we heard

Capabilities and needs – mo' solutions, mo' problems?

As our poll reflects, more than 70 percent of respondents indicated they want more clarity on people analytics technologies’ capabilities and a framework for thinking about the different solution providers. This sentiment was echoed in our interviews: when we asked folks to provide a way to categorize the market, we received as many different frameworks as interviews!

Our conversations revealed that while some group people analytics solution providers into categories of data storage, analysis, and visualizations, others think of them in terms of solutions that perform transactional/reporting versus analytics functions. Still others separated solutions into pure-HR players versus ones that cater to business intelligence capabilities.

Ultimately, both technology providers and organization leaders recognize the opportunity to work together to identify and use technology in a meaningful and purpose-driven manner. To do so, buyers and users should have a realistic and accurate understanding of the technology.

A good starting point for the users could be clarity, consistency and agreement on how they define people analytics and success within the organization. Buyers could also work with the vendors more, especially at the beginning of implementations, to put in place processes to ensure insights and findings from using the technology are shared broadly in an actionable way.

Similarly, vendors need a clear understanding of the business and talent challenges as well as the objectives of the organization. One of the vendors we spoke to stated,

“ typically start by saying that they have a data problem…and are trying to wrangle the data etc. Once we talk to them though, the problems that they are trying to solve for are business problems, such as turn over issue and are trying to wrangle the data to figure out what is driving the turnover.”

This points to the need for customers to spend time clarifying the business problems they are trying to solve and for vendors to spend time ensuring they understand those problems before implementing a solution.

Data Integration – Connecting the Dots

A topic that frequently popped up in our interviews was the opportunities and challenges of data integration from different sources. Some of the key questions we heard from practitioners during our interviews included

1. Are there people analytics tools that can help practitioners integrate their data and allow for data interoperability?
2. Which technologies speak to each other and can be used as an integrated suite?
3. Do the existing tools require data be within one system or should it be siloed in different systems?

While some of these concerns tie in with the issue of capabilities that we touched upon earlier, it is clear that data integration is a major challenge  Practitioners we spoke to mentioned that a growing number of vendors claim to offer multiple services and solutions for all things people analytics, but fail to provide that. For example, one person stated,

“ really wish the technology vendors would stop promising a one-stop shop for all your analytics need…These technologies require a lot of thoughtfulness in implementation to have them be useful.”

Further, other practitioners indicated a frustration with vendors who have incremental additional charges for additional data sources. While no one likes being “nickled and dimed,” these practitioners were especially sensitive, as they felt that for the extra time, effort, and cost to integrate these additional data, they could build the analysis capabilities themselves for roughly the same cost, and with additional flexibility.

Data Security and Ethics – Collect, Measure, Share?

A recurring concern expressed by practitioners and vendors alike centered around the issue of compliance and ethics in data collection, use, and sharing — almost a third of those who responded to our poll listed issues of ethics, access, privacy, ownership, and security as a top issue they want to better understand. Vendors we spoke to also brought up issues of data democratization such as how do businesses view their position in terms of data transparency and what should they be doing with it.

One of the vendors we spoke to expressed concerns about not keeping employees in the loop with regards to the data collected on them.  As tools that monitor and collect passive data (which can often be sensitive ) become more popular, managing and sharing that data with employees is increasingly becoming an issue of concern and was frequently brought up in our conversations.

Another vendor we spoke to referred to the recent data privacy breach incidents at Facebook and Google and their impact on the conversation about data sources by bringing it to the forefront.

Some of the concerns raised by people analytics practitioners and vendors regarding this issue include these questions:

1. How transparent are the solution providers regarding the data they are collecting and providing access to among the various levels within the customer's organization?
2. What are some of the decisions made by solution providers to help customers protect their employees?
3. How much of the data do practitioners tell their employees they are collecting?
4. How involved and informed are the business leaders about the data that people analytics practitioners have?

Implementation and End-User Value – Will the Real People Analytics Solution Please Stand Up?

While there is a lot of marketing about the value of people analysis technology, practitioners want to understand – ideally from other practitioners – others' implementation experiences and how quickly and efficiently those solutions delivered value.

In particular, practitioners would like to see more real customer case studies and to have opportunities to discuss the “how-to” and “lessons learned” with other practitioners.  This will allow them to align their expectations and objectives and achieve greater value from these solutions.

In addition, practitioners would like more information about whom the technology is designed to serve. For example, they reported that they will often hear that a solution can be made accessible to HR Business Partners, only to find that it is too technical or difficult for someone who is not especially data-savvy. By having a more realistic understanding about whom the system is really designed for, people analytics practitioners will be better able to understand and highlight the value of specific technologies.

Vendors are also interested in gaining greater clarity on who the organization leaders see as the end-user of the product and the expectations and practical ways organizations intend to use their solutions. They want to understand from practitioners not only what they want, but for whom, and why they want it. As one of the interviewees we spoke to put it,

“ require a level of sophistication from HR folks to go into the tool and identify the questions they should be asking….The people analytics team, the HRBPs, or the communications… have such varied levels of maturity.”

And while upskilling and building data capabilities for the wider section of users is crucial, it should be matched by a clear identification of the primary user.

Overall Impressions

Both the conversations and poll results are reflective of our findings from the literature review: the people analytics technology landscape is a rapidly growing space with great potential. However, there remains an urgent need for greater clarity and knowledge sharing on issues of various technology capabilities, ethics and security, data integration, and transparency about real use cases on scaling insights for business value and custom solutions that fit business needs.

Our Next Steps

We recently closed our online vendor survey (and thank the more than 35 vendors who participated!). We are analyzing that data and holding vendor briefings over the course of summer. We will also be conducting a second round of interviews with practitioners during the summer which, along with the survey results, will inform our final findings. These findings will be presented in the fall at PAFOW Philadelphia conference, with the final report published later in the fall.