Events

Findings from our People Analytics Tech Interviews

Posted on Tuesday, July 2nd, 2019 at 5:43 PM    

Introduction

In the spring of 2019, when summer still felt like a distant dream, we conducted an online poll and more than 20 interviews with people analytics practitioners and solution providers to understand their current challenges and what they most wanted to know about the people analytics technology market.

We did this so we could better understand what our overall research – including our online survey and literature review – on people analytics technology should focus on. We figured you’re likely curious to know about what we learned, so we’ve summarized our findings in this article, specifically focusing on

  • Poll results
  • Interviews: what we heard
  • Overall impressions
  • Our next steps

Poll results

Our poll revealed some interesting insights into what is on top of minds when folks think of people analytics technology. We asked respondents (vendors, practitioners, and others with an interest in the field) to tell us what they think we should 1) cover in our research, 2) not cover in our research, and 3) any other feedback or ideas they wanted to share. The findings reaffirmed our suspicions: the top issue is better understanding the solution categories and capabilities for people analytics technologies.

Findings from our People Analytics Tech Interviews

Interviews: What we heard

Capabilities and needs – mo' solutions, mo' problems?

As our poll reflects, more than 70 percent of respondents indicated they want more clarity on people analytics technologies’ capabilities and a framework for thinking about the different solution providers. This sentiment was echoed in our interviews: when we asked folks to provide a way to categorize the market, we received as many different frameworks as interviews!

Our conversations revealed that while some group people analytics solution providers into categories of data storage, analysis, and visualizations, others think of them in terms of solutions that perform transactional/reporting versus analytics functions. Still others separated solutions into pure-HR players versus ones that cater to business intelligence capabilities.

Ultimately, both technology providers and organization leaders recognize the opportunity to work together to identify and use technology in a meaningful and purpose-driven manner. To do so, buyers and users should have a realistic and accurate understanding of the technology.

A good starting point for the users could be clarity, consistency and agreement on how they define people analytics and success within the organization. Buyers could also work with the vendors more, especially at the beginning of implementations, to put in place processes to ensure insights and findings from using the technology are shared broadly in an actionable way.

Similarly, vendors need a clear understanding of the business and talent challenges as well as the objectives of the organization. One of the vendors we spoke to stated,

“ typically start by saying that they have a data problem…and are trying to wrangle the data etc. Once we talk to them though, the problems that they are trying to solve for are business problems, such as turn over issue and are trying to wrangle the data to figure out what is driving the turnover.”

This points to the need for customers to spend time clarifying the business problems they are trying to solve and for vendors to spend time ensuring they understand those problems before implementing a solution.

Data Integration – Connecting the Dots

A topic that frequently popped up in our interviews was the opportunities and challenges of data integration from different sources. Some of the key questions we heard from practitioners during our interviews included

  1. Are there people analytics tools that can help practitioners integrate their data and allow for data interoperability?
  2. Which technologies speak to each other and can be used as an integrated suite?
  3. Do the existing tools require data be within one system or should it be siloed in different systems?

While some of these concerns tie in with the issue of capabilities that we touched upon earlier, it is clear that data integration is a major challenge  Practitioners we spoke to mentioned that a growing number of vendors claim to offer multiple services and solutions for all things people analytics, but fail to provide that. For example, one person stated,

“ really wish the technology vendors would stop promising a one-stop shop for all your analytics need…These technologies require a lot of thoughtfulness in implementation to have them be useful.”

Further, other practitioners indicated a frustration with vendors who have incremental additional charges for additional data sources. While no one likes being “nickled and dimed,” these practitioners were especially sensitive, as they felt that for the extra time, effort, and cost to integrate these additional data, they could build the analysis capabilities themselves for roughly the same cost, and with additional flexibility.

Data Security and Ethics – Collect, Measure, Share?

A recurring concern expressed by practitioners and vendors alike centered around the issue of compliance and ethics in data collection, use, and sharing — almost a third of those who responded to our poll listed issues of ethics, access, privacy, ownership, and security as a top issue they want to better understand. Vendors we spoke to also brought up issues of data democratization such as how do businesses view their position in terms of data transparency and what should they be doing with it.

One of the vendors we spoke to expressed concerns about not keeping employees in the loop with regards to the data collected on them.  As tools that monitor and collect passive data (which can often be sensitive ) become more popular, managing and sharing that data with employees is increasingly becoming an issue of concern and was frequently brought up in our conversations.

Another vendor we spoke to referred to the recent data privacy breach incidents at Facebook and Google and their impact on the conversation about data sources by bringing it to the forefront.

Some of the concerns raised by people analytics practitioners and vendors regarding this issue include these questions:

  1. How transparent are the solution providers regarding the data they are collecting and providing access to among the various levels within the customer's organization?
  2. What are some of the decisions made by solution providers to help customers protect their employees?
  3. How much of the data do practitioners tell their employees they are collecting?
  4. How involved and informed are the business leaders about the data that people analytics practitioners have?

Implementation and End-User Value – Will the Real People Analytics Solution Please Stand Up?

While there is a lot of marketing about the value of people analysis technology, practitioners want to understand – ideally from other practitioners – others' implementation experiences and how quickly and efficiently those solutions delivered value.

In particular, practitioners would like to see more real customer case studies and to have opportunities to discuss the “how-to” and “lessons learned” with other practitioners.  This will allow them to align their expectations and objectives and achieve greater value from these solutions.

In addition, practitioners would like more information about whom the technology is designed to serve. For example, they reported that they will often hear that a solution can be made accessible to HR Business Partners, only to find that it is too technical or difficult for someone who is not especially data-savvy. By having a more realistic understanding about whom the system is really designed for, people analytics practitioners will be better able to understand and highlight the value of specific technologies.

Vendors are also interested in gaining greater clarity on who the organization leaders see as the end-user of the product and the expectations and practical ways organizations intend to use their solutions. They want to understand from practitioners not only what they want, but for whom, and why they want it. As one of the interviewees we spoke to put it,

“ require a level of sophistication from HR folks to go into the tool and identify the questions they should be asking….The people analytics team, the HRBPs, or the communications… have such varied levels of maturity.”

And while upskilling and building data capabilities for the wider section of users is crucial, it should be matched by a clear identification of the primary user.

Overall Impressions

Both the conversations and poll results are reflective of our findings from the literature review: the people analytics technology landscape is a rapidly growing space with great potential. However, there remains an urgent need for greater clarity and knowledge sharing on issues of various technology capabilities, ethics and security, data integration, and transparency about real use cases on scaling insights for business value and custom solutions that fit business needs.

Our Next Steps

We recently closed our online vendor survey (and thank the more than 35 vendors who participated!). We are analyzing that data and holding vendor briefings over the course of summer. We will also be conducting a second round of interviews with practitioners during the summer which, along with the survey results, will inform our final findings. These findings will be presented in the fall at PAFOW Philadelphia conference, with the final report published later in the fall.


Employee Experience: What the Experts Say

Posted on Thursday, June 27th, 2019 at 6:41 PM    

Why is employee experience so sizzling?

Employee experience (EX) is hot. Don’t believe us? Just check out how it has trended on Google over the last decade.

While the term “EX” just sounds smart and snappy, there are more substantive reasons for the increased focus on employee experience, such as changing demographics,1 a hypercompetitive talent market,2 and business’ ravenous need for innovation.3 The heightened emphasis on EX is such that, in a global 2018 study of five hundred CHROs, 83% of organizational leaders emphasized a positive employee experience as crucial to organizational success.4

All this EX talk led us to some “burning” questions like, what exactly is this secret “employee experience” hot sauce? And if your organization chooses to focus on it, how do you approach it? And will it live up to all the hype?

To get our answers, we read the most relevant published works on employee experience, ranging from as early as 2011 to as recently as a few months ago (though the bulk of the articles were from the past three years). We also looked at related topics, such as employee engagement, customer satisfaction, and technology. In all, we examined over 60 scholarly and business articles, reports, blogs, and books.

This article gives you the highlights of what we learned and suggests publications for further reading. Here’s what you’ll find below:

  • What we saw
  • What caught our attention
  • Additional literature we recommend

What we saw:

We identified six primary themes within the literature we reviewed:

  1. Employee experience: A combination of employees’ perceptions and interactions
  2. Employee experience and engagement: Related but different
  3. Symbiosis: At the heart of employee experience
  4. Multi-method listening: Key to understanding employee experience
  5. HR: Not the only one responsible for employee experience
  6. Technology: Clarifying and enhancing employee experience

1. Employee experience: A combination of employees’ perceptions and interactions

It turns out, lots of folks have different employee experience “recipes” – meaning there’s no universal definition for employee experience. Instead, we found a range of definitions, such as:

  • How difficult it is to get work done and how people are expected to behave5
  • How the work environment and work habits enable employees to perform their jobs6
  • How employees interact with the physical environment, social connections, and work tasks7

We found the most convincing definitions, though, to be those that took a macro-level perspective and argued that employee experience originates in employees’ perceptions and interactions.8, 9 These perceptions are subjective interpretations of what employees encounter, observe, or feel in their interactions with the organization.10 Interactions are further described as the intersection of cultural, technological, and physical elements of the work environment with employees’ expectations, needs, and wants.11

2. Employee experience and engagement: Related but different

Employee experience and engagement are often used as synonyms, but our review of research-based articles underscores that these are different (though related) concepts.

For example, engagement and employee experience differ in their top-down vs. bottom-up approach.12, 13 Employee engagement is seen as a top-down concept because organizational leaders choose strategies that impact engagement scores. On the other hand, employee experience is seen as a bottom-up concept because it focuses on those perceptions and interactions employees have – not just on the measured engagement score.

Another difference between employee experience and employee engagement is that experience is described as the cause, whereas engagement is described as the effect.14 The combination of what organizations do, say, and give (experience) influences what employees see, feel, and hear (engagement).

These perspectives bring the employee front and center. They highlight the influential role that organizations play in how employees behave and approach their work.

3. Symbiosis: At the heart of employee experience

Just like honeybees and flowers, a positive employee experience results in a mutually beneficial relationship for both employees and organizations. This relationship is based on the impact that a positive employee experience has on engagement:

  • Organizations with transformational leaders and a supportive work environment – both aspects of employee experience – have better employee engagement, based on a comprehensive review of 214 studies.15
  • Employees with a high level of engagement are healthier and feel an overall sense of wellbeing. In contrast, employees with a low level of engagement experience more illnesses, stress, and burnout.

While this is great news for employees, they are not the only ones that benefit. While there is plenty of research that indicates a positive relationship between employee engagement and business outcomes, there is also research that indicates organizations receive business value directly from a positive employee experience. Specifically, companies with a strong employee experience have twice the innovation and customer satisfaction and have higher profits than organizations with a weaker employee experience.16

4. Multi-method listening: Key to understanding employee experience

More organizations are starting to recognize the dynamic nature of employee experience and the need to account for spontaneous and deliberate interactions.17 A dynamic employee experience evolves when employees interact with the organization in organic ways or through specific touchpoints.

To view the entire employee experience picture, organizational leaders increasingly want effective ways to measure both constant and sporadic interactions. Thus, a multi-method listening approach is gaining popularity.18 In addition to pulse or touch-point triggered surveys, organizations are also exploring the use of wearable technology devices and social networking applications to collect frequent data from various sources.19, 20

5. HR: Not the only one responsible for employee experience

Employee experience is often viewed as the primary responsibility of human resources or talent management functions. However, that viewpoint is changing.

HR does not own most aspects of employee experience, whether you measure it via “touchpoints” (42%) or “moments of truth” (78%).21 Instead, business managers or other non-HR support functions own them.

Thus, some authors recommend building a cross-functional coalition where members can approach employee experience from different, yet complementary angles based on their expertise.22 This cross-functional coalition includes shared accountability across HR, marketing, IT, finance, and facilities to design a positive employee experience.

6. Technology: Clarifying and enhancing employee experience

Technology provides an opportunity to better understand employee experience. It clarifies what employees want, value, and need. In fact, most CHROs (57%) believe that technology enables them to prioritize budget and time investments on employee experience.23

Organizational leaders also recognize that technology enables them to analyze, automate, and collaborate when designing a positive employee experience. For example, more technology vendors are offering artificial intelligence capabilities that provide a deeper view of employee experience than ever before.24

There is also an increased focus on providing a personalized digital experience that resembles what employees encounter as consumers. Employees want technology that enables them to communicate and connect with others in the organization.25 They also want technology that anticipates their needs and streamlines administrative processes. A large global survey highlights that most workers prefer digital as opposed to personal interactions to handle HR tasks.26

But there is a large gap between the technology that employees encounter at work and the one they encounter as consumers. To address it, organizations want to offer digital solutions that enable collaboration, anticipate employees’ needs, and appeal to individual preferences.

What caught our attention:

Of the literature we reviewed, we found the articles below particularly intriguing and useful. We learned from their perspectives and encourage you to do the same.

Maximizing the Employee Experience: How Changing Workforce Dynamics are Impacting Today’s Workplace

Dr. Brad Harrington

"Taking a systems-oriented approach as the foundation of an organization’s people strategy will require more cross-functional collaboration with HR’s various functions and much greater alignment and integration with organizational leaders."

Highlights:

  • Summarizes the significant workplace trends that influence organizations trying to positively impact employees' experience.
  • Describes the importance of developing and establishing an integrated people strategy that considers the organization’s mission, values, and workforce expectations.
  • Suggests that using a seven-step change management model may help in maximizing a positive employee experience.

Building Business Value with Employee Experience

Kristine Dery and Ina Sebastian

“In our research, companies with great employee experience  (i.e., low work complexity, and strong behavioral norms for collaboration, creativity, and empowerment) were more innovative and profitable and had higher levels of customer satisfaction.”

Highlights:

  • Discusses findings from MIT Sloan Center for Information Systems Research’s recent global survey on employee experience.
  • Defines employee experience according to work complexity and behavioral norms.
  • Connects a positive employee experience to innovation, customer satisfaction, and profitability.
  • Provides recommendations on how to build a positive employee experience through digital capabilities and leadership.

Employee Experience vs. Engagement, and 3 Things You Should Start Thinking About Now

David Sturt and Todd Nordstrom

“As leaders, employee experience is something we all should start looking at seriously… because, like it nor not, there’s a silent revolution taking place with employees all over the world. And, if we do our best, we will see those engagement scores move in the right direction.”

Highlights:

  • Discusses differences between employee experience and employee engagement.
  • Provides examples of ways in which organizations have tried to foster a positive employee experience.
  • Offers suggestions to build a positive employee experience by focusing on organizational culture, flexibility, and storytelling.

With CX, Engaged Employees Mean Everything

Bruce Temkin

“In companies that lead in CX, 75% of employees are highly or moderately engaged.”

Highlights:

  • Connects customer experience to organizational culture and internal processes.
  • Substantiates the notion that a positive customer experience is linked to highly or moderately engaged employees.
  • Recommends specific actions to increase engagement via the “Five I’s of Employee Engagement.”

Designing Employee Experience: How a Unifying Approach Can Enhance Engagement and Productivity

Eric Lesser, Janet Mertens, Maria-Paz Barrientos, and Meredith Singer

"Applying a holistic, iterative design approach to change can help ensure that employees see improvements relevant to their work and can set their expectations for continuous intervention."

Highlights:

  • Identifies factors that shape employee experience such as work-based relationships, the design of employees’ physical work environment, and the tools employees use in their work-related activities.
  • Emphasizes the importance of personalization, transparency, simplification, authenticity, and organizational responsiveness as ways for organizations to enhance employee experience.
  • Offers suggestions to design a positive employee experience through using analytics, understanding and investing in crucial touchpoints, and building a coalition.

How AI Can Help Redesign The Employee Experience

Forbes Insights

“The hope isn’t simply for AI to ensure smarter, faster hiring, but to improve the entire lifecycle of the employee experience, from job application to exit interview. In other words, can AI make workers happier?”

Highlights:

  • Identifies ways in which AI can create a positive employee experience.
  • Mentions specific tech vendors that gather data and provide insights into an organization’s employee engagement, manager effectiveness, communication, talent development, and organizational culture.
  • Describes how AI technologies can provide a “consumerized” employee experience to address HR needs.

Additional Recommended Reads:

  1. The Employee Experience in 2019,” Charong Chow, SocialChorus, 2019.
  2. The Heart of the Matter: Emotions in the Employee Experience,” Christina Zurek, ITA Group, 2019.
  3. Employee Experience: Enabling Your Future Workforce Strategy,” Kristine Dery, Nick van der Meulen, Ina Sebastian, MIT Center for Information Systems Research, 2018.
  4. The New CHRO Agenda: Employee Experiences Drive Business Value,” Pat Wadors, ServiceNow, 2019.
  5. Employee Experience Reimagined,” Michael Liley, Patricia Feliciano, and Alex Laurs, Accenture, 2017.
  6. 2018 Employee Experience Report,” Toluna Group & Udemy, 2018.

While we have identified the articles above as being the most crucial for readers to review, we read many others. If you’d like a full list of the articles we covered, please do not hesitate to reach out to us at [email protected].

Footnotes

 


Women and Men: Different Networks, Different Outcomes

Posted on Thursday, May 30th, 2019 at 3:34 PM    

Introduction

It’s old news that gender diversity matters.1 Yet, while the discussion around and evidence for gender diversity in organizations has grown, the number of senior women – especially women of color2 – in organizations has not.

For years, organizations have tried approaches such as mentoring and sponsorship programs for women and providing women-focused employee resource groups. Given that representation numbers remain stubbornly low, we clearly need a different approach.

We think at least part of the solution lies in better understanding employees’ professional connections. Networks, and the informal information they facilitate, are one of the primary ways people learn about career advancement and development opportunities. Research3 suggests that women and men's networks – and the information within them – are different, thus networks could vastly influence opportunities for career development and advancement. Understanding these connections between people – who knows whom and why – could help organizations understand why some employees rise and why others do not.

We looked at approximately 50 academic and business articles, reports, and books to better understand network dynamics, how they differ by gender, and the potential opportunity for technology to democratize or accelerate women's effective use of their networks. This short article will summarize the literature we reviewed in terms of the following:

  • What we saw
  • What we didn't see
  • What we learned

What we saw:

This literature review focused on research covering the fundamental differences between the networks of men and women and the resulting different outcomes. We also reviewed literature on the approaches organizations can use to help women create networks that support their advancement. The findings from this review fell into three major themes:

  1. Professional networks are different between men and women
  2. Differences in networks result in different outcomes
  3. Specific network characteristics matter for advancement

Professional networks are different for men and women

Men and women have different networks, at least partly due to traditional social dynamics4. Traditionally, women have been primarily responsible for life at home and, therefore, created more ties with the community, family, and friends. Likewise, men have traditionally been responsible for the financial stability of the family and created more professionally-focused networks.

While these traditional social dynamics have helped shape professional networks, other variables at play in organizations likely have a more immediate impact on the structure and composition of professional networks.

For example, men tend to hold a larger percentage of leadership positions and to be promoted more often than women. As a result, when men move up the ranks in organizations, they are more likely to be surrounded by other men (rather than women) in similar leadership or influential roles. As information and job opportunities tend to be shared within networks, men continue to have better opportunities due to their participation in these networks, perpetuating the gender imbalance at higher organizational levels.

Women, on the other hand, are, generally speaking, less likely to be promoted into senior roles. Those that are promoted tend to find themselves as part of a male-dominated professional network (only one in five C-suite executives is a woman5).  Women who do not move into leadership roles often sit in less influential, lower-status positions in organizations.6 These positions do not afford women the same connections into higher-status networks, curtailing access to information and sponsorship for better development and career opportunities.

Differences in networks result in different outcomes

While on the surface network differences may not seem like that big of a deal, those differences affect how individuals move around and up through an organization.  There is a tendency for professional networks to be closed, meaning individuals in one group are all connected to each other and the individuals within those networks have access to and share the same information. Therefore, new and unique information often does not flow into the network.

Closed networks tend to impact women differently than they impact men.7 For men, as they are more likely to be in networks with others in higher-status roles, information that is shared is likely to be important or strategic, which benefits them. Whereas for women, who are less likely to be in high-power networks, closed networks result in less important, less strategic information sharing.

Specific network characteristics matter for advancement

The literature indicates there are several things individuals should consider when developing networks to facilitate career growth. While these practices are not unique to women’s networks, being aware of these characteristics will enable women to better leverage their professional networks for advancement.

First, women need a small “inner group” of connections who can provide immediate support. Individuals benefit from being a central figure in a close-knit group. The literature indicates that these inner groups are crucial because they create a sense of communality and can tap into a woman’s shared experience.

Second, women need to become “brokers” – people connected to fundamentally different networks that can provide them with new and diverse sources of information.8 An individual who can position themselves as a broker between two high-powered groups, or who is directly connected to a broker, has a higher likelihood of accessing unique information and opportunities. The literature also hints that while this is an acceptable and even coveted position for men, women acting as brokers may experience some backlash (which could impact performance9).

This brings us to the third characteristic. To counteract backlash, women may also need to become “energizers” in their network. An energizer is someone that engages with others in a way that builds trust, instills a sense of purpose, and fosters an environment of psychological safety where people are not worried that they will be judged or dismissed. Because of this, energizers are presented with new opportunities and information more often than non-energizers. Research10 suggests that being seen as an energizer is a critical factor in advancement.

What we didn't see:

There were also a couple of thing we expected to see that just didn't show up:

What causes differences

The literature offered little insight into the actual mechanisms that cause differences in networks of men and women. So little has been written on the topic that we found ourselves accessing research in tangential fields to understand how and why these gaps may occur. This gap likely impedes organizational efforts to develop innovative and effective interventions.

Likewise, most of the research geared toward practitioners largely focuses on “fixing” women’s networks through mentoring, sponsorship, and Employee Resource Groups (ERGs). On the surface, these solutions seem like great ideas; however, because they are often not intentionally managed or designed, they may serve as a forum for the shared experience of women rather than a resource for helping women build networks and advance careers.

How technology can help

There was also little research that discussed using technology as a part of the solution to directly address the advancement of women. In fact, we feel that the gaps in current research have impacted the current technology solutions available. While there are technologies with network analysis capabilities, many – if not all – offer only descriptive reports on the structure and makeup of various networks. Descriptive statistics are a good start, but don’t point organizations toward potential drivers of these differences or suggest solutions.

What caught our attention:

In the process of putting this literature review together, we came across several articles that simplified complex ideas, offered unique solutions, or expanded our understanding of the topic.

The first three articles provide insight into social networks, how they impact aspects of our lives including work, and how different groups of women may face very different realities in the workplace. The final two articles are more academic and provide insight into characteristics of professional networks and their consequences for women. We’ve also included a bonus article that highlight network practices that are critical to the success of women in organizations. Finally, a list of additional readings is provided to offer further insight on the topics covered in this literature review.

Social Network Theory: Explaining Society

Tobias Stone

"…understanding the underlying mechanisms of social networks allows you to use them more effectively. Also, this descriptive analysis can explain some of the weirder things happening in our lives."

Highlights:

  • Provides a high-level overview of social networks
  • Breaks down basic aspects of social network theory and related concepts
  • Explains how social networks impact almost every aspect of our lives

This three-part series (provided via text and/or audio) gives an interesting look at social networks and dives into how they impact our world – from happiness to politics. It also asks the question as to whether technologies that tap into our networks are friend or foe.

Mentorship, Sponsorship, and Networks: The Power and Value of Professional Connections

Center for Women and Business at Bentley University

"…when it comes to professional networks, women and men find themselves in very different positions."

Highlights:

  • Provides a brief overview of important concepts and the benefits of mentorship and sponsorship to individuals and organizations
  • Outlines a framework for understanding networking
  • Offers insight on how to establish a mentorship culture for organizations and how to navigate networks for individuals

Not only is this a great review of the current research on sponsorship and mentoring, but it also brings in a network-based perspective. In addition, the list of resources and articles provided is a robust set of materials to help you further understand this topic.

Connections that Count: The Informal Networks of Women of Color in the United States

Catalyst / Katherine Giscombe

"Depending on the work environment, it may be difficult to form relationships at all with dissimilar colleagues, which would then lead women of color to turn to similar colleagues or people from outside the work organization for advice."

Highlights:

  • Dives into the increased complexities of professional networks for women of color
  • Discusses different network strategies women of color may use when dealing with barriers to organizational advancement
  • Provides recommendations for organizations to think past a "one-size fits all" mentality when addressing the professional networks of women

This article highlights the complex network challenges that women of color face. It forces the reader to see that solutions designed for women do not always apply to all women and solutions should not advance one group of women over others.

Do Women Suffer from Network Closure?

Mark Lutter

"Females … have a higher risk of career failure than do their male colleagues when affiliated in cohesive networks, but women have better survival chances when embedded in open, diverse structures."

Highlights:

  • Discusses the importance and impact of social capital, along with the two main ways that social capital is built (or borrowed) – cohesive and weak ties
  • Illustrates differences in female and male networks and the practical impact that has on women's careers

This is an examination of how women's networks impact career advancement and is an example of what other research is starting to uncover.

To Land Top Jobs, Women Need Different Types of Networks than Men

Kellogg Insight

"While it's true that highly connected women tend to land better jobs, the most successful women also have something they cannot get through ‘beers with the boys’…"

Highlights:

  • Summarizes the findings of a recent empirical investigation11 on the impact of differences between men and women’s networks among MBA students
  • Illustrates that women need a fundamentally different network to find career success
  • Discusses the difference in business networking between men and women

If you can get your hands on the original empirical article that this summary is based on, we highly recommend it. However, if you cannot, this overview hits all the major points and interviews one of the researchers to add context.

Bonus: Invisible Network Drivers of Women’s Success: How Successful People Manage Collaborative Overload

Inga Carboni, Rob Cross, Aaron Page and Andrew Parker

“If the ultimate goal of gender diversity efforts is to build organizations in which employees work together and evolve as professionals without regard to gender, then let’s focus on the relationships themselves.”

  • Suggests that understanding gender diversity requires organizations to think beyond implicit bias
  • Presents the BEST model to help organizations understand the critical networking practices that distinguish high-performing women, including boundary-spanning, efficiency, stickiness, and trust
  • Provides a number of tips for organizations to put their findings into practice

Additional Reading:

    • “What’s in the ‘old boys’ network? Accessing social capital in gendered and racialized networks,” S. McDonald, 2011.

Where Are We in the Great Performance Management Experiment?

Posted on Thursday, May 23rd, 2019 at 10:42 AM    

It has been a tad shy of a decade since organizations began redesigning their approaches to performance management (PM). While previous approaches focused mainly on top-down, annual, staid processes, organizations now have to go beyond simply managing performance: they need to enable it. More specifically, organizations need practices that motivate, engage, and develop employees through a more collaborative, dynamic, and personalized process.

Yet, with all the changes made to PM, people are still unhappy, unmotivated, and disengaged. In addition, practices hailed as innovated and forward-thinking haven’t shown themselves to be the “cure-all” they may have been touted as.

So, the question remains – Where are we in the great PM experiment?

We have partnered with Glint to answer this question and looked at 40 academic and business articles, reports, and books for this literature review.

What we saw in the literature

Not surprisingly, the literature shows a general consensus that the traditional annual PM process isn’t enough. As it turns out, most traditional processes don’t drive engagement, often don’t encourage development, and don’t focus on the employee experience. While the research recognizes that some of the more traditional aspects of PM are still very necessary, organizations have experimented with ways to better engage employees in the process.

And experiment they have. We found literature on everything from completely ditching the performance review to facilitating continuous conversations, to adaptable goals. And while the sentiment is right, we think that there is an overemphasis in the literature on the process (i.e., the nuts and bolts of how we conduct performance management and its individual components), and an under-emphasis on the changing relationship between employee and employer (manager and up) and what that means for  performance. Specifically, four general themes emerged:

Traditional approaches are no longer appropriate

The traditional model of performance management – the one introduced in the mid 1990s and discussed ad nauseum as an evil necessity – is unlikely to yield the results organizations want because it doesn’t focus on feedback as an informal, real-time way to engage and develop talent. It also disregards the importance of developing and maintaining a relationship between manager and employee, and instead, rigidly sticks to a standardized cycle that is not aligned with how and when work gets accomplished or when feedback is needed.

A “one-size-fits-all” approach to performance management can’t handle the highly dynamic and customized world we live and work in because it doesn’t take into account the work type nor the people in the organization.

Because of this, organizations are shifting performance away from a complex, top-down system to a mechanism that can enhance employee experience – turning something that happens to the employee into something that happens for the employee.

Ratings aren't the problem – we are

Over and over again, authors, particularly in more recent publications, urge caution about removing ratings. In fact, some authors say that organizations have reached premature conclusions about ratings: instead of fixing them, they’re buying into the myth that they serve no purpose and should be removed.

However, the literature also indicates that removing ratings does not exonerate organizations from providing feedback or evaluation; and it may make it harder. Evaluation is necessary to provide meaningful and personalized feedback so employees can improve.

A bigger problem appears to be managers’ inability or unwillingness to diagnose and confront performance problems1. This is particularly challenging in situations where ratings have been removed. And as organizations remove ratings, they need to rely more heavily on more frequent feedback in order to guide employees. Unfortunately, many organizations aren’t sure that their managers are either willing or able to have those feedback discussions2. Which brings us to our third theme.

Relationships are increasingly important

As organizations have replaced annual performance reviews with more regular feedback conversations, there has been a necessary renewed focus on the relationships in organizations – particularly those between employees and managers.

To do this effectively, organizations have to trust their managers to move past just managing projects to truly managing people. This has expanded the role of managers from beyond simply assigning work to one that also includes motivating and engaging team members and holding individuals accountable.

The literature also addresses the growing trend of peer reviews – the practice of employees providing open feedback to each other. This has prompted organizations to begin to think about how best to create a “culture of feedback” where everyone is able to provide quality feedback.

Fairness matters

Finally, fairness. As organizations adopt more frequent feedback and more open conversations, they also need to think through how they create an environment of trust and fairness. More specifically, employees need to feel that the feedback they get is credible and fair – regardless of whether it comes from a peer or a manager.

The literature points out that in most PM processes, subtle forms of bias exist, and these biases can create different outcomes for different groups. For example, similarity bias may subtly influence a manager to provide slightly higher ratings for someone more like them (i.e., same likes/dislikes, same gender and race, similar background). These biases are particularly relevant when talking about the relationship between manager and employee. Most PM systems are not yet set up to protect against them.

Interestingly, removing ratings doesn’t remove the potential for bias – it can actually increase it. When organizations remove ratings, they often replace them with fairly ambiguous criteria for evaluation which allow for much broader interpretation. This broader interpretation can lead to perceptions of unfairness and violate norms of trust.

Articles that caught our eye

Of the literature we reviewed, several pieces stood out to us. Each of the following pieces explored ideas that we found useful and interesting. We found them helpful in expanding the way we have been thinking about PM, its challenges, and its possible solutions.

Performance Management: A Marriage Between Practice and Science – Just Say “I Do”

Paul E. Levy, Steven T. Tseng, Christopher C. Rosen and Sarah B. Lueke

“Spoiler alert: the fix is not to blindly get rid of ratings.”

This chapter discusses recent criticisms of traditional PM practices and reviews them in light of academic research. In an effort to reduce the gap between practice and science in PM, the chapter highlights what organizations can do to improve their PM practices and where scholars should focus their research efforts.

Highlights:

  • Argues that practitioners are driving the criticism of PM and that the gap between science and practice needs to be addressed
  • Suggests that solutions to address criticisms of PM should come from both a practical and research-based point of view
  • Advocates that removing ratings should be the rare exception and not the general rule

"Re-Engineering Performance Management"

Ben Wigert and Jim Harter / Gallup, Inc.

“Performance management has buckled because organizations have prioritized measurement over development.”

This report presents research on why traditional practices are not working, insights on how to improve them, and expectations that today’s employees have for their employing organizations.  The authors recommend that organizations should create a culture of performance development by establishing expectations, continually coaching, and creating accountability.

Highlights:

  • Presents the research behind why traditional PM is not effective in today’s organizations
  • Discusses the changing nature of what employees expect from their organizations and how organizations can think through what (if any) changes are necessary in their approach to PM

"Straight Talk About Employee Evaluation and Performance Management"

Lucia Rahilly, Bryan Hancock and Bill Schaninger / McKinsey

“…there is still no substitute for the direct feedback and coaching that happens day in and day out…”

This podcast, with transcription provided, discusses recent research by McKinsey on what drives effective PM. The discussion focuses on the role of the manager to engage in quality performance and development conversations with direct reports, the need for some sort of evaluative component, and the finding that perceptions of fairness impact the degree to which PM is seen as effective.

Highlights:

  • Discusses the current trends in PM and the necessary reliance on the ability of managers to provide coaching and feedback
  • Explains that people still want to know how they’re performing and that some sort of evaluative component is likely necessary
  • Illustrates the importance of perceptions of fairness in the PM approach

"3 Biases that Hijack Performance Reviews and How to Address Them"

Beth Jones, Khalil Smith, and David Rock

“…not all biases make us actively malicious. The key is how we manage our biases.”

The article discusses bias from a neuroscience perspective, highlighting that bias is our brain’s constant search for efficiency. While bias is not inherently bad, it can lead to negative outcomes if left unexplored. The authors discuss three biases – expedience, distance, and similarity – and how managers and organizations can mitigate their impact on performance appraisal.

Highlights:

  • States that bias negatively impacts performance appraisal and briefly discusses the impact of three prevalent biases
  • Provides high-level suggestions on how to mitigate the influence of these biases in performance appraisal

"Putting the System into Performance Management Systems: A Review and Agenda for Performance Management Research"

Deidra J. Schleicher, Heidi M. Baumann, David W. Sullivan, Paul E. Levy, Darel C. Hargrove and Brenda A. Barros-Rivera

"…much work is yet to be done in developing a body of scientific knowledge about performance management systems that can better inform practice."

While this article isn’t particularly provocative or stirring (it's why we put it last), it does provide a foundational summary of current PM research, which is helpful in understanding more progressive and innovative perspectives. The authors present a model of PM and summarize research from 1980 to 2017. Based on this review, they provide recommendations for future research in PM. This article is great for leaders and practitioners that want to geek out on the history of PM.

Highlights:

  • Presents a model of PM to organize components of PM and to integrate perspectives
  • Suggests that there are only seven core tasks involved in PM
  • Illustrates the importance and value of both formal and informal components of PM
  • Builds a case that we’ve excluded the examination of important variables in PM
  • Argues that more research is still needed on PM

Overall Impressions

The literature on PM is vast and varied, and there are many, many smart people with different perspectives. We’re pretty sure no one perspective is the “right” perspective. That said, we’re starting to see large-scale agreement for the notion that traditional, top-down, annual-driven PM is less likely to reign supreme in the workforce of the future. With this shift, we think we’ll also see an increased emphasis on the role of relationships in organizations, the expectations of managers, and the importance of trust and fairness in PM approaches.


Managing and Enabling Performance

Posted on Thursday, April 25th, 2019 at 6:04 AM    

It feels like we have been talking about performance management forever. In our conversations, organizations are telling us that performance management practices are not just ineffective; in some instances, they’re painful. In fact, when we recently asked managers in one organization how performance management was handled, our two favorite answers were: “with a pointy stick” and “like it’s 1979.”

Why this is important:

We are at least 7 years into the performance management “revolution”, and yet, many (rightfully) have a healthy skepticism about whether the new approaches are any better than the old. In fact, recent research by McKinsey & Company found that 54% of respondents still don’t believe that performance management has a positive impact.

Despite myriad articles and case studies about simplified ratings (or no ratings at all), separate performance and compensation conversations, real-time feedback and the technologies that support it, agile goal-setting, and coaching and development as a part of performance management, most organizations would admit that performance management is still a struggle.

Today, we’re launching a formal research study, sponsored by Glint, into what organizations are doing with respect to performance practices and how effectives those practices are.

Hypothesis:

Peformance shouldn't just be managed; it should also be enabled.

The premise for this project is that we have moved from an era where organizations only had to concern themselves with performance management to one where enabling performance is equally as important.

In the beginning…

A hundred years ago, performance management was much less complicated than it is today. Organizations were looking for a way to understand the return on their people investment. During the first industrial revolution, as assembly lines and other methods for increasing efficiency made an appearance, people were plugged into those systems and processes.

The worth of an employee was much more easily quantified than it is today. Tasks had a “right” way, and because measures associated with those tasks were objective (e.g., cycle time, number of widgets), it was easy to determine how well an individual performed based on those numbers. It created a transparent and equitable way to compensate employees.

The times, they are a-changin’…

However, we’re not living through the first industrial revolution; we’re living through the fourth. And there are some differences.

First, the work organizations are asking employees to do is different and much harder to measure. There are increasingly fewer ‘right’ ways to do a task because employees are continuously adjusting to changes in both internal and external environments. Instead of a set way of doing things with a set way of measuring things, employees are often left with nebulous job responsibilities and organizations are left with even more nebulous ways of assessing them.

As a result, “performance” has necessarily become less of a technical term. Instead of objective measures, shifting goals and strategies make it necessary to use more subjective measures when appraising employee performance.

Second, employee expectations have also shifted. As our societies continue to evolve toward a knowledge and information economy, compensation has become only one of many factors that keep employees with an organization. Employees now choose roles and tasks, not solely based on compensation, but also on things like how interesting they find the work, whether there is room for growth, opportunities for recognition, and what types of skills they can learn.

These expectations have prompted organizations to supplement traditional performance management processes. They are no longer just worried about the outcome or throughput; if organizations invest in individuals, they also want to measure their propensity to learn and their attitudes as a part of their overall performance.

The result…

As a result of these changes, many organizations have implemented a hodgepodge of old practices that don’t quite fit current conditions, and new practices that are applied haphazardly. As organizations try to blend the old with the new, gaps are introduced that result in frustration and uncertainty, both for the organization and for their employees. We currently see that frustration in the market. Gaps include:

Focusing on management or enablement, but not both

It is likely that during this last performance management revolution that we threw the baby out with the bathwater. Performance appraisals are an important part of how organizations compensate and create fairness, and throwing them out has caused havoc in more than one company.

As we begin to adopt some of the new ways of doing things, it becomes doubly important that we figure out how we fulfill some of the more traditional reasons for performance management. (By the way, if you’ve managed to create performance management practices that balance both of these, we would love to talk to you.)

Treating performance as a stand-alone challenge

Currently, most organizations look at performance and performance management as a separate set of practices and measures. However, how employees perform is ultimately the culmination or result of all other people practices.

This means that incentives, recognition, compensation, wellness programs, career management, coaching and mentoring, and engagement programs impact the performance of the organization. Truly enabling performance means taking a holistic approach to people practices by looking at all parts of the organization and understanding how they impact individuals and their ability (or willingness) to perform.

Missing the perspective of the employee

Most modern performance management practices have been conceived of by leaders of organizations and management consultants. Strangely, a lot of organizations limit upstream feedback to engagement surveys and the occasional 360. In the case of performance management, we think employees are in a unique position to tell us what would be useful to them in order to perform better. As a part of this study, we will be asking them.

Ignoring inherent bias

Performance management processes as implemented in most organizations are inherently biased. Most processes are largely subjective, no matter how many times a year a conversation is conducted. While a manager’s and peers’ opinions are important and valid to appraise performance, new technologies and access to new data can provide additional information that may create more fairness in the process and provide the organization with much more information about who is performing well at what tasks and in what instances.

We want to explore that further.


What's the Link Between Employee Experience and Customer Experience?

Posted on Thursday, April 18th, 2019 at 1:10 AM    

Why we care:

It seems like everywhere we look folks are talking about employee experience (EX). In fact, a global 2018 study of five hundred CHROs, found that 83 percent of organizational leaders believe a positive employee experience is crucial to the organization’s success.1

Why is this the case? Some folks mention the hyper-competitive talent market and the expanding need for innovation. However, an equally critical reason is financial: many believe there is a clear connection between employee experience and customer experience (which should then drive revenue).

While this makes sense intuitively, there are still many unanswered questions:

  1. What, exactly, comprises employee experience?
  2. What is the relationship between employee experience and employee engagement?
  3. How does employee experience impact customer experience (CX)?2
  4. What is the measurable impact of employee experience on customer experience?

Understanding the answers to these questions will allow leaders to make much more strategic employee experience investments.

This is why we are launching a new research initiative, sponsored by Medallia, focused on these questions. We have already begun our analysis of existing literature and are actively looking to interview organizational leaders (is that you? Email us!) who can share their stories (the good, the bad, and the ugly) of how they have used employee experience to impact customer experience.

Hypotheses:

There is not a clear definition of employee experience. For example, some believe that employee experience is the result of connection, meaning, impact, and appreciation that employees find in their jobs – and builds on the foundations of culture and engagement3. Others see employee experience as being more akin to customer experience – using technology to make employee life more personal, predictive, and seamless4.

Though neither of those definitions of employee experience are aligned, they are different from employee engagement, which itself does not necessarily have a clear definition. For example, some5 defined employee engagement as the harnessing of organization member’s selves to their work roles. Others,6 stated it is a positive fulfilling, work-related state of mind that is characterized by vigor, dedication, and absorption. And, yet another academic7 defined it as an “employee’s willingness and ability to contribute to company success by freely giving the extra effort on an ongoing basis.”

Finding clarity on the differences and similarities of employee experience and engagement is our starting point.

Despite the competing definitions, research8,9 indicates that both employee engagement and experience influence customer experience and satisfaction, but the way they do that may be different. To better understand where and how organizations should invest, we need to have a framework for thinking about all of these concepts.

In addition, we believe there are insights that can be drawn from the customer experience world,10 such as thinking about experience as both static (e.g., a specific point-in-time or interaction) and dynamic (developing collectively over time), that can be applied to our understanding of employee experience.

Finally, we believe that organizations where employees can take action on employee experience and engagement insights tend to see better customer experience. To create a culture that enables these actions, organizations need to consider desired behaviors, leadership activities, information sharing protocols, decision-making rights, incentives and technology.

This Project:

Some of the ideas this research project will explore include:

What’s the Link Between Employee Experience and Customer Experience?

Please take a few moments to share your comments with us!


Learning Impact: Anything New?

Posted on Friday, March 8th, 2019 at 1:55 AM    

Introduction

If you have been following our Learning Impact project, you know that the main premise of this research is that we’re evaluating “learning” in organizations all wrong. Therefore, in conducting a fairly in-depth review of existing literature on the topic, we were not at all surprised about the state of learning impact – only somewhat disappointed.

We looked at over 50 academic and business articles, reports, and books for this literature review, which has given us a decent understanding of the known world of evaluating learning. This short article will summarize:

  • What we saw
  • What we learned
  • Overall impressions

Learning Impact: Anything New?

Word cloud of the learning impact literature: Most prevalent words in literature reviewed.

What we saw

What we hoped to see in the literature were new ideas – different ways of defining impact for the different conditions we find ourselves in. And while we did see some, the majority of what we read can be described as same. Same trends and themes based on the same models with little variation. We have highlighted four of those themes below.

Models, models, models!

Much of the literature focused on established evaluation models. These articles generally fell into three main categories: use cases, suggested improvements, or critiques.

By far, the most common model addressed in the research is Kirkpatrick, although, given the number of articles written on how to use it effectively, adaptation is still a challenge. It was frequently called the “industry standard” (including by Kirkpatrick Partners themselves).1

Articles on Kirkpatrick appeared to be fairly passionate, either for or against. While many authors doubled down on it, we also ran across several articles, like this one or this one, that offer what we consider to be fair critiques of the model, including that there is a lack of empirical evidence for existing learning evaluation models and their tie to business results.

Other models, including Phillips’ Chain of Impact, Kaufman’s 5 Levels of Evaluation, Brinkerhoff’s Success Case Method, and Anderson’s Return on Expectation, among others, were also explored. In total, we looked at over 20 models. They are summarized in the table below.

Evaluation Model Summary

 

Model Year Incredibly simplified steps  Read more
Kirkpatrick 4 Levels of Training Evaluation

Kirkpatrick

1976 Termed “the industry standard” by many of the articles we read, Kirkpatrick’s four levels are used widely to determine learning effectiveness.

  • Reaction
  • Learning
  • Behavior
  • Results
“The Kirkpatrick Model”
Kaufman’s 5 levels of Evaluation

Kaufman, Keller, Watkins

1994 Kaufman’s model adapts Kirkpatrick’s original model to include 5 levels and is used to evaluate a program from the employee’s perspective.

  • Input and process
  • Acquisition
  • Application
  • Organization Output
  • Societal outcomes
“What Works and What Doesn’t: Evaluation Beyond Kirkpatrick”
Success Case Method

Robert Brinkerhoff

2006 Particularly effective in assessing important or innovative programs. It focuses on looking at extremes: most successful and least successful  cases, and examining them in detail.

  • Needs assessment
  • Program plan and design
  • Program operation and implementation
  • Learning
  • Usage and endurance of training
  • Payoff
“Success Case Method”
Chain of Impact

Jack Phillips

1973 Adapts the Kirkpatrick model by adding a fifth step: ROI. Purpose is to translate business impact of learning into monetary terms so that it can be compared more readily.

  • Evaluation
  • Reaction
  • Learning
  • Behavior
  • Results
  • ROI
ROI Institute
Value of Learning Model

Valerie Anderson

2007 Consists of a three-stage cycle applied at the organizational level. One of the few models that does not necessarily use the course or initiative as the unit of measurement. Anderson also introduced the term, “Return on Expectation” as a part of her work.

  • Determine current alignment against strategic priorities
  • Use a range of methods to assess and evaluate the contribution of learning including learning function measures, ROE – expectation, ROI, and Benchmark and capacity measures
  • Establish the most relevant approaches for your org
A new model of value and evaluation: A new model of value and evaluation
CIPP Model Daniel Stufflebeam 1973 Framework was designed as a way of linking evaluation to program decision-making (i.e., making decisions about what happens to the program). Has a use case for resource allocation and/or cost-cutting measures. Utilizes the following areas:

  • Context
  • Input
  • Product or results
  • Analyze
“The CIPP Evaluation Model: How to Evaluate for Improvement and Accountability”
UCLA Model

Marvin Alkins

Dale Woolley

1969 Five kinds or need areas of evaluation – each designed to provide and report information useful for making judgments relative to the categories:

  • Systems assessment
  • Program planning
  • Program Implementation
  • Program Improvement
  • Program certification
“A Model for Educational Evaluation”
Discrepancy Model 1966 Used in situations where there is an understanding that a program does not exist in a vacuum, but instead within a complex organizational structure.

Program Cycle Framework:

  • Design
  • Installation
  • Process
  • Product
  • Cost-benefit
“ABCs of Evaluation”
Goal Free Evaluation

Michael Scriven

1991 Focuses on actual outcomes of a program rather than only those goals that are identified. Scriven believed that goals of a particular program should not be taken as a given.

  • Goals and objectives
  • Processes and Activities
  • Outcomes
“The ABCs of Evaluation”
LTEM: Learning Transfer Evaluation Model

Will Thalheimer

2018 Designed to help organizations get feedback to build more effective learning interventions and validate results.

  • Tier 8: Effects of Transfer
  • Tier 7: Transfer
  • Tier 6: Task Competence
  • Tier 5: Decision-making Competence
  • Tier 4: Knowledge
  • Tier 3: Learner Perceptions
  • Tier 2: Activity
  • Tier 1: Attendance
“The learning-transfer evaluation model: Sending messages to enable learning effectiveness.”

Justification as the goal

Much of the literature reviewed focused on utilizing learning evaluation, measurement, and analytics to either prove L&D’s worth to the organization or to validate L&D’s choices and budget. Words and phrases like “justify” and “show value” were used often.

Interestingly, according to David Wentworth at Brandon Hall Group, the pressure to defend L&D’s decisions and actions appears to be coming from the L&D function itself (44%), rather than other areas of the business (36%),2 which means, while business leaders may not be explicitly asking for “proof”, L&D departments most likely feel the need to quantify employee development in order to have that proverbial seat at the table.
Literature also focused heavily on Return on Investment, or ROI. How-to articles and research in this space continues to attempt to tie the outcomes of a specific program or initiative to financial business results.

Course-focused

Almost all of the literature we reviewed utilized the ‘course’ or ‘program’ as the unit of measurement. While several models address the need to take into account environment and other variables, they appear to do so in order to either control the entire experience or be able to isolate the “learning” from everything else.

To date, we have not been able to find any literature that addresses evaluating or measuring continuous learning as we understand it (i.e., individuals utilizing the environment and all resources available to them to continuously develop and improve). We feel that this is a shortfall of the current research and should be addressed.

Finally, the research focused heavily on learning from the L&D function’s point of view. Few appear to be looking at the field of learning evaluation/measurement/analytics from a holistic viewpoint. We expected to see more literature addressing L&D’s role in delivering the business strategy, or at lease providing information to other functions that could be helpful to them in making decisions.

Aged ideas

While we do not disparage any of the great work that has been done in the area of learning measurement and evaluation, many of the models and constructs are over 50 years old, and many of the ideas are equally as old.

On the whole, the literature on learning measurement and evaluation failed to take into account that the world has shifted – from the attitudes of our employees to the tools available to develop them to the opportunities we have to measure. Many articles focused on shoe-horning many of the new challenges L&D functions face into old constructs and models.
We realize that this last finding may spark a heated conversation – to which we say, GOOD! It’s time to have that conversation.

5 articles you should read

Of the literature we reviewed, several pieces stood out to us. Each of the following authors and their work contained information that we found useful and mind-changing. We learned from their perspectives and encourage you to do the same.

Article 1:  Making an Impact3

Laura Overton and Dr. Genny Dixon at Towards Maturity

“96% are looking to improve the way they gather and analyze data on learning impact. However, only 17% are doing it.”

Highlights:

  • Points out areas where L&D functions measure and what is important to them
  • Provides an interesting discussion on evidence to understanding impact
  • Shows compelling data about benefits of those who measure vs. those who guess
  • Gives some good hints for getting started

Towards Maturity’s 2016 report provides some interesting statistics about the world of learning metrics / measurement / analytics / evaluation. This article provides a sound platform for continued research on learning measurement and evaluation, as it provides a good summary of how learning leaders are currently thinking about the space.

Article 2:  Human Capital Analytics @Work4

Patti Phillips and Rebecca Ray at The Conference Board

“Aspirational organizations use analytics to justify actions. Experienced organizations build on what they learned at the aspirational level and use analytics to guide actions.”

Highlights:

  • Outlines an analytics maturity model for organizations to gauge their evolution when it comes to using data.
  • Provides a good discussion on “converting data to money”, or utilizing data to provide a comparison of cost savings
  • Identifies four key elements to help organizations make analytics work: Frameworks, Process Models, Guiding Principles, and Capability, all of which should be considered when putting together a learning strategy
  • Recounts some good examples and case studies

This article broadens the discussion about learning measurement to people analytics in general – something that L&D functions should be considering as they revamp their measurement and evaluation methods.

Article 3: The Learning-Transfer Evaluation Model5

Will Thalheimer at Work-Learning Research, Inc.

“For too long, many in the learning profession have used these irrelevant metrics as indicators of learning.”

Highlights:

  • Honest (yet biting) assessment of the current 4-level models and their success to this point in time
  • Section about the messages that measurement can send
  • Discussion on measuring negative impact, as well as positive impact
  • Introduction of the first new model for learning evaluation in about 10 years

Will addresses several points that have evolved our thinking. On top of that, Will is a witty writer who is easy to read and downright entertaining.

Article 4: Making data analytics work for you – instead of the other way around6

Helen Mayhew, Tamin Saleh, and Simon Williams

“Insights often live at the boundaries.”

Highlights:

  • Emphasizes the importance of focusing on “purpose-driven” data, or data that will help you meet your specific purpose.
  • Introduces the idea that large differences can come from exploiting and amplifying incrementally small improvements.
  • States that incomplete information is not useless and should not be treated as garbage – it has value and can be essential in helping people connect the dots
  • Provides a good discussion on using feedback loops instead of feedback lines

This article addresses data analytics in general, but provides several applicable points that L&D departments can incorporate.

Article 5: Leading with Next-Generation Key Performance Indicators7

Michael Schrage and David Kiron

“Measurement Leaders look to KPIs to help them lead — to find new growth opportunities for their company and new ways to motivate and inspire their teams.”

Highlights:

  • Provides a decent discussion on Key Performance Indicators and what they currently mean in organizations
  • Points to Chief Marketing Officers and their increasing accountability for growth-oriented objectives (we think CLOs and L&D in general are close behind).
  • Has an excellent discussion on leading versus lagging indicators, and the importance of both in “measuring”.
  • Recounts several good case studies that helped us think differently about what a KPI is and how it can be used

We found this article eye-opening. While it is not geared specifically to “learning”, it provides several, adaptable ideas that we feel will be important for next-generation learning measurement and evaluation.

Bonus: 4 Measurement Strategies That Create the Right Incentives for Learning8

Grovo

“As human beings, we are compelled to adapt our behavior to the metrics we are held against.”

Highlights:

  • Has a great discussion on how even the act of measuring learning can be a motivation.
  • Introduces several non-traditional ways to “measure” learning
  • Makes the point that measurement strategy should be a part of the learning strategy, not as a way to measure its effectiveness.

Yes, we know it’s a blog, and yes, we realize it was written by a vendor. But this piece made some interesting points – particularly, how what we measure impacts the business.

Overall Impressions

If we were to sum up all we read into a short statement, it would be this: L&D has a long way to go. That said, we are also hopeful. As L&D functions further integrate into the rest of the business, as tools for analytics and measurement get better, and as we begin to define new models that incorporate new ways of learning and new environmental variables, we can imagine a world, in the not too distant future, where we finally – after more than 50 years of trying – maybe crack this nut.

We would love to hear what you think – what did we miss? What else should we be looking at? Comment below.


D&I Tech: The Rise of a Transformative Market

Posted on Tuesday, February 5th, 2019 at 10:47 PM    

In this Research:

Diversity and inclusion is not a new idea for today's corporations, but over the last 18 months, the slow D&I burn has turned into a flashpoint, in part due to the #MeToo moment. Leaders across organizations are asking: "How can we systematically challenge the status quo, and build a more diverse and inclusive workforce?"

D&I Technology Rise of a transformative market

It is upon this foundational question that technology companies have begun to construct dozens of new and innovative ideas to support equity, diversity and inclusion in the workplace—recognizing that new technological capabilities, paired with this increased urgency, represents an opportunity to address D&I challenges in novel ways.


A Smart First Buy: Culture Amp Acquires Zugata

Posted on Wednesday, January 30th, 2019 at 11:58 PM    

Culture Amp announced today it acquired Zugata, a continuous performance management technology company. I was thrilled to hear about this acquisition – Culture Amp’s first – as it has real potential to both increase the competitiveness of the talent management technology landscape and significantly benefit both Culture Amp and Zugata’s customers. In this blog, I’m going to explain what is happening more broadly in the market and then specifically focus on this new acquisition.

As I wrote last year, talent management practices are converging; in particular, the following areas are coming together:

  • Performance management (PM)
  • Employee engagement
  • Recognition
  • Learning / career management

This is happening because organizations are trying to better “listen” to employees by more holistically collecting feedback (on employee engagement, experience, and performance) and using that insight to drive culture. This shift is a natural outcome of the massive changes we’ve seen in performance management across the last eight years or so, where more frequent performance conversations and seeing the manager / employee relationship as critical to employee engagement have become the norm. In this new world, the employee and manager are the center of these talent management practices, and they are using technology tools to regularly check in on, enable, and encourage performance, engagement, and experience.

Unfortunately for HR technology customers, though, there has been a hole in the technology market. Many of the traditional talent management systems vendors have not been able to create this employee-centered system that enables continuous, circular feedback and development opportunities. Further, the best-of-breed solutions have – as you would expect – only focused on their primary silo, and not offered enough capabilities to provide the experience either.

But that began to change about 18 months ago. We’ve recently seen a wave of acquisitions and capability build-outs to meet these multiple needs (and I’m sure this list is not comprehensive):

So, in short, you can see that a lot of previously best-of-breed vendors are bringing together at least three of these four capabilities.

Which brings us to today’s announcement. I was thrilled to hear about this acquisition for many reasons (detailed below), but the biggest one is that it allows Culture Amp to go beyond its traditional employee engagement / experience offering and now provide performance management, recognition, and some aspects of learning, via Zugata.

Let’s talk about why this makes sense for Culture Amp first. For years I have urged Culture Amp to think more about how to serve the performance management space, given the above trends and the strong demand from their customers for a PM solution (many Culture Amp customers have been customizing their solution for years to meet performance management needs). Second, Culture Amp’s customers are deeply passionate about their product and driving meaningful, data-driven change in their organizations. If there is a technology vendor that could further help the performance management revolution, Culture Amp is it. Finally, Culture Amp is competing in the highly-competitive employee engagement / experience market, and a performance management offering helps them further differentiate their product.

So why Zugata? For the last few years, I have had a short list of innovative PM solutions, and I’ve told anyone that asked that they should check out Zugata (someone finally listened!)  There are lots of reasons I like Zugata, in addition to the fact that they offer performance management, recognition (praise only, no rewards), and learning.

First, I love how Zugata incorporates organizational network analysis (ONA) into its continuous feedback technology. By doing this, Zugata can help identify the people an individual has worked with the most in a given period, and then solicit ongoing feedback from the most relevant folks. Further, since Zugata is doing this, it can avoid over-asking for feedback from any one individual, which is a real challenge in the continuous PM era. In addition, this built-in ONA capability can help people understand who they work with the most and the composition of their own network – which can be useful for understanding where the network may need to be built more.

Second, Zugata offers natural language processing (NLP) of performance feedback and allows leaders to analyze key themes. For example, an organization can use NLP to identify the feedback given to high performers in the company to understand the behaviors / attributes that are being rewarded the most (frequent positive mentions) versus the least (infrequent or negative mentions). The company can then compare those highly rewarded behaviors / attributes to the published values / behaviors / attributes to see if the company is “walking the walk” in terms of what is valued within the culture. Further, as I’ve written about in our research on diversity and inclusion (D&I) technology, Zugata allows organizations to see if specific words or capabilities are being commented on for one gender more than the another (e.g., focusing on work output more for men and personality more for women). This can help an organization begin to root out unconscious bias that is occurring within performance management.

Third, Zugata uses the data within its system to suggest specific learning opportunities, offering an expanded development library from the likes of Lynda.com and Skillshare. While there are a lot of technologies that do this, what has impressed me most about Zugata is that it uses the information on who has what specific strengths and then can suggest mentor matches for folks who need to develop those specific strengths. Further, if a mentor is not available within a company, organizations can leverage Zugata’s relationship with The Muse, and given their employees access to career coaches. This represents a focus on extending beyond content to recommending relationships, which is an important shift.

Finally, I am excited about this acquisition because I think there is a strong culture fit between these two companies. Culture Amp remains laser-focused on culture (as you would expect), and in particular cares deeply about diversity and inclusion. As you may expect from my commentary on Zugata’s D&I capabilities, Zugata also cares about making the corporate world more diverse, inclusive, and equitable.

This acquisition is likely to force the market to advance. It will no longer be enough for these providers to loosely stitch together the different capabilities (engagement / experience assessments, performance management, recognition, and learning), given the number of players who will offer those capabilities. Instead, technology vendors will have to provide an increasingly seamless solution that has a clear vision of the value it provides to workers, managers, leaders, and HR. This will benefit everyone.

I should not conclude without mentioning, as always, there will be challenges for both companies in managing and executing the integration. And, as we all know, most acquisitions are not successful. However, I am very bullish on this one and wish Didier and Srinivas and their teams well as they continue their efforts to become one big Culture Amp team.

Disclosure: We do not serve on the Board of Directors of any companies mentioned and are not receiving any compensation for this post. We have no positions in any companies mentioned and no plans to initiate any positions.

Women, Networks, and Technology

Posted on Saturday, January 19th, 2019 at 7:50 PM    

Why this is important right now:

Research abounds showing the positive impact of diversity – and gender diversity in particular – on an organization’s outcomes. For example, a recent McKinsey study showed 47% higher return on equity for companies with women on executive committees.1

However, women are unique in that they are the only historically disadvantaged group who make up nearly 50% of the workforce. Despite that, they are woefully underrepresented at top levels. Fewer than 5% of S&P 500 CEOs are women and only 26% of senior management positions are occupied by women.2

To address this, organizations have recently invested in unconscious bias training in droves. However, it is not at all clear that unconscious bias is the villain. One large-scale analysis of more than 80 research studies and 17,000 individuals found no reliable relationship between measures of unconscious bias and actual behavior.3 And even if unconscious bias did affect behavior in some cases, simple awareness cannot remove implicit bias. It cannot be trained away. Diversity training, in fact, is one of the least effective methods to promoting diversity and inclusion.4 It may even make matters worse.5

Hypotheses:

Exclusion from informal professional networks has been identified as one of the greatest barriers to career success.6 One multinational study of over 240,000 men and women found that while 81% of women report some form of exclusion at work—astonishingly—92% of men don’t believe that they are excluding women at all!7

However, research shows that men’s and women’s networks do not seem to follow consistent patterns, revealing that solving the problem is not so as easy as simply identifying new ways in which women should build their networks. Instead, we believe organizations may need to re-think work partitioning, training, mentoring, sponsorship programs, and collaborative technologies to create opportunities for professionals to develop effective working relationships built on understanding and trust.

To that end, RedThread Research is excited to announce our new research initiative on how women use their networks to advance in organizations and the potential opportunity for technology to amplify those network behaviors. This research is being supported by GSV AcceleraTE. The final research report will be previewed at the 2019 ASU GSV Summit, April 8-10 in San Diego, and published shortly thereafter.

This Project:​

This research will focus on identifying how women can more effectively use existing opportunities, overcome factors that hinder performance, examine the role of technology, and make recommendations on what can be done by women and men as individuals, and organizations as the system in which people work, to improve women’s likelihood to rise in companies.

More specifically, we will examine the following topics:

Women, Networks, and Technology: Premise

Participate:

The above list represents our initial hypotheses as to what the study will cover. However, one of our core values at RedThread is collaboration and we need you to be a part of the process. We are collaborating with Dr. Inga Carboni, Associate Professor of Organizational Behavior at the College of William and Mary, to conduct interviews now through the end of March. If your organization is doing anything interesting on this topic, we encourage you to reach out to us and share your input at [email protected].

We are currently looking for folks to participate in our research around these topics:

  • How organizations help women advance in their organizations
  • How organizations help women build and develop important professional relationships inside and outside their organizations
  • How organizations are helping women design their networks intentionally
  • The importance of women’s networks and relationships in enabling them to advance within the organization
  • The role or potential opportunity of technology to democratize or accelerate women effectively using their networks

If you have an interesting story to share about the topics above, wish to participate in an interview, or have recommendations, please contact us.

RedThread Research is an active HRCI provider