A lot of change practitioners are extremely comfortable with saying that change management is about attitudes, behaviours, and feelings and therefore we cannot measure them. After all, a big chunk of change folks are more interested in people than numbers. This metaphor that change management is ‘soft’ extends into areas such as leadership and employee engagement whereby it may not be easy to measure and track things. However, is it really that because something is harder to measure and less black and white that there is less merit in measuring these?
“If you can’t measure it you can’t improve it” Peter Drucker”
The ‘why’ behind a lot of industry changes in our day and age come from the fact that data is now dominating our world. Data is a central part of everything that is changing in our world. Since we are now more reliant on the internet for information, the data that can be collected through our digital interactions around our lives are now driving change.
Home assistants like Alexa or Google Assistant can recognize our voices and tell us what we want to know. We can be identified through street cameras. Our Google usage leads to better-targeted advertisements and product promotions. Our Facebook usage leads to a deep understanding of our preferences and lifestyles, and therefore we become targetted by advertisements for what we may find value in (according to Facebook data and algorithms).
At work, we are surrounded by work functions and departments that rely on data to run and manage the business. HR, Finance, Operations, Manufacturing, Risk, Procurement, etc. The list goes on. In each of these departments data is an essential part of the day to day running of the function, without which the function cannot be run effectively.
Now with AI, companies are focused on data at an even greater level more than ever. Without data, AI cannot work nor add value to organisations.
So if our world is surrounded by data, why are we not measuring it in managing change? To answer this question let’s look at what we are or are not measuring.
Starting at a project level, these are some of the common ways in which change is often measured:
1. Change readiness surveys
Change readiness surveys are usually online surveys sent by a project owner to understand how stakeholder groups are feeling about the change at different points in time throughout the project. It can be in the form of a Likert scale or free text. Most results are summarized into a quantitative scale of the degree in which the group is ready for change. A simple SurveyMonkey or Microsoft Form could be set up to measure stakeholder readiness for change.
It used to be that change readiness surveys were quite long and wordy. Nowadays, a lot of change practitioners prefer to have shorter ‘pulse’ surveys as a way to regularly check on the stakeholder sentiments for readiness. However, shorter surveys could mean a lack of depth in the feedback you are receiving and limited data to use to pivot as necessary to address any concerns. So, you may find out if your stakeholders are ready for change, but not why. Ensure you balance ease and speed with insight and outcome.
2. Training evaluation surveys
These evaluations are normally based on participant satisfaction across various categories such as content, instructor effectiveness, usefulness, etc. In a face-to-face training format, these surveys are normally paper-based so as to increase the completion rate. For online or virtual training, ratings may be completed by the user at the conclusion or after the session.
Considering most organisations use virtual training formats, it is good practice to incorporate training evaluation at the conclusion of the session before the participants leave (after which it is almost impossible to get the satisfactory level of participant responses).
With the range of digital/AI-enabled tools on offer now, you can design training sessions in a way that requires much less and effort and gives you better results (to read more check out this link from Forbes). Some of these features include:
– gamifying training content to make it more engaging, interesting and fun
– easily creating micro-courses with little instructional design expertise
– incorporate a range of media such as videos and pictures with little effort
– using avatars as instructors to host the content
– easily create quizzes and assessments
3. Communications metrics
One way in which communications may be measured is the ‘hit rate’ or the number of users/audience that views the article/material/page. This may be easily tracked using Google Analytics which not only tracks the number of views per page but also viewership by the time of day/week as well as audience demographic information as such gender and geographical locations.
There is also a range of digital tools on offer to track the effectiveness of communication efforts. With Microsoft applications such as Yammer and Teams, there is already rich analytics capabilities on offer. These include user/group activity, device type usage, etc. Speak to your IT counterpart to access Microsoft Viva Engage which help you measure your community’s reach and engagement. You can find out more about the people, conversations, and questions & answers that make up your targeted communities.
There are also ways to A/B Test your communications message, whereby you have 2 different messages and test this with a smaller group fo audience to see which ones resonate or lead to more action. You can also create 2 different versions of the same intranet page and test messaging this way. When you have concluded the test you can then select the ‘winning’ version to the broader set of audience.
4. Employee sentiments/culture surveys
There are some organizations that measure employee sentiments or culture over the year and often there are questions that are linked to change. These surveys tend to be short and based on a Likert scale with fewer open-ended questions for qualitative feedback. Since these surveys are often sent across the entire organization they are a ‘catch-all’ yardstick and may not be specific to particular initiatives.
There is now a range of AI tools to do text and sentiment analysis if your survey contains text items. All the major technology providers such as Microsoft, Amazon and IBM already provide these tools (some are even free). These are some of the ways you can use AI tools right now:
– detect a range of emotions such as anxiety, anger, and disgust and based on response statistics
– cluster topics based on key response themes
– identify any data anomalies that you may want to exclude
– identify and label tone of voice of the responses, and classification such as positive, neutral, negative
– analyse trends over time
Data analysis and reporting can also be easily leveraged with the range of digital tools on offer. Data analysis tools using AI can automated generate charts and dashboards for you with little effort. Change Automator contains rich survey features that do exactly this, including:
– Easily selecting chart type with one click
– Leverage from AI-suggested data insights
– Generate predictive trends based on existing data
– Easily share charts and dashboards using different ways, including a URL link
5. Change heatmaps
Some organizations devise change heatmaps on excel spreadsheets to try and map out the extent to which different business units are impacted by change. This artifact speaks to the amount of change and often leads to discussions concerning the capacity that the business has to ‘handle/digest’ change. The problem with most heatmaps is that they are usually categorised and rated by the creator of the artifact (or a limited number of people making judgments), and therefore subject to bias. Data that is based on 1 person’s opinions also tend not to have as much weight in a decision-making forum.
In fact, we highly recommend that you don’t use change heat maps as the only way to track change volume. Instead, there is a range of other visuals such as bar charts, and timeline charts that are just as easy to interpret and are more insightful from a decision-making perspective.
Heatmaps are also by design categorical and not particularly precise. It may be useful at a high level for understanding hot spots, but not one to use to make specific decisions concerning business capacity levels and corresponding challenges.
The following is an example of a Change heatmap that uses the standard red, amber green traffic light coding scheme. This may play into the psychological bias of your audience interpreting red as bad and only focus on ‘alleviating’ the red.
6. Change initiative benefit tracking
In addition to typical change management measures, there are various initiatives-specific measures that focus on the actual outcome and benefit of the change with the goal of determining to what extent the change has taken place. Some examples of this include:
System usage rates
Cost reduction
Revenue increase
Transaction speed
Process efficiency
Speed of decision-making
Customer satisfaction rate
Employee productivity rate
Incidents of process violation
Non-initiative based change management measures
There are two other measures that are used within an organizational vs. initiative-specific context, change leadership assessment and change maturity assessment. In the next section, we will discuss these two areas.
Change leadership assessment
David Miller from Changefirst wrote about 3 types of change leaders.:
1. The sponsor whose role is to drive the initiative to success from the beginning to the end. This involves possessing competencies in rallying and motivating people, building a strong network of sponsors, and communicating clearly to various stakeholder groups.
2. The influencer whose role is to leverage their network and influence to market and garner the traction required to make the initiative successful. Four types of influencers as identified by Changefirst includes:
a) Advocates who are great at promoting and advocating the benefits of the change
b) Connectors who are able to link and leverage people across a part of the organization to support the change
c) Controllers who have control over access to information and people and these could include administrators and operations staff
d) Experts who are viewed by others in the organization as being technically credible
3. The change agent is someone who is tasked with supporting the overall change in various ways, including any promotional activities, gaging different parts of the organization on the change and be able to influence, up, down and sideways across the organization to drive a successful change outcome. Some call this the ‘change champion’. They can be your key to influencing across the organisation.
Whilst there isn’t one industry standard tool for assessing change leadership competencies and capabilities. There are various change leadership assessment tools offered by Changefirst as well as other various smaller consulting firms. Some of the ways in which you can assess change leadership may include categories such as Goal Attainment, Flexibility, Decision Making, and Relationship Building.
Some of the key competencies critical in change leadership have been called out by Pagon & Banutal (2008), and include:
Goal attainment
Assessing organizational culture and climate
Change implementation
Motivating and influencing others
Adaptability
Stakeholder management
Collaboration
Build organizational capacity and capability for change
Maneuvering around organizational politics
There is a range of change leadership assessment offerings from various consulting firms. Whichever one you choose, ensure that it is not overly simplistic and not ‘tested’ and therefore not reliable. Assessments will only be useful if they have gone through the rigour of being tested, with the results showing that they are reliable can be trusted. Anyone can ‘invent’ a simple survey with various leadership categories, but this does not mean they are actually valid. Afterall, if you are asking your leaders to spend time to fill in an assessment survey, you want to be confident that the outcome of the assessment will provide sufficient insight.
Change maturity assessment
Organisations are increasingly realising that managing change initiative by initiative is no longer going to cut it as it does not enable organizational learning and growth. Initiatives come and go and those who rely on contractor change managers often find that their ability to manage change as an organization does not mature much across initiatives, especially across time.
Change maturity assessment is focused on building change capability across the organization across different dimensions, whether it be project change management, operational change or change leadership. The goal of conducting a change maturity assessment is to identify areas in which there may be a capability gap and therefore enable structured planning to close this gap. The meaning of ‘capability’ does not just refer to people skills, but also to process and system capabilities.
Change maturity assessment results may prompt focus and action to improve change management capabilities if used in the right channels to influence the leadership and the business.
There are 2 major change maturity assessment models available in the market. The first is by Prosci and the second is by the Change Management Institute (CMI). Read up more about CMI’s Organisational Change Maturity Model here. To read more about change maturity assessment read out article A New Guide for Improving Change Management Maturity, where we outline how to improve change maturity throughout different business units across the organization.
A comprehensive model of Change Management Measures
In this diagram various change management measures are represented along two axes, one being the different phases of the initiative lifecycle, and the other being different organizational levels of project, business and enterprise in which change management measures fall into.
In the broad initiative phases of Plan, Execute and Realise there are various change measurements and assessments that may be applicable. At the Business and Enterprise levels, these measurements and assessments are not so much split according to initiative phases. Instead, they may be conducted periodically, for example change capacity and impost tracking may be done on a monthly basis, with change maturity assessment conducted at an annual basis.
Project level measures
1. ‘Plan’ phase
In this phase of the project, the team is discovering and scoping what the project involves and what the change is. As a result, the details are not known clearly at the commencement of the phase. Later in the phase the scope becomes much clearer and the team starts to plan what activities are required to implement the change.
The change complexity assessment evaluates how complex the project is. It looks at how many people could be impacted, what the size of the impact could be, how many business units are impacted, whether multiple systems and processes are impacted, etc.
Change resourcing costing. At the planning phase of the project cost required for the change management stream of the work is required. This includes such as any contractors, communication campaigns, learning cost, travel, and administration cost, just to name a few.
Change readiness assessment is usually conducted prior to the change and during the change. Usually, the same set of questions is asked of various stakeholder groups to assess their readiness for change.
2. ‘Execute’ phase
The execute phase is one of the most critical parts of the project. Activities are in full flight and the project is busy iterating and re-iterating changes to ensure successful execution to achieve project goals.
Communication and engagement tracking. Effective engagement of stakeholders in the change is absolutely critical. Stakeholder interviews, surveys, communication readership rates are all ways in which engagement may be tracked.
Learning tracking. Measuring learning is critical since it tracks to what extent the new competencies and skills have been acquired through learning interventions. Typical measurements include course tests or quizzes in addition to course evaluations. On the job performance may also be used to track learning outcomes and to what extent learning has been applied in the work setting.
Change readiness assessment continues to be critical to track during the execution phase of the project
3. ‘Realise’ phase
In this phase of the project the change has ‘gone live’ and most project activities have been completed. It is anticipated in this phase that the ‘change’ occurs and that the benefits can then be tracked and measured.
Change benefit tracking measures and tracks the extent to which the targeted benefits and outcomes have been achieved. Some of these measures may be ‘hard’ quantitative measures whilst others may be ‘soft’ measures that are more behavioural.
Business level measures
Business level measures are those that measure to what extent the business has the right ability, capacity, and readiness for the change.
Change heatmaps can help to visualize which part of the business is most impacted by 1 project or multiple projects. The power of the change heatmap is in visualizing which part of the business is the most impacted, and to compare the relative impacts across businesses. As the number of change initiatives increase so would the complexity of the change. When facing this situation organisations need to graduate from relying on excel spreadsheets to using more sophisticated data visualization tools to aid data-based decision making. To read more about change heatmaps and why this is not the only way to understand business change impact, go to The Death of the Change Heatmap.
Sponsor readiness/capability assessment can be a critical tool to help identify any capability gaps in the sponsor so that effort may be taken to support the sponsor. A strong and effective sponsor can make or break a change initiative. Early engagement and support of the sponsor are critical. Both Prosci, as well as Changefirst, have sponsor competency assessment offerings.
Change champion capability assessment. Change champion or change agent are critical ‘nodes’ in which to drive and support change within the organizational network. A lot of change champions are appointed only for one particular initiative. Having a business-focus change champion network means that their capability can be developed over time, and they can support multiple initiatives and not just one. Assessing and supporting change champion capability would also directly translate to better change outcomes.
Change leadership and change maturity assessment – refer to the previous section
Change capacity assessment.
In an environment where there is significant change happening concurrently, careful planning and sequencing of change in balance with existing capacity are critical. There are several aspects of change capacity that should be called out in the measurement process:
Different parts of the business can have different capacity for change. Those parts of the business with better change capability, and perhaps with better change leadership, are often able to receive and digest more changes than other businesses that do not possess the same level of capability.
Some businesses are much more time-sensitive and therefore their change capacity needs to be measured with more granularity. For example, call centre staff capacity is often measured in terms of minutes. Therefore, to effectively plan for their change capacity, the impacts of change needs to be quantified and articulated in a precise, time-bound context so that effective resourcing can be planned in advance.
The change tolerance or change saturation level for business needs careful measurement in combination with operational feedback to determine. For example, it could be that last month a part of the business experienced significant change impact across several initiatives happening at the same time. The operational indicators were that there was some impact on customer satisfaction, productivity, and there were negative sentiments reported by staff that there was too much change to handle. This could mean that the change tolerance level may have been exceeded. With the right measurement of change impact levels for that part of the business, next time this level of change is seen, previous lessons may be utilized to plan for this volume of change. Utilise measurement and data visualization tools such as the Change Compass to track change capacity.
Enterprise level change measures
At an enterprise level, many of the business unit level measures are still applicable. However, the focus is comparing across different business units to sense-make what each part of the business is going through and if the overall picture is aligned with the intentions and the strategic direction of the organization. For example, typical questions include:
Is it surprising that one part of the business is undergoing significant change whilst another is not?
Is there a reason that one business unit is focused on a few very large changes whilst for other business units there is a larger set of changes each with smaller impacts?
Is the overall pace of change optimum according to strategic intent? Does it need to speed up or slow down?
What is the process to govern, report and make decisions on enterprise level change, prioritization, sequencing and benefit realization?
Is there one business unit that is able to manage change more effectively, faster with greater outcomes? How can other business units leverage any internal best practices?
As mentioned in the Change Management Measures diagram, some enterprise level change measures include:
Change capacity assessment – Does one business unit’s change capacity limits mean that we are not able to execute on a critical strategy within the allocated time? How do we create more capacity? Ways in which to create more capacity could include more resources such as staff, or initiative funding, more time is given, or more talent to lead initiatives
Change maturity assessment – At an enterprise level, the concern is with the overall change maturity of the organization. How do we implement enterprise level interventions to build change maturity through programs, networks, and exchanges, such as:
Enterprise change capability programs
Enterprise change analytics and measurement tools
Enterprise change methodology
Enterprise network of change champions
Strategy impact map – Change management need not be focused only on project execution or business unit capability. It can also demonstrate value at an enterprise level by focusing on strategy execution (which by definition is change). The way in which different strategies exert impact on various business units may be visualized to help stakeholder understand which initiatives within which strategic intent impact which business units. To illustrate this please refer to the below diagram which is an example of a strategy impact map. In this diagram, each of the organisation’s strategy is displayed with different initiatives branching out of each strategy. The width of each initiative correlates with the level of impact that the initiative has on the business over a pre-determined period of time. Therefore, the width of each strategy also indicates the overall relative impact on the business.
This data visualization artifact can be valuable for business leaders and strategic planning functions as it depicts visually how the implementation of various strategies is impacting business units. This helps planners to better understand strategy implementation impacts, potential risks and opportunities, and balancing change pace with strategy goals at various points in time.
Predictive indicators on business performance – We started this article talking about how data is all around us and we also need to better manage change using data. With quantitative data on change impact, it is possible to ascertain any correlations with operational business indicators such as customer satisfaction, service availability, etc. For those business indicators where there is a significant correlation, it is possible to hence use predictive reporting to forecast performance indicator trends, given planned change impacts.
In the below graph you can see an example of this whereby using historical data it is possible to establish correlations and therefore forecast future impact on business indicators. This example is focused on the customer contact centre (CCC) and key business indicator of average handling time (AHT) is utilized as an illustration.
This type of predictive performance forecasting is extremely valuable for organisations undergoing significant change and would like to understand how change may impact their business performance. By demonstrating the impact on business indicators, this puts the importance of managing change at the front and centre of the decision-making table. At The Change Compass, we are developing this type of measurement and reporting function. This is the frontier for change management – to be established as a key business-driving function (versus a standard back-office function).
Change can be measured and this article has outlined various operational and strategic ways in which change measurement can demonstrate significant value. Most corporate functions cannot exist without data and analytics. For example, Human Resources relies on people and pay data. Marketing cannot function without measurement of channel and campaign effectiveness. For Information Technology, pretty much everything is measured from system usage, to cost, to efficiency. It is time we start utilizing data to better visualize change to better plan and make business decisions.
Have a chat with us if you are looking for ways to streamline how you capture, visualise data for decisions, and leverage AI to easily generate insights. This includes the ability to easily do forecasting, ask data questions using natural language and get instant answers.
Change management professionals are increasingly requested to provide measurement, data, and insights to various stakeholder groups. Not only does this include tracking various change outcomes such as business readiness or adoption, but stakeholder concerns also include such as change saturation and visibility of incoming initiative impacts.
To become better at working with data there is much that change managers can learn from data scientists (without becoming one of course). Let’s explore how change management can benefit from the practices and methodologies employed by data scientists, focusing on time allocation, digital tools, system building, hypothesis-led approaches, and the growing need for data and analytical capabilities.
1. Time Allocation: Prioritizing Data Collection and Cleansing
Data scientists spend a substantial portion of their time on data collection and cleansing. According to industry estimates, about 60-80% of a data scientist’s time is dedicated to these tasks. This meticulous process ensures that the data used for analysis is accurate, complete, and reliable.
In the below diagram from researchgate.net you can see that for data scientists the vast majority of the time is spent on collecting, cleansing and organising data.
You might say that change managers are not data scientists because the work nature is different, and therefore should not need to carve out time for these activities? Well, it turns out that the type of activities and proportions of time spent is similar across a range of data professionals, including business analysts.
Below is the survey results published by Business Broadway, showing that even business analysts and data analysts spend significant time in data collection, cleansing, and preparation.
Lessons for Change Management
a. Emphasize Data Collection and Cleansing: For change managers, this translates to prioritizing the collection of reliable data related to change initiatives. This might include stakeholder feedback, performance metrics, impact data and other relevant data points. Clean data is essential for accurate analysis and insightful decision-making. Data projects undertaken by change managers are not going to be as large or as complex as data scientists, however the key takeaway is that this part of the work is critical and sufficient time should be allocated and not skipped.
b. Allocate Time Wisely: Just as data scientists allocate significant time to data preparation, change managers should also dedicate sufficient time to gathering and cleaning data before diving into analysis. This ensures that the insights derived are based on accurate and reliable information.
It also depends on the data topic and your audience. If you are presenting comparative data, for example, change volume across different business units. You may be able to do spot checks on the data and not verify every data line. However, if you are presenting to operations business units like call centres where they are very sensitive to time and capacity challenges, you may need to go quite granular in terms of exactly what the time impost is across initiatives.
c. Training and Awareness: Ensuring that the change management team understands the importance of data quality and is trained in basic data cleansing techniques can go a long way in improving the overall effectiveness of change initiatives. Think of scheduling regular data sessions/workshops to review and present data observations and findings to enhance the team’s ability to capture accurate data as well as the ability to interpret and apply insights. The more capable the team is in understanding data, the more value they can add to their stakeholders leveraging data insights.
2. Leveraging Digital Tools: Enhancing Efficiency and Accuracy
Data scientists rely on a variety of digital tools to streamline their work. These tools assist in data collection, auditing, visualization, and insight generation. AI and machine learning technologies are increasingly being used to automate and enhance these processes.
Data scientists rely on various programming, machine learning and data visualisation such as SQL, Python, Jupyter, R as well as various charting tools.
Lessons for Change Management
a. Adopt Digital Tools: Change managers should leverage digital tools to support each phase of their data work. There are plenty of digital tools out there for various tasks such as surveys, data analysis and reporting tools.
For example, Change Compass has built-in data analysis, data interpretation, data audit, AI and other tools to help streamline and reduce manual efforts across various data work steps. However, once again even with automation and AI the work of data checking and cleansing does not go away. It becomes even more important.
b. Utilize AI and Machine Learning: AI can play a crucial role in automating repetitive tasks, identifying patterns, data outliers, and generating insights. For example, AI-driven analytics tools can help predict potential change saturation, level of employee adoption or identify areas needing additional support during various phases of change initiatives.
With Change Compass for example, AI may be leverage to summarise data, call out key risks, generate data, and forecast future trends.
c. Continuous Learning: Continuous learning is essential for ensuring that change management teams stay adept at handling data and generating valuable insights. With greater stakeholder expectations and demands, regular training sessions on the latest data management practices and techniques can be helpful. These sessions can cover a wide range of topics, including data collection methodologies, data cleansing techniques, data visualisation techniques and the use of AI and machine learning for predictive analytics. By fostering a culture of continuous learning, organizations can ensure that their change management teams remain proficient in leveraging data for driving effective change.
In addition to formal training, creating opportunities for hands-on experience with real-world data can significantly enhance the learning process. For instance, change teams can work on pilot projects where they apply new data analysis techniques to solve specific challenges within the organization. Regular knowledge-sharing sessions, where team members present case studies and share insights from their experiences, can also promote collective learning and continuous improvement.
Furthermore, fostering collaboration between change managers and data scientists or data analysts can provide invaluable mentorship and cross-functional learning opportunities. By investing in continuous learning and development, organizations can build a change management function that is not only skilled in data management but also adept at generating actionable insights that drive successful change initiatives.
3. Building the Right System: Ensuring Sustainable Insight Generation
It is not just about individuals or teams working on data. A robust system is vital for ongoing insight generation. This involves creating processes for data collection, auditing, cleansing, and establishing governance bodies to manage and report on data.
Governance structures play a vital role in managing and reporting data. Establishing governance bodies ensures that there is accountability and oversight in data management practices. These bodies can develop and enforce data policies, and oversee data quality initiatives. They can also be responsible for supporting the management of a central data repository where all relevant data is stored and managed.
Lessons for Change Management
a. Establish Clear Processes: Develop and document processes for collecting and managing data related to change initiatives. This ensures consistency and reliability in data handling.
b. Implement Governance Structures: Set up governance bodies to oversee data management practices. This includes ensuring compliance with data privacy regulations and maintaining data integrity. The governance can sponsor the investment and usage of the change data platform. This repository should be accessible to stakeholders involved in the change management process, promoting transparency and collaboration. Note that a governance group can simply be a leadership team regular team meeting and does not need to be necessarily creating a special committee.
c. Invest in system Infrastructure: Build the necessary system infrastructure to support data management and analysis that is easy to use and provides the features to support insight generation and application for the change team.
Data scientists often use a hypothesis-led approach, where they test, reject, or confirm hypotheses using data. This method goes beyond simply reporting what the data shows to understanding the underlying causes and implications.
Lessons for Change Management
a. Define Hypotheses: Before analyzing data, clearly define the hypotheses you want to test. For instance, if there is a hypothesis that there is a risk of too much change in Department A, specify the data needed to test this hypothesis.
b. Use Data to Confirm or Reject Hypotheses: Collect and analyze data to confirm or reject your hypotheses. This approach helps in making informed decisions rather than relying on assumptions or certain stakeholder opinions.
c. Focus on Actionable Insights: Hypothesis-led analysis often leads to more actionable insights. It is also easier to use this approach to dispel any myths of false perceptions.
For example: Resolving Lack of Adoption
Hypothesis: The lack of adoption of a new software tool in the organization is due to insufficient coaching and support for employees.
Data Collection:
Gather data on the presence of managerial coaching and perceived quality. Also gather data on post go live user support.
Collect feedback from employees through surveys regarding the adequacy and clarity of coaching and support.
Analyse usage data of the new software to identify adoption rates across different departments.
Analysis:
Compare adoption rates between employees who received sufficient coaching and support versus those who did not.
Correlate feedback scores on training effectiveness with usage data to see if those who found the training useful are more likely to adopt the tool.
Segment data by department to identify if certain teams have lower adoption rates and investigate their specific training experiences.
Actionable Insights:
If data shows a positive correlation between coaching and support, and software adoption, this supports the hypothesis that enhancing coaching and support programs can improve adoption rates.
If certain departments show lower adoption despite completing coaching sessions, investigate further into department-specific issues such as workload or differing processes that may affect adoption.
Implement targeted interventions such as additional training sessions, one-on-one support, or improved training materials for departments with low adoption rates.
5. Building Data and Analytical Capabilities: A Core Need for Change Management
As data and analytical capabilities become increasingly crucial, change management functions must build the necessary people and process capabilities to leverage data-based insights effectively.
Lessons for Change Management
a. Invest in Training: Equip change management teams with the skills needed to manage data and generate insights. This includes training in data analysis, visualization, and interpretation.
b. Foster a Data-Driven Culture: A lot of organisations are already on the bandwagon of Encourage a culture where data is valued and used for decision-making. The change management needs to promote this equally within the change management function. This involves promoting the use of data in everyday tasks and ensuring that all team members understand its importance. Think of incorporating data-led discussions into routine meeting meetings.
c. Develop Analytical Frameworks: Create frameworks and methodologies for analyzing change management data. This includes defining common key metrics, setting benchmarks, and establishing protocols for data collection and analysis for change data. Data and visual templates may be easier to follow for those with lower capabilities in data analytics.
Practical Steps to Implement Data-Driven Change Management
To integrate these lessons effectively, senior change practitioners can follow these practical steps:
Develop a Data Strategy: Create a comprehensive data strategy that outlines the processes, tools, and governance structures needed to manage change management data effectively.
Conduct a Data Audit: Begin by auditing the existing data related to change management. Identify gaps and areas for improvement.
Adopt a Hypothesis-Led Approach: Encourage the use of hypothesis-led approaches to move beyond descriptive analytics and derive more meaningful insights.
Invest in Technology: Invest in the necessary digital tools and technologies to support data collection, cleansing, visualization, and analysis.
Train the Team: Provide training and development opportunities for the change management team to build their data and analytical capabilities.
Collaborate Across Functions: Foster collaboration between change management and data science teams to leverage their expertise and insights.
Implement Governance Structures: Establish governance bodies to oversee data management practices and ensure compliance with regulations and standards.
By learning from the practices and methodologies of data scientists, change management functions can significantly enhance their effectiveness. Prioritizing data collection and cleansing, leveraging digital tools, building robust systems, adopting hypothesis-led approaches, and developing data and analytical capabilities are key strategies that change management teams can implement. By doing so, they can ensure that their change initiatives are data-driven, insightful, and impactful, ultimately leading to better business outcomes.
Successful change management relies on having the right metrics to measure progress, gauge impact, and communicate with stakeholders. Moreover, the right metrics can drive continuous improvement and help directly achieve change outcomes. However, not all metrics are beneficial, and some can mislead or fail to meet stakeholder needs. Let’s check out the top change management metrics to avoid and go through examples to take note.
Understanding the Disconnect: Change Managers vs. Business Stakeholders
A significant reason certain change management metrics fall short is the differing perspectives between change managers and business stakeholders. Change managers are trained to view metrics through the lens of change management frameworks and methodologies, focusing on detailed assessments and structured approaches. These include applying ratings and judgments on aspects such as impact levels.
In contrast, business stakeholders prioritize business operations, strategic outcomes, and practical implications. The busy business stakeholder is often looking for practical implications from metrics that can be used to directly drive decision making, meaning “what do I do with this data to improve the ultimate business outcome”.
Of course, different stakeholder have different data needs, and you need to show the right metric to the right type of stakeholder. For example, operations focused stakeholders expect fairly detailed metrics and data and what that means in terms of organisation, coordination, capacity and performance perspectives. Senior managers may prefer higher level data with focus on strategic impacts, overall progress and adoption indicators.
This disconnect can lead to the use of metrics that do not resonate with or are misunderstood by stakeholders.
Metrics from a Change Manager’s Perspective
Change managers may leverage metrics that are derived from the various change management documents such impact assessments, training plan or communications plan. Metrics also often chosen for ease of use and ideally are not overly complicated to execute.
For example, impact assessments typically involve rating stakeholder groups and initiatives on a traffic light system (red, amber, green) based on their impact. While this approach is systematic, it can be problematic for several reasons:
Lack of Sufficient Stakeholder Context: Business stakeholders might not understand the practical implications of these ratings. For instance, an “impact rating per initiative” may not clearly convey what the rating means for day-to-day operations or strategic goals. For example, if an initiative has a red impact rating, stakeholders might not grasp the specific operational changes or strategic adjustments needed, in essence, “what do I do with this?”.
Misinterpretation of Traffic Light Ratings: The red, amber, green system can be misleading. Stakeholders might interpret red as an indicator of alarm or imminent risk, while green may be seen as a sign that no action is needed. This is because stakeholders are trained to interpret traffic light ratings this way (from the various project/business updates they’ve attended). In reality, red might simply mean high impact, requiring focused attention, and green might indicate a low impact but still require monitoring. For instance, a red rating might indicate significant process changes that need careful management, not necessarily a negative outcome.
Hard to defend ratings if prompted: Business stakeholders may also want to drill into how the ratings are determined, and based on what basis. They may expect a logical data-backed reasoning of how each colour scheme is determined. If a rating is based on an overall ‘personal judgment’ this may be hard to defend infront of a group of stakeholders.
Examples of Potentially Misleading Metrics
Certain metrics, although straightforward, can be easily misinterpreted and fail to provide a realistic picture of change impacts. Often these are selected because they are easy to report on. However, easy, make not give you the outcome you are looking for.
Number of Go-Lives: Tracking the number of Go-Lives over time might seem like an effective way to represent change volume. However, the most significant impacts on people often occur before or after the Go-Live date. For example, the preparation and training phase before Go-Live and the adoption phase afterward are critical periods that this metric overlooks. A Go-Live date might indicate a milestone but not the challenges, progress or impacts faced during the implementation phase.
Number of Activities Implemented: Similar to Go-Lives, this metric focuses on quantity rather than quality. Simply counting the number of activities does not account for their effectiveness or the actual change they drive within the organisation. For example, reporting that 50 training sessions were conducted does not reveal whether employees found them helpful or if they led to improved performance.
Number of impacts or stakeholders impacted: Again, using a numerical way to indicate progress can be very misleading, or unmeaningful. This is because it may be ‘interesting’ but with no real action for your stakeholder to take in order to somehow lead to a better overall change outcome. If metrics do not result in some kind of action, then over time it will not shape your change(s) toward the targeted outcomes. Or worse, your stakeholders may lose interest and lose confidence in the strategic impact of these metrics.
Another common way to report change metrics is to use the number of impacts or number of stakeholders impacted. This can be in terms of the following:
Number of divisions impacted
Number of stakeholder groups impacted
Number of employees impacted
Number of initiatives per division/stakeholder
Metrics That May Be Too Operational
Metrics that are overly operational can fail to capture meaningful progress or adoption. Perhaps if the metric are for reporting within the Change Management team that may be OK. However, when you are showing metrics to stakeholders, a different set of expectations should be cast.
If you are presenting metrics to senior managers, you need to ensure that they hit the mark for that audience group. If the group is more interested in strategic impact, and higher level progress outcomes, you need to tailor accordingly.
Examples of metrics that may be too operational include:
Number of Communications Sent: This metric measures activity but not effectiveness. Sending numerous emails or messages does not guarantee that the message is received, understood, or acted upon by stakeholders. For instance, stakeholders might receive 100 emails, but if the content is unclear, the communication effort is wasted. Or worse, the emails may not even have been read.
Number of Training Sessions Attended: This one is a classic. While training is crucial, the number of sessions attended does not necessarily reflect the attendees’ understanding, engagement, or the practical application of the training. For example, employees might attend training but not apply the new skills if the training is not relevant to their roles for various reasons.
Number of workshops/meetings: Another way of articulating the change management progress in terms of activities is the number of workshops or meetings conducted with stakeholders. Again, this may be good to track within the change management team. However, presenting this metric to stakeholders may not be appropriate as it may not mee their needs.
The way metrics are presented is just as important as the metrics themselves. Poor visualization can lead to misinterpretation, confusion, and misguided decisions. Here are some common pitfalls to avoid:
Ineffective Use of Pie Charts
Pie charts can be misleading when used to show data points that are not significantly different. For example, using a pie chart to represent the percentage of divisions impacted by a change might not effectively communicate the nuances of the impact if the differences between the divisions are minimal. A pie chart showing 45%, 30%, and 25% might not convey the critical differences in impact levels among divisions.
Misleading Traffic Light Ratings
Using red, amber, and green to indicate high, medium, and low impacts can send the wrong message. Stakeholders might associate these colours with good and bad outcomes rather than understanding the actual levels of impact. Stakeholder may be used to interpreting these in the context of their usual project or business updates where red indicated alarm and ‘bad’. This can lead to unnecessary alarm or complacency. For instance, a green rating might suggest no need for action, while in reality, it might require ongoing monitoring.
Overuse of Colours
Using too many colours in charts and graphs can overwhelm stakeholders, making it difficult to discern the key message. Using colours in data visualisation can be two-edged sword. Colour can effectively point your stakeholders are the area where you want them to focus on. But, too many colours can lose your audience. A cluttered visual can obscure the critical data points and lead to misinterpretation. For example, a graph with ten different colours can confuse stakeholders about which data points are most important.
Practical Takeaways for Senior Change Managers
To ensure that change management metrics are effective, consider the following practical takeaways:
Align Metrics with Stakeholder Perspectives
Understand Stakeholder Priorities: Engage with stakeholders to understand their priorities and concerns. Tailor your metrics to address these aspects directly. For example, if stakeholders are concerned about operational efficiency, focus on metrics that reflect improvements in this area.
Use Business Language: Frame your metrics in a way that resonates with business stakeholders. Avoid change management jargon and reference, and ensure that the implications of the metrics are clear and actionable. For example, instead of using technical terms, explain how the metrics impact business outcomes. Think in terms of business activities, milestones, busy periods, and capacity challenges.
Focus on Meaningful Metrics
Measure Outcomes, Not Just Activities: Prioritize metrics that reflect the outcomes and impacts of change, rather than just the activities performed. For example, instead of counting the number of training sessions, measure the improvement in employee performance or knowledge retention post-training.
Example: Instead of reporting that 100 employees attended training sessions, report that 85% of attendees showed improved performance in their roles after training, or that certain level of competencies were gained.
Track Engagement and Adoption: Monitor metrics that indicate the level of engagement and adoption among stakeholders. This could include surveys, feedback forms, or direct measures of behaviour change.
Example: Use post-training surveys to measure employee confidence in applying new skills or managerial rating of application of learnt skills. Track the percentage of employees who actively use new tools or processes introduced during the change.
Improve Metric Visualization
Simplify Visuals: Use clear, simple visuals that highlight the key messages. Avoid clutter and ensure that the most important data points stand out.
Example: Use bar charts or line graphs to show trends over time rather than pie charts that can be harder to interpret.
Contextualize Data: Provide context for the data to help stakeholders understand the significance. For example, instead of just showing the number of Go-Lives, explain what each Go-Live entails and its expected impact on operations. Or better, focus on showing the varying levels of impact on different stakeholders across time within the initiative.
Example: Accompany a Go-Live count with a visual showing the varying impact level of various implementation activities of the changes.
Narrative Approach: Combine metrics with a narrative that explains the story behind the numbers. This can help stakeholders understand the broader context and implications.
Example: Instead of presenting raw data, provide a summary that explains key trends, successes, and areas needing attention.
Educate your stakeholders: Depending on stakeholder needs you may need to take them on a phased approach to gradually educate them on change management metrics and how you ultimately want them to drive the outcomes.
Example: You may start the education process to focus on more simplistic and easy-to-understand measures, and as your stakeholders are more change-mature, move to drill into more detailed metrics that explain the ‘why’ and ‘how’ to drive outcome success.
Continuously improvement: Provide regular updates on key metrics and adjust them based on feedback from stakeholders. Continuous communication ensures that everyone remains aligned and informed.
Example: Hold monthly review meetings with stakeholders to discuss the latest metrics, address concerns, and adjust strategies as needed.
Examples of Effective Metrics
Employee Adoption and Engagement
Percentage of Employees Adopting New Process/System: This metric measures the rate at which employees are using new processes or systems introduced during the change. High adoption rates indicate successful integration.
Implementation: Use software usage analytics or surveys to track tool adoption rates.
Visualization: A graph showing adoption rates over time.
Employee Feedback Scores: Collect feedback on change initiatives through surveys or stakeholder ratings to measure sentiment/feedback and identify areas for improvement.
Implementation: Conduct regular surveys asking employees about their experience with the change process. Do note that depending on the change you may expect negative feedback due to the nature of the change itself (vs the way it was implemented).
Visualization: Bar/Line charts comparing feedback scores across different departments or time periods. Bar/Line charts are the standard go-to for data visualisation. They are easy to understand and interpret.
Impact on Business Outcomes
Improvement in Key Performance Indicators (KPIs): Track changes in KPIs that are directly impacted by the change initiatives, such as productivity, customer satisfaction, or financial performance.
Implementation: Identify relevant KPIs and measure their performance before and after change initiatives.
Visualization: Use line/bar graphs to show trends in KPI performance over time.
Operational Efficiency Metrics: Measure improvements in operational processes, such as reduced cycle times, error rates, or cost savings.
Implementation: Track specific operational metrics relevant to the change initiatives.
Visualization: Bar charts or heatmaps showing improvements in efficiency metrics across different operational areas.
Effective change management requires metrics that not only measure progress but also resonate with business stakeholders and accurately reflect the impact of change initiatives. Avoiding common pitfalls such as relying on easily misinterpreted or overly operational metrics is crucial. By aligning metrics with stakeholder perspectives, focusing on meaningful outcomes, improving visualization, and communicating effectively, senior change and transformation professionals can ensure that their metrics truly support the success of their change initiatives.
The top change management metrics to avoid are those that fail to provide clear, actionable insights to business stakeholders. By understanding and addressing the disconnect between change managers and business stakeholders, and by prioritizing metrics that truly reflect the impact and progress of change, you can drive more effective and successful change management efforts by influencing your stakeholders in your organisation.
Chat with us if you would like to discuss more about leveraging AI and technology to generate high-impact change management metrics and data for your stakeholders, both at project and portfolio levels.
Scaled Agile Framework (SAFe) has emerged as a leading methodology to address organisational change demands of fostering flexibility, collaboration, and continuous improvement. A cornerstone of SAFe is the principle of “Measure and Grow,” which emphasizes using data and fact-based decisions to enhance change outcomes over time. Despite its centrality, SAFe does not explicitly detail the change management components essential for its success. Here we outline how change management practitioners can effectively apply the “Measure and Grow” principle to lead change and improve outcomes to support the Scaled Agile environment.
The “Measure and Grow” Principle in Scaled Agile
“Measure and Grow” is integral to SAFe, focusing on systematic measurement and continuous improvement. By leveraging data and analytics, organizations can make informed decisions, identify areas needing attention, and iteratively enhance performance. For change management professionals, this principle translates into a structured approach to evaluate the effectiveness of change initiatives, pinpoint areas for improvement, and implement necessary adjustments.
In a Scaled Agile environment, “Measure and Grow” is a core tenant or principle that applies in all types of agile environments. By continuously assessing and refining change efforts, organizations can align their initiatives with strategic objectives, mitigate risks, and ensure sustained success.
In practice, a lot of organisations have not pinpointed exactly how change management measures can make or break the outcome of the change, and in a SAFe environment, across the program, portfolio as well as enterprise.
The ‘Measure and Grow’ principle as a core part of SAFe (From Scaled Agile Framework)
Key Elements of Measuring and Growing Change Outcomes
To operationalize the “Measure and Grow” principle in change management, it is crucial to establish a set of metrics and assessment frameworks. Here are some broad categories of different types of change measurements that are relevant. Note that since we are talking about SAFe, it is not just at the initiative level that we are talking about metrics. More importantly, it is about establishing a system to promote change improvement across the organisation.
Change Management KPIs and OKRs
Key Performance Indicators (KPIs) and Objectives and Key Results (OKRs) are essential tools for tracking the success of change management initiatives. KPIs provide quantitative measures of performance, while OKRs align change efforts with broader organizational goals. A change management stream or function should focus on establishing KPIs or OKRs to achieve laser focus on achieving change outcomes.
Examples of Initiative-Level Change Management KPIs that may roll out to form portfolio views
Employee Engagement Levels: This KPI assesses how change impacts employee morale and engagement, providing insight into the overall acceptance and support of the change initiative.
Learning Achievement Rates: This can include tracking the percentage of employees who have completed necessary training programs, as well as achieving the target level of competence to ensure that the workforce is adequately prepared for the change.
Feedback Scores: Collecting feedback from stakeholders through surveys or feedback forms helps gauge perception and identify areas needing improvement. It is important to note that depending on the change context, stakeholders may not be happy with the content of the change. However, understanding and tracking this perception is still important.
Change Adoption Rate: This KPI measures the percentage of stakeholders who have adopted the change. High adoption rates are the ultimate goal for initiatives.
Issue Resolution Time: Measuring the time taken to resolve user-related issues related to the change highlights the efficiency of support mechanisms and the responsiveness of the change management team. This is especially important during an agile environment where there may be constant changes.
Change Readiness and Stakeholder Engagement Metrics
Evaluating change readiness and stakeholder engagement is crucial to the success of any change initiative. These metrics help assess the organization’s preparedness for change and the level of involvement and support from key stakeholders. Readiness and engagement rates can also roll up at a portfolio level to provide oversight.
Change Readiness Metrics
Readiness Assessments: Conduct surveys or interviews to gauge the organization’s preparedness for the impending change. This can include evaluating awareness, understanding, and acceptance of the change.
Resource Availability: Measure the availability of necessary resources, such as budget, personnel, and tools, to support the change initiative.
Communication Effectiveness: Assess the clarity, frequency, and effectiveness of communication regarding the change to ensure stakeholders are well-informed and engaged.
Stakeholder Engagement Metrics
Engagement Scores: Use surveys or feedback forms to measure the engagement levels of stakeholders, indicating their commitment and support for the change.
Participation Rates: Track stakeholder participation in change-related activities, such as workshops, meetings, and training sessions, to gauge their involvement.
Influence and Support: Assess the influence and support of key stakeholders in driving the change, ensuring that influential figures are actively endorsing the initiative.
By monitoring these metrics, change management professionals can identify potential barriers to change and take proactive steps to enhance readiness and engagement.
Stakeholder Competency Assessment
Successful change initiatives rely on the competence and readiness of key stakeholders. Assessing stakeholder competency involves evaluating the capability of sponsors and change champions to support and drive the change.
Sponsor Readiness/Capability Assessment
Sponsor Engagement: Measure the level of engagement and commitment from sponsors, ensuring they are actively involved and supportive of the change.
Decision-Making Effectiveness: Assess the ability of sponsors to make timely and effective decisions that facilitate the change process.
Resource Allocation: Evaluate the sponsor’s ability to allocate necessary resources, such as budget and personnel, to support the change initiative.
Change Champion Capability Assessment
Training and Knowledge: Measure the knowledge and training levels of change champions to ensure they are well-equipped to support the change.
Communication Skills: Assess the ability of change champions to effectively communicate the change message and address stakeholder concerns.
Influence and Leadership: Evaluate the influence and leadership capabilities of change champions, ensuring they can effectively drive and sustain the change.
By conducting these assessments, change management professionals can ensure that key stakeholders are prepared and capable of supporting the change initiative.
Change Adoption Metrics
Change adoption metrics provide insight into how well the change has been accepted and integrated into the organization. These metrics help assess the effectiveness of the change initiative and identify areas for improvement. At a portfolio level, there may be different levels of change adoption set for different initiatives depending on priority and complexity.
Key Change Adoption Metrics
Adoption Rate: Measure the percentage of stakeholders who have adopted the change, indicating the overall acceptance and integration of the new processes or systems.
Usage Metrics: Track the usage of new tools, processes, or systems introduced by the change to ensure they are being utilized as intended.
Performance Metrics: Assess the impact of the change on key performance indicators, such as productivity, efficiency, and quality, to determine the overall success of the change initiative.
By monitoring these metrics, change management professionals can gauge the success of the change initiative and identify opportunities for further improvement. To read more about change adoption metrics check out The Comprehensive Guide to Change Management Metrics for Adoption.
Change Impact and Capacity Metrics
Understanding the impact of change and the organization’s capacity to manage it is crucial for successful change management. Change impact metrics assess the effects of the change on the organization, while capacity metrics evaluate the organization’s ability to manage and sustain the change.
Change Impact Metrics
Aggregate impacts: Aggregate impacts across initiatives to form a view of how various teams and roles are impacted by various changes.
Risk Assessments: Identify potential risks associated with the change and evaluate their impact, ensuring that mitigation strategies are in place. A particular focus should be placed on business performance during change, across initiatives.
Capacity Metrics
Resource Capacity: Assess the availability of resources, such as personnel, budget, and tools, to support the change initiative.
Change Fatigue: Measure the risk for potential fatigue within the organization and its impact on stakeholders, ensuring that change initiatives are paced and driven appropriately.
Support Structures: Evaluate the effectiveness of support structures, such as training programs, information hubs, and help desks, in facilitating the change. Support structures may also include change champion networks.
By assessing change impact and capacity, change management practitioners can ensure that the organization is well-equipped to manage and sustain the change initiative.
Change Maturity Assessment
Change maturity assessments provide a comprehensive evaluation of the organization’s capability to manage change effectively. These assessments help identify strengths and weaknesses in the organization’s change management practices and provide a roadmap for improvement.
The Change Management Institute (CMI) Change Maturity Model is a comprehensive framework that takes a holistic approach to enhancing an organization’s change management maturity. It’s divided into three core functional domains, each playing a vital role in the overall journey toward maturity:
Project Change Management
Business Change Readiness
Strategic Change Leadership.
These domains serve as the foundation for achieving higher levels of maturity within the organization.
Within each of these domains, the CMI model outlines a structured path, consisting of five distinct maturity levels. These levels represent a continuum, starting at Level 1, which serves as the foundational stage, and progressing all the way to Level 5, the zenith of maturity and effectiveness. This multi-tiered approach offers organizations a clear roadmap for growth and development, ensuring that they have the tools and insights necessary to navigate the complexities of change management.
By conducting regular change maturity assessments, change management professionals can identify areas for improvement and develop targeted strategies to enhance the organization’s change management capability.
The “Measure and Grow” principle is a powerful tool for improving change outcomes in a Scaled Agile environment. By leveraging data and fact-based decision-making, change management professionals can ensure that change initiatives are effective, aligned with strategic objectives, and continuously improving. Establishing robust metrics and assessment frameworks, such as KPIs, OKRs, change readiness and stakeholder engagement metrics, stakeholder competency assessments, change adoption metrics, change impact and capacity metrics, and change maturity assessments, is essential to applying the “Measure and Grow” principle effectively.
Incorporating these metrics and assessments into change management practices enables organizations to identify areas for improvement, make informed decisions, and drive continuous improvement. By doing so, change management professionals can enhance the effectiveness of change initiatives, ensure successful adoption, and ultimately achieve better business outcomes.
