Measuring behaviours as a part of change adoption is the ultimate measurement that change practitioners should incorporate as a part of tracking to what extent the change has been adopted by impacted stakeholders.
Whilst there could be a wide range of different behaviours depending on the initiative in concern, what are some of the tips in selecting the right behaviours to measure?
Check out our infographic on the top 4 elements to pay attention to when measuring behaviours as a part of change adoption metrics.
Change saturation is a common term used by change practitioners to describe a picture where there may be too many changes being implemented at the same time. The analogy is that of a cup with limited capacity, where if too much change is poured into a fixed volume, the rest will not stay in the cup or be ‘embedded’ as adopted changes.
At the end of 2020, Pivot Consulting conducted extensive research where they asked a range of different roles in organisations about implementing change. When questioned about key challenges to executing strategy and driving change, change fatigue or employees being overwhelmed by multiple initiatives is identified as one of the top 2 most critical challenges. It can be seen that change saturation is not just a popular discussion topic but a serious focus area that is posing significant challenges to a range of organisations.
Research from Pivot Consulting, 2020
There are many common ways of understanding and approaching change saturation. However, many of these are not always correct with some being quite misleading. In this article, we aim to review the 5 key incorrect assumptions about change saturation that are downright misleading and should be directly challenged. These may be assumptions that are widely held and assumed to be ‘facts’ and are not questioned.
Incorrect assumptions:
In the following, we outline the key assumptions that should be challenged when approaching change saturation.
1. Change is disruption
The first assumption is that change is always ‘disruption’. Change can be dynamic. There is also a range of different types of changes. Therefore, change does not always need to be negative and cause chaos or impede normal ways of working.
Take, for example, agile teams. A part of the work of an agile team is to drive continuous improvement. The team establishes regular routines to try something new, i.e. a change. They then execute it and examine the data to see the effect of the change on business. For these teams, ‘planned’ changes are just part of normal ways of working, and therefore not necessarily viewed as ‘disruptions’ to their work since this is part of their work.
On the other hand, change is also not always ‘negative’. Some changes may be there to make it easier for the employee or the customer. For example, it may be that the organisation is implementing system-driven automation to save employees time in entering manual information. These changes are typically welcomed by the impacted employees and are not perceived as ‘disruptions’ to their work. Instead, they are typically perceived as positive changes.
As a result, change needs to be understood by its specific impact on the various stakeholders, and not by its ‘disruption’. A more useful way to understand the impact of the changes on end stakeholders may be to understand the various activities required for them to undergo the change and shift their behaviours.
For example, it could be that a customer service rep may need to undergo training sessions, team briefing sessions, review documentation, and receive team leader feedback, in the overall change journey. These activities may be ‘on top’ of existing normal business routines, or they may be a part of existing business routines, and therefore not ‘adding’ to the ‘saturation level’.
2. Change capacity is determined by capability
It is a commonly held belief that change capacity is determined by change capability at individual, team and organisational levels. Yes, factors such as change leadership, individual change capability and skills can improve change capacity. However, change capacity is not only determined by capability.
Indeed, there are other factors that determine change capacity.
a. Biological.
Humans are designed to have a limited attention span. When there are too many things happening at the same time, we can only focus on a limited number of things at the same time. There are many studies that show if we keep switching focus between different tasks, we are likely to not have full focus and attention which will leave us to making mistakes.
This also applies to learning. The more we focus on multiple tasks, the more we are not able to tune out and therefore engage in deeper processing and learning.
What about thinking about multiple initiatives? According to University of Oregon researchers, professors Edward Awh and Edward Vogel, the human brain has a built-in limit on the number of discrete thoughts it can entertain at one time. The limit for most individuals is four. It does not matter how much capability development one focuses on, there is a limit to how much capacity can be created. Therefore, there is a cap on to what extent capability may lift change capacity. After all, no matter how skillful someone is, biological tendencies and restrictions remain.
b. Expectation.
The level of expectation of the extent to which one can change can determine the outcome. Studies have shown the individual negativity or positivity can impact the outcome. The more negative an individual of the outcome, the more negative the outcome becomes. However, if the expectations are unrealistically high, they may lead to disappointment.
Think back to the impacts of Covid, and how what would have seemed almost impossible in terms of virtual working has suddenly become a reality overnight. Often what companies had imagined taking 10 years to achieve, is suddenly achieved overnight out of necessity. The expectation that there is no other way and that there is no choice leads to the acceptance of the change scenario.
3. Basing saturation points purely on opinions
As change practitioners, we often aim to be the ‘people’ representative. Many think of themselves as the ‘social worker’ or ‘welfare worker’ who are there to be the voice of employees. Whilst, it is true that we need to be the voice of people, the definition of ‘people’ should not just include employees, but a range of stakeholders including managers.
Especially when the change environment is complex and challenging, there may be a tendency for people to ‘over-inflate’ the reality of the situation. Sometimes it may be easier to call out that there is too much change in the hope that this feedback will result in less change volume, thereby making work ‘easier’.
Change practitioners need to be aware of political biases or tendencies for people to report on feedback that is not substantiated by data. Interviews with stakeholders may need to be supplemented by surveys or focus groups to test the validity of the results. We should not simply assume that anything stakeholders tell us are ‘truths’ per se, especially since there is political motivation in biased reporting.
Example from The Change Compass – Plotting change saturation line against change impact levels
4. Focus on capability vs systems and processes to manage saturation
An overt focus on capability, knowledge and skills, may lead to gaps in the overall ability to manage change saturation. This is because skills and competencies are just one of many elements that supports change execution. Beyond this, effective organisations also need to focus on having the right systems and processes established to support ongoing change execution.
Systems and processes include such as:
Learning operations processes whereby there is a clear set of steps for the business to communicate, undertake, and embed training/learning activities. These include the right channel to organise people capacity to attend sessions, communication channels regarding the nature of scheduled training sessions and monitoring the effectiveness of these sessions
Communication processes include having a range of effective channels that promote dynamic communication between employees and managers, as well as across different business units and teams.
Data and reporting mechanisms to visualise change impacts, measurement on change saturation levels, and report on change delivery tracking and change adoption progress
Governance established to examine change indicators including change saturation, risks identified, and make critical decisions on sequencing, prioritisation, and capacity mitigation
Skills and competencies are one element, but without processes and systems established to execute the change and track/report on change saturation, there will be limited business outcomes achieved.
Outlined in this article are just 5 of the common assumptions about change saturation that are misleading. There are many more other assumptions. The key for change practitioners is not to blindly rely on ‘methodologies’ or concepts, but instead to focus on data and facts to make decisions. Managing change saturation needs to be data-driven. Otherwise, stakeholders may easily dismiss any change saturation claims (as is often the case with senior managers). Armed with the right data and insights, the change practitioner has the power to influence a range of change decisions to achieve an optimal outcome for the organisation.
An important part of measuring change is to be able to design change management surveys that measure what it has set out to measure. Designing and rolling out change management surveys is a core part of what a change practitioner’s role is. However, there is often little attention paid to how valid and how well designed the survey is. A survey that is not well-designed can be meaningless, or worse, misleading. Without the right understanding from survey results, a project can easily go down the wrong path.
Why do change management surveys need to be valid?
A survey’s validity is the extent to which it measures what it is supposed to measure. Validity is an assessment of its accuracy. This applies whether we are talking about a change readiness survey, a change adoption survey, employee sentiment pulse survey, or a stakeholder opinion survey.
What are the different ways to ensure that a change management survey can maximise its validity?
Face validity. The first way in which a survey’s validity can be assessed is its face validity. Having good face validity is that in the view of your targeted respondents the questions measure what they aimed to measure. If your survey is measuring stakeholder readiness, then it’s about these stakeholders agreeing that your survey questions measure what they are intended to measure.
Predictive validity. If you really want to ensure that your survey questions are scientifically proven to have high validity, then you may want to search and leverage survey questionnaires that have gone through statistical validation. Predictive validity means that your survey is correlated with those surveys that have high statistical validity. This may not be the most practical for most change management professionals.
Construct validity. This is about to what extent your change survey measures the underlying attitudes and behaviours it is intended to measure. Again, this may require statistical analysis to ensure there is construct validity.
At the most basic level, it is recommended that face validity is tested prior to finalising the survey design.
How do we do this? A simple way to test the face validity is to run your survey by a select number of ‘friendly’ respondents (potentially your change champions) and ask them to rate this, followed by a meeting to review how they interpreted the meaning of the survey questions.
Alternatively, you can also design a smaller pilot group of respondents before rolling the survey out to a larger group. In any case, the outcome is to test that your survey is coming across with the same intent as to how your respondents interpret them.
Techniques to increase survey validity
1. Clarity of question-wording.
This is the most important part of designing an effective and valid survey. The question wording should be that any person in your target audience can read it and interpret the question in exactly the same way.
Use simple words that anyone can understand, and avoid jargon where possible unless the term is commonly used by all of your target respondents
Use short questions where possible to avoid any interpretation complexities, and also to avoid the typical short attention spans of respondents. This is also particularly important if your respondents will be completing the survey on mobile phones
Avoid using double-negatives, such as “If the project sponsor can’t improve how she engages with the team, what should she avoid doing?”
2. Avoiding question biases
A common mistake in writing survey questions is to word them in a way that is biased toward one particular opinion. This assumes that the respondents already have a particular point of view and therefore the question may not allow them to select answers that they would like to select.
Some examples of potentially biased survey questions (if these are not follow-on questions from previous questions):
Is the information you received helping you to communicate to your team members
How do you adequately support the objectives of the project
From what communication mediums do your employees give you feedback about the project
3. Providing all available answer options
Writing an effective survey question means thinking through all the options that the respondent may come up with. After doing this, incorporate these options into the answer design. Avoid answer options that are overly simple and may not meet respondent needs in terms of choice options.
4. Ensure your chosen response options are appropriate for the question.
Choosing appropriate response options may not always be straightforward. There are often several considerations, including:
What is the easiest response format for the respondents?
What is the fastest way for respondents to answer, and therefore increase my response rate?
Does the response format make sense for every question in the survey?
For example, if you choose a Likert scale, choosing the number of points in the Likert scale to use is critical.
If you use a 10-point Likert scale, is this going to make it too complicated for the respondent to interpret between 7 and 8 for example?
If you use a 5-point Likert scale, will respondents likely resort to the middle, i.e. 3 out of 5, out of laziness or not wanting to be too controversial? Is it better to use a 6-point scale and force the user not to sit in the middle of the fence with their responses?
If you are using a 3-point Likert scale, for example, High/Medium/Low, is this going to provide sufficient granularity that is required in case there are too many items where users are rating medium, therefore making it hard for you to extract answer comparisons across items?
Example of survey design interface and reporting from Change Automator
5. If in doubt leave it out
There is a tendency to cram as many questions in the survey as possible because change practitioners would like to find out as much as possible from the respondents. However, this typically leads to poor outcomes including poor completion rates. So, when in doubt leave the question out and only focus on those questions that are absolutely critical to measure what you are aiming to measure.
6.Open-ended vs close-ended questions
To increase the response rate, it is common practice to use closed-ended questions where the user selects from a prescribed set of answers. This is particularly the case when you are conducting quick pulse surveys to sense-check the sentiments of key stakeholder groups. Whilst this is great to ensure a quick, and painless survey experience for users, relying purely on closed-ended questions may not always give us what we need.
It is always good practice to have at least one open-ended question to allow the respondent to provide other feedback outside of the answer options that are predetermined. This gives your stakeholders the opportunity to provide qualitative feedback in ways you may not have thought of.
Writing an effective and valid change management survey is often glanced over as a critical skill. Being aware of the above 6 points will get you a long way in ensuring that your survey is designed in a way that will measure what it is intended to measure. As a result, the survey results will be more bullet-proof to potential criticisms and ensure the results are valid, and provide information that can be trusted by your stakeholders.
Change saturation is one of the popular search items when it comes to measuring change management. How do we effectively measure change saturation without resorting to personal opinions? And how might we formulate effective recommendations that are logical and that stakeholders can action immediately?
Use this recipe to measure change saturation using The Change Compass.
Measuring change has become increasingly popular within the change management discipline. It used to be that change practitioners were more comfortable thinking and proposing that they are all about people, and therefore people factors are not hard and easy to measure. Areas such as change leadership, training, communication, and engagement were often the key tenants of a change professional’s key focus areas.
With increasing digitisation and focus on data and metrics, change management is also not exempt from this trend. Business leaders are now demanding that change management, just like most other business disciplines, demonstrate their value and work outcome through measurable metrics.
Using change management software to measure change
Even before the more recent trend of focusing on various aspects of change, change management software products have emerged. 10 years, when the basic idea for The Change Compass started, there was only 1-2 change management software in the market. Several years after that, there started to be 2-4 product offerings emerging in the marketplace.
Over the years, many of these software products have closed shop, or been sold to other companies. At the time, the need for change management software to measure change was not strong in the marketplace. It was perhaps an immature market where a lot of practitioners saw little need.
Types of change management software
There are many types of change management software. First, let’s spell out that we are not talking about technical change management software such as ITIL or technical change tracking software. The focus of this article is on organisational change management software.
The different types of change management software include:
Change project adoption measurement – Such as ChangeTracking that focuses on measuring the extent to which the project is progressing on track from a stakeholder perception and adoption angle
Change project implementation measurement – Such as Change Automator that provides a platform to automate project change delivery work that change practitioners focus on in capturing change deliverables, and overall change reporting and tracking
Change portfolio measurement – Such as The Change Compass that offers data visualisation for a change portfolio (collection of change projects) to help make portfolio level decisions on prioritisation, sequencing and delivery risks
Other organisational measurement – Such as OrgMapper that provides organisational network analysis maps and networks to understand relationship networks across individuals and groups
Data capture and automation
One of the key value propositions of a digital change management software is to provide some levels of automation in the capture of change management data, so that the data may be easily analysed and visualised. However, in our analysis of available change management software in the market, we found that there is a wide range of various levels of automation. Some have almost zero automation, whilst others have significant levels of automation.
In searching for an effective digital change management tool, ensure that you take into account the following in data capture:
To what extent is the data capture flexible and can be easily tailored
What are the system features to automate data entry?
Is there a range of data analytics from the data entered
Is the tool just a data depository without insight generation and data analysis features
Is the data capture too categorical? E.g. Agile vs Waterfall? And how useful are the data fields in terms of making decisions or generating insights?
Here is an example of automation from Change Automator where stakeholder data can be pulled from the company’s Microsoft Azure system to reduce the significant time required to input stakeholder details.
Change Automator example of using automation to save time
Data analysis & reporting
The power of digital software is that it can easily calculate, track, and visually show the metrics that we are focused on. Looking at raw data is meaningless if it cannot be turned into highly engaging and meaningful charts that generate an understanding of some kind of insight into the organisational situation with regard to initiatives.
Some change management software reports simple figures that may provide limited usefulness. For example, the number of impacts affecting each stakeholder group may be interesting but there is not much we can do with the data. This is because the number of impacts doesn’t indicate the overall severity or volume of the impacts.
Data visualisation should also support ‘drill-through’, where the user is able to click on the chart and drill into more details about that particular part of the data to better understand what contributes to it. This is a critical part of data analysis and understanding the story that the data is telling us.
Effective data visualization
Data visualisation formats are also critical. With the wrong data visualisation design, it becomes very difficult for people to understand and interpret the data. Ideally, the user should see very clearly what the data is showing them visually. For example, pie charts have become very popular in reporting. However, pie charts are only useful to contrast a few different data points. When there are too many data points and the data is too similar, the human eye finds it difficult to compare and contrast any differences.
Effective data visualisation should also allow the user to highlight a part of the data to create a visual emphasise to support a particular point. Making the visuals simple for the user is ultimately the most important part of chart design. The more complex it is, the harder it is for you to get your point across to your stakeholders.
To learn more about designing the right data visualisation to create optimal impact check out our infographic.
Insight generation & decision making
Ultimately, the change management software should be designed to provide insight into what is going to happen to the impacted people (whether it be employees, customers, or partners). The data should help you zoom into where is the source of the problem or the risk area, what the risk is, and potentially how to make a recommendation to resolve it. The drill-through capability is critical to support the insight generation.
The data visualisation should also directly support you or your business stakeholders to make business decisions on change. If the data was just ‘interesting’ it will not have much impact and after a while business stakeholders will lose their interest in the data. Instead, data and reporting should form a core part of regular business decision-making. Decision making using change data can be:
Within a project in making roll out and implementation decisions
Within a portfolio in making prioritisation and sequencing decisions
Within a business unit in making capacity prioritisation, business readiness and operational planning decisions
Across the enterprise in PMO and change governance settings on prioritisation, sequencing, benefit realisation and enterprise planning
Tailoring of data visualization
For those users who are more advanced with change analytics, there may be stakeholder requests to tailor charts in different formats. It could also be that for a specific organisational scenario, the user would like to create a tailored chart to show the specific problem that is not represented in existing off-the-shelf report designs.
In this case, the software should have the flexibility to allow these users to select their desired data fields and even types of charts that they want to work with to design the tailored chart without too much effort, and ideally not from scratch.
Here is an example from Change Automator where the user is able to easily tailor a chart by selecting the data fields, experiment with different charting, to come up with the ideal chart to influence stakeholders.
Example from Change Automator on easily tailoring charts
Trend analysis and predictive analysis through machine learning
Reading and interpreting individual charts can yield significant insights. However, the real power of analytics is to look at historical trends and even predict future trends based on data. Therefore, having the right data, over time, can create significant value. This is why investing in data is so critical, and why not just technology companies, but most industries are focused on digitising and leveraging the power of data.
The same thing applies to change analytics and change data. Invest in change data and the benefits can be enormous. By better understanding data trends with the assistance of machine learning, the system can highlight and draw your attention to critical observations and findings that you may have skipped.
With sufficient data, you’re also able to utilise machine learning to generate predictive data trends. Some examples of situations in which this can provide significant value include:
Typical times in which the business unit or team are busy with changes or operational challenges
Typically how initiatives of different complexities take to adopt and embed within the business
Typical delays in forecasted versus actual change implementation timeline
Stakeholder groups that tend to show the highest resistance or lowest engagement to initiatives
Predicted time it takes to realise targeted benefits
Investing in a change management software can create significant value for your organisation by measuring change and making it visual and easier to understand. Selecting the right tool is critical since there is a variety of options on the market. Examine closely the functionalities and how they enable you to make business decisions since not all charts may be useful. With the right software support, you will be able to not only tell a compelling, data-backed story of what is going to happen to the business, but also the logical recommendations that stakeholders find hard to dispute.
Change Automator example of leveraging machine learning in change analytics
