An important part of measuring meaningful change is to be able to design effective communication effectiveness change management surveys that measure the purpose of the survey it has set out to measure the level of understanding of the change. Designing and rolling out change management surveys is a core part of what a change practitioner’s role is. However, there is often little attention paid to how valid and how well designed the survey is. A survey that is not well-designed can be meaningless, or worse, misleading. Without the right understanding from survey results, a project can easily go down the wrong path. This is how this survey can be a powerful tool to ensure smooth transition for the change initiative.
Why do change management surveys need to be valid?
A survey’s validity is the extent to which it measures what it is supposed to measure. Validity is an assessment of its accuracy. This applies whether we are talking about a change readiness survey, a change adoption survey, employee engagement, employee sentiment pulse survey, or a stakeholder opinion survey.
What are the different ways to ensure that a organizational change management survey can maximise its validity and greater success?
Face validity. The first way in which a survey’s validity can be assessed is its face validity. Having good face validity is that in the view of your targeted respondents the questions measure what they aimed to measure. If your survey is measuring stakeholder readiness, then it’s about these stakeholders agreeing that your survey questions measure what they are intended to measure.
Predictive validity. If you really want to ensure that your survey questions are scientifically proven to have high validity, then you may want to search and leverage survey questionnaires that have gone through statistical validation. Predictive validity means that your survey is correlated with those surveys that have high statistical validity. This may not be the most practical for most change management professionals.
Construct validity. This is about to what extent your change survey measures the underlying attitudes and behaviours it is intended to measure. Again, this may require statistical analysis to ensure there is construct validity.
At the most basic level, it is recommended that face validity is tested prior to finalising the survey design.
How do we do this? A simple way to test the face validity is to run your survey by a select number of ‘friendly’ respondents (potentially your change champions) and ask them to rate this, followed by a meeting to review how they interpreted the meaning of the survey questions.
Alternatively, you can also design a smaller pilot group of respondents before rolling the survey out to a larger group. In any case, the outcome is to test that your survey is coming across with the same intent as to how your respondents interpret them.
Techniques to increase survey validity
1. Clarity of question-wording.
This is the most important part of designing an effective and valid survey. This is a critical part of the change management strategy. The question wording should be that any person in your target audience can read it and interpret the question in exactly the same way.
Use simple words that anyone can understand, and avoid jargon where possible unless the term is commonly used by all of your target respondents
Use short questions where possible to avoid any interpretation complexities, and also to avoid the typical short attention spans of respondents. This is also particularly important if your respondents will be completing the survey on mobile phones
Avoid using double-negatives, such as “If the project sponsor can’t improve how she engages with the team, what should she avoid doing?”
2. Avoiding question biases
A common mistake in writing survey questions is to word them in a way that is biased toward one particular opinion which may lead to biased employee feedback. This assumes that the respondents already have a particular point of view and therefore the question may not allow them to select answers that they would like to select.
Some examples of potentially biased survey questions (if these are not follow-on questions from previous questions):
Is the information you received helping you to communicate effectively to your team members through appropriate communication channels?
How do you adequately support the objectives of the project
From what communication mediums do your employees give you feedback about the project
3. Providing all available answer options
Writing an effective employee survey question means thinking through all the options that the respondent may come up with regarding the upcoming change. After doing this, incorporate these options into the answer design. Avoid answer options that are overly simple and may not meet respondent needs in terms of choice options.
4. Ensure your chosen response options are appropriate for the question.
Choosing appropriate response options may not always be straightforward. There are often several considerations, including:
What is the easiest response format for the respondents?
What is the fastest way for respondents to answer, and therefore increase my response rate?
Does the response format make sense for every question in the survey?
For example, if you choose a Likert scale, choosing the number of points in the Likert scale to use is critical.
If you use a 10-point Likert scale, is this going to make it too complicated for the respondent to interpret between 7 and 8 for example?
If you use a 5-point Likert scale, will respondents likely resort to the middle, i.e. 3 out of 5, out of laziness or not wanting to be too controversial? Is it better to use a 6-point scale and force the user not to sit in the middle of the fence with their responses?
If you are using a 3-point Likert scale, for example, High/Medium/Low, is this going to provide sufficient granularity that is required in case there are too many items where users are rating medium, therefore making it hard for you to extract answer comparisons across items?
5. If in doubt leave it out
There is a tendency to cram as many questions in the survey as possible because change practitioners would like to find out as much as possible from the respondents. However, this typically leads to poor outcomes including poor completion rates. So, when in doubt leave the question out and only focus on those questions that are absolutely critical to measure what you are aiming to measure.
6.Open-ended vs close-ended questions
To increase the response rate of change readiness survey questions, it is common practice to use closed-ended questions where the user selects from a prescribed set of answers. This is particularly the case when you are conducting quick pulse surveys to sense-check the sentiments of key stakeholder groups. Whilst this is great to ensure a quick, and painless survey experience for users, relying purely on closed-ended questions may not always give us what we need.
It is always good practice to have at least one open-ended question to allow the respondent to provide other feedback outside of the answer options that are predetermined. This gives your stakeholders the opportunity to provide qualitative feedback in ways you may not have thought of. This may include items that indicate employee resistance, opinions regarding the work environment, new ways of working, or requiring additional support.
Writing an effective and valid change management survey best practices for a specific change initiative is often glanced over as a critical skill. Being aware of the above 6 points will get you a long way in ensuring that your survey addresses areas of concern in a way that aligns with your change management process and strategy and will measure what it is intended to measure. As a result, the survey results will be more bullet-proof to potential criticisms and ensure the results are valid, providing information that can be trusted by your stakeholders.
Change saturation is one of the popular search items when it comes to measuring change management. How do we effectively measure change saturation without resorting to personal opinions? And how might we formulate effective recommendations that are logical and that stakeholders can action immediately?
Use this recipe to measure change saturation using The Change Compass.
Measuring change is no longer a nice to have. It’s a must-have for a lot of organisations. A lot of stakeholders are now demanding to see and understand what is happening in the world of change. With the enhanced volume of change and therefore the increased investment made by the organisations, it’s no wonder.
Why are stakeholders demanding to see change data?
When we look across the room amongst the various disciplines, data forms an integral part of any function. Finance – tick. HR – tick, yes pretty much all aspects of people are tracked and reported. Operations – tick, as we have all types of performance KPIs and efficiency indicators. Technology – tick, since every part of technology can easily be measured and reported. Marketing – tick, as marketing outcomes are tied to revenue and customer sentiments.
With Covid it is even more the case that data is integral. We can no longer ‘walk the factory’ to sense what is happening. To see what is happening and what is going to happen stakeholders revert to data. In our virtual working environment, stakeholders require a constant dashboard of data to track how things are progressing.
Why is measuring change not an activity?
In the past it used to be that measuring change is only something you do in a project when you want to see if stakeholders are ready for the change. No more. Most organisations have a multitude of changes running concurrently. There is no choice to select 1 or 2 changes to roll out. With significant business challenges, most organisations are finding that running with multiple changes is the norm.
With multiple changes, increased stakeholder demands and appetite, measuring change is no longer just an activity. Measuring change takes a set of structured routines. It requires effective governance design. It takes experience and analytical expertise. Most of all, it is not a once-off event, it is a continual building of organisational muscle and capability. We are heading into the world of change analytics capability.
What is change analytics capability and how do I attain this?
Here are 7 core components of building and maturing change analytics capability:
1. Establishing change data management procedures and practices
This is about setting up the right steps in place so that change data can be identified, collected, and documented. This includes identifying the types of change data you would like to collect and how to go about collecting them. It will be easier to start with the core set of data required and then build from these as needed. This will reduce the risk of overwhelming your stakeholders.
After the right metrics and collection channels have been identified then it’s about building the regular routines to collect and document the metrics.
2. Sponsorship and leadership of change analytics
To really reap the value of change analytics you will need to gain the blessing and sponsorship of your leaders. Well, at least in time. In the beginning, you may need some time to come up with compelling data that tell the story that you want them to before you show your leaders. Eventually, without strong leadership buy-in, change data will not be effectively leveraged to make business decisions.
Getting your leaders’ blessing isn’t just a verbal exercise. It means that they are signing-up to regularly review, discuss and utilise change data to realise business value.
3. Build talent and organisation to support change analytics
Think about the various stakeholders and what you need them to understand in terms of change data. The way you educate stakeholders will be different to how you educate operations managers or the PMO. Plot out how you plan to help them get familiar with change data. Do you need particular roles to support data analysis? Is it a Change Analyst who is focused on the regular upkeep and consolidation of change data? What roles do you need other team members to play?
4. Insight generation
With a full set of change data infront of you, it’s now time to dive into them to generate insights. What is the data telling you? How do they support other data sources to form a clear picture of what is happening in the workforce? Is the data accurate and updated? Generating insights from the data takes skills and experience. It takes the ability to integrate different sources of data outside of change data themselves.
5. Insight application
This is about setting up the right routines and processes so that any insights generated may be discussed and applied. It could be through various governance forums, leadership or planning meetings that insights are shared and socialised. An integral part of this step is applying the insight by making business decisions. For example, do we delay the initiative roll out or invest more to support leaders? Are there reasons for us to speed up roll out to support the workforce?
6. Change analytics capability development
Change analytics is a capability.
With good change data emerging, you also need to have the right people with the right skills to collect, process and interpret the data. You may also want to think about which teams need what analytical skills. Do you have people in the team who are sufficiently analytical and data-oriented? Do they know how to interpret the data to form trends and predictions?
You may want to think about organising capability sessions or training to strengthen data analysis skills. Are there members in the different governance bodies that need support to be more confident in using change data?
7. Realising business value through change analytics
The last part of the equation is realising business value through change analytics. This is about tracking and documenting the value realised through using change analytics. It could include incidents where the business decision made has lead to significant risk reduction or operations protection. It could be enhanced leadership confidence mitigating risks in negative customer experience. Tracking value generated is critical to make clear to stakeholders the value of the overall investment.
If you found this article useful please share.
Do you have questions on measuring change for your organisation? Ping us on our chat.
Change Management outcome is the holy grail, and virtually all organisations are undergoing change. Now more than ever, companies are challenged with multiple layers of driving change simultaneously. What is applicable in this situation is not about a particular methodology of implementing a change program. It is all about implementing simultaneous changes, at the same time. There is no luxury of just focusing on one change at a time, the result of competitive, industry, and environmental challenges.
As change practitioners we work closely with our colleagues in Operations to get ready for, implement, and fully embed changes. So how do our colleagues in operations view and manage change initiatives?
Operations as a function is focused on managing performance and delivery to ensure that the business runs smoothly, with little disruptions, and that performance measures are achieved. Operations is focused on resource management, efficiency, and achieving the various operational indicators whether it’s customer satisfaction, turn-around time, average handling time, or cost target.
When times are hectic and a lot is going on with multiple change initiatives, the key focus for Operations is on managing people’s capacity. Key questions would be “Do we have sufficient time to cater for the various changes?”, and “Will we exceed our change saturation level?”. This is a critical question to answer since the business still needs to run and deliver services without negative change disruptions.
From an Operations planning perspective ‘change capacity‘ is often reduced to the time element, especially those impacting frontline staff.
For example:
What are the times required to reschedule the call centre consultants off the phone to attend training?
How much time is required in the team meeting agenda to outline the changes that are being rolled out?
What is the time involvement of change champions?
Though these are all critical questions clear answers will help Operations plan better to face multiple changes. However, this is not adequate. There is more to planning for multiple changes than just focusing on the time element.
Using the lego analogy to manage multiple changes
We all know LEGO as kids. To build a car we start one brick at a time and see how we go. We experiment with different colours, shapes, and sizes. We make do with the bricks we have and use our imagination to come up with what a car would look like. Sometimes we get stuck and we may need to tweak our bricks a little, or sometimes start from scratch.
It is the same as implementing change initiatives. In order to take people along the journey, we implement a series of activities and interventions so that our impacted stakeholders are aware, ready, committed, and embed the change. The design on the change journey is the process of determining what LEGO bricks to choose. There is no shortcut. It is not possible to build a building without each necessary brick to raise the building up. In implementing change, we also need to lay out each step in engaging our stakeholders.
McKinsey studies over decades have told us that one of the most critical factors to focus on in ensuring change outcome success is clear organisation-wide ownership and commitment to change across all levels. This means that when we design each change brick we need to ensure we target every level of impacted stakeholders.
For example:
Team Leaders: How often do we want Team Leaders to talk about the changes to their teams before the rollout? What content do we want them to use? Do they know how to translate the message in a way that resonates? Do we want them to tell compelling stories that talk to the what, why, and how of the change?
Managers: How are managers made accountable? What metrics are they accountable for? What mediums do we want them to use to engage their teams? What are the consequences of not achieving the outcomes?
Senior Managers: Through what mediums do we expect senior managers to engage their teams about the changes? How do we ensure that they are personally accountable for the success of the change? How are they involved to ensure they own the change?
Looking at the above you can see that for complex change there may need to be a lot of bricks in place to ensure the change outcome is successful!
Going back to the issue of facing into multiple changes, how do we play around with the bricks to ensure that multiple changes are successful? The same way that we play with LEGO bricks!
Look at the colours of the bricks. Do certain colours belong together? When we look across different initiatives, are there similar or common behaviours that can be better linked together to tell a compelling story? Do they support the same strategy? Can there be a joint campaign for these changes?
Is the overall LEGO structure going to be intact? What are the impacts of the various changes happening at the same time in terms of focus, performance and change outcome? Have we exceeded the likely ‘mental capacity’ for people to stay focused on a core set of changes at any one time? Will the pieced-together structure collapse due to having too many elements?
Look at the sizes of the LEGO structures. During implementation when we have both larger and smaller initiatives being executed at the same time, will the larger ones overshadow the smaller ones? If so what are the risks if any?
Re-jig or re-build parts of the LEGO structure as needed to see what it looks like. In a situation where we want to see what the changes look like before we action it, it makes sense to visualise what would happen if we move timelines or change implementation tactics
Example of data visualisation of ‘re-jigging’ change implementation timeline with The Change Compass using different scenarios.
Just like in building LEGO, for change initiatives we need to be agile and be flexible enough to play with and visualise what the change outcome could look like before pulling the trigger. We also need to be able to tweak as we go and adjust our change approaches as needed. In facing the multitude of changes that the organisation needs to be successful, we also need to be able to play with different implementation scenarios to picture how things will look like. Each brick needs to be carefully laid to reach the overall outcome.
Careful consideration also needs to be how all the bricks connect together – the analogy that the change outcomes across initiatives can be determined by how we’ve pieced together various pieces of LEGO for them to make sense, and result in the ownership and commitment of stakeholders.
