Measuring change is not just an activity, it is about achieving a change outcome.  The particular outcome can range from understanding how ready stakeholders are for the change, how change mature they are, whether they have adopted the change, or whether they have the capacity for change.  Achieving this change outcome is not just about selecting the right measure or collecting the right data.  Even after analysing the data and interpreting the results, there is a number of steps required to achieve the targeted change outcome.

In the first part of ‘The Ultimate Guide to Measuring Change’ the focus is on outlining the various areas in which change may be measured.  These include initiative-based change management measures such as training evaluation or communications effectiveness.  Non-initiative-based change management measures include change leadership assessment, change saturation assessment or change maturity assessment.

In this part 2 of the same topic, we focus on the various steps involved in achieving your change outcome when you embark on your change measurement exercise.  So, you have already selected the change management measures.  You know how the measurement works and how to collect the data.  What next?

Dashboard design

Developing a good dashboard can make or break the success of your initiative.  A good dashboard draws the attention of your stakeholders to the right areas of focus and can prompt action.  As your stakeholders receive ongoing feedback on how the measures are tracking in the dashboard, this provides them with evidence of whether any actions they have taken have influenced the measures.

On the other hand, a badly designed dashboard can be overwhelming, difficult to understand, and most importantly do not prompt the right attention and action required from stakeholders.  In this case, the dashboard may be skimmed over by stakeholders and not taken seriously.  This is absolutely what we want to avoid.

Some key considerations in designing dashboards

1. Focus on the critical few.  More is not better when it comes to dashboard design.  The dashboard should comprise of key measures that tell a good picture (not necessarily a thorough one) of what is happening and if things are on track.  For example, if you would like to present a readiness dashboard, key measures could be:

• Overall readiness score

• Readiness by business unit

• Readiness by roles

• Training completion rates

• Yammer chat trends

2. Determining visual formats in your dashboard.  Not all data visualisation formats are created equally.  Some tend to be harder for users to understand than others.  Overly colourful charts can also distract the user from focusing on what you want him/her to focus on.  

A key question you should always ask yourself is ‘can the audience understand what this chart is showing in 5 seconds without context’.  This makes it challenging, right?  This is exactly what we need to aim for to make it easy for the audience.

For more information and tips on choosing the right visual formats, visit our article Making an impact with change management charts.

3. Create a balance of different types of measures that, in total, capture core aspects of what the audience would like to find out about.  For example, in the above example, there is a balance of measures such as surveys, stakeholder ratings during meetings, completion rates and trend analysis.  

Your dashboard could also have a mix of visuals from charts, figures, quotes, insights, and data tables.  It does not always need to be in the form of charts alone.

4. Experiment with different types of visuals and see which ones are the most appealing in terms of generating insight and action from your stakeholders.  Depending on the stakeholders you work with, some data visualisation formats may not resonate as much as others.  So it may take experimenting to figure out the most impactful set for your particular stakeholders.

5. Allow the user to drill through the data, where possible.  When you present the dashboard to your stakeholders, this will naturally prompt a series of questions and discussions.  Essentially, by allowing drill-through of the data, you are answering questions around ‘why’.

Common questions include what data comprise this component?  Real examples could be ‘why is this business unit not ready for the change’, ‘why is the training completion so low for last month’, and ‘why are stakeholders not confident about our overall readiness’.  To facilitate queries such as these, it is always very helpful to be able to click into the data points that comprise a particular chart. 

6. Your dashboard should form an overall picture of the story you are trying to tell on a regular basis.  Designing any dashboard requires you to be clear about the key story you are trying to tell to your stakeholders.  Some potential stories could be:

• We are slowly getting ready for this change, but not quite there yet
• Certain business units or roles are finding this change challenging and ongoing support and engagement is required to overcome their potential resistance
• We are incrementally making headways to engage the impacted employees, but some are more engaged than others
• We have not made significant grounds in our change maturity, though we are laying the right foundations to support the shift

For more information on this check out our article The ultimate guide to storytelling with change management data.

7. Automating dashboard data collection.  If you are running a less complex initiative or in a smaller organisation it may make sense to resort to Excel to create dashboards.  Many also use Tableau or Power BI.  However, both can be quite technical and would require time and effort to set up and use.  Experts in Tableau and Power BI within the organisation may likely be pulled into multiple demands already.  Leveraging off-the-shelf data visualisation platforms such as The Change Compass may provide a way to save significant time to enable plug-and-play options for complex change environments.  In this case, no expert knowledge is required to set up or maintain dashboards.

Making recommendations

So you’ve selected the change management measures, created the dashboard, and allowed the ability to drill-through the data as needed.  The most important next step is to prepare your recommendations.  You already have the what, the why, and now it’s time to answer the ‘so what’.  This is the part where you deliver your biggest value and make recommendations to improve the current status.

There are several things to consider in making your recommendations:

1. The recommendations should be very logical and a natural extension of the metrics and results you’ve presented.  It should almost be a ‘no-brainer’ for your stakeholders in your recommendations.  On the other hand, if your recommendation is not substantiated by the metrics you’ve just presented, or there is data missing, your stakeholders may easily challenge the basis of your recommendations.
2. Practice writing out the reasoning for your recommendations to test its deductive and logical reasoning.  Break this down into a step-by-step series of premises in which you can analyse and ‘prove’ the recommendation where possible.

For example:

• The Finance department has the lowest training completion rate
• Many participants from Finance commented that they did not understand why they are doing the training.
• There seem to be more questions about the initiative from Finance employees on Yammer
• The number of engagement sessions held in Finance is lower than in other departments
• Overall readiness scores for Finance are also lower than other departments
• Workload in Finance is not high during readiness survey nor during the training rollout
• The last employee engagement surveys showed that Finance employees had lower engagement scores than other departments
• Your recommendation:  Interview select Finance employees to understand why they are less engaged.  Since it’s unlikely due to workload, it may be due to leaders not putting the time and focus to engage employees about this initiative.  As a result, there is less engagement in the Finance department.  So the premise that needs to be tested is if the lower engagement is due to Finance leadership not engaging the group about the initiative.

The key call out here is to be able to sequence together a series of deductions that lead to the recommendation.  Each data visualisation shown is a step in this deduction that leads the user to agree with your recommendation.

3. Your recommendation must be actionable, and helps to move the needle toward improving the change outcome.  It should not be just a hypothesis, a conclusion or worse, an opinion.  It must be something that can be acted on by someone.  For example, a set of actions to remediate the change engagement, to motivate leaders to resolve employee resistance, to celebrate success with the team in the current progress, etc.

Ensuring data integrity

There is nothing worse than standing in front of your audience and being challenged by the data you are presenting because of data integrity.  If you are caught in the position of presenting data that is not accurate then you may only have a small window of opportunity to salvage your reputation.  To avoid this, ensure you have done your homework on the integrity of the data.

There are several things you can do in ensuring data integrity:

1. Look for patterns that may not make sense.  There are many ways in which you can easily zoom in on potential errors by checking:
   • Variance from previous data, e.g. Is there an unusual deviation from previous months’ report that may not make sense in terms of engagement scores?
   • All data points have been updated, e.g. Are you missing any survey data from particular business units? Have all initiative impacts been updated?
   • Patterns that may not make sense, e.g. You would have expected that Marketing would be the less change ready but the data suggests they are on-par with other business units.  This may prompt additional investigation
   • Drill-through significant findings to check that it makes sense.  E.g. The bar chart shows that March has the highest impact level on the business.  Drill into March data to double-check that this matches the observation and that it is not a chart error.
2. Regular data audits.  If your dashboard will be presented every month, aim to start auditing your data at least a few days before the presentation to leave enough time to do any follow-up on chasing the right data.
3. Send your dashboard to a ‘friendly stakeholder’ whom you can confide in without being challenged or critiqued.  Ask the stakeholder for feedback to check if there is any glaring error and if the data makes sense.

Storytelling

Having a set of metrics presented in a dashboard is exactly what it is … a set of data.  It may be presented in a way that is easy to read and easy to understand.  However, the purpose of dashboards is so that they can be presented and discussed so as to facilitate regular monitoring and decision-making.  This means that to really get the full impact out of the dashboards you need to be able to tell compelling stories that will grab the attention of your stakeholders and motivate them into action.

If you are making a full presentation, then you may want to follow the following sequence:

1. Key summary of findings and recommendations

This is the Barbara Minto pyramid model of starting with a quick summary of all your points including recommendations to quickly grab the attention of your audience.  After this, they will be clear in terms of what you will be talking about in your segment.  This is a form of executive summary and is very effective for senior executives who may have a low attention span and can easily get lost in the details.

2. Key findings and insights

Walk through your key findings and insights.  Ideally, stick to no more than 3 major points.  This is because with too many points the audience is unlikely to remember them and may be lost in the details.  If you have a lot to talk about, group them into themes so that this makes it easy for your audience to understand and remember.  Again, aim for no more than 3 themes if possible.

3. Why – substantiated by data

This is probably the hardest part to facilitate because it is likely that your audience may want to dive straight into the details of the data and start to ask all kinds of questions.  You may want to guide the conversation by telling the audience exactly the flow of what you will be going through to minimise any major disruptions in your storytelling.  

Avoid overwhelming your audience with too much data.  A part of your storytelling is to balance having just enough data to support your observations and lead them to agree with your recommendations in the next step. 

Balance discussion with your overall storytelling flow by allowing time and space for reflection and questions.  To maximise engagement, prompt certain members of your audience to see if the data resonates.  This can add weight to what you are presenting.  

4. Recommendations

If you are making a regular update at a recurring meeting such as a monthly planning meeting, committee meeting, or change governance meeting, then this could be the sequence that you will be following.  In this context, you may have limited time to highlight key observations and may not have enough time to dive into the details of all the data.  The focus is for a short and sharp call out of key findings, highlighting data that supports your observations, followed by a logical set of recommendations that are action-based.

This is the sequence for your presentation for updates:

1. Key findings and observations
2. What is new and different since the last meeting, or what trends to note
3. Why – substantiated by data
4. Recommendations

Change governance

It would be great if you can easily leverage existing meetings and business routines to incorporate your charts and dashboards so that they can be used.  However, it could be that existing meeting chairpersons do not currently see the value or that there is no current meeting that could serve this purpose.  Getting the value out of your dataset means that you need to think about the overall system of change governance.

If you are working in 1 project you can leverage existing project reporting rhythms such as monthly meetings for business stakeholders or meetings with impacted business units, project team and the project sponsor.  If you find that certain important stakeholders may not have a way to receive this information or that they should be involved in decision-making, you may need to work with your project manager and project sponsor to address this gap. 

Ensuring that your change metrics are used in the right change governance forums may take time to influence.  Often the ‘gatekeeper’ of the forums may not be focused on change management and be reluctant to provide additional agenda within an already crowded meeting.

To influence and get your metrics in the key decision-making forums, depending on your organisation, try influencing these key stakeholders:

• Portfolio manager/PMO
• Project managers/project sponsors
• Senior leaders from impacted businesses that see the value of your change metrics
• Business planning/Strategy managers that may be able to leverage your change data for better planning
• Risk partners who are responsible for influencing the business.  They may be particularly interested in risk-in-change 
• HR business partners who may be concerned about people capacity, engagement, and sentiments

Building capability in change analytics

Not all of your stakeholders will understand the data you are presenting straight away.  Especially in the beginning, it could take a bit of time to induct them to what the data is showing, how to interpret it and the significance of the data.  Some stakeholders may want to spend most of the session critiquing and diving into the details of the data.  Others may want to focus on parts of the data that are not what you want them to focus on.  Or, it could be that they need more coaching and support to decipher what the dashboard is telling them.  Be aware and ready to pre-empt and guide the flow of the conversation.

To build the capability in change analytics within your organisation:

1. Start by calling out the fact that you are presenting a new set of data and that you will be focused on spending time to support the audience in understanding and interpreting the data in the beginning
2. Use your business rhythms and regular meetings/committees to start your meeting segment by walking through step by step the art of interpreting the data
3. Provide coaching or drop-in sessions for a broader set of stakeholders to provide guidance as needed
4. Create an intranet page for the program or portfolio where stakeholders can access the data and ‘self-help’ in building their change analytics capability 

To read up more about building capability in change analytics visit our article How to Build Change Analytics Capability.