Evaluation

Overhead view of a group dance exercise class in a community centre

We we want to understand not just what change has happened, but how and why this has happened, as well as explore how this might differ in different contexts.

Substance consortium (Substance and Sheffield Hallam University) were commissioned in March 2019 as an evaluation partner for the Greater Manchester (GM) Place Partners work (previously known as the local pilot). They have employed a mix of realist, developmental and participatory evaluation methods to help us to understand and explain the extent to which system conditions, referred to as the five enablers of change, in a place can enable active lives for all long-term.

This was for two key reasons, it remains difficult, due to the complexity and the distance between cause and effect, to say direct contributions from the actions of GM Moving partners are moving us towards our goal. And yet the mixed methods data gives us confidence that efforts are plausibly contributing to shifts within the system and generating instances of impact.

The methodology involves three broad stages to develop, test and refine theories for each of the five enablers of change. These follow a cyclical process that has been repeated over seven times throughout the past four and a half years.

To test the theories, the evaluation team employed a range of methods and data sources. Amongst other things, these included:

  • Embedded researchers to capture qualitative observations of meetings or events.
  • Deep dives on emerging topics through in-depth qualitative interviews with wider partners or community members.
  • Extensive “Counts and Measures” spreadsheets combining qualitative and quantitative data and work examples around the five enablers of change. These are updated every six months.
  • Reflective practice forms following three steps: What (description of the story or problem to reflect on); So what (challenges, learning, what worked or did not work); Now what (what are the next steps or plans to use that learning). These are connected to the five enablers and examples from the Counts and Measures.
  • The yearly perceptions survey sent to stakeholders in each locality to capture their views on how embedded or “mature” the five enablers of change are in their place.
  • Secondary data analysis on the National Active Lives data.

After each cycle, insights from these and other data sources are then used to refine the theories to better incorporate learning on patterns of ways of working might lead to outcomes in different places.

This involves a complex process of synthesising and coding the various numerical and rich qualitative data sources. An important part of refining theories were the “sensemaking sessions” in which place-based partners discussed their data and outputs with the research team. Part of these discussions were also how to scale up, deepen, or expand insights and learnings from place-based work. Following the writing of insight and impact reports, a new cycle would begin to develop and test new and adapted theories.

You can find the process evaluations from:

To understand more about features of system working, how and why we think they may support active lives, and what we are learning about trying to work in this way read below to find out about each of the enablers of change.

Further details please contact

Enablers of change

Over the last 5 years we have been trying to understand how to get more people moving through a whole system approach.

To develop our understanding we have examined existing evidence from other whole system approaches, looked at alternative ways to capture the value in our work, and had lots (and lots!) of conversations with our partners across Greater Manchester.

From this, we have developed five enablers of change, which are system conditions we know need to be in place in order for sustainable and equitable change to happen. The five enablers not only help us to understand how we can work in a joined-up way to create changes that enable active lives for all, but they also can be used to help plan and guide our work going forward.

Watch the video (right) to discover what the five enablers of change are, and use the links below to learn more about each of them.