2023 so what?

By GM Moving | 06 June 2024 | TAGS: Local Pilot, Evaluation, "Research and learning"

Since baseline in 2019, the consortium have prepared annual evaluation and learning and interim bi-annual process learning update reports using a mix of evaluation methods to test and refine theories cyclically for each of the five enablers of change (ways of working). 

This update covers annual partner perceptions and reference to active lives data trends, alongside the regular C&Ms, reflective practice and deep dives against the enablers of change. 

The evaluation work is part of our Place Partnership (previously Local Pilot) work, find out more here, which adopts a whole-system, place-based approach to behaviour change in relation to increasing physical activity levels.   

Physical activity: Active Lives analysis  

The latest active lives data (2022-23) trends suggest a simultaneous increase in the percentages of adults and children who are classed as ‘active’ and ‘inactive’.

This might indicate growing inequality in the sense that some people who were already ‘fairly active’ have become more active, while others may have reduced their activity levels.

Within this there is also variation across places and people.  

Programme theory and system maturity   

The Place Partnership survey and the Counts and Measures qualitative tool aim to capture how ‘established’ certain ways of working – mainly viewed through the enablers of change – are in localities. This is one way of assessing system maturity. 

The Place Partnership survey captures how people in localities view various aspects around physical activity and place-based work in their area, and by extension: the work of the Place Partnership core teams. The Counts and Measures tool looks at this from the perspective of Place Partnership teams who list various work examples across the ‘enablers of change’ and rate how established these are.  

The results continue to show growing maturity since the initial baseline in 2020. The scores across both methods of collection in 2023 are overall slightly higher than the 2022 scores (9 out of the 10 measures being higher than last year – all five Counts and Measures scores and four out of five PP survey scores except involving local people and growing assets).   

Even with a proactive approach to intentionally widen the diversity of respondents (123 to 288) than in previous years. Some localities had concerns that opening up the Place Partnership survey to wider audiences might lead to lower scores on key variables and enabler summary scores.

This is because views on localities’ work across the enablers might now be shared by people not as close to the work as previous survey audiences. However, high-level survey results are in line with, and often indicating slightly higher system maturity compared to, the results of the previous year.  

In regards to the five specific enablers of change, the following observations were made: 

Strategic leadership enabling collective leadership 

This programme theory identifies that where there is a spine of influence, from community, local officer, management, and senior levels, a sense of ‘collective leadership’ is promoted.

• The average score in the Place Partnership survey increased compared to last year and has seen a 17% relative increase compared to 2020. 

• The Counts and Measures summary indicates that various work examples are increasingly established. There has been a 18% relative increase in the summary score compared to 2020.  

• 58% of examples were perceived as ‘established’ and 18% ‘in progress’. This suggested that a high proportion of leadership examples are well established practices for localities. Only 13% of examples were ‘getting started’ and 7% were ‘in planning’. 

It is important to consider further whether the distributed leadership principles expressed via leadership training translate into wider values and perspectives.

Subsequent interviews with front-line staff will explore whether employees at all levels feel able act on and make their own decisions when they return to deliver their role, post training.

To date it is apparent that sometimes having the courage and belief to act in a certain way may be at odds with a wider, external system.

Often the external structural, political or economic environment within which GMM is operating presents preventative barriers or hinders the operationalisation of system leadership principles.  

Deep dive - systems leadership to follow

Effective work across and between sectors

This programme theory is founded on the belief that cross-sector working is about collaboration and teamwork, having explicit recognition that the pooling of energy, resources and ideas to work towards a common agenda is likely to be more effective in coherent and sustainable changes.2 

• The average score in the Place Partnership survey increased compared to last year and has seen a 15% relative increase compared to 2020.  

• The Counts and Measures summary indicates that various work examples are increasingly established. There has been a 38% relative increase in the summary score compared to 2020. 

• Nearly half (47%) of examples were classified as established, showing the maturity of relationships built within and between sectors. There was a more even distribution of examples ‘in progress (23%) and ‘getting started’ (21%), providing evidence that new collaborations are starting and are progressing to become more effective. 

Although survey respondents of the past two years perceived these sub-questions as more embedded than in 2020-21, these scores still suggest that there is some space for improvement.

Qualitative work and conversations suggest, for instance, that (policy) changes require structural support. Individual relationships between partners might not be sufficient to lead to effective change without support from key positions and facilitative governance.  

Deep dive - GM Moving conference. 

Learning and adapting

This programme theory is centred on the knowledge that the Place Partnership and previous LP had no explicit KPIs, and as such emphasis was placed on using data and insight to inform approaches, and testing and learning.3 

• The average score in the Place Partnership survey increased compared to last year and has seen a 11% relative increase compared to 2020.  

• The Counts and Measures summary also illustrates that various work examples are increasingly established. There has been a 45% relative increase in the summary score compared to 2020. 

• 26% were perceived as ‘established’, and a similar share (25%) were classified as ‘in planning’. 23% were ‘getting started’ and 25% ‘in progress’. 

The in-depth examples from Oldham and Wigan might also indicate that continuous learning requires time and capacity, and needs to be reinforced by line managers creating value by taking small but important steps.

This could lead to developing a ‘learning culture’ to enable spaces in which people can genuinely share learning on which can be built. 

Deep dive - Wigan Reflective practice and Research Ambassador Training in Oldham.

Involving local people and growing assets

This programme theory is founded on the assumption that involving local people in the work would help to ensure local ownership and effective communication about physical activity between people in the local area.

This, in turn, would lead to more appropriate investments and/or shifts in local culture which enable people to be more active.4 The importance of dedicated roles and facilitating community leadership was highlighted when last formally updated in March 2023. 

• The average score in stakeholder perceptions in the Place Partnership survey decreased compared to last year (2022) but has seen a 19% relative increase compared to 2020.  

• The Counts and Measures summary shows that examples are increasingly established, showing a 24% relative increase in the summary score compared to 2020. 

• Over half (51%) of examples were classified as established, showing the maturity of practices related to community involvement and asset-based approaches. There was a more even distribution of examples ‘in planning’ (17%), ‘getting started’ (13%) and ‘in progress’ (17%). 

Overall, the scores from GM may suggest that ‘Involving local people and growing assets’ has reached a plateau. The reality is that for most places the efforts to maintain engagement and replace individuals and organisations that drop away from the work (fragile/precarious) means that it can take a lot of time and work to ‘stand still’. 

Transforming governance and processes

This programme theory centres on the belief that governance processes should be designed to support place-based inter-sectoral collaboration or distribute resources easily around the system.5 

• The average score in the Place Partnership survey increased compared to last year and has seen a 15% relative increase compared to 2020.  

• The Counts and Measures summary illustrates that examples are increasingly established, showing an 18% relative increase in the summary score compared to 2020. 

• 29% of examples were ‘in planning’ and 27% ‘getting started’, indicating that many current practice examples related to governance and processes are in early stages and/or new projects. 

Summary and next steps 

• Explore how different systems leadership approaches work in different contexts to put knowledge and learning into practice
• Explore how co-designing evaluation and learning approaches bespoke to locality leads can embed a culture of learning to action
• Explore how investing in community capacity building in Wigan and Bury can support more sustainable local leadership
• Explore how ways of working can be embedded across sectors beyond test and learn through facilitative and agile governance