Local Pilot Process Evaluation Summary March 2020

By GreaterSport | 31 March 2020 | TAGS: Local Pilot, Research and learning

Find the Local Pilot process evaluation below. In the evaluation report, completed by evaluators Substance and Sheffield Hallam University, there were five emerging theories that have been identified as priority areas to drive system change.

Download the Local Pilot Process Evaluation Summary March 2020 here

New forms of leadership are required to motivate the workforce and local population to take action to enable active lives. Leaders – at all levels – not only have to be seen to be supportive but need to have an authentic interest and belief in the importance of physical activity.

This theme highlights the importance of engaging and continuously involving senior leaders and partners to align agendas. It also shows the importance of finding others with an authentic interest in the values, principles and ways of working.

Strategic Leadership

Since the last report, we now know how important it is to recognise and grow genuine, or authentic, interest in the work, to identify who can lead the way. A lack of authentic interest has slowed progress in the work, localities found that not everyone cared about physical activity as much as they did and didn’t know what leadership characteristics were required to lead.

There has been continued support from leaders in Greater Manchester, including Andy Burnham, the Mayor of Manchester, however at a local level, there was a feeling that senior leaders have become disengaged. Having senior leaders supporting was seen to be a key factor in whole systems change; but leaders needed to be engaged emotionally, rather than drawn in through the investment.

In response, a strategic lunch was held, hosted by GM Moving Chair Steven Pleasant. It was recognised that inviting senior leaders to be part of an exclusive conversation may reinforce hierarchies, and went against the local pilot principles, however it was judged that they needed to influence within an existing system. The lunch was a one-off event, and strategic leads were invited to attend the ongoing bi-weekly Local Forum meetings. Every locality was represented, and attendees pledged to support a whole system approach to reducing physical activity, including promoting cross sector working and supporting local leads in their work.

To build skills and capacity around system leadership, 10GM/MACC were commissioned to deliver system training. The initial sessions will increase skills, knowledge and capabilities of local pilot leads and partners by exploring values, public narrative, and an introduction to systems leadership.

Collective leadership

The LP work is based on the principle that system change cannot occur if led by one person or organisation, therefore collective or distributed leadership is required. There are a number of implications emerging from this:

• Leaders must be encouraged to emerge in a system where vision and principles are aligned
• Individuals should be supported to develop their own test and learn approaches
• How to challenge someone if they’re not upholding the principles
• How senior leaders can continue to be accountable for the work whilst letting go of control.

To address some of these issues we have seen:

1. Changes to the format of LP meetings. Moving from two meetings: one focussed on operations and another for strategy, to a Local Forum which has an open agenda and more frequent peer review session.
2. Steering group refresh. In some localities, steering groups were pressuring locality leads to spend money, and in some instances to spend money to fill gaps in provision. It was felt that this way of working did not align with the Local Pilot principles and consequently, some localities have refreshed their steering groups, even when it risked damaging relationships with existing partners.

Joining work up across the system involves an interwoven process of change leading us through:

• Relationship building;
• Connecting up the system;
• Working together on common projects;
• Productive partnership working around common goals

Relationship building

Relationships need to be built in all directions; with senior managers and leaders, with colleagues within organisations and with new organisations and sectors. We know that this requires persistence in order to lead to change, but new learning suggests that it is also important not to add to someone’s workload and to follow up on actions.

Connecting up across the system

There appears to be a balance between having the openness to pursue opportunities as they come up and work becoming overwhelming. It may work to have a number of focussed areas with the flexibility to pursue alternatives if opportunities arise.

Working together on a common project

The communities targeted in the local pilot, including those who are the most inactive, are also of interest to other public health interventions. It has been suggested by some localities, that there would be benefits to joining up to facilitate larger scale and more sustainable interventions.

Productive partnership working around a common goal

Partnerships appear to be driven by people who are able to influence and who can find alternative ways to solve problems. It also appears important to have people who are able to critically reflect on their work and processes, who are prepared to compromise and who can identify ways to do things differently.

Some of the benefits of working effectively across and between sectors include:

• Avoiding duplication
• New and innovative initiatives
• More sustainable funding

Example of productive partnership working around a common goal:

Brinnington is a residential estate adjacent to the M60 but also adjoining Reddish Vale Country Park and, as such, provides residents with access to walks and open countryside. This natural community asset is underused by Brinnington residents.

Following discussions at the Brinnington Forum highlighting that signage was not facilitative to encouraging physical activity in Brinnington, the group concluded that a community map might better enable such activity. Members of the Forum identified that, in Brinnington, people can be hyper-local in outlook and that if a community asset isn’t right on their doorstep or on their route to the local shop or school, then they might not be aware of it.

The decision was made to work in conjunction with TfGM to create a sustainable journeys application for the mapping and signage work to link Brinnington with the Town Centre and surrounding amenities, including the Vale. TfGM attend the Brinnington Forum so had built up a relationship with the community and LP staff.

A brief was created and early discussions with TfGM left the group concerned that the mapping output might not reflect exactly what the community had envisaged. To ensure the community ideas were reflected, TfGM ‘pitched’ their ideas and prototype maps with the Forum group. At this session in February, TfGM highlighted that they acknowledged they did not have the skills to respond to the ask from Brinnington residents following concerns raised by LP staff, noting:

TfGM maps were good, but they didn’t quite fit in with the community, so we went back and asked the design team to create something else. They said we can’t, so we went to tender an external illustrator.”

Two separate responses to the brief by illustrators were presented and the Forum considered these maps in table discussions. Following these group discussions around accessibility and readability, a map was approved. The map which was rejected was felt to be too pictorial in nature with one resident adding:

“It’s way too busy, I can’t understand it. It would be nice as a mural on the side of First House. It’s more like a work of art and a Beatles album cover or a promotional image for a community and not a map. It doesn’t tell you where to walk.”

The map was approved by the Forum and the illustrator is currently working on the final version ahead of sharing across Brinnington. This case illustrates the importance of engaging community ideas in shaping system-decisions in order to achieve behaviour change. The LPs intervention in acting as mediator between strategic partners from other parts of the system (TfGM) and the community have been vital in this process of influencing change.

TfGM, who benefited from being able to engage with local people through the forum, were able to reflect on the feedback and change their approach. This was thought to be an idea that could transfer to other areas, for example, in Partington, Trafford, a similarly isolated community which was well connected to green space. Consideration is now being given to how to evaluate the influence of the map on people’s ideas, engagement and usage of active travel / walking and biking for leisure in and around Brinnington.

Knowledge, effort and willingness all need to be in place to find ways to adopt a whole system approach which can also be accountable.

Whole system approaches to physical activity should be person-centred, flexible and responsive, promoting joined-up thinking with integrated policy and planning across organisations.

However, much of the local pilot work to date has suggested that current governance processes are not designed for place-based working, shared policies and or/ investing in local VCFSE organisations, and therefore create barriers to the LP’s test and learn approach.

Governance processes have caused blockages in the work at all levels of the system: from Sport England to GreaterSport; GreaterSport to Locality; and Locality to Locality System.

Changing processes requires clarity of purpose (why you need to change it), persistence, support from senior managers, patience from partners and flexibility within teams. Many of the changes in processes seen have been ‘work arounds’ rather than long term solutions to the problem, that involve the risk being pushed down onto the individual.

There are lessons to be learnt from the VCFSE sector, but also from others with experience of governing within whole system approaches.

Whilst learning environments which stimulate test and learn approaches are vital, stakeholders learn at variable pace and in different ways, with ‘penny drop’ moments emerging from many sources.

Learning and adapting can be further broken down into:

1. Using local data and insight

Many localities have undertaken local insight to inform their strategies, however, there seems to be a lack of confidence about how best to use the local insight. It may be that in some localities it is seen as too much of a risk to develop new approaches based on local insight.

2. Learning from Best Practice

Learning from outside the GM Local Pilot has been facilitated by the Sport England Community of Learning events. Learning has been shared from other pilots around specific topics.

3. Adapting Based on Emerging Theory and Evidence

Some people are becoming more comfortable with the conceptual thinking associated with whole system approaches. There is recognition, and acceptance, that the system is complex, dynamic, open and that outcomes are the result of many interacting and interlocking parts. This is seen where the LP values and principles are used to encourage changes to be made to the conditions that enable active lives.

We know is that people learn in different ways and at different paces. I, in this work we have seen people learning through:

• Immersing themselves in theory and conceptual discussions, and reading widely and/or
• Experience and sharing practical examples or challenges with each other

Both of these processes take time.

Critically reflecting on what is working and whether this can be improved appears to facilitate faster progress in this work.

There is a need to engage and involve local people and organisations of all types and size, using asset-based approaches and co-production. This might involve the Voluntary, Community, Faith and Social Enterprise (VCFSE) sector and others in a range of ways.

1. Investing in the VCFSE sector

We have previously discussed the different ways localities had engaged with the VCFSE sector, and in the last 6 months the evaluation team have been exploring how well each approach worked to engage with the targeted local community.

Localities have different relationships with their VCFSE organisations: in some there is an established anchor organisation, core funded by the local council. It is thought this adds stability allowing organisations to build strong links with communitiesy. Strong relationships with locality leads and VCFSE organisations has facilitated place-based working and the involvement of local communitiesy.

Localities without a core funded VCFSE sector have worked to create processes and structures so that they can invest into local VCFSE organisations on a smaller scale.
There is an acceptance among localities that work with local communities must be done at a local or hyper local level; particularly if they are to test and learn at the same time. A number of localities have taken the strategy of starting with one target audience or location and possibly moving on.

2. Asset-based approaches

Localities are engaging people by building on what local people consider to be existing assets. Assets may be human, social, political, financial, cultural or natural assets.

3. Engaging, involving and co-producing

It is important to involve local people at all stages of planning and delivery. We know that it takes time to build relationships with both individuals and partners to work in this way, and that partners must be motivated to develop solutions that meet the identified needs and wants of local people.
It is thought that involving local groups may maintain energy and lead to more effective and sustainable solutions.

Example of engagement, involvement and co-production

In Salford, LP investment has gone directly to Salford CVS which has given a £500 investment to Barton Athletics Club. The LP worker has linked the Barton Athletics Club to Eccles Girls Group – who were unaware of each other before this introduction.

After consultation and with the girls, it was established that they would like to try boxing and wrestling. Barton AC were approached and agreed to work in coproduction for this project which is primarily designed to target girls who are socially isolated and not engaging in physical activity but also to enable Barton Athletics Club become a destination for girls and young women, increasing the male to female ratio.

The sessions design has been coproduced and the 1st session allowed attendees to decide what activities they would like to try (e.g. boxing) and set out a timetable for the following weeks. In the 1st session all stakeholders including the attendees had a discussion and completed a Journal which explored what their barriers were to being active, and what they hope to get from the group. This made all stakeholders aware of each other and their needs. They were also encouraged to log their experiences of attending the group during the girl’s group. Sticker charts were discussed as an idea for motivation; however, the girls became motivated without this external factor.

This is different to what has been done before because it is user-centred and focuses on activities that the girls’ group would like to try, rather than being expert led. The girls attend their girls group at the venue and then walk to Barton AC to participate in the activities for one hour every Wednesday. They have also been on 2 trips, one to challenge4change – with a high ropes assault course, to build teamwork and the other trip to Chill Factory to celebrate International Women’s Day.

The reason Salford are focusing on this is to try and reduce social isolation and create a new 'norm' in that area as well as creating partnerships between different local community groups. The girls in the group can bring a friend or adult with them - the hope is that this will help with maintenance and sustainability.

The partnership has now grown in strength and trust and they are in the process of applying for funding to permanently hold and run the girls group at Barton AC; this will allow girls/ young women to access the gym/classes and the boys/ young men to access youth workers and councillors. They will also hold the LGBTQ+ night at Barton AC again offering much wider opportunities for the groups’ attendees. This means that there will be a session for 3 hours on a Wednesday with sport activities and alternative activities such, as counselling, art, drama. Making it ‘the norm’ to be in a gym setting and giving girls more opportunities to be active and learn a new skill.