Blog: Helping to strengthen evaluation capacity in the third sector: the perspective of an intermediary
Sara Redmond, Partnership and Practice Development Programme Manager, The ALLIANCE writes
I have worked for the Health and Social Care Alliance Scotland (the ALLIANCE) for three years and a central feature of my roles has been to support the third sector to feel better able to use their learning and demonstrate the impact of their work in order to generate investment and make improvements. I personally believe in the importance of evaluation as a process to help organisations learn from what they do and the difference it makes, as much as to communicate this difference to others.
Over the last two years I’ve been involved in some work with Evaluation Support Scotland to develop its approach to strengthen the evaluation capacity across the third sector. This approach was delivered as a mixture of training for trainer style sessions along with some action learning. Those of us who participated were supported to deliver four of Evaluation Support Scotland’s (ESS’s) core courses: What are my Outcomes; Collecting Information to Report on Outcomes; Using Creative Approaches to Evaluate my Project; and Telling My Story: Analysing and Reporting on Outcomes. A condition of participating in the programme was that we had the capacity and reach to offer the training to the third sector.
We recently held our final action learning session where we reflected on our experience of the approach and the ways we have begun to implement it through our various networks.
I have particularly enjoyed the opportunity to learn from my peers – exploring with others the different ways of talking about and guiding learning in evaluation. The approach Evaluation Support Scotland adopts is critical to their success – creating a learning environment which is designed to build upon the skills and experiences of the people being trained. We each described how we had applied the learning to our own organisations which I think highlights there will always be improvements we can make to our evaluation efforts, but to use the ESS mantra “good enough is good enough.”
It was interesting to hear that each of us had adapted the delivery method to suit the needs of our members and stakeholders. A significant issue we each identified is the capacity of groups to attend formal training. I would suggest most third sector organisations understand the importance of evaluation but can only commit to short sessions and therefore information about evaluation must be targeted to their development needs. In addition, most are receiving funding from several funders, each with different reporting timescales and commitments. Evaluating their project activity and impact is typically an add-on to someone’s job and therefore capacity and time are factors to take into account. Making evaluation relevant to the work organisations deliver is therefore essential.
Another issue which was raised was the question of ‘who’ does the evaluation. We found that there can often be disconnect between the staff who deliver the project activity and gather the data for the evaluation and the person who analyses the data and writes the report. Where this is the case the person involved in writing the report can find it a daunting task to identify the key learning to draw out from the data. In addition, without being involved in the whole evaluation process, operational staff may find they are missing the essential feedback loop to support them reflect upon what has been working and where changes need to be made. Who we target, therefore, to help build capacity is another key question.
Speaking from a health and social care perspective, with the changing landscape around commissioning of services there is an increasing demand for third sector organisations to take on a greater strategic role to add into this complex picture.
This is where I see the value of an informal network of evaluation support that can be identified as a local resource - so that organisations can have these initial conversations about evaluating their work and help signpost people on to the more in-depth support of Evaluation Support Scotland.
I would suggest that much of what is needed is a common way to describe the contribution of the third sector to health and wellbeing and of describing the uniqueness of how we do it. This is not always about evaluating more but about making the links between what we do and why we do it. I think to do this we need to challenge the notion that what we do is ‘soft’ and difficult to evidence. For me this is about ensuring we invest in the capacity of the third sector to evaluate what it does in order to learn what works, in what circumstances and why. We need to see funding to the third sector for what it is – an investment to realise social impact.