Mentoring and coaching are complex interactions between people and any evaluation needs to, not only look at outcomes, but also the process that led to the outcomes. Enter realist evaluation!
Evaluation is an essential part of any mentoring or coaching scheme. Why? Because, how can you improve the scheme without feedback?
Often in organisations the so-called softer areas get downplayed or dismissed as anecdotal or subjective. But, mentoring and coaching are subjective and individualistic and pretending otherwise is to misunderstand them.
A key question, when evaluating mentoring or coaching is ‘what is the truth about the programme?’ There could of course be many answers to that question. Recently I watched that classic film ‘Twelve Angry Men’. It demonstrates that truth can be a subjective process in itself!
Realist evaluation of mentoring and coaching interactions
Realist evaluation sheds light on the full picture of change in a mentoring or coaching programme. It is a process that is about ‘what works’ and that has appeal for many managers.
Mentoring and coaching happen through personal and interpersonal engagement in developmental relationships and purposeful conversations. The outcomes, in the main, are individually defined and therefore have meaning to those individuals. Each relationship is unique but, at the same time there are aspects that link to other mentoring relationships across the scheme and to the rest of the organisation—mentoring and coaching create ripples! Realist evaluation looks at those ripples!
Realist evaluation is about theory and as Lewin (1951) points out, ‘there is nothing as practical as a good theory’. There is theory behind every practice in organisations.
Pawson and Tilley (1997) were the first to make the case for realist evaluation. Realist evaluation is about the ‘black box’ in mentoring and coaching. It looks at the theory behind the intervention and sees if it stands up or not! So, what are the steps?
Step 1: Establish the theory of the programme
What is it supposed to do? Who is it for? This may be some kind of ‘theory of change’ framework that attempts to link intent with the hoped-for outcomes. There may be ‘levels’ of theory, for example:
- The coach/mentor
- The mentee/coachee
- The organisation
- The educational input for mentors/coaches
- Customers and clients
Step 2: Consider the interaction of context, mechanism and outcome
Let’s consider the following three interacting variables:
- What is the context?
- What are the mechanisms
- What are the outcomes?
And create a logic chain which asks, What is it about X (context) that gives rise to a causal pathway (mechanism) which leads to Y (outcome)? Here is an example from the NHS:
Step 3: Gather the knowledge
Formulate appropriate questions to gather data to support the theoretical outcomes and your logic chain. Then create appropriate data collecting tools i.e. survey, interviews, satisfaction questionnaires, focus groups etc.
Step 4: Perform the Realist Evaluation
Relate the data gathered to the theoretical outcomes and your logic chain and write your report!
In this way, the realist evaluation process helps to identify the parts played in the scheme by the various participants and how these happen. Realist evaluation is about appreciating the patterns in the way the linked elements play out, through the contexts, mechanisms and outcomes. Inevitably this is a dynamic process because change is dynamic. Having a good theory of change within any scheme creates good practice in both mentoring and coaching.
So contact us to see how realist evaluation can help your organisation. For a fuller article see Coaching at Work Vol 14 Issue 6 pp 54-55.
References
Lewin, K. (1951) Field Theory in Social Science: Selected Theoretical Papers (ed. D. Cartwright). New York: Harper & Row.
Pawson, R., and Tilley, N. (1997). Realistic Evaluation. London: Sage
Enjoyed this article?
Subscribe to Newsletter…I want to know more!
Get in touch and one of the team will contact you to see how we can help.
