by Esteban Tapella
Dear colleagues and friends of the EvalParticipativa community, in this article we will present two tools to facilitate the construction of a shared conceptual framework around Participatory Evaluation.
These tools were designed within the framework of our community’s capacity building strategy, and are aimed at project leaders, socio-cultural animators and professionals working in the field of programme and project management and evaluation.
EVALUATION AND SOCIAL PARTICIPATION
The relationship between evaluation and participation started to take shape in the 1960s as a response to criticism of classical evaluation models. These models were characterised by a rational, technical and positivist approach that focused on quantitative evaluation and a clearly top-down perspective. Up until this point, evaluation had been viewed as a purely technical exercise, reliant on measurement tools and the external perspective of an expert evaluation team capable of drawing objective conclusions about the functioning and outcomes of programmes and projects.
Not until the mid-1990s, within an international context that promoted ‘human development’ were participation and evaluation considered together for the first time, sparking a theoretical and methodological debate. Since the middle of this decade, several articles, books and manuals have explored evaluation practices under different names. In these different interpretations, social actors are connected to the intervention in dynamic and consistent ways.
Different evaluation approaches have recognised the importance of involving a wide variety of stakeholders in the evaluation process, as mentioned in the Sowing & Harvesting handbook (2022). The inclusion of civil society voices is currently considered to be a central element not only to respond to evaluation needs but also to obtain high-quality, credible, feasible and evidence-based evaluations.
When we decide to embark on an evaluation that integrates the participation of social actors, the first question we have to ask is how to define participatory evaluation. There are a wide range of definitions and approaches that combine social participation and evaluation. This variety is evidenced in the different handbooks and guidelines that we have collected in our EvalParticipativa repository. These varied perspectives comment on the ‘what’ and ‘how’ of an evaluation that involves numerous social actors.
In this context, EvalParticipativa decided to establish a context-specific conceptualisation. In other words, we decided to work with participating stakeholders, namely the evaluation team, to define what we understand participatory evaluation to be and how it should be implemented. Thus, we worked with attendees of the First Gathering of Participatory Evaluation Experiences in Latin America and the Caribbean (2019) to draw up a definition of participatory evaluation alongside seven key governing principles. In this way, the conceptual reference framework presented in Sowing & Harvesting was constructed on the basis of 12 representative evaluations carried out across the region.
The tools can be used together (first one, and then the other) or separately (either one or the other) depending on the level of prior knowledge and experience that the team members have of this kind of evaluation.
THE TOOLS
Tool 1. Defining Participatory Evaluation
The tool is designed to be used with a group of between five and ten people. The participants receive a set of 106 words written on small cards or slips of paper, some of which are related to the concept of participatory evaluation and others are not. The cards are placed on the table face up so that everyone can see them. The facilitator asks the participants to identify and select four words that they think are relevant to the concept of participatory evaluation.
Once everyone has chosen four words, they take it in turns to explain their choices. The participants take it in turns to show their card and read the words they have chosen to the others. They say why they have chosen them and what they believe they mean. Through dialogue, debate and discussion, the group discards those words that are least representative, leaving those that they consider to be most important. The group should reach a point where only seven cards are left, having first eliminated repeated or similar ideas, followed by those that are less relevant or for which there is no consensus in the group. Finally, the group agrees on how to order the seven words to form a definition of participatory evaluation. This definition will reflect the feeling of the group or the evaluation team.
Tool 2. The Participatory Evaluation Playing Card
The game uses a pack of 40 playing cards which contain phrases related to the profile of an evaluator in a participatory evaluation and the principal characteristics of this kind of evaluation. There are also blank cards that act as wildcards on which participants can write their own phrases during the game or which can be used to recover a card that has already been discarded.
Once all the cards have been dealt out equally (each player should receive between four and seven cards depending on the group size) each participant selects a card that they think best reflects their ideas on the topic. Each participant takes turns to show a card that they have selected and explain what it means and why they think it is an important aspect of this kind of evaluation. The group should discuss this freely and choose one card that everyone agrees on from the preselected cards. In the second round, the group repeats the procedure, selecting another card that is also representative of this kind of evaluation. Round by round (between four and seven) the group will select the cards that best characterise this kind of evaluation. The cards that are not selected in each round can no longer be used unless a participant uses a wildcard to recover one. At the end of the game, the group will have a list of the main characteristics of a participatory evaluation and these will form the reference framework for this team or working group.
MATERIALS NEEDED TO PLAY
To use these tools, you have to download the materials by clicking on each element of the game (there are 5 PDF files). Next to that, we suggest you how to print each element.
In this link, you can download the folder containing both tools. You have to print it in A3 format, 250 grams paper, preferably in colour.
You need to download the instructions for tool 1 and the instructions for tool 2. These instructions can be printed in A4 or letter format, in black and white or colour.
For tool 1, you need to download and print the key word cards. For technique 2, you need to download and print the content and wild cards. These need to be printed on A3 format, 250 gram paper, preferably in colour.
RECOMMENDATIONS
It is important to review the contents of the cards and adapt them to the context in which they will be used. In the following video, I present the background to these games, describing how they were developed and the different ways they can be used. I also provide some basic recommendations on how to adapt them to different groups and contexts.
