Making Every Voice a Protagonist

Conventional evaluations have got us used to the users or beneficiaries of the programs and public policies becoming involved only as key informants. In a participatory evaluation the parties involved are the protagonists of the whole process. They define who will participate, what will be evaluated and when, what data collection and analysis methods will be used and how the results will be communicated.

Karla Zalazar, who has vast experience in facilitating participatory processes, points out that it is essential to foreground multiple stakeholders throughout the process. “If we are going to talk about participatory evaluation, we have to recognise the necessity of creating spaces for true participation, where every voice and perspective is taken into account. This implies being in close contact with the communities, with the different stakeholders and their views, and not just asking questions”, says the Costa Rican social facilitator.

Karla Salazar is a psychologist with a Master’s in Criminology and a Master’s in Political Science. She currently works as an academic coordinator and researcher at the Latin American Faculty of Social Sciences (Costa Rica Headquarters). She is a lecturer and an independent consultant in the fields of research and evaluation. Her career has been characterised by the direct contact with communities, social organizations and people living with multiple vulnerabilities based on gender, violence and social exclusion, and therein lies her passion for the active participation of people in building knowledge and evidence.

The social collective as a unit of analysis

Participatory evaluation is intrinsically collective and qualitative, that is its essence. In this short testimony, Carmen Luz Sánchez (Calu) emphasises that the key to participatory evaluation is to train the different stakeholders to ensure their assimilation of the tools needed to carry out the entire process.

Calu has over four years’ experience in participatory evaluation with the Servicio País program, which was implemented in Chile by the Foundation for Overcoming Poverty (Fundación para la Superación de la Pobreza). In her testimony (with English subtitles), she claims that this approach to evaluation must go hand in hand with an intervention strategy that allows the users of the program to take centre stage.

Carmen Luz Sánchez is from Santiago, Chile. A sociologist from the Pontificia Universidad Católica de Chile and Master of Arts in Sociology from the University of Sydney, she has specialised in quantitative and qualitative methods in social research and program evaluation. She has worked as a lecturer and researcher in both fields. Her main interests are poverty, urban sociology and participatory tools. She is currently the Evaluation and Program Management Coordinator of the Servicio País program for the Foundation for Overcoming Poverty (Chile), a civil society organisation in partnership with the EvalParticipativa virtual community. Over the past five years she has worked in the design, development and implementation of participatory evaluation in social interventions.

 

WE NEVER STOP LEARNING! PRINCIPLES FROM FEMINIST PARTICIPATORY EVALUATION IN COLOMBIA

Dear friends and colleagues, we are still receiving responses to our open call for meaningful lessons in participatory evaluation. This time from Alexandra Santillana (senior evaluator at Global Affairs Canada) together with Fabiola Amariles and Ana Isabel Arenas (consultants at Learning for Impact). Below they share with us a rich reflection on feminist and participatory evaluation taken from their experience working with rural community development projects in Colombia. Many thanks for your contributions and let’s keep learning!

THE EXPERIENCE

This experience, which took place between May and November 2018, was not strictly speaking a participatory evaluation, but provides reflection on some lessons learned from applying principles from feminist evaluation and participatory methods in rural development community projects. This pilot was run within the framework of a conventional evaluation led by the Canadian Ministry of Global Affairs (GAC).

Inspired by Canada’s Feminist International Assistance Policy (FIAP)¹ and taking advantage of an evaluation already underway as part of the Canadian cooperation program in Colombia, a mixed team of evaluators was formed including Canadian and Colombian specialists to design and implement the pilot.

Continue reading

PARTICIPATORY EVALUATION AND RIGOUR

By Osvaldo Néstor Feinstein

Participatory Evaluation (PE) gives voice to stakeholder perspectives on policy, programme or project processes and results in order to limit or avoid technocratic bias. Furthermore, it promotes ownership of the evaluative process and results which makes the evaluation more widely accepted. These are two of the arguments in support of PE.

On the other hand, PE has been criticised by the argument that it is not a rigorous approach due to its qualitative methods which capture “impressions” and anecdotes but do not provide rigorous quantitative procedures. Sometimes random control trials (RCTs) are used as an example of rigour in evaluation.

Continue reading

More significant learning from Participatory Evaluation. This time from the Costa Rican Caribbean.

Many valuable contributions have been made in the ongoing debate around Participatory Evaluation in times of COVID 19, and today we would like to share another contribution on meaningful lessons ‘in’ and ‘from’ Participatory Evaluation.

This time, we would like to take an example from Costa Rica and listen to Karla Salazar Sánchez, a social researcher and participatory evaluation facilitator. Remember, the invitation to participate remains open and we are interested in hearing from voices on the ground; those who have practice-based knowledge. All contributions are welcome. The many journeys made and times of stumbling and of getting back up; the great discoveries and joys can be made accessible to everyone if you share them with a brief post in this community.

Continue reading

SOCIO-EDUCATIONAL FUNCTIONS AND STRATEGIES THAT COMMUNITY SPECIALISTS CAN USE IN PARTICIPATORY EVALUATION PROCESSES FOR COMMUNITY INITIATIVES

Dear EvalParticipativa friends and colleagues, we hope this post finds you all well!

A few days ago, we symbolically crossed over the big pond to meet Héctor Núñez who lives and works in Barcelona. Héctor has worked exclusively on themes related to Participatory Evaluation (EP) for a long time. In this post, we would like to share his professional profile with you as well as some of his academic work. We are convinced it will enrich the training and reflection that we have been doing around these topics in our community of practice and learning.

Héctor Núñez qualified as a social educator at the University of Santiago de Compostela. He is a pedagogue and holds a PhD in Education from the Autonomous University of Barcelona. His professional experience mainly centres on social pedagogy, non-government organisations and the public sector. Héctor has participated in different research and consultancy projects in Spain and beyond. He is currently professor in the Serra Húnter programme in the Department of the Theory and History of Education at the University of Barcelona.

The main reason we contacted Héctor was to learn about participatory evaluation processes in community initiatives. As we talked, Héctor told us that the working perspective is wider with regard to evaluation. It views the role of Participatory Evaluation as part of the social pedagogy discipline and as a professional strategy within social education. We realised that there were large crossovers between what he told us and the issues that arose in our debates on the ‘whats’ and ‘whys’ of PE in our Gathering in Ecuador (2019).

 

Continue reading

LESSONS TAKEN FROM THE WORKING GROUP EXPERIENCE IN THE SANTA TERESA INFORMAL SETTLEMENT (TECHO-CHILE)

Dear friends, we are continuing to learn from each other and this time we will hear some thoughts from Fernanda Arriaza, TECHO member. This organisation works in 19 Latin American countries and seeks to overcome the poverty experienced by millions of people living in informal settlements through initiatives which unite the efforts of settlement dwellers and young volunteers.  In this post, you can read about the lessons they learnt in an experience that was recently documented as part of an initiative in and with the Working Group.

We would like to remind you that our invitation to share your meaningful lessons remains open and we would love to receive and share your them. Greetings!

Continue reading

WHY IS PARTICIPATION IMPORTANT IN EVALUATION? LESSONS FROM AN EXPERIENCE IN MEXICO

A couple of months ago, we got in touch with Giovanna Montagner, follower of our community of practice, and invited her to share some of her thoughts on evaluation and participation.

She opted to share on the following initiative and writes together with the evaluation team. It is an adaptation of Outcome Harvesting, developed over two years and carried out in close collaboration with the programme’s staff. The evaluation process included telephone interviews to all the small agricultural producer leaders who were participants in the initiative, visits to a “sample” of the groups on the ground where interviews, observation and group activities were held. In addition, they carried out semi-structured interviews with the other programme participants (partner companies and NGOs). This all meant that they could seek to identify results from the perspective of the participants which was an innovative approach for the programme and which completed a pre-existing quantitative evaluation. Here are the lessons learnt from the experience.

Continue reading

HAVE YOU ASKED YOURSELF WHAT YOUR ROLE IS AS FACILITATOR IN CHANGE PROCESSES THAT ARE DRIVEN BY OUR INITIATIVES?

Here we share a new experience in response to the invitation to share meaningful lessons from Participatory Evaluation. This time the reflection focuses on emotive lessons in participatory monitoring by Dagny Skarwan and her work at ONG – IDEAs. Here, she tells us in first person about her valuable lesson.

….

For around three years, I was involved in coordinating the project ONG-IDEAs with German and Latin American colleagues in Central America as well as NGOs in Colombia and Mexico. It was an experience that made a lasting impression on me.

For me, learning how to guide NGOs and facilitator groups in Participatory Evaluation processes and advising them on how to use tools to create a culture of evaluation in their target groups has been a great learning experience. It has also involved growth that has been charged with intense emotions and memories.

Going beyond the formal monitoring proposals (such as creating transparency and accountability), the participatory monitoring focus that we have developed at ONG – IDEAs includes our proposal to generate processes that empower and provide moments of learning for all those involved, focusing on the perspectives, self evaluation and commitment of target groups.

ONG-IDEAs’ approach focuses on several tools that are able to generate information and enable analysis. They work when those who facilitate work together with those who are motivated and available to change their outlook, behaviour, attitudes and practices.It is important to clearly express what I want to change, what my objectives are, why I participate in the group or present an idea for a project.

Continue reading

AND WE CARRY ON LEARNING… THIS TIME VISITING A PARTICIPATORY EVALUATION OF HEALTH SERVICES PROVIDED BY A LATIN AMERICAN NGO (ARGENTINA)

We are still receiving reflections and lessons from the invitation we launched a few days ago. This time, Olga Nirenberg, a colleague EvalParticipativa member through CEADEL, shares some of her thoughts with us on a participatory evaluation experience evaluating the health services of a Latin American NGO.

Remember that everyone can participate with their thoughts and lessons!

Prologue

Here I’m going to share an experience that at the time I found negative but which, looking back now, I think was a good lesson for me, the team and especially the women who participated in this “evaluation saga”.

I think that evaluation often clearly portrays differences in stakeholder perspectives and the importance of having their interests and influences mapped out. Furthermore, all stakeholders should always be fully involved. In other words, there should be a multi-stakeholder focus with clear decisions around which voices will be given priority.

The recommendation that arises from the experience that will be shared below touches on the ethical issues involved in participatory evaluation. This means that those of us who are evaluation professionals and who know about methodological issues, have the moral obligation to amplify the quietest voices and those that are not always heard, especially when it is claimed that these same people are the “recipients” of the initiative in question.

Continue reading