Indigenous evaluation and its challenges to the evaluation field: does participative evaluation has a word to say?

27 July, 2021Leave a comment

By Larry K. Bremner (*)

I would like to commend Evalparticipativa in establishing a community for professional dialogue and learning – well done! I would also like to thank you for asking me to contribute to this important dialogue.

I am Métis which is one of the three recognized Indigenous groups in Canada, the other two groups are First Nations and Inuit. As an evaluator I have had the privilege to work with dedicated people across Canada and internationally who are trying to improve and empower their communities. I believe evaluation is a tool for use in the empowerment of communities and peoples. For too long evaluation has been used as a tool to help maintain and justify a colonial and paternalistic approach to Indigenous communities. Transient evaluators, not familiar with the cultural, political, and social histories of the communities, have arrived in Indigenous communities to evaluate programs about which they are unfamiliar. Furthermore, they do not understand or follow community protocols. For too long evaluation has been an extractive process which has taken from the communities often giving nothing back in return. This has led me to advocate for Indigenous communities to take control of their evaluation agenda. I have outlined some things I have found that helps me when undertaking evaluation with Indigenous peoples and communities.

Continue reading →

WE ARE MEASURING EVALUATION CAPACITIES IN COUNTRIES IN THE REGION, WILL YOU HELP US?

14 July, 202114 July, 2021Leave a comment

Some of the EvalParticipativa member entities such as Techo, ReLAC, DEval or PETAS, have participated in a working group since 2018 to create an index to measure each country’s evaluation capacities.

This index, based on 76 indicators, has been named INCE and is highly useful for guiding how evaluation is developed at national level. It is also useful for academic production, identifying good practices, opening up opportunities for collaboration within and between nationals, etc.

The participation of Techo and other social organisations in the working group has enabled civil society’s perspective to be integrated into the definitions of the dimensions that make up the INCE. This collective, important and fundamental in how evaluation develops in reality, is also expected to participate in the periodic measurements that are taken in the region’s countries that request it.

Continue reading →

EVALPARTICIPATIVA’S ONGOING JOURNEY: PLANS, ACTION AREAS AND DREAMS

12 July, 202114 July, 2021Leave a comment

Since 2019, the EvalParticipativa initiative, a community of practice and learning for participatory evaluation in Latin America and the Caribbean, has been run by PETAS, the only academic centre in the region with a research and training programme specialising in collaborative and participatory evaluation approaches; together with the Focelac+ project, run by the German Institute for Development Evaluation (DEval); and the Costa Rican Ministry of Planning (Mideplan). After the first two years of the project came to a close, the UNSJ and Focelac+ renewed their agreement so they could run new activities in 2021-2022 with the aim of continuing in the same spirit and deepening the initiative’s scope.

In line with the 2030 Agenda and the Sustainable Development Goals, we centre our focus on strengthening the role of civil society in policy and programme evaluations that directly affect them. We are guided by our desire to increase the participation of various different social stakeholders and give them leading roles when evaluating initiatives that affect and involve them. This means we have to deepen knowledge around conditions and mechanisms that facilitate or complicate effective citizen participation in evaluation processes.

Over the next 18 months, we intend to continue learning from concrete experiences of participatory evaluation in the region, maximising their scope of influence and sharing methods and tool. This work stems from our conviction that these approaches:

- - strengthen participating organisations so that they have greater control over their own development;
  - improve their capacity to reflect, analyse and propose solutions,
  - benefit from new and different knowledge held by relevant stakeholders in order to create better policies, programmes and projects; and
  - contribute to building more inclusive and equal societies.

We are guided by our objective to maximise the inclusive involvement of civil society in evaluation processes by strengthening and consolidating EvalParticipativa as a community of practice and learning, facilitating the multiplication and institutionalisation of this evaluation approach and initiating training processes on the same topic.

The activities that we have planned can be grouped into the following categories:

1. MANAGING THE COMMUNITY OF PRACTICE AND LEARNING

Evalparticipativa’s online platform has been the hub and meeting place for this evaluation community. We will work on maintaining and keeping its different sections up to date, adding posts, handbooks and tools as well as identifying and documenting new experiences of high-quality Participatory Evaluation and meaningful lessons on the topic in the region.

The space for exchanging and sharing information will be hosted online through thematic forums as requested by members of the EvalParticipativa community. And we will extend the practice of sharing our web content through social networks and platforms: Instagram, Facebook, LinkedIn.

2. PROMOTING PE INSTITUTIONALISATION IN THE REGION

We intend to offer talks to disseminate information on the reality of participatory evaluation in conferences, postgraduate courses and events. We will recover some of the valuable articles and experiences documented on the EvalParticipativa platform in order to publish an online document or booklet.

We also want to set up the EvalParticipativa Prize for the best academic production regarding Participatory Evaluation, open to evaluations, research projects, essays and theses on participatory evaluation.

3. DISEMMINATING AND FACILITATING THE USE OF THE SOWING AND HARVESTING HANDBOOK

We will reinforce activities that present and disseminate the Sowing and Harvesting handbook. We will create audio-visual material that illustrates its content with real examples, both in terms of participatory evaluation cases and in experiences of meaningful lessons. In addition to the printed version of the handbook, we will publish the English version digitally to enable us to further dialogue with stakeholders from other regions.

4. COURSES TO FACILITATE PARTICIPATORY EVALUATIONS

We will develop participatory evaluation training sessions, both online and in-person. The latter will be delivered to agencies of our main funder, the German Cooperation Ministry, with an initial focus on Ecuador and Colombia that will then be expanded to the rest of Latin America and the Caribbean.

We will begin a process of identifying organisations interested in this in order to combine efforts and implement it. These may be non-governmental development organisations, academic institutions, civil organisations, etc. The course will enable us to present and implement the Teaching Guide with the participants in order to facilitate participatory evaluation courses.

As can be seen, the future is sure to be busy and exciting for those of us who are seeking to make evaluation experiences increasingly participatory. We trust that the members of our community of practice and learning, EvalParticipativa, will be increasingly active and that we will gain new members interested in the topic. We are very pleased with the support and attention received so far, and we would like to invite you to continue to contribute your experiences so we can mutually help each other advance. We will keep you informed of advances and news. See you soon.

How Are Participating Stakeholders Empowered in a Participatory Evaluation?

8 June, 2021Leave a comment

by Emma Rotondo (*)

In this contribution to EvalParticipativa I will describe the characteristics of the empowerment of multiple stakeholders during participatory evaluations. But first I should clarify what I understand by participatory evaluation (PE).

PE evaluation involves the participation of the parties interested in a programme, project or policy. On the basis of valuing local knowledge and wisdom, it seeks to generate learning about the changes in order to empower people and social groups to make decisions and to strengthen their capacity.

There are three basic elements common to all participatory evaluation processes: (i) a transformative approach, that is, a proposal to produce social and political change; (ii) a joint or associated work between different stakeholders; and (iii) social learning and use of the outcomes of an evaluation. In this context, what is understood by empowerment of stakeholders in a PE process is the broadening of both individual and collective capacity (or faculties) to act; the provision of tools for reflection, dialogue and co-building; and the generation of knowledge to learn socially, all of this with the intent to use the evaluation and transform their environment.

Continue reading →

AN EVALUATION WITHOUT EVALUATORS

7 May, 202126 May, 2021Leave a comment

We are beginning the second stage of EvalParticipativa with a new activity plan that we will share soon. In the meantime we invite you to listen to the testimony of Juan Carlos Sanz, who represents DEval in the Coordinating Team of this regional initiative.

In this video (English subtitles) Juan highlights the particular features of participatory evaluation, such as the main role that the stakeholders play in analysing and assessing an intervention in which they themselves have been involved. He also emphasises the importance of thoroughness in participatory evaluation as this will provide credibility and will help to adopt the changes and improvements to the evaluated project. Lastly, he extends an invitation to join the Community for Practice and Learning EvalParticipativa so as to exchange lessons learnt and improve abilities in this evaluation approach.

Continue reading →

SIMILARITIES BETWEEN SYSTEMATISATION OF EXPERIENCES AND EVALUATION

19 April, 202119 April, 2021Leave a comment

by Olga Nirenberg (*)

Introduction

As I mentioned in my brief comment on the article published the 22nd of March on EvalParticipativa, “Systematisation of Experiences and Evaluation: Similarities and Differences“, by Oscar Jara Holliday, my first reaction upon reading it was of surprise. I felt it was paradoxical that certain arguments regarding evaluation would appear on the blog on Participatory Evaluation, it seemed like an oxymoron, a contradiction in itself.

I am familiar with the works published by the author of the article, OJH, which are widely known and have been of great influence on those of us who work in evaluation in Hispanic American countries. I have even had the opportunity to debate directly with him on the resemblance between systematisation and evaluation. Our conversation was very important to me, so much so that in the book I published shortly after (Nirenberg, 2013) I included a chapter (chapter 8) dedicated almost entirely to systematisation of experiences in which I highlighted its points of contact with non-traditional evaluation approaches.

Although I have acquainted myself with the approach and methodologies of systematisation of experiences and have even put them into practice in many occassions, here I will discuss mainly the references to evaluation present in the cited article, since that has been, for more years than I would like to admit, the central focus of my professional career.

I am very grateful to my colleagues and coordinators of EvalParticipativa, Pablo Rodriguez-Bilella and Esteban Tapella (PETAS/National University of San Juan, Argentina) and Carlos Sanz (DEval), for giving me the opportunity to expand and enrich this debate.

Continue reading →

Making Every Voice a Protagonist

7 April, 202119 April, 2021Leave a comment

Conventional evaluations have got us used to the users or beneficiaries of the programs and public policies becoming involved only as key informants. In a participatory evaluation the parties involved are the protagonists of the whole process. They define who will participate, what will be evaluated and when, what data collection and analysis methods will be used and how the results will be communicated.

Karla Zalazar, who has vast experience in facilitating participatory processes, points out that it is essential to foreground multiple stakeholders throughout the process. “If we are going to talk about participatory evaluation, we have to recognise the necessity of creating spaces for true participation, where every voice and perspective is taken into account. This implies being in close contact with the communities, with the different stakeholders and their views, and not just asking questions”, says the Costa Rican social facilitator.

Karla Salazar is a psychologist with a Master’s in Criminology and a Master’s in Political Science. She currently works as an academic coordinator and researcher at the Latin American Faculty of Social Sciences (Costa Rica Headquarters). She is a lecturer and an independent consultant in the fields of research and evaluation. Her career has been characterised by the direct contact with communities, social organizations and people living with multiple vulnerabilities based on gender, violence and social exclusion, and therein lies her passion for the active participation of people in building knowledge and evidence.

SYSTEMATISATION OF EXPERIENCES AND EVALUATION: SIMILARITIES AND DIFFERENCES

22 March, 202119 April, 20214 Comments

In Latin America and the Caribbean, the systematisation of experiences approach is widely known[1]. In some fields, such as popular education, it is even better known than evaluation. We ask ourselves, what differences and similarities are there between evaluation and systematisation of experiences? Is it possible to identify common aspects between systematisation and some types of evaluation?

At EvalParticipativa we wish to open the debate and for that we have invited Oscar Jara Holliday, one of the most renowned figures in the field, to kick it off.

The Context

According to specialised literature, systematisation of experiences aims at establishing learning as an essential element of any intervention policy. And he does so by rising to the challenge of promoting, designing and conducting learning processes in experiences that were probably not conceived with that purpose in mind. But is the search for learning exclusive to systematisation? Are there similar purposes in the field of evaluation?

Continue reading →

A Bridge Between Community and Decision Making

10 February, 202119 April, 2021Leave a comment

In this video (with English subtitles), Matthias Casasco from TECHO Chile highlights the importance of participatory evaluation as a tool with the potential to provide a bridge between the locals’ voices and decision making.

He also points out how important it is to successfully generate spaces for reflection in the communities to discuss their problems and projects as well as community organising. The main goal of the participatory evaluation process, Matthias claims, is that local stakeholders take ownership of the process.

Matthias Casasco has a Master’s degree in Political Science from Sciences Po Rennes (France). He has specialised in housing and urban development policies. Matthias has been living in Santiago de Chile for nine years and is now in charge of the program for Housing Solutions at the TECHO-Chile foundation. In this capacity, he joins the communities of popular settlements on their journey to their right to adequate housing and connects them with the housing programs of the Chilean state. As a member of the EvalParticipativa community, he has worked on the design and implementation of a participatory evaluation pilot program in the Santa Teresa camp, on the outskirts of Santiago de Chile.

SOWING AND HARVEST. A Handbook for Participatory Evaluation

1 February, 202119 April, 2021Leave a comment

2020 will remain engraved on our memories as the year when the COVID-19 pandemic irrupted into our lives. We experienced on a global scale the depth of our connection and interdependency as well as how closely intertwined our realities are.

In such a context, we are very happy to present this handbook for participatory evaluation which was put together as our response from the field of evaluation. In the face of fragility and the limits of self-sufficiency, we intend to foreground the multitude of voices and experiences of the people involved in development processes.

This book is the result of joint work. It is at the same time sowing and harvest of multiple experiences and knowledge. Its pages mirror the collective thinking, feeling and learning of a great many colleagues who have been working on the subject in Latin America.

Continue reading →