Integrative evaluation and participatory evaluation

by Osvaldo Néstor Feinstein

“Integrative Evaluation” (IE) is an approach that mitigates polarising discourse by integrating various seemingly contradictory perspectives and/or hypotheses. Participatory evaluation (PE) allows for the incorporation of the population’s perspectives on the processes and results of policies, programmes and/or projects, limiting or avoiding technocratic bias.

Evaluations are affected by perspective both in terms of the questions posed and the answers obtained. As populations are heterogeneous, it is very likely that they will contain a range of perspectives, which may also differ from those of experts. How should this type of situation be handled?

Continue reading

Applying inclusive rigour to participatory evaluation

by Marina Apgar

While celebrating a greater openness toward participatory evaluation (PE), many evaluators continue to adhere to traditional ways of understanding “rigour”. Within these frameworks, quality standards are based on the supposed existence of a methodological hierarchy, in which objective and quantitative methods are placed higher than others, considered to be less “rigorous”.

These traditional approaches to rigour are manifested in evaluation designs that select one central method —which may be quantitative or qualitative— and add other less important methods if required, creating a mixed methods approach. If we follow this approach, our role as evaluators is to faithfully and strictly apply a protocol based on the standards established by our central methodology. In this context, participatory methods are considered to be weak, lacking in rigour and prone to bias. The only way to overcome their perceived weakness is to add “objective” methods to increase the “rigour” of the participatory design and so minimise its bias.

Continue reading

A few thoughts on the similarities and differences between social research and participatory evaluation

by Luisa Graffigna

While I have always found Participatory Evaluation attractive, until recently I viewed it as irrelevant to my work. That changed when I listened to Marina Apgar’s presentation, Evaluación Participativa y rigor en el marco de una evaluación transformadora [Participatory Evaluation and rigour in the context of a transformative evaluation, only available in Spanish] at the EvalParticipativa international seminar in December 2021. Her talk helped me realise how the criteria of rigour in evaluation that she spoke of are echoed in a field with which I am more familiar: social research.

All of us carry with us our own set of lived experiences. In my case, my academic training as a sociologist and a period several decades ago of collective work, reflection and practice linked to processes of Popular Education -together with some of the people who coordinate EvalParticipativa today- contributed to my interest in, and understanding of, what Marina was saying. From this experience, the “participatory” part resonates with me, but my background positions me more firmly within the field of social research and it is from this position that I will share some thoughts about the ways these two processes, each with their own logic, converge and diverge.

Continue reading

The tools, on their own, do not make the difference

by Dagny Skarwan

The need to carry out an evaluation in order to discover what results have been achieved by an intervention is appreciated by organisations, projects and the teams responsible for implementing them. In turn, monitoring is often linked to accountability, generally through a weekly or annual report.

Moreover, in the field of NGOs, monitoring is usually understood as reporting activities, in other words, accounting for everything that has been done, within a set period, in relation to the operational plan.

Even when projects have a logical framework or results matrix, and even when they have developed a theory of change, it is not unusual for organisations and local teams to be surprised by the instruments they come across when they start getting involved in participatory monitoring of outcomes and impacts. In this type of monitoring, I usually help teams reflect on how outcomes are measured, how impacts can be recognised and measured, and -from there- recognise the different contributions of the project. Questions also arise concerning the purposes of monitoring that go beyond the need of a project coordinator to provide accountability, and include questions such as where to start when monitoring a project and how to know when it is the right moment to do so.

Continue reading

PARTICIPATORY EVALUATION: AN OPPORTUNITY TO ADVANCE IN HUMAN RIGHTS, INCLUSION AND EQUITY

by Julia Espinosa Fajardo

In participatory evaluation, people and their diverse needs are put at the centre of evaluation processes, and consequently, public policies and programmes. The active inclusion of the different voices throughout the whole evaluative exercise opens up a space to highlight the violation of rights, processes of social exclusion and the structural inequalities that exist in each context.

In this sense, it is an opportunity to make visible the different situations of discrimination and vulnerability, and move towards public actions that address these realities to a greater extent and have more transformative power. In this way, participation in evaluation is a key aspect in the process of deepening democracy and ensuring rights, while at the same time, leaving no one behind.

What does EvalParticipativa reveal to us about the Latin American experience in this regard? How can we promote evaluation practices that have a positive impact on rights, inclusion and equity? What challenges are posed in the region?

Continue reading

A Participatory Process in the New Methodology for Spanish Cooperation Country Partnership Framework Evaluation

The Country Partnership Framework, hereafter CPF, (MAP in Spanish) is the tool used by Spanish Cooperation, hereafter SC, for bilateral geographic strategic planning to ensure that SC actions contribute to sustainable development. Through the CPFs, dialogue is established between SC and the partner countries to benefit the development strategies and plans of these countries.

With this tool, the SC contributes to the implementation of the 2030 Agenda within the framework of the Sustainable Development Goals (SDGs); and in particular, SDG 17 Partnership for the Goals, promoting synergies and building partnerships at all levels, both with the partner country and among the SC actors. The CPF seeks to obtain a strategic, global and coherent vision of the Spanish Cooperation as a whole, and seeks to avoid merely compiling a list of interests held by the different actors. It is precisely the strategic approach that differentiates this tool and provides an interesting added value to CPF evaluation.

Continue reading

Humble Evaluations: the evaluator role and attitude in Participatory Evaluation

by Silva Ferretti

The stereotype evaluator is an expert. S/he can capture, scientifically, what doesn’t work in a program. S/he can provide wise recommendations to fix issues… and manager shall respond to them! It is a position of professional authority.

The whole evaluation system pushes evaluators and their commissioner to conform to this stereotype. It seems convenient that, at some point in time, the expert can come in, validate a program and provide the right recommendations and solutions to improve it.

Continue reading

Community Ownership in Evaluation. The Experience of Asia Pacific Evaluation Association (APEA)

by  Rituu B. Nanda & Randika de Mel

Let us join hands EvalParticipativa! Greetings from India!

Congratulations on the brilliant work you have been doing on promoting participation of communities in evaluation.

We are of the Asia Pacific Evaluation Association (APEA) action group on Community Ownership in Evaluation. We held an online Consultation in July 2021 in which 90 people participated from different parts of the world to create awareness of the importance of strengthening community ownership in evaluation and to develop an action plan for community ownership in evaluation in the Asia Pacific Region.

The highlight was participation of communities in the consultation. Two indigenous youth from India (supported by Faith Foundation) accepted the Evaluation torch. A young youth leader presented her experience in girl-led research from EMpower.

Continue reading

Evaluating 15 years of Experiencia Sur

by Belén Rodríguez Navas & Juan José Clavaín Nuño

Entreculturas is a Jesuit-sponsored international cooperation NGO that works to promote justice and social transformation. It defends education as a human right and upholds the right to a dignified life for migrants and refugees. It also seeks to construct committed global citizenship, gender equality and the reconciling of humans with nature. It seeks to contribute to the development of the most vulnerable communities, appealing to values such as solidarity and equal rights, and involving all types of stakeholders (citizens, companies, governments etc.) that share the responsibility of tackling these global challenges.

Continue reading

How to ensure that all voices are heard. Predefined criteria vs. stakeholder questions in evaluation

by Laura Porrini

For a while now, I have been pondering some key aspects that, in my experience, shape Monitoring, Evaluation and Learning (MEL) practice in the Global South. It is within this context that I have decided to set out some ideas that could be incorporated into practice criteria. One of the ideas that I have focused on is the existing tension between the increasingly felt need to ensure that all voices are heard in the evaluation process and predefined evaluation criteria, both in terms of their content and use.

Continue reading