Integrative evaluation and participatory evaluation

by Osvaldo Néstor Feinstein

“Integrative Evaluation” (IE) is an approach that mitigates polarising discourse by integrating various seemingly contradictory perspectives and/or hypotheses. Participatory evaluation (PE) allows for the incorporation of the population’s perspectives on the processes and results of policies, programmes and/or projects, limiting or avoiding technocratic bias.

Evaluations are affected by perspective both in terms of the questions posed and the answers obtained. As populations are heterogeneous, it is very likely that they will contain a range of perspectives, which may also differ from those of experts. How should this type of situation be handled?

Continue reading

Applying inclusive rigour to participatory evaluation

by Marina Apgar

While celebrating a greater openness toward participatory evaluation (PE), many evaluators continue to adhere to traditional ways of understanding “rigour”. Within these frameworks, quality standards are based on the supposed existence of a methodological hierarchy, in which objective and quantitative methods are placed higher than others, considered to be less “rigorous”.

These traditional approaches to rigour are manifested in evaluation designs that select one central method —which may be quantitative or qualitative— and add other less important methods if required, creating a mixed methods approach. If we follow this approach, our role as evaluators is to faithfully and strictly apply a protocol based on the standards established by our central methodology. In this context, participatory methods are considered to be weak, lacking in rigour and prone to bias. The only way to overcome their perceived weakness is to add “objective” methods to increase the “rigour” of the participatory design and so minimise its bias.

Continue reading

SOWING & HARVESTING, THE DOCUMENTARY. PARTICIPATORY EVALUATION WITH A HUMAN FACE

Dear colleagues and friends of EvalParticipativa: after a little over a year of work, we are delighted to officially launch and share with you the documentary series, SOWING &  HARVESTING, an accompaniment to the participatory evaluation handbook of the same name.

Its five episodes, based on experiences from Mexico, Guatemala, Costa Rica, Colombia and Chile, aim to illustrate the lessons learned from participatory monitoring and evaluation practices developed by different organisations in Latin America. To do this, we have selected emblematic and noteworthy cases from the region to illustrate their different nuances, levels of participation, processes and the participatory tools they use.

Continue reading

Evaluating with rights-holders. Lessons from the Updated Human Rights Appraisal in Mexico City

by Marcia Itzel Checa

It is increasingly common to hear of participatory evaluations that give leading roles to a large range of actors affected by a particular intervention. This allows evaluations to be carried out using a more comprehensive vision, one that recovers the different perspectives involved.

Likewise, Mexico City’s Human Rights Appraisal and the Human Rights Programme associated with it are one of a kind, for the following reasons: the broad participation of multiple political and social actors in the elaboration, execution, monitoring and evaluation stages; its institutional design, which has matured over time and has, indeed, been incorporated into the city’s new constitution; and the fact that, despite its ups and downs, it has survived three periods of municipal government.

Continue reading

“TELL ME AND I FORGET, TEACH ME AND I MAY REMEMBER, INVOLVE ME AND I LEARN”

EVALPARTICIPATIVA’S CAPACITY DEVELOPMENT STRATEGY

The Benjamin Franklin quote that we have used as the title is perhaps the best way to introduce this post, in which we wish to share some of EvalParticipativa’s recent experiences as part of its capacity development strategy in the region. Our contribution to capacity strengthening in Participatory Evaluation (PE) has made use of both online and face-to-face formats, and has focused on specific personal and organisational contexts.

The aim of these capacity development sessions was to ensure that participants were familiar with conceptual and methodological features of PE, based on participants’ own experiences and the contents of the “Sowing & Harvesting” handbook. Didactic tools and documentary videos prepared specifically for each context were combined so that participants could:

      • develop their knowledge, skills and capacities concerning the aims, steps and critical moments involved in this kind of evaluation;
      • acquire an initial understanding of how to implement a participatory approach and facilitate inclusive processes; and
      • gain a basic understanding of how to use the methodology, both to improve their own evaluation practice and to contribute to development processes across the region.

In this post, we share an account of three capacity development workshops, held in late July and early August in Ecuador, Colombia and Costa Rica. These training experiences were organised and facilitated jointly with local teams drawn from different academic and social organisations. Around sixty people were trained in three countries. This account speaks of the joint efforts and lessons learnt, of networking and synergy, of real-life challenges that were overcome, and of hopes about the possibility of constructing a Latin American society that is more just and inclusive and where, one day, nobody will be left behind.

Continue reading

A few thoughts on the similarities and differences between social research and participatory evaluation

by Luisa Graffigna

While I have always found Participatory Evaluation attractive, until recently I viewed it as irrelevant to my work. That changed when I listened to Marina Apgar’s presentation, Evaluación Participativa y rigor en el marco de una evaluación transformadora [Participatory Evaluation and rigour in the context of a transformative evaluation, only available in Spanish] at the EvalParticipativa international seminar in December 2021. Her talk helped me realise how the criteria of rigour in evaluation that she spoke of are echoed in a field with which I am more familiar: social research.

All of us carry with us our own set of lived experiences. In my case, my academic training as a sociologist and a period several decades ago of collective work, reflection and practice linked to processes of Popular Education -together with some of the people who coordinate EvalParticipativa today- contributed to my interest in, and understanding of, what Marina was saying. From this experience, the “participatory” part resonates with me, but my background positions me more firmly within the field of social research and it is from this position that I will share some thoughts about the ways these two processes, each with their own logic, converge and diverge.

Continue reading

PARTICIPATION, A KEY FOCUS IN OUTCOME HARVESTING: LESSONS FROM CHILE

by Andrea Peroni Fiscarelli

Homogenous evaluations vs differentiated evaluations

In order to build a solid framework and evidence, state-led evaluations have become more standardised over time, and are thus increasingly gaining in credibility. The problem that remains is the need to recognise that not all public programmes share the same characteristics, and therefore differentiated types of evaluations should be considered.

This is evident in programmes that centre around the delivery of goods with no interaction with the beneficiaries (such as in the case of subsidies, plans and vouchers), and others that seek to provide tools and/or develop competencies and skills aimed at increasing the social inclusion of individuals, especially from vulnerable sectors.

For these types of programmes, traditional evaluation methodologies have proved to be insufficient as they only seek to assess the level of effectiveness or efficiency and do not manage to capture the complex and diverse reality, practice and results that are present. Furthermore, as the programmes deal with human and social behaviours, the complexity of their contexts should also be recognised.

Continue reading

The tools, on their own, do not make the difference

by Dagny Skarwan

The need to carry out an evaluation in order to discover what results have been achieved by an intervention is appreciated by organisations, projects and the teams responsible for implementing them. In turn, monitoring is often linked to accountability, generally through a weekly or annual report.

Moreover, in the field of NGOs, monitoring is usually understood as reporting activities, in other words, accounting for everything that has been done, within a set period, in relation to the operational plan.

Even when projects have a logical framework or results matrix, and even when they have developed a theory of change, it is not unusual for organisations and local teams to be surprised by the instruments they come across when they start getting involved in participatory monitoring of outcomes and impacts. In this type of monitoring, I usually help teams reflect on how outcomes are measured, how impacts can be recognised and measured, and -from there- recognise the different contributions of the project. Questions also arise concerning the purposes of monitoring that go beyond the need of a project coordinator to provide accountability, and include questions such as where to start when monitoring a project and how to know when it is the right moment to do so.

Continue reading

PARTICIPATORY EVALUATION: AN OPPORTUNITY TO ADVANCE IN HUMAN RIGHTS, INCLUSION AND EQUITY

by Julia Espinosa Fajardo

In participatory evaluation, people and their diverse needs are put at the centre of evaluation processes, and consequently, public policies and programmes. The active inclusion of the different voices throughout the whole evaluative exercise opens up a space to highlight the violation of rights, processes of social exclusion and the structural inequalities that exist in each context.

In this sense, it is an opportunity to make visible the different situations of discrimination and vulnerability, and move towards public actions that address these realities to a greater extent and have more transformative power. In this way, participation in evaluation is a key aspect in the process of deepening democracy and ensuring rights, while at the same time, leaving no one behind.

What does EvalParticipativa reveal to us about the Latin American experience in this regard? How can we promote evaluation practices that have a positive impact on rights, inclusion and equity? What challenges are posed in the region?

Continue reading

A Participatory Process in the New Methodology for Spanish Cooperation Country Partnership Framework Evaluation

The Country Partnership Framework, hereafter CPF, (MAP in Spanish) is the tool used by Spanish Cooperation, hereafter SC, for bilateral geographic strategic planning to ensure that SC actions contribute to sustainable development. Through the CPFs, dialogue is established between SC and the partner countries to benefit the development strategies and plans of these countries.

With this tool, the SC contributes to the implementation of the 2030 Agenda within the framework of the Sustainable Development Goals (SDGs); and in particular, SDG 17 Partnership for the Goals, promoting synergies and building partnerships at all levels, both with the partner country and among the SC actors. The CPF seeks to obtain a strategic, global and coherent vision of the Spanish Cooperation as a whole, and seeks to avoid merely compiling a list of interests held by the different actors. It is precisely the strategic approach that differentiates this tool and provides an interesting added value to CPF evaluation.

Continue reading