THE ‘CHIVA’, A USEFUL PARTICIPATORY TOOL FOR EVALUATION PROCESSES: LESSONS FROM COLOMBIA

by Carmen Lucía Jaramillo

Creating spaces where -regardless of their levels of education- people feel they can truly participate in planning and evaluation processes, in an informed and active way, has been a principal methodological concern during my work with communities, particularly in rural areas.

Beyond discourses on empowerment, horizontal relationships and recognition of the value of the knowledge and experience of local actors, it is always challenging to combine the demands of methodological rigour (structures, formats and technical language) and the need for fluent communication with protagonists in the transformation of the challenging realities they face in their territories. Generally speaking, structural socio-economic problems and indifference on the part of those in power are standard features in such environments. That is why it is always a challenge to “[…] create a space for debate, that is, a truly respectful space. Not the simple tolerance derived from indifference and scepticism, but a positive appreciation of differences” (Zuleta, 1985).

For this reason, in this continual pursuit of in-depth analysis and debate based on the use of simple language, I often opt for methods rooted in analogies that are familiar to the contexts and daily lives of the people with whom I carry out participatory planning or evaluation. One of the analogies that I have been able to adapt to multiple situations is that of a journey in a chiva, a form of transport used in rural Colombia to carry both passengers and goods. The image of the chiva is also very useful, because each one is a unique representation of what its owners want to say about their region. This is why they are covered in colourful images as a hallmark of pride and identity.

Continue reading

Knowledge and social representations in Participatory Evaluation

by Sergio Martinic

Participatory Evaluation experiences value and validate the interpretations of participants concerning the projects, policies and interventions that are being analysed (Fetterman, 2005). These experiences create space for participants to share their knowledge and particular ways of looking at the real-life situations and problems that the interventions under evaluation are seeking to address (Gil & Heras, 2010; Menéndez, Torralbo & Luque, 2021; Paño, Rébola, & Suarez, 2019).

This constitutes one of the principal contributions of this evaluative approach: knowing and understanding the actions from the point of view of the other allows for a deeper and more realistic examination of the results and impacts of the programme or action under evaluation.

The integration of participant perspectives brings with it significant conceptual and methodological challenges. This article argues that the opinions of participants should be analysed on the understanding that they are part of broader social representations of the culture to which the subjects belong. From a methodological point of view, qualitative data and discourse analysis strategies are good tools for analysing interpretations and knowledge expressed during a participatory process. Continue reading

Integrative evaluation and participatory evaluation

by Osvaldo Néstor Feinstein

“Integrative Evaluation” (IE) is an approach that mitigates polarising discourse by integrating various seemingly contradictory perspectives and/or hypotheses. Participatory evaluation (PE) allows for the incorporation of the population’s perspectives on the processes and results of policies, programmes and/or projects, limiting or avoiding technocratic bias.

Evaluations are affected by perspective both in terms of the questions posed and the answers obtained. As populations are heterogeneous, it is very likely that they will contain a range of perspectives, which may also differ from those of experts. How should this type of situation be handled?

Continue reading

Applying inclusive rigour to participatory evaluation

by Marina Apgar

While celebrating a greater openness toward participatory evaluation (PE), many evaluators continue to adhere to traditional ways of understanding “rigour”. Within these frameworks, quality standards are based on the supposed existence of a methodological hierarchy, in which objective and quantitative methods are placed higher than others, considered to be less “rigorous”.

These traditional approaches to rigour are manifested in evaluation designs that select one central method —which may be quantitative or qualitative— and add other less important methods if required, creating a mixed methods approach. If we follow this approach, our role as evaluators is to faithfully and strictly apply a protocol based on the standards established by our central methodology. In this context, participatory methods are considered to be weak, lacking in rigour and prone to bias. The only way to overcome their perceived weakness is to add “objective” methods to increase the “rigour” of the participatory design and so minimise its bias.

Continue reading

SOWING & HARVESTING, THE DOCUMENTARY. PARTICIPATORY EVALUATION WITH A HUMAN FACE

Dear colleagues and friends of EvalParticipativa: after a little over a year of work, we are delighted to officially launch and share with you the documentary series, SOWING &  HARVESTING, an accompaniment to the participatory evaluation handbook of the same name.

Its five episodes, based on experiences from Mexico, Guatemala, Costa Rica, Colombia and Chile, aim to illustrate the lessons learned from participatory monitoring and evaluation practices developed by different organisations in Latin America. To do this, we have selected emblematic and noteworthy cases from the region to illustrate their different nuances, levels of participation, processes and the participatory tools they use.

Continue reading

Evaluating with rights-holders. Lessons from the Updated Human Rights Appraisal in Mexico City

by Marcia Itzel Checa

It is increasingly common to hear of participatory evaluations that give leading roles to a large range of actors affected by a particular intervention. This allows evaluations to be carried out using a more comprehensive vision, one that recovers the different perspectives involved.

Likewise, Mexico City’s Human Rights Appraisal and the Human Rights Programme associated with it are one of a kind, for the following reasons: the broad participation of multiple political and social actors in the elaboration, execution, monitoring and evaluation stages; its institutional design, which has matured over time and has, indeed, been incorporated into the city’s new constitution; and the fact that, despite its ups and downs, it has survived three periods of municipal government.

Continue reading

“TELL ME AND I FORGET, TEACH ME AND I MAY REMEMBER, INVOLVE ME AND I LEARN”

EVALPARTICIPATIVA’S CAPACITY DEVELOPMENT STRATEGY

The Benjamin Franklin quote that we have used as the title is perhaps the best way to introduce this post, in which we wish to share some of EvalParticipativa’s recent experiences as part of its capacity development strategy in the region. Our contribution to capacity strengthening in Participatory Evaluation (PE) has made use of both online and face-to-face formats, and has focused on specific personal and organisational contexts.

The aim of these capacity development sessions was to ensure that participants were familiar with conceptual and methodological features of PE, based on participants’ own experiences and the contents of the “Sowing & Harvesting” handbook. Didactic tools and documentary videos prepared specifically for each context were combined so that participants could:

      • develop their knowledge, skills and capacities concerning the aims, steps and critical moments involved in this kind of evaluation;
      • acquire an initial understanding of how to implement a participatory approach and facilitate inclusive processes; and
      • gain a basic understanding of how to use the methodology, both to improve their own evaluation practice and to contribute to development processes across the region.

In this post, we share an account of three capacity development workshops, held in late July and early August in Ecuador, Colombia and Costa Rica. These training experiences were organised and facilitated jointly with local teams drawn from different academic and social organisations. Around sixty people were trained in three countries. This account speaks of the joint efforts and lessons learnt, of networking and synergy, of real-life challenges that were overcome, and of hopes about the possibility of constructing a Latin American society that is more just and inclusive and where, one day, nobody will be left behind.

Continue reading

A few thoughts on the similarities and differences between social research and participatory evaluation

by Luisa Graffigna

While I have always found Participatory Evaluation attractive, until recently I viewed it as irrelevant to my work. That changed when I listened to Marina Apgar’s presentation, Evaluación Participativa y rigor en el marco de una evaluación transformadora [Participatory Evaluation and rigour in the context of a transformative evaluation, only available in Spanish] at the EvalParticipativa international seminar in December 2021. Her talk helped me realise how the criteria of rigour in evaluation that she spoke of are echoed in a field with which I am more familiar: social research.

All of us carry with us our own set of lived experiences. In my case, my academic training as a sociologist and a period several decades ago of collective work, reflection and practice linked to processes of Popular Education -together with some of the people who coordinate EvalParticipativa today- contributed to my interest in, and understanding of, what Marina was saying. From this experience, the “participatory” part resonates with me, but my background positions me more firmly within the field of social research and it is from this position that I will share some thoughts about the ways these two processes, each with their own logic, converge and diverge.

Continue reading

PARTICIPATION, A KEY FOCUS IN OUTCOME HARVESTING: LESSONS FROM CHILE

by Andrea Peroni Fiscarelli

Homogenous evaluations vs differentiated evaluations

In order to build a solid framework and evidence, state-led evaluations have become more standardised over time, and are thus increasingly gaining in credibility. The problem that remains is the need to recognise that not all public programmes share the same characteristics, and therefore differentiated types of evaluations should be considered.

This is evident in programmes that centre around the delivery of goods with no interaction with the beneficiaries (such as in the case of subsidies, plans and vouchers), and others that seek to provide tools and/or develop competencies and skills aimed at increasing the social inclusion of individuals, especially from vulnerable sectors.

For these types of programmes, traditional evaluation methodologies have proved to be insufficient as they only seek to assess the level of effectiveness or efficiency and do not manage to capture the complex and diverse reality, practice and results that are present. Furthermore, as the programmes deal with human and social behaviours, the complexity of their contexts should also be recognised.

Continue reading

The tools, on their own, do not make the difference

by Dagny Skarwan

The need to carry out an evaluation in order to discover what results have been achieved by an intervention is appreciated by organisations, projects and the teams responsible for implementing them. In turn, monitoring is often linked to accountability, generally through a weekly or annual report.

Moreover, in the field of NGOs, monitoring is usually understood as reporting activities, in other words, accounting for everything that has been done, within a set period, in relation to the operational plan.

Even when projects have a logical framework or results matrix, and even when they have developed a theory of change, it is not unusual for organisations and local teams to be surprised by the instruments they come across when they start getting involved in participatory monitoring of outcomes and impacts. In this type of monitoring, I usually help teams reflect on how outcomes are measured, how impacts can be recognised and measured, and -from there- recognise the different contributions of the project. Questions also arise concerning the purposes of monitoring that go beyond the need of a project coordinator to provide accountability, and include questions such as where to start when monitoring a project and how to know when it is the right moment to do so.

Continue reading