IS OUR EVALUATION PARTICIPATORY? THE PARTICIPATION SCANNER

by Sara Vaca

After studying business and spending my early career working in the private sector, discovering what “participation” meant, albeit theoretically, was key. I understood that “participation” meant that several stakeholders had a voice (and vote) in decision making during the four phases of the evaluation: the design, data collection, analysis, and interpretation… seemed simple enough!

From that moment (back in 2012), I began to notice that most evaluation reports included in their summary or methodology something along the lines of: “this is a (highly) participatory evaluation”, and some even mentioned it in the title itself (“participatory evaluation of…”), when what they really meant was that they had consulted many people or groups, but only as informants…! I wondered why they called it participation when what they actually meant was that they had consulted a wide sample.

Continue reading

Building and sustaining meaningful engagement of youth in evaluation

 

The deadline to deliver the ambitious 2030 Agenda for Sustainable Development is approaching fast. The world is challenged and struggling to gain a foothold with the COVID-19 crisis still looming large. Bold, ambitious and inclusive actions can turn it around for the people and the planet. At the same time, solutions that have the highest transformative power to change people’s lives must be scaled up. With a world population younger than ever before, engaging with youth in development processes, including in evaluation, can provide the impetus and the multiplier effect to get the Sustainable Development Goals back to course.

The imperative

Inclusive and meaningful engagement of youth in evaluation provides an unparalleled opportunity to make development programmes responsive to the needs and demands of the youth. It raises youth voices and agency and recognizes them as active leaders and contributors in building a sustainable world. When the power of youth is harnessed in evaluation through meaningful ways, it can bring innovation, increase evaluation quality, enhance the relevance and the transformational power of evaluation.

Continue reading

Inclusion in Evaluation. The Importance of Participatory Techniques

One of the challenges of participatory evaluation is that of including a broad diversity of stakeholders, many of whom have no training or previous knowledge of evaluation.

In this testimony Olga Nirenberg highlights the importance of the tools and techniques employed. These must be at the same time effective in tackling the topic and simple so as to be within the reach of all the participants. Including the voices of a vast and diverse array of stakeholders in the process is the best way towards a useful, transformational and high quality evaluation.

Olga Nirenberg has a PhD in Social Sciences (UBA, 2005) and a diploma in Public Health (UBA, 1976). A founding member of the Local Development Support Centre – CEADEL -(https://ceadel.org.ar​, 1986 – 2020), she developed the Self-Assessment Tool for Education Quality project – IACE- (UNICEF/CEADEL, 2007-2017). She has worked as a consultant/evaluator for UNICEF Argentina, the ARCOR Foundation, the Pan American Health Organization (PAHO), the World Health Organization (WHO) and the W.K. Kellogg Foundation. She has also worked in different social areas of the national government in Argentina. She has been an evaluator of extension projects and a teacher in both public and private universities and has published books and articles on social planning and evaluation. She is a member of the Argentine Network of Evaluation (EvaluAR) and of the Latin American and Caribbean Monitoring, Evaluation and Systematization Network (ReLAC). She is currently collaborating with the EvalParticipativa initiative.

Indigenous evaluation and its challenges to the evaluation field: does participative evaluation has a word to say?

By Larry K. Bremner (*)

I would like to commend Evalparticipativa in establishing a community for professional dialogue and learning – well done! I would also like to thank you for asking me to contribute to this important dialogue.

I am Métis which is one of the three recognized Indigenous groups in Canada, the other two groups are First Nations and Inuit.  As an evaluator I have had the privilege to work with dedicated people across Canada and internationally who are trying to improve and empower their communities. I believe evaluation is a tool for use in the empowerment of communities and peoples. For too long evaluation has been used as a tool to help maintain and justify a colonial and paternalistic approach to Indigenous communities. Transient evaluators, not familiar with the cultural, political, and social histories of the communities, have arrived in Indigenous communities to evaluate programs about which they are unfamiliar. Furthermore, they do not understand or follow community protocols. For too long evaluation has been an extractive process which has taken from the communities often giving nothing back in return. This has led me to advocate for Indigenous communities to take control of their evaluation agenda.  I have outlined some things I have found that helps me when undertaking evaluation with Indigenous peoples and communities.

Continue reading

How Are Participating Stakeholders Empowered in a Participatory Evaluation?

by Emma Rotondo (*)

In this contribution to EvalParticipativa I will describe the characteristics of the empowerment of multiple stakeholders during participatory evaluations. But first I should clarify what I understand by participatory evaluation (PE).

PE evaluation involves the participation of the parties interested in a programme, project or policy. On the basis of valuing local knowledge and wisdom, it seeks to generate learning about the changes in order to empower people and social groups to make decisions and to strengthen their capacity.

There are three basic elements common to all participatory evaluation processes: (i) a transformative approach, that is, a proposal to produce social and political change; (ii) a joint or associated work between different stakeholders; and (iii) social learning and use of the outcomes of an evaluation. In this context, what is understood by empowerment of stakeholders in a PE process is the broadening of both individual and collective capacity (or faculties) to act; the provision of tools for reflection, dialogue and co-building; and the generation of knowledge to learn socially, all of this with the intent to use the evaluation and transform their environment.

Continue reading

SIMILARITIES BETWEEN SYSTEMATISATION OF EXPERIENCES AND EVALUATION

by Olga Nirenberg (*)

Introduction

As I mentioned in my brief comment on the article published the 22nd of March on EvalParticipativa, “Systematisation of Experiences and Evaluation: Similarities and Differences“, by Oscar Jara Holliday, my first reaction upon reading it was of surprise. I felt it was paradoxical that certain arguments regarding evaluation would appear on the blog on Participatory Evaluation, it seemed like an oxymoron, a contradiction in itself.

I am familiar with the works published by the author of the article, OJH, which are widely known and have been of great influence on those of us who work in evaluation in Hispanic American countries. I have even had the opportunity to debate directly with him on the resemblance between systematisation and evaluation. Our conversation was very important to me, so much so that in the book I published shortly after (Nirenberg, 2013) I included a chapter (chapter 8) dedicated almost entirely to systematisation of experiences in which I highlighted its points of contact with non-traditional evaluation approaches.

Although I have acquainted myself with the approach and methodologies of systematisation of experiences and have even put them into practice in many occassions, here I will discuss mainly the references to evaluation present in the cited article, since that has been, for more years than I would like to admit, the central focus of my professional career.

I am very grateful to my colleagues and coordinators of EvalParticipativa, Pablo Rodriguez-Bilella and Esteban Tapella (PETAS/National University of San Juan, Argentina) and Carlos Sanz (DEval), for giving me the opportunity to expand and enrich this debate.

Continue reading

SYSTEMATISATION OF EXPERIENCES AND EVALUATION: SIMILARITIES AND DIFFERENCES

In Latin America and the Caribbean, the systematisation of experiences approach is widely known[1]. In some fields, such as popular education, it is even better known than evaluation. We ask ourselves, what differences and similarities are there between evaluation and systematisation of experiences? Is it possible to identify common aspects between systematisation and some types of evaluation?

At EvalParticipativa we wish to open the debate and for that we have invited Oscar Jara Holliday, one of the most renowned figures in the field, to kick it off.

The Context

According to specialised literature, systematisation of experiences aims at establishing learning as an essential element of any intervention policy. And he does so by rising to the challenge of promoting, designing and conducting learning processes in experiences that were probably not conceived with that purpose in mind. But is the search for learning exclusive to systematisation? Are there similar purposes in the field of evaluation?

Continue reading

Bravo EvalParticipativa!

By J. Bradley Cousins and Hind Al Hudib
University of Ottawa, Canada

EvalParticipativa – what an amazing space! We are longtime fans, researchers, and purveyors of participatory evaluation but in our experience, EvalParticipativa is unparalleled as a space for professional exchange, capacity building, and learning in this domain. It is our very great honour to contribute to, and become part of, the EvalParticipativa community.

Participatory evaluation (PE) has been near and dear to our hearts for a very long time. One of us (Cousins) has been writing about this topic for almost 3 decades. While our contributions have been mostly research on PE, we’ve always had an interest in translating research-based knowledge into practice. What an amazing opportunity EvalParticipativa provides in this regard! But perhaps even more compelling is the reverse; what a fabulous opportunity to turn expert practice into research! Lessons learned will surly advance evaluation theory and practice.

Continue reading

PARTICIPATORY EVALUATION AND RIGOUR

By Osvaldo Néstor Feinstein

Participatory Evaluation (PE) gives voice to stakeholder perspectives on policy, programme or project processes and results in order to limit or avoid technocratic bias. Furthermore, it promotes ownership of the evaluative process and results which makes the evaluation more widely accepted. These are two of the arguments in support of PE.

On the other hand, PE has been criticised by the argument that it is not a rigorous approach due to its qualitative methods which capture “impressions” and anecdotes but do not provide rigorous quantitative procedures. Sometimes random control trials (RCTs) are used as an example of rigour in evaluation.

Continue reading