Evaluating the impact of AI on society: why we need a range of perspectives

by Gabriela MunaresRebeca Lucas

Cities around the world, including many in Latin America, are incorporating artificial intelligence solutions into their urban services: video surveillance and facial recognition systems for public safety, traffic management, citizen services, waste collection and tourism promotion.

In many cases, this trend is advancing faster than the mechanisms for evaluating its social consequences. A good example is the case of security: cities such as Lima, Buenos Aires and Mexico City have deployed AI systems for the surveillance of public spaces without any systematic processes in place to evaluate who is disproportionately affected, what biases may be embedded in their algorithms, or how the rights of those being monitored are protected.

Continue reading

When Participation Becomes Data

Reflections on Artificial Intelligence and the Future of Participatory Evaluation

by Angie Pereira Calvo

A few months ago, I sat for more than two hours with a group of women, listening to their experiences with a social program. There was no recorder running. I held no questionnaire. There was silence, pause, chocolate, and a trust that had taken weeks to build. In that conversation, I learned more about the real determinants of access to health services than from any dataset I had ever processed.

I genuinely wonder what part of that conversation an artificial intelligence (AI) system would have captured. And I also wonder what part of it we would have sacrificed if the methodological emphasis had been placed on the efficiency of data processing rather than on the quality of listening. This, in my view, is the central question the Latin American evaluation community must ask itself right now: before the adoption of AI tools silently redefines what we mean by participation.

Continue reading

Evaluating to strengthen communities: lessons from a participatory evaluation in urban neighbourhoods in southern Chile

                                         by Alba Ximena Zambrano Constanzo, Vaite Trujillo Burgos, Mauricio García Ojeda, Francisca Román Mella

In contexts of urban vulnerability, evaluation is never neutral. Every evaluation reflects a way of seeing reality, deciding what counts as a problem, what is considered an achievement, and who gets to interpret it. From this perspective came the article Community Strengthening in Urban Neighbourhoods: Psychosocial Dynamics in Vulnerable Contexts, which reports on a participatory evaluation of community strengthening processes carried out alongside urban communities in southern Chile.

The article is part of a broader comparative study involving five urban neighbourhoods: three in Chile and two in Brazil. This publication focuses specifically on the Chilean cases, developed in the city of Temuco, in the La Araucanía Region, areas marked by socio-spatial segregation, urban precarity, and a historically unequal relationship with public policy.

Continue reading

Inclusive Rigor as a Transformative Compass: An Experience from Colombia

The challenge of evaluating what is already participatory

by Ángela María Báez-Silva Arias

When we set out to evaluate Co-Inspira, a peacebuilding initiative in Colombia, we knew we were facing an unconventional challenge. This was not about assessing a traditional intervention, but something far more complex: a Systemic Action Research (SAR) process that was, in itself, already a collective exercise in knowledge generation.

How do we evaluate a process that is, by nature, an exercise in Participatory Research? How do we avoid overburdening participants with an additional layer of evaluation work when they are already leading their own knowledge-generation process? In the chapter Lessons from practicing Inclusive Rigor in the evaluation of a peace initiative in Colombia”, published in the book Evaluation, Democracy and Transformation”, we share how we approached this methodological challenge through an evaluative learning process guided by the Inclusive Rigor framework.

Continue reading

A PATH OF NO RETURN TOWARDS PARTICIPATION

A Costa Rican experience of participatory evaluation that transformed perspectives and practices

by Karol Cruz Ugalde, Eddy García Serrano and Juan Murciano

What happens when those who have historically been evaluated become co-evaluators? How can such a change be fostered, managed, and capitalized upon? Is it possible to promote participatory evaluation from within public institutions? How?

These were some of the questions that prompted us to write this chapter about the evaluation of the Program for the Promotion of the Autonomy of Persons with Disabilities in Costa Rica. We did not want to recount a technical process full of methodologies and results, but rather to share, from within, a living, challenging, and deeply human experience.

Within these pages, readers will find the Costa Rican context —a country with a legal framework that recognizes the rights of persons with disabilities, but which still faces great challenges in making those rights a part of daily life. Presenting this backdrop was essential, as it grounds the very purpose of our work: to show how evaluation can be a tool for transformation and democratic strengthening.

Continue reading

READINESS FOR PARTICIPATORY EVALUATION

by Ricardo Ramírez & Joaquín Navas

‘Readiness’ is an English language term that is difficult to translate into Spanish as it encompasses various themes: preparation, availability, propensity, timeliness, agility, competency, willingness – among others.  In the Spanish version of this entry, we use ‘availability’ (‘disposición’) with the understanding that it covers the other themes. In the field of evaluation, readiness is foremost across the first two steps of Utilization-focused Evaluation (UFE) as proposed by Michael Quinn Patton (2008). Readiness is also referenced in other fields, for instance in communication strategy design (Ramelan, 2014).

While the goal of UFE is the actual use of evaluation, the approach relies on the direct participation of ‘evaluation users’; it is their direct engagement that provides them with a sense of ownership over the process and the findings. In other words, UFE confirm that it is very much part of other collaborative approaches to evaluation (Ramírez & Brodhead, 2020).

Continue reading

WHEN EVALUATION LEADS TO CHANGE: A PIONEERING EXPERIENCE OF PARTICIPATORY EVALUATION IN JALISCO (MEXICO)

by Mónica Ballescá

I’m genuinely delighted to share a personal take on Chapter 7: Participatory evaluation of a public transport support policy: an inclusion and transformation perspective – Jalisco (Mexico), recently published in the book Evaluation, Democracy and Transformation: Experiences of Participatory Evaluation in Latin America. I co-wrote it with Sugey Salazar and Selene Michi, and together with other colleagues we reflect on what it really means to carry out public policy evaluation using participatory approaches from within the public sector.

This chapter sets out the details, practical insights and lessons from an evaluation process that, in my view, marked a real turning point in government practice in Mexico. It was one of the first participatory evaluations promoted from within the public sector at subnational level in the country, and the first to use a methodology that was later formalised and replicated in the years that followed.

Continue reading

Design of a participatory evaluation of a public service for older adults in Ecuador

by Carmen Lucía Jaramillo 

The invitation we received to share the experience of designing the first participatory evaluation promoted from within the public sector in Ecuador represented a challenge both for me, as the facilitator of the process, and for Joselyn Corrales, who at that time was serving as Undersecretary for Evaluation in the state body responsible for evaluation.

The challenge lay in communicating clearly not only the methodological and conceptual aspects, but also the uncertainties, challenges and lessons learned arising from the design of the participatory evaluation of the Socialisation and Meeting Spaces for Older Adults, an intervention implemented by the Ministry of Economic and Social Inclusion.

Continue reading

Foceval and Focelac: 10 Years of Cooperation for the Development of Participatory Evaluation Capacities in the Region

by Andrea Meneses Rojas

In Chapter 13 of the book Evaluation, Democracy, and Transformation, I invite you to explore how, over ten years of international cooperation, diverse intentions and shared learning have come together so that participatory evaluation has moved from being a mere ideal to becoming a transformative practice in Latin America and the Caribbean.

Throughout this decade, cooperation between the German Institute for Development Evaluation (DEval) and Costa Rica’s Ministry of National Planning and Economic Policy (Mideplan) has established participatory evaluation as a strategic commitment in actions carried out in the region. Between 2014 and 2024, this joint effort was channeled through two projects (Foceval and Focelac) with a common purpose: to strengthen evaluation capacities and contribute to evidence-based decision making.

Continue reading

ELEVATING TSAATAN VOICES: SOWING THE SEEDS OF EVALUATION IN MONGOLIA

 by Gereltsetseg Adiya, Oyuntulkhuur Jukov, Itgemjit Gankhuyag and Azjargal Amarsanaa

With the enforcement of Mongolia’s Law on Special Protected Area (*),  97% of Tsagaannuur soum in Khuvsgul province where the Tsaatan live has been declared a strictly protected zone. This has placed major restrictions on their traditional way of life: hunting, seasonal migration, reindeer herding, and pasture use have all become increasingly limited. These restrictions directly affect their food security, cultural traditions, and quality of life.

Recognizing that the Tsaatan voices have been systematically excluded from designing, implementing and evaluating policies, this participatory and culturally responsive evaluation demonstrates how meaningful participation can transform the evaluation of conservation policies – by centering Tsaatan people’s traditional knowledge and ensuring their perspectives directly shape evaluations and are considered in policy making, that affect their ancestral lands.

Continue reading