Can digital approaches to monitoring, evaluation, research and learning support participation?

by Linda Raftree 

Back in the 2010s, the world was buzzing about the potential for digital tools to revolutionize and democratize most everything – SMS for community feedback loops, mobile phones for citizen journalism, open data to improve transparency and accountability, social media platforms for people to make themselves heard without needing to be part of the elite, and networks so that social movements could organize in resilient ways at very low cost.

A decade later, there have been enormous shifts and changes that have benefited participation. However as often happens, these spaces have also been appropriated and taken advantage of by those who are seeking to maintain their status and power.

Continue reading

Participatory Assessment in Education: A step closer to student-centred education

by Tulio Barrios BullingRicardo Cristi López

What is not measured cannot be improved”. This idea is popular in the world of educational assessment, and especially common in the field of learning assessment. However, the statement does not help us determine what is assessed, how it is assessed or who carries out the assessment.

While traditional summative and normed psychometric assessment is effective when it comes to measuring learning, it does not allow for comprehensive or individualised responses to each question in the assessment or take into account every factor involved in the teaching-learning process. Thus, new evaluative trends are emerging in an effort to fill these gaps and introduce more formative and comprehensive aspects to learning assessment. These include Assessment for Learning, Authentic Assessment and more recently, the Transformation Assessment Framework.

These three new approaches share the belief that assessment should not be limited to providing a grade. Instead, according to the logic of continuous improvement, it should consist of a process that monitors student learning in order to offer continuous feedback and suggest changes to the teacher’s input. The hope is that these interventions significantly improve student learning. The assessment activities involved in these approaches are formative, and often simple, quick, low-stake and applicable during class time. Teachers can access the results immediately and can therefore make adjustments to their lessons in real time. Feedback also serves as formative assessment.

Unlike summative assessment, its formative counterpart is not only used to measure and quantify students’ learning levels, but also to provide teachers with relevant information that they can use to improve their educational practice. In this sense, the level of student involvement in assessment processes is fundamental in order that the results are more meaningful and representative.

So, what does student participation look like? Studies have consistently shown that their role is often limited to the single task of answering questions in a test. This represents a low level of student participation in the assessment process. We believe that students should be involved in all three of its stages: the design, the implementation and the grading.

For Quesada, García-Jiménez and Gómez-Ruiz (2017), who follow the Transformation Assessment approach, the first stage should involve agreeing on the focus of the assessment and the different planning aspects involved. These should include the assessment criteria, the tasks to be completed and who should carry them out, the assessment instruments and the weighting of each item. The second stage, implementation, should take into consideration both the person who is assessing the learning and the person whose learning is being assessed, in order to determine the most appropriate assessment modality and thereby ensure the most appropriate instrument is chosen given the required information and the characteristics of the area being assessed. The final stage, grading, should include the information provided by the people responsible for the assessment (feedback) and this should be reflected in the final grades obtained in the subject, which should in turn be translated into decision making to help improve pedagogical processes and strengthen future learning outcomes.

Beyond the ways in which students participate in the assessment of their learning, it is advantageous to include them for a number of reasons. First, student participation in the assessment process leads to more reliable evidence of learning, as it makes it less likely that the assessment will be carried out using traditional techniques or a single method. Self-assessment or peer-to-peer methods save significant amounts of time for teachers who no longer need to spend hours assessing the work of their students. This newly available time can be reinvested in providing deeper and more meaningful feedback to each of their students.

Another significant advantage of participatory assessment is that it strengthens students’ metacognition as they become more conscious of their cognitive processes, gaining understanding and autonomy. This enables students to identify and value the demands of a particular task or activity, assess their knowledge of the topic, plan how to approach the task at hand, monitor the process and apply adjustments where necessary.

A third advantage of participatory assessment is that it encourages students to commit to their own learning process, which in turn has a positive impact on their levels of motivation. Commitment and motivation provide the student with a stronger desire to finish the tasks or activities they have been assigned well. This gives them more ownership over the learning and teaching processes in which they are involved.

Finally, students develop skills and qualities as a result of this participatory process (critical thinking, collaborative working, autonomy, resilience, communication, tolerance, respect, empathy, etc.) which will remain with them for the rest of their lives and which are transferable to other areas of personal and occupational development.

If the benefits of participatory assessment are known, why are they so seldom used? Many teachers probably feel afraid of losing the control they have over the teaching-learning process and over the assessment process itself. Some may fear losing their central role, while others were trained under a teacher-centred teaching approach. Others will not feel prepared to handle the methodological and conceptual changes involved in introducing a participatory assessment approach.

Certain experiences have shown that engaging in student participation can be physically and emotionally exhausting for teachers. On the other hand, positive experiences of applying participatory assessment may be found too, such as at Ikasolas (especially in the privately-run schools funded by the state) in the Basque Autonomous Community, or in schools in Catalonia, which have used different strategies to promote the active participation of students in assessments of the teaching-learning process. This means that students can assess their own progress, participate in the assessment of their peers and collaborate in decision-making about improvement strategies.

Achieving open and decisive student participation in assessment processes may lead to significant changes in the culture of a school. This effect is not welcomed by all and may be resisted by many teachers, making it a difficult change to achieve. However, convincing leadership by a strong management team can lead to significant advances that in turn lead to better learning.

It is also important to note that a certain level of resistance may even come from the students themselves. The requirement to participate in new ways will move them out of the comfort zone of being passive spectators. Many students see the development of their autonomy as a challenge that they do not feel ready for. We must face these challenges head-on if we want participatory assessment to be the next step forward in education.


Reference
Quesada, V., García-Jiménez, E., & Gómez-Ruiz, M.Á. (2017). Student Participation in Assessment Processes: A Way Forward. Pennsylvania: IGI Global

BOOK ON EVALUATION, INCLUSION AND DEMOCRACY IN LATIN AMERICA: EXPERIENCES AND LESSONS

OPEN CALL

A while back, we began working with colleagues in the region on the concepts of evaluation, inclusion and democracy in Latin America and the Caribbean. In the new EvalParticipativa project (2023-2024) we plan to revisit these ideas and produce a book on the subject. Our aim is to identify participatory evaluation experiences in Latin America and the Caribbean, analysing dimensions of inclusion and democracy, and emphasising the learning components generated by these experiences.

The book’s target audience is made up of individuals who are responsible for designing and carrying out evaluations, in the hope of increasing their awareness of the possibility and feasibility of conducting evaluations with civil society participation. We seek to bring this awareness both into their consciousness and onto their agendas. The book is intended for Latin American bodies that provide capacity building and training in evaluation, in the hope that it will make available real cases of evaluations carried out with the participation of civil society in the region. Our target readers are teachers and students in diploma courses, master’s degrees or other specialisations focused on evaluation and social planning, in the hope of influencing the training of professionals who will in the future be conducting, commissioning, supervising and assessing participatory evaluations in the region.

how to participate?

In this document we present a general, preliminary overview of the aspects we intend the book to address. We are clear that the focus should be on presenting one or more existing evaluation experiences as the core data for analysis, without excluding the possibility that some general remarks or proposals may be added where appropriate. In other words, a substantial part of each chapter should be devoted to a descriptive and analytical account of one or several evaluation experiences, while a smaller part may be dedicated to more general or propositional reflections that build on the first section.

This brief post is therefore an invitation to friends and colleagues of EvalParticipativa who are interested in contributing a chapter to the book to email their ideas to eval.participativa@gmail.com.

We will select the chapters that will be included in the book from the submissions we receive. We thank you in advance for sharing this announcement with colleagues and contacts interested in these topics.

Best wishes,
The EvalParticipativa team

INTRODUCING THE BOOK ‘EXPERIENCIAS EN EVALUACIÓN PARTICIPATIVA’

by Maria José Garcia Oramas

The book ‘Experiencias en Evaluación Participativa’ (Participatory Evaluation Experiences, available only in Spanish) was published by the Veracruz Ministry of Education, Mexico, in 2009. It is a compilation of participatory evaluations carried out in community and educational settings carried out by diploma students at the Institute of Psychological Research, Veracruz.

Long-standing problems in education, health, environmental care, gender equality and inclusion continue to exist in 2023 and we remain obliged to seek creative ways of resolving inequalities of gender, ethnicity and class derived from patriarchal, racist and capitalist political systems that are still prevalent in our Latin American countries.

Continue reading

EVALPARTICIPATIVA AT GLOBAL EVALUATION WEEK – GLOCAL2023

Greetings fellow EvalParticipativa colleagues,

Earlier this week we took part in the gLOCAL 2023 roundtable discussion, CHALLENGES IN PARTICIPATORY EVALUATION: A FEMINIST AND DECOLONIAL APPROACH. The activity was organised jointly with Kalidadea, a feminist consultancy network, working in the field of international cooperation and development.

Evaluations entail several different challenges including ensuring that the process empowers those involved by achieving a sufficient level of effective participation across a diverse set of participants. Another challenge is to ensure meaningful gender mainstreaming and a move towards feminist evaluations rooted in human rights.

The aim of the roundtable was to create space for reflection on these challenges and on how to integrate participatory and feminist approaches successfully into our evaluations in a way that is relevant, effective and appropriate to the groups we work with, be they local communities or the general public.

Our speakers—Esteban Tapella from PETAS, Lara González Gómez, director of Kalidadea and Alejandra Lucero Manzano, member of PETAS and Kalidadea— introduced some of these challenges and suggested ways to overcome them. They shared concrete strategies and ecofeminist principles intended to contribute new perspectives for developing empowering evaluation processes based on a decolonial perspective. The roundtable was moderated by Vanesa Castro (a PETAS researcher). You can click the link below to watch the recording which includes a Q&A session with the participants.

The gLOCAL Evaluation Week is a unique knowledge-sharing event, connecting a global community of people across sectors and regions. Over the course of a week, participants join events—in their own regions and further afield—to learn from each other on a vast number of topics and experiences. By helping participants understand how their work fits in with monitoring and evaluation (M&E) ecosystems in their region and the larger M&E community worldwide, gLOCAL helps to inspire and energise a global movement: individuals and organisations that value the power of evidence to improve people’s lives. gLOCAL, is convened and supported by the Global Evaluation Initiative (GEI), a network of organisations and experts that supports countries with strengthening their M&E systems in order to help governments gather and use evidence that improves the lives of their citizens.

We were delighted to take part in this event again this year!

STAKEHOLDER MAPPING: A PARTICIPATORY EVALUATION TOOL

by Esteban Tapella

Over the last fifteen years, public discourse on development policies and programmes has emphasised the need to reinforce the leading role of civil society in the interventions that implicate them. Increasingly, concepts such as ‘participation’, ‘accompaniment’ and ‘stakeholder perspective’ are heard.

This trend can also be detected in the field of evaluation, in approaches such as Democratic Evaluation, Systematisation of Experiences, Most Significant Change, Collaborative Approaches to Evaluation and -of particular interest to us- Participatory Evaluation, all of which seek to increase the involvement of a range of social actors. These approaches reflect an emerging sensibility in the field of evaluation that seeks to capture stakeholder perspectives when it comes to assessing outcomes and impacts and attributing them (or not) to a particular intervention.

We describe an evaluation as participatory when the parties involved in a programme or project define what will be evaluated, what the objectives will be, when the evaluation will be carried out, what data collection and analytical methods will be used and how the results will be communicated. Thus, a participatory evaluation aims to encourage an active and conscious incorporation of the so-called key stakeholders that are linked to the intervention being evaluated. But who are these key stakeholders? How are they related to each other and to the programme or project? In this post, I will attempt to respond to these questions and at the same time present the working document ‘Key Stakeholder Mapping’ (KSM) which introduces this tool, which I consider to be useful in evaluations that value social inclusion and participation.

Continue reading

PARTICIPATORY MONITORING AND EVALUATION OF SUSTAINABLE LAND MANAGEMENT PRACTICES TO ENCOURAGE SOCIAL LEARNING AND LARGE-SCALE ADOPTION

by Raquel Luján Soto

Participatory Action Research (PAR) emerged in the 1970s as an alternative to the technocratic methods of top-down research, which continue to be widely used to this day in the field of agricultural sciences.

These technocratic methods have not proved capable of involving farming communities in sustainable land management and are often abandoned at the end of the research process.

On the one hand, PAR arises from the need to recover the local knowledge of grassroots farming communities and recognise the value of the diversity of the agricultural practices and natural resource management methods they use in food production, biodiversity conservation and in creating a multifunctional landscape, as well as the ways they have maintained agroecosystems sustainably over centuries.

On the other hand, it also emerges from the necessity that researchers and local communities jointly identify customised solutions capable of addressing their needs and objectives in order to ensure they are more positively received and more widely adopted in the long term.

The PAR approach involves the development of horizontal relations between farmers and researchers based on the premise that research should be carried out through a ‘dialogue of knowledge’ and with recognition and respect for the rural communities, their knowledge and the ways they manage their relationship with nature. This idea will be further explored in this post.

Continue reading

POLITICISING EVALUATION: TRANSFORMATION THROUGH GENDER RESPONSIVE PARTICIPATORY EVALUATIONS

by Andrea Meneses

Can we say that an evaluation is truly participatory if it does not involve a gender perspective? Can we carry out gender responsive evaluations without the active participation of the groups involved in the evaluation process? Are we politicising evaluation by incorporating an inclusive perspective?

Far from providing answers, I want to briefly look at the ideas put forward by various researchers and evaluators who invite us to reflect on the topic, the task of evaluation, and its transformative potential.

Continue reading

NEW TRAINING SET: “SOWING AND HARVESTING HANDBOOK” IN WHITEBOARD-ANIMATION FORMAT

It is with great joy that we share the news of the launch of a new resource in our community of practice and learning EvalParticipativa, aimed at accompanying and illustrating the Sowing and Harvesting, participatory evaluation handbook .

This is a set of five short videos, one for each core chapter of the manual, done in Whiteboard-animation format.

The Whiteboard-animation format (or whiteboard animation) has gained popularity as a form of communication and distance training. The logic of this tool is to tell an illustrated story, simulating that it unfolds and develops on a blackboard. The animations are accompanied by a scripted narration with the purpose of explaining, in an entertaining and simple way, concepts that, explained in another way, would not be easy to grasp in a few minutes.

Continue reading

A methodology that fosters a participatory approach

Lessons from using a Theory of Change in a Participatory Evaluation

by Viola Cassetti and Joan J. Paredes-Carbonell

Our professional paths crossed in Valencia in 2016, when Viola was finishing her European Master in Public Health (EuroPubHealth) and about to start her PhD at the University of Sheffield (UK) and Joan was working as Deputy Director General of Health Promotion at the Valencia regional health authority.

Our first project was to adapt the NICE guidelines on community involvement to the Spanish context using a collaborative approach. We spent two years co-coordinating a group of more than 80 professionals who actively participated in the project. You can acess the guide (only in Spanish) by cliking on it (Cassetti et al., 2018).

Continue reading