Interest in evaluating public policies has grown significantly in the last fifteen years. This is reflected in theoretical and methodological production as well in the increase of national evaluation policies in countries of all continents. It is also seen in the increasing institutionalisation of evaluation and a surge of initiatives focused on making this practice more professional. Since the 2015 ‘International Year of Evaluation’ evaluation has become a global trend. This has made the main international development organisations converge their interests and actions with regional evaluation networks (VOPEs), foundations, various government bodies, non-governmental organisations and academia; all of whom are interested in maximising evaluation as an instrument for improving public policies.

In addition to this, the new Sustainable Development Goals (SDGs) were an important step forward for evaluation theory and practice. With regard to our context, the idea is to generate a new agenda of global evaluation priorities from 2016 to 2030 which aims to bridge the gap between the community of evaluators (supply) and the community of decision makers (demand), giving a central role to civil society as way of ensuring that evaluations do not only respond to the needs of end users but also meet evidence-based quality standards which are credible and can be used to create and manage public policies.

But the topic of participation and the leading role of society in evaluation is nothing new.  This has been developing over years under other names such as Participatory Monitoring or Tracking, Democratic Evaluation, Systematisation or Capitalisation of Experiences,  More Significant Change, Systemic Approach in Evaluation and Complex Evaluation, to name but a few. The change is seen in the fact that today, participation is consistently featured in many different approaches. While some nuances differ, the context and perspective of stakeholders is seen as increasingly important when valuing and attributing effects and results to a specific intervention.

FROM ‘PARTICIPATIVITIS’ TO CITIZEN PARTICIPATION

As we have said, the need for citizen participation has been increasingly emphasised not only in public policy design but also in their evaluations. Civil society has therefore been given a more central role in evaluative practice. Concepts such as ‘participation’, ‘support’, ‘stakeholder perspective or perception’ are increasingly frequent in the evaluative field.

However, besides the discourse and good intentions, the practice of evaluation does not always reflect this participatory ‘vocation’ as the notion of participation can have different connotations. Often, programmes and projects which tend to ‘foster’ participation ignore the fact that participation activity can easily become simply a symbolic simulation if there is not the conscious awareness that power needs to be redistributed if a process is to be truly participatory. Therefore, in many cases, PE tends to be limited to mere consultations which do not offer local stakeholders the possibility of influencing decisions regarding the evaluative agenda. These recurrent practices are evidence of a lack of theoretical clarity and good quality instruments in many evaluations which claim to be participatory. The reasons for this deficit are not understood.

So how do we ensure that stakeholders in society participate and are given a leading role in evaluating programmes or projects that affect or involve them?

This question makes us reflect on which conditions and mechanisms facilitate effective citizen participation in evaluation processes. It also leads us to deepen evaluative experiences to ensure that they understand aspects which make a difference in evaluation in terms of real and effective social participation as well as exploring possible contributions of participatory and collaborative approaches to conventional evaluations.

LET ME DO IT! TEACH ME HOW TO DO IT!

In my opinion, the notion of participation is the result of two merging dynamics: the opportunity to participate and the capacity to participate. The first is determined by the institutional and political willingness of those who design and conduct evaluations to create spaces for real participation. The second is determined mainly by the attitudes and skills that civil society stakeholders have developed in previous experiences. It is possible to talk about participation when these two dynamics come together: in other words, the capacity of residents to participate are subject to the opportunities ‘opened up’ by the institution, programme or project.

Regarding the opportunity to participate, experience tells me that not all institutions/programmes that instigate participatory evaluation are truly willing to facilitate and support these processes. Among other challenges, I would like to highlight a few which I feel are the most relevant:

• • it is important that representatives of different levels of the evaluated programme are truly committed to this type of evaluation and that they are aware of the implications involved in multi-stakeholder evaluation and are open to listen to and adopt, from the very beginning, recommendations which may emerge;
  • the institution/programme needs to have enough time and resources. Creating real spaces so that civil society has the leading role implies long, slow and expensive processes and not all institutions are willing to take this on;
  • it is really important to be able to rely on the support of a local facilitator who knows firsthand the reality and idiosyncrasies of the participants in addition to having a good grasp on communication and socio-cultural activity tools;
  • a determining feature is if the institution requests or generates capacity-building activities for local stakeholder participants. In most cases, participatory and evaluative experience of the main interested stakeholders is very limited;
  • and it is important to form a relatively small evaluation team which consists of representatives from all groups involved. It should be a central part of all processes starting with prioritising the most relevant topics and formulating the questions and objectives through to communicating results and making sure they are put into action. It should also be part of defining indicators, collecting and analysing data and forming conclusions and recommendations.

With regard to the capacity to participate, I feel that those of us who encourage these processes are confronted with a big challenge. Especially in contexts like those in Latin America which have been reticent and inflexible about creating participatory experiences for a long time. Without a doubt, capacity for participation is created when an institution offers opportunities for participation and encourages settlement residents and other stakeholders to take a central role in evaluative practice. But, in addition to the existing ‘participation opportunities’ which act as basic motivation for the participants, it is necessary to initiate an active capacity-building process throughout the process.

In my experience, the main challenges are:

• • the effort required to match moments of participation with stakeholder capacity, seeking common interests that are within the reach of the participants (for example, do not expect members of a grassroots organisation to get involved in designing a statistical survey for an impact evaluation) and supporting them when challenges arise;
  • the confidence to use a varied set of tools for democratic and participatory evaluation which exist today and be able to use them appropriately and/or replicate them so that they are useful for participatory processes which require evaluation (creating suitable tools, games or activities which guarantee that everyone remains at the centre of the process, rather than just the most extroverted);
  • the recognition that the tools ‘can’t do everything’; that a participatory vocation must be evident in the gathering in addition to dialogue among the participants. This vocation is expressed in tolerance for mistakes, willingness to ‘explain again’, the ability to revisit agreements and redesign working plans etc.;
  • and a clarity around our role as external agents, even though we are participants, throughout the process; this means:  ‘motivating’ without ‘pushing’, ‘reflecting’ with the group without ‘conditioning’ conclusions, ‘sharing ideas’ without ‘imposing them’ and ‘asking questions’ without ‘suggesting answers’.

BY WAY OF CONCLUSION

When we think about what PE is and what its implication is for civil society, an endless amount of possible definitions arise. Some are more ambitious than others in terms of involving multiple stakeholders. Part of the process of collective construction proposed by this community of learning is understanding what PE is, but we would also like to think that in this upcoming gathering we could explore how evaluation is participatory when the different parties involved determine what will be evaluated, who will participate, when it will take place, what data collection and analysis methods will be used and how the results will be communicated. We hope that future PE favours the active and conscious incorporation of organisation members in the evaluative process.

And as in every evaluation, a participatory approach should be helpful for gaining knowledge, making changes and taking corrective measures to gain better results; adding or removing activities or simply changing organisational strategy. In other words, evaluation should help contribute to new and different knowledge for policy and programme creation. Yet, ensuring that PE is capable of strengthening organisations so they can have more control over their own development is just as important. This type of evaluation should also act as a tool to build capacity among various stakeholders as they reflect on, analyse and propose solutions from multiple perspectives.

In a participatory evaluation workshop held many years ago in Costa Rica, we collectively compiled a list of principles which could help guide this practice. The result of that activity was enlightening. I personally feel that those ideas reflect the spirit of EvalParticipativa. We concluded that participatory evaluation:

• • recognises the value of experience and the knowledge and perception held by the local population involved;
  • does not limit itself to consulting the population, but rather incorporates them into decision-making throughout the whole process;
  • creates multiple spaces for information gathering, analysis and use by diverse stakeholders involved;
  • provides procedures, tools and user-friendly methodologies which facilitate participation among multiple stakeholders, who usually have different capacities and strengths;
  • and does not rely on ‘evaluators’ but rather professionals who work as facilitators in the process (design, data collection, processing, information analysis and use) to develop local potential.

We hope that the first gathering of PE experiences for Latin America and the Caribbean to be held in Quito, Ecuador next November will allow us to analyse these ideas, replicate them and move towards consolidating a new evaluation paradigm which integrates social participation.

Esteban Tapella | Researcher at the National University of San Juan,  Argentina, co-director of the Social and Environmental Labour Studies Programme (PETAS)