by Sergio Martinic & Luis Soberón
This article describes the drafting process for the recently published revised and expanded Standards for Latin America and the Caribbean. We give a brief description of the methodology followed, highlighting its participatory nature and the most significant milestones and results. We then examine in more detail questions concerning participation, a cross-cutting thread of particular interest that is woven through all four dimensions and the twenty standards that make up the Standards document.
The document itself can be accessed in English, Spanish and Portuguesse.
Participatory construction of the standards
In 2017 and 2018, the ReLAC Standards Working Group encouraged and promoted presentations on the Standards document that was drafted in 2016, and subsequent discussions. Evaluators from the region participated in this process at seminars, conferences and workshops. The Standards document was thus disseminated and could later be developed and updated thanks to the suggestions that were received in response. In 2019, the ReLAC Executive Committee organised round tables concerning the Standards and invited ReLAC thematic working group participants to have another detailed look at the document so their contributions could be used in its revision.
The thematic group participation was organised into four areas: gender and human rights; indigenous peoples; academia and professional competence; and young evaluators[1]. Each area held three meetings: the first was for the general exchange and brainstorming of topics of interest for each thematic block, while the second and third meetings were used to delve more deeply into the most interesting or concerning issues. A summary report was prepared, specifying the main contributions, questions and suggestions, both for the document as a whole, and for each of the dimensions and standards. An annex included the summaries for each respective thematic block and the list of participants[2].
In 2020, the Standards Working Group was expanded to include new participants (coordinators and opinion leaders from the discussion groups), enabling the inclusion of a greater diversity of perspectives, experiences and countries from the region[3]. The Working Group was set up as a Committee of Experts, taking on the task of revising and developing a new standards document based on input from a large number of evaluators in the region.
As part of the revision process, the following steps were carried out: gathering and reviewing the reports from the previous stages of the discussion; preparing questions to guide the reading of the material and identifying the main issues and problems; developing comparative matrices with proposals for responses to identified questions and topics; and developing a shared perspective, drafting new texts and validating the result. The process focused on three key aspects that had emerged from the discussions: the social and cultural context of Latin America; making the principles and conceptual framework more explicit; and the cross-cutting topics (gender, human rights and interculturality and indigenous peoples).
A draft of the new document was discussed and validated within the working group and later in a consultation with evaluators from the ReLAC working groups. The suggestions made were summarised in a matrix[4] that served as an input for a last revision and final drafting of the new version of the document, released publicly on 30 August 2021.
Participation in the Standards
Participation is a key theme in the Standards, as specifically mentioned in the relevance and usefulness dimension, under the concept of effective participation. It refers to evaluation that ‘takes into account the perspectives of the different stakeholders concerned’. This means making an effort to identify such stakeholders, recognise them, promote their participation from the initial evaluation design phase through to the communication of results, and anticipate the most relevant moments and spaces for their participation in the work plan.
In the Standards, the four dimensions that promote high-quality evaluation are *Rigour, *Ethics and Legal Principles, *Cultural Understanding, and *Relevance and Usefulness. An interrelated set of twenty standards are organised into these four dimensions as follows: 5 – 5 – 3 – 7 respectively. These dimensions and standards have been constructed taking into account human rights, gender and interculturality approaches, as well as the principles of ethics and professional integrity, equality, transparency, inclusion and transformation of reality.
The need to carry out effective participation is fundamentally concerned with the rights and interests of the communities, populations and social sectors that development interventions seek to support and that, in the Latin American and Caribbean region, are largely subordinated and left without adequate levels of access to public resources and services. These same groups also often endure conditions of poverty and are negatively affected by their identities, cultural heritage and discrimination.
Due to the intersecting of approaches and principles, effective participation should indeed occur in the very design and implementation of the development intervention. It is essential in the evaluation so that diverse perspectives and value criteria are included. It is also important to promote empowerment and prepare the stakeholders to take on board the evaluation results in line with their development interests. For these reasons, participation is considered a central element and condition. It is also deemed to be key because of its ramifications and the way it is woven through all the dimensions and standards proposed in the document.
The following standards are the ones that are most clearly relevant in terms of effective participation.
In dimension 1. Rigour: ‘Relevant and agreed evaluation questions’.
In dimension 2. Ethics and legal principles: ‘Respect for people’s rights’.
In dimension 3. Cultural understanding: ‘Equality and equity’ and ‘Reciprocity and interculturality’.
In dimension 4. Relevance and usefulness: obviously ‘Effective participation’ but also ‘Mutually agreed purposes’.
If we merge the above standards to form a definition, effective participation implies that concerned stakeholders participate in the evaluation process by deciding what questions will guide the evaluation and the equal rights of individuals and communities are recognised in the evaluation implementation process. However, specific detail on when participation can be deemed effective or not is still to be developed.
By way of attempting to define when participation is effective or not, the idea of ‘active learning’ can be taken from the field of education. This refers to learning that occurs when learner participation involves cognitive processes of reflecting, analysing, applying and transferring the subjects studied in the teaching-learning process. Following this logic, it could be said that effective participation in evaluation occurs when the community or population – through representation and processes of collective dialogue – analyses, discusses and reflects on the purposes, questions, and processes of the evaluation at different moments, in relation to their interests and perspectives, concerning the way the current and future development intervention has an affect and influence on them.
Conclusions
ReLAC considers the Standards document to be a living document that at the same time as guiding professional practice, should be continuously revised and updated. With regard to participation, the expectation is that experiences of applying the standards and systematising them, and debate on the diversity of situations experienced, can lead to a broader and more astute idea of what effective participation means in the evaluation context.
More broadly, with regard to the document as a whole, a periodic revision and update is planned and, in the meantime, practical experiences of applying it, and conceptual aspects that enable depth and precision, suggested guidelines and definitions will be collected and systematised. Thus, a common guiding framework will be built through participation and dialogue that is key for the consolidation of evaluation culture and professional practice in Latin America.
For an introduction to the Standards, you can watch the following animated video.
[1] The groups and their coordinators were: Gender and Human Rights: Pablo Rodríguez Billela; Indigenous Peoples and Evaluation from the South: Andrea Peroni; Academics and Professional Competence: Luis Soberón; Youth: Andrea María Wherle and Sergio Martinic.
[2] ReLAC-Standards Working Group. Round Table Discussion July – December 2019. Global systematisation of the contributions of the meetings. Prepared by Luis Soberón (MSc)
[3] The following people make up the working group: Fabiola Amariles (Colombia), Marcia Itzel Checa (Mexico), Celeste Ghiano (Argentina), Ana Luisa Guzmán (Costa Rica), Sergio Martinic (Chile), Andrea Peroni (Chile), Luis Soberón (Peru) and Andrea María Wehrle (Paraguay).
[4] The summary workshop and matrix were created by Emma Rotondo (Peru)