by Luisa Graffigna

While I have always found Participatory Evaluation attractive, until recently I viewed it as irrelevant to my work. That changed when I listened to Marina Apgar’s presentation, Evaluación Participativa y rigor en el marco de una evaluación transformadora [Participatory Evaluation and rigour in the context of a transformative evaluation, only available in Spanish] at the EvalParticipativa international seminar in December 2021. Her talk helped me realise how the criteria of rigour in evaluation that she spoke of are echoed in a field with which I am more familiar: social research.

All of us carry with us our own set of lived experiences. In my case, my academic training as a sociologist and a period several decades ago of collective work, reflection and practice linked to processes of Popular Education -together with some of the people who coordinate EvalParticipativa today- contributed to my interest in, and understanding of, what Marina was saying. From this experience, the “participatory” part resonates with me, but my background positions me more firmly within the field of social research and it is from this position that I will share some thoughts about the ways these two processes, each with their own logic, converge and diverge.

To begin with, and viewing the two fields in historical context, it is my perception that evaluation works more closely with actors on the ground and with the organisations that work with them, while, for a long time -at least in my experience- social research has been more circumscribed to the academic field. More recently, these paths, which had seemed to be separate, have become intertwined in a way that has been mutually enriching. It is also worth mentioning that a current of thought has emerged within the field of social research over the last three or four decades that has promoted perspectives that move away from the long-held positivist assertion that there is only one scientific method. Here, I am referring in particular to the development of a range of qualitative research approaches in the social sciences and to profound discussions about the validity of the methods they employ. This has led to the development of other ways of doing research according to positions and a logic that differ from those governing the natural, or so-called “hard” sciences.

In my opinion, these points of convergence between “evaluation” and “research” do not occur at the general level but rather between certain approaches of the two fields that share a particular view of the social aspects. This entails the adoption of an epistemological stance on the construction of knowledge that focuses on intersubjectivity -the understanding of diverse realities, themselves dynamic and complex- and, of course, on the corresponding methodological strategies used to advance these processes. This convergence of perspectives implies:

• • • A holistic vision of complex and ever-changing realities and scenarios.
    • The inclusion of processes that are dynamic and interactive and that promote involvement.
    • Attentiveness to a range of actors and knowledge systems, a positive view of collective construction processes and the minimisation of formal hierarchies.
    • The creation of situated and context-dependent knowledge.
    • A recognition of the transformative intentionality of political processes.
    • Reflexivity vis à vis processes for reviewing the practices that have been developed.
    • Consideration of ethical aspects.

Building on these shared issues –each of which merits its own debate and reflection– further differentiations between participatory evaluation and social research are apparent. I would like to focus specifically on the question of objectives, as these constitute an area of particular divergence. From a research perspective, an intrinsic goal of “generating knowledge” for its own sake rather than to be “useful” is no mere detail. This can lead to discussion about the characteristics of the process, the type of knowledge generated, or its goals. By contrast, my understanding is that participatory evaluation focuses on assessing processes of intervention or transformation in different contexts. This makes it possible to identify different logical underpinnings of actions and processes.

Based on this (very) general framework, I would like to express some ideas about two points that both paths have integrated, albeit with their individual nuances: the “criteria of rigour”, on the one hand, and the “participatory” focus, on the other.

Criteria of rigour and quality

From the perspective of participatory evaluation, a particular concern for rigorous processes is reflected in the criteria mentioned by Marina Apgar: the ability to respond, usefulness in relation to the final use the results will be put to, credibility, critical reasoning, and transferability. It is clear that these criteria are, indeed, very pertinent to ensuring a deeper examination of the processes involved.

Likewise, from the perspective of social research, recent reviews of practice have raised questions about the validity of the processes. In order to differentiate their work from that based on conventional criteria, many authors have opted to refer to research “quality” and have established other principles that go beyond a focus on results in order to highlight each of the moments and decisions involved in the process.

It is particularly important to mention the criterion of reflexivity, as this makes it possible to examine the practices and positions used to conduct a process, whether for the purposes of research or evaluation. Reflexivity assumes that the once-prized “value neutrality” does not exist, recognising instead that we form a part of the contexts in which we become involved in pursuit of knowledge and, therefore, we act from a certain position that must be made explicit and taken into consideration at all times. At times, an ethical aspect also emerges, insofar as researchers and evaluators should keep a critical eye on the way their own values might impose/hide/modify the views expressed by the actors they are interested in, preventing them from being able to give meaning to (or deal honestly with) the things that emerge.

In this regard, both the criteria of rigour in participatory evaluation and of quality in social research go beyond conventional forms of validating results to analyse other aspects that support, lend coherence and give meaning to entire processes. Some of these are: the use of multiple methods, participant validation, intersubjectivity and, of course, reflexivity.

Participation

The need to look for other criteria to examine the validity of processes is also related to the consideration of multiple and diverse actors. Particularly in social research, it used to be standard practice for the research team to make all the decisions and guide the entire process, while other actors were traditionally treated as “informants” or recipients of results. Increasingly -and above all in the field of Participatory Action Research- stress is placed on a whole range of aspects associated with the collective construction of meaning during the entire evaluation or research process. These include the co-construction of knowledge; the recognition of diverse knowledge and know-how, much of which arises from lived experiences; and the flexibility of processes that emerge from the exchanges that occur.

Personally, I believe that further questions may be asked to facilitate deeper reflection on the kind and depth of involvement of the different actors. These include:

• • • How effectively are their different voices incorporated?
    • What power relations exist between them?
    • Do local actors only provide “data” or can they have an effective say in the evaluation/research team’s proposals (critical reasoning is important here)?
    • What happens when evaluators/researchers and local actors do not “look at” the process according to the same criteria, or even have opposing perspectives?

In qualitative research -particularly when it is developed using a participatory perspective- there is still a long way to go in terms of recognising the people involved in processes of knowledge generation, and reflecting on the ethical considerations that this entails.

To sum up, it might be said that “we are still walking” both in participatory evaluation and in social research. So far on this walk, we have found that the more critical approaches of the two fields have converged. At the same time, they have become increasingly complex, as reflections and questions have emerged on ethical matters associated with understanding the processes that occur, including the ways in which the different participants are involved in processes, the methods employed, and the final use to which the knowledge generated will be put.

---