by Andrés Peregalli

Tell me how you evaluate and I will tell you what kind of education you promote

The complex world that we inhabit values measurements and their virtues, often to an excessive degree. We thus live with the constant pressure to capture and make sense of learning that is taking place among individuals, groups and institutions.

Within such a context, the evidence that results from an evaluation instrument will be used as a proxy. In the best-case scenario, it is a solid and robust proxy. I once heard it said that evaluation in a university setting involves one person who is terrified (the student) and another who is bored (the lecturer). While all images are unfair to a point, this characterisation (still very common) is useful for identifying concepts/representations that allude to ways of understanding education, learning, teaching, evaluation and participation, and the effects they have on those who experience them. To put it another way: tell me how (what, why, when, who, where…) you evaluate and I will tell you what kind of education you promote.

This post describes a participatory evaluation experience in an undergraduate university setting. More specifically, it describes a Theoretical-Practical Seminar (TPS) within the department of Philosophy and Theology at the Catholic University of Argentina (UCA) that has been running for 10 years and is based on the Service-Learning (SL) pedagogy. The experience exemplifies a more profound epistemological and evaluative approach and another way of understanding teaching and learning. These political-pedagogical and didactic foundations are consistent with the educational model we are striving to pursue: a process of humanisation centred around the people who teach and learn. In this approach, there is no room for terror or boredom. Rather, the focus is on agency so that learning can flourish, be enjoyed and celebrated.

Evaluation and participation: an unusual pairing

The pairing of evaluation/participation, which may appear to be a mismatch and is rejected by many, invites us to look at the shape and depth of the instruments we use, and even to rethink education. We understand evaluation as a complex, political field (of power), in dispute, where diverse traditions and approaches are evident. Emerging perspectives, such as participatory evaluation, seem to be making inroads in higher education, though less so than expected. Within this framework, we face the challenge of creating conditions that make evaluation fairer and more viable. In keeping with the political-pedagogical dream that inspires us, brings transformation and transforms us, this type of evaluation is an opportunity for learning.

To frame this approach, we will base our understanding of evaluation on the ideas of Duro & Nirenberg (2018) and say that it is a programmed activity that explores and seeks to understand the value of something and reflects on it with the intention of grasping the complexity of the teaching and learning processes involved (the being, doing, knowing; the head, the heart, and the hands, holistically).

We conceive evaluation to involve active and extensive participation of the actors, and as an ongoing process with specific moments of feedback, reflection and self-evaluation, from a collective viewpoint. Evaluation occurs through the following systematic procedures: data collection, analysis and interpretation of information; comparisons against defined parameters; making informed and communicable evaluative judgements about processes and results. It involves developing the ability to ‘learn to always be learning’, generating a culture of evaluation and making visible thinking drawn from different inputs (Narrative, Quantitative/Logical, Philosophical, Experiential, Collaborative).

For the case at hand (TPS), the reason for the evaluation is that we need to pinpoint, with quantitative and qualitative information, the complex dynamics involved in processes that are intended to achieve holistic education. Our intention is to improve ourselves, what we do and what we know, to contribute to knowledge generation, make better evidence-based decisions, formulate recommendations to improve action, learn from all those involved and provide transparency in management. In this process, it is key to understand the concept of participation as the ability to get involved, take part, make decisions that lead to personal transformation and take responsibility for learning, because through the roles that each person plays in education, we all learn and we can all teach.

TPS Social Issues and Social Commitment for a Culture of Engagement

The TPS is an Inclusive Learning Experience, referred to as EFI (from the Spanish) by the UCA to curricular areas that implement SL pedagogy. It is worth mentioning three programmatic features of the pedagogy: it is a compulsory part of the curriculum, it is student-led, and it seeks to solve social problems jointly with the community (Tapia, 2002). Specifically, participation is understood as a principle, a practice and a process of tangible protagonism. It is a way of forming personal experience and coexistence in dignity and solidarity (Cussianovich, 2003).

The TPS has been organised every four months since 2014. It is interdisciplinary and inter-faculty in nature and is run by the Philosophy and Theology department. Over the past 10 years, it has involved 20 cohorts and more than 1,000 students from 15 degree programmes and 20 social organisations (community partners). The TPS offers students a theoretical/practical way of learning. The course provides them with content that enables them to approach problems posed by social organisations with the aim of putting a culture of engagement into practice (co-designing, co-implementing, co-evaluating) (Francisco, 2020). Together, they seek concrete and real solutions by designing and implementing an SL project. The TPS aims to strengthen the academic, personal and professional development of university students, encouraging the development of transformational spirituality. The epistemic approach that underpins it refers to the complexity involved in recognising systems and sub-systems that produce knowledge in non-traditional or largely unrecognised spaces (for example, social organisations and partners). Thus, ‘learning does not take place only at the university, or from professors’, but also through participation, engagement and encounters with realities and experiences that teach us to be professional experts in humanity. In this process, the recording and systematisation of information and joint reflection encourage everyone to become involved in an ongoing evaluation that seeks improvements.

SL projects are designed using a specific planning tool: the roadmap, which anticipates the formulation of learning objectives and partnership objectives (Bridi & Puglisi, 2022). The quality of learning is evaluated in relation to: the extent to which the approach taken to tackle the social issue is linked to subject content; the way interdisciplinary content is integrated; the degree to which skills and values are applied; new knowledge acquisition; and the ability to provide feedback and self-evaluation. The quality of the service is evaluated in relation to: the way it promotes cultural recognition and demands, the frequency of activities, community satisfaction, the creation of networks between institutions and local organisations, the sustainability of the proposal and achievements in terms of medium and long term social change objectives. More specifically, the roadmap envisages five stages (motivation, diagnosis, design and planning, implementation and completion) and three crosscutting processes (a: reflection, b: recording, systematisation and communication, and c: evaluation). In SL projects, the concept of horizontal solidarity is important. In other words, it focuses on the recognition of the other and in the process, recognising them not merely as ‘beneficiaries’ but as ‘co-constructors’ of a shared project. Participatory evaluation and active participation are strongly related to the SL pedagogy, which should be reflected in the design of specific evaluation instruments when carrying out these types of evaluation.

As this text addresses the intersection between evaluation and participation, the following table illustrates an example of how this interrelation takes place in this specific experience. We have opted to illustrate this issue according to three ‘classic’ evaluation moments in a social intervention: before (design), during (monitoring) and after (impacts). In the example of the evaluation/participation link, we focus on: the participants (subjects), the object of the evaluation at each stage (what), the techniques and instruments used (how), and the formal evaluation stages involved (when). This is evidence of a participatory evaluation approach in which each actor involved finds the opportunity to participate while deploying and developing (learning) their capacity to do so.

Evaluating according to a participatory approach means learning together, with clear, methodical actions and criteria that are specific and consistent with the educational approach that underpins the practice. This understanding of evaluation and participation, which fosters a culture of evaluation and creates a community of learning, partly explains the outcomes of this experience (TPS) and the experiences of the students and community partners. The following quotes express effects generated by this way of educating. In terms of their experience and its ‘impact’ on their training, the students highlight that:

‘The theoretical-practical seminar was a wonderful experience. It would be good to have opportunities like this in earlier years (…) It was one of the most enriching experiences I have had as a professional (…). A space that trains professionals who know how to act with awareness, generating the belief that if there is no global, deconstructed and broad vision of social contexts that are alien to us, we will not be able to propose anything in the future. We seek to use our faith to work so that people, groups and societies become more sensitive and mature, more just and caring (…). I had to think, create and develop a project for children in a centre where they go to play, yet it did much more in me than that. I learnt to let go of my privileged perspective and the belief that I knew ‘what was best’ for the children and listen to what the children were looking for. (…). It is a much more difficult task than others, it involves learning about experiences that the rest of society ignores. However, the centre achieves this and the ‘Seminar on social problems’ encourages us to make a difference’ (UCA-TPS, Student, 2022).

The outlook and perspective of the community partners reflects the construction of a culture of engagement and co-management as collaborative knowledge building:

It is fundamental that educational institutions such as this one train students in humanity (…) from an interdisciplinary perspective (…) What was central to the first contact, and which must be related to this culture of engagement that you refer to, was that you didn’t come with a fixed proposal, but rather listened carefully from the very beginning to what we said. This meant we could tell you ‘OK, this is the theory but this is the reality that we, as community-based social actors, experience in different spheres’ (…). And so this very first exchange set in motion the idea of a diagnostic stage; this brought together people who had not gathered together before and that was enriching (…) we have to continue to work on developing this critical perspective every day (…) this question of the process and being perceptively critical and intuitive is what is going to enhance each project. (…) What we take away from this project is that we managed to create a process of listening, of empathy, that the students did not come with the answers. Instead, the issue was dealt with from a collective viewpoint and through collaborative discussion. From there, the concepts and activities were under constant review. That is what nourishes us the most as an institution, and we believe in the importance of generating these links and sustaining them (…) I can only express my gratitude, because they really took the work seriously and this exchange took place with the freedom to say yes, no, like this, like that’. (UCA, TPS, Community partner, 2022)

More participatory evaluation for more and better education

The TPS experience encourages us to dream of universities that are ‘more network and less bubble’, ‘more on the ground and less behind a desk’. The EFIs, and the way they interpret evaluation and participation, challenge traditional models of understanding the roles of teaching, dissemination and research. This implies envisaging/developing the teaching role according to the training model in the framework of institutions that co-manage the experience and specific institutional capacities (skills) that generate conditions to promote participatory evaluation approaches. It also invites discussion on certain prejudices and tensions (for example, ‘These experiences are highly complex and therefore cannot be carried out’; ‘these experiences are of lower academic quality than a master class’; ‘it is not the role of the university to engage in social action’; ‘students cannot participate in the construction of evaluation criteria’; ‘community partners cannot be evaluators’; ‘evaluation is only carried out in university classrooms’, etc.).

The implementation of the TPS shows that it is possible to generate profound learning experiences when tackling social problems in order to build a fairer and more inclusive world, and to provide more and better training for future professionals and today’s citizens. The inclusion of everyone in this world is not the task of a few disciplines, areas or individuals. Rather, it is the responsibility of everyone; it is built from institutional cultures that are inclusive and that evaluate and improve. The strengthening of evaluative and participatory capacities in social engagement is a political and strategic activity that contributes to:

• • • making visible what we do, who we are, and what we know.
    • researching and producing knowledge,
    • building a culture of evaluation, and
    • democratising knowledge.

The results show that the TPS is a significant training experience that contributes to building our shared home and the dream of including everyone, underpinned by a holistic higher education and the central aspect of co-management that enables institutional (ad intra) and inter-institutional (ad extra) learning. These include creating a new way of knowing-doing-being at the university, a culture of engagement. By going out to the (social and existential) margins, the university encounters itself, its identity and mission, and its deepest motivations. This does not mean that it renounces its functions, but rather that it undergoes a subjective and organisational metanoia (deep conversion).

There is still a long way to go before evaluation and participation become part of the landscape, practices and institutional culture of universities; but these experiences are not isolated cases. They illustrate extensive movements that encourage new and better ways of educating; they extend the boundaries of what is thought to be possible (unprecedentedly viable: Freire) and they move towards the celebration of shared learning without fear and without boredom.

---

REFERENCES

Bridi G., y Puglisi S. (2022) Aprendizaje y Servicio Solidario para una Educación Intercultural. Manual para docentes, Buenos Aires, CLAYSS.

Cussiánovich Villarán, A. (2003) Protagonismo, participación y ciudadanía como componente de la educación y ejercicio de los derechos de la infancia, en Historia del pensamiento social sobre la infancia. Lima: Universidad Nacional Mayor de San Marcos. Fondo Editorial de la Facultad de Ciencias Sociales.

Duro, E. y O. Nirenberg (2018) Instrumento de Autoevaluación de la Calidad Educativa (IACE), Buenos Aires, CEADEL-UNICEF.

Papa Francisco (2020). Carta encíclica Fratelli tutti sobre la fraternidad y la amistad social. 3 de octubre de 2020.

Tapia, M. N. (2002) El aprendizaje-servicio en América Latina, en CLAYSS Centro Latinoamericano de Aprendizaje y Servicio Solidario. Aprender sirve, servir enseña. Buenos Aires.

UCA (2022) STP. Seminario Teórico-Práctico Problemáticas Sociales y Compromiso Social, Registros de campo, Buenos Aires, Mimeo.