Dear friends and colleagues, we are still receiving responses to our open call for meaningful lessons in participatory evaluation. This time from Alexandra Santillana (senior evaluator at Global Affairs Canada) together with Fabiola Amariles and Ana Isabel Arenas (consultants at Learning for Impact). Below they share with us a rich reflection on feminist and participatory evaluation taken from their experience working with rural community development projects in Colombia. Many thanks for your contributions and let’s keep learning!
THE EXPERIENCE
This experience, which took place between May and November 2018, was not strictly speaking a participatory evaluation, but provides reflection on some lessons learned from applying principles from feminist evaluation and participatory methods in rural development community projects. This pilot was run within the framework of a conventional evaluation led by the Canadian Ministry of Global Affairs (GAC).
Inspired by Canada’s Feminist International Assistance Policy (FIAP)¹ and taking advantage of an evaluation already underway as part of the Canadian cooperation program in Colombia, a mixed team of evaluators was formed including Canadian and Colombian specialists to design and implement the pilot.
The evaluation team identified some participatory methods for collecting the data (the Most Significant Change Technique and Outcome Mapping) with a gender focus to enable participation by the project beneficiaries and related organisations. The team organised workshops and reflective debates with stakeholders involved in the projects. The workshops, differentiated by gender, focused on the changes taking place in the communities thanks to the project, especially those related to the care economy, financial independence and the empowerment of women and girls.
After collecting and analysing the data, participatory workshops were organised to validate the findings as the first stage in sharing back the information with those involved. These workshops were facilitated by local evaluators and allowed stakeholders to reflect on the findings and contribute to developing learning products. The content and format of these products was defined by the same implementation agencies and project participants. Infographics were created for the implementation agencies whilst the project participants preferred a teaching pamphlet format. 
LESSONS
The team was made up of members from both the north and south and this also contributed to ensuring that the evaluation took into consideration gender, local power attitudes and dynamics in addition to institutional biases and influences.
Other factors that contributed to this learning included the open and receptive attitude of the Canadian evaluation team members who took on the role of learner, delegating the feminist reflection design as a contribution to the evaluation and taking on board cultural recommendations with regard to organising and running the pilot experience.
As feminist evaluation requires that the project is highly participatory and inclusive, a key factor to achieve this was to insist that the implementing agencies invite “stakeholders who are difficult to access”. This meant that they needed to facilitate transport and accommodation so that the women and men from the project’s target areas could travel from their different rural areas to participate in the workshops and reflective debates in the city. The result was that a far greater number of participants were gathered.
Lessons learned from designing the pilot exercise
Feminist evaluation critically examines those who participate in the evaluations, how they evaluate and how their contributions are valued.
Designing this exercise revealed some lessons that the evaluation team and Canadian cooperation evaluative processes could take away:
-
- Including the project management group (implementing agency) as a main contributor for collecting data at the beginning and end of the process meant that this group grew in empathy and more readily owned the results. The group also recognised that the feminist reflections had taught them to strengthen strategies and results by seeking gender equity and equality in the project.
-
- Validating findings and feeding back information. The exercise of validating findings made it easier for those who contributed information to the evaluation to own the findings when feedback was given.
Lessons taken from the tools, participatory process methods
-
- Separating men and women in the focus groups for small group work and for sharing results back to the larger group guarantees that all opinions are expressed. It has been demonstrated that when reflection is carried out in mixed small groups, women tend to participate much less and end up accepting the opinion of the men in the group. These methodologies contribute to equal opportunities for both women and men to express their opinions so that women’s voices are taken into account.
-
- Using “Outcomes Mapping (OM)”and “the Most Significant Change (MSC)” tools and protocols was useful for detecting changes that resulted from the project, whether intentional or not and as advocacy tools. For example, the MSC allowed us to detect more accurately qualitative changes that were taking place in the women’s lives. Likewise, stakeholder analysis using the OM tool made it possible to visualise impact strategies with these stakeholders with regard to gender equity and equality and women’s rights in the region’s rural sector.
-
- Learning products created, which describe the feminist evaluation approach and findings and which were co-created by the project participants. The resulting leaflets and posters went on to become tools used later in advocacy work for gender issues in the project’s area of influence.
RECOMMENDATIONS
We would like to highlight a few aspects which are valid for future evaluations.
-
-
- Generating trust and fostering the conditions to achieve institutional and community political will in the evaluation process, facilitates the gathering of direct information, and the identification of both positive aspects and difficulties presented in the execution of the project and its overcoming.
-
- Direct dialogue and opinion exchange with the various process agents, management and operational institutional staff, both those responsible for the process and participate in specific areas and the women and men who participate and express their own opinions.
-
- Specify the most significant change or transformation with regard to gender to inspire the developed project. In addition to the aforementioned, it is worth highlighting the significant changes that occur regarding the sexual division of labour, financial independence for the women and their greater participation in decision-making moments and a greater awareness among the men of the need to transform relationships between women and men.
[1] In recent years, Canadian Cooperation has been applying and promoting its Feminist International Assistance Policy (FIAP) with the object of eradicating poverty and constructing a more peaceful, inclusive and prosperous world. Canada firmly believes that promoting gender equality and empowering women and girls is the most efficient approach to achieve this objective.