Can digital approaches to monitoring, evaluation, research and learning support participation?

by Linda Raftree 

Back in the 2010s, the world was buzzing about the potential for digital tools to revolutionize and democratize most everything – SMS for community feedback loops, mobile phones for citizen journalism, open data to improve transparency and accountability, social media platforms for people to make themselves heard without needing to be part of the elite, and networks so that social movements could organize in resilient ways at very low cost.

A decade later, there have been enormous shifts and changes that have benefited participation. However as often happens, these spaces have also been appropriated and taken advantage of by those who are seeking to maintain their status and power.

In the monitoring, evaluation, research, and learning (MERL) space, the role of digital technologies and approaches (“MERL Tech”) became clear early on, spurred by the simple mobile phone. The dream was that people who were not previously reachable could finally be a part of conversations and could more easily and directly participate in development dialogues, in consultations and decision-making, and in feedback on what was and was not working. At the level of organizations, digital data collection, storage, analysis, and use showed great potential to improve efficiency and accountability. By the mid 2010s, evaluators had started dreaming about the potential for big data to allow for prediction of behaviors based on patterns in data. The idea was that this could enable more efficient and effective funding and programming, because we would know from the data what interventions would work and for whom and could thus invest in areas where there would be the biggest impact.

At the same time, the world began waking up to all the ways that data was being extracted by big tech companies. People became more alert to the dangers of personal and sensitive data – it could be easily used to surveil, manipulate, and do harm. And of course, along came COVID-19 in 2020, forcing a rapid shift to the use of digital tools and approaches because remote programming and remote MERL were the only options in some contexts where the possibility of spreading the virus was a huge risk and lock downs and quarantines were restrictive. Now the amounts of personal and sensitive data being collected were even greater – and our dependence on digital tools and platforms even more cemented.

During this time, some evaluators have been questioning the ways that digital approaches are changing the field of MERL. As remote methods became the norm, discussions about what is gained and lost with digital only and digital heavy approaches also became more common. Issues like exclusion, difficulties in building trust and relationships with communities, loss of contextual awareness, and concerns about privacy and protection were raised. These concerns have led to questions about data quality in some cases – if we have little trust, poor understanding of context, and little assurance that people are safe when we engage remotely with them, how does this affect the quality of data we are collecting?

At the same time, discussions on ‘localization’ and ‘decolonization’ have become front and center in development, humanitarian, and evaluation spaces. COVID realities opened these discussions in philosophical as well as practical ways. International development agencies and evaluation firms located outside of ‘program countries’ suddenly realized just how reliant they were on local organizations and local staff. This is not to mention that the perceived gap in ‘development’ closed somewhat as COVID affected all countries in the world, with many of the so-called “developed” countries doing a poor job of managing it.

In this context of the COVID pandemic, Data Innovators and MERL Tech conducted a landscape review of technology-enabled MERL approaches and solutions in African contexts. The study was supported by the Mastercard Foundation, who have a primary focus on the African continent, youth, indigenous ways of knowledge, and future-focused impact approaches. In our landscape study, we looked at what types of digital MERL tools and approaches are being used by African-led and -based organizations and for what purposes, including whether they were enabling community-centric and youth-led processes, and how they were considering aspects like data protection. We spoke with a wide range of African evaluators NGOs, technology companies and others to better understand the benefits, risks, barriers, and challenges they experienced in developing and designing responsible MERL Tech solutions and approaches. We identified almost 200 entities that were in some way focusing on “MERL Tech” in the African context. Understanding more about the landscape, and what promotes and hinders home-grown technology-enabled MERL tools and approaches will provide insights into where investments, support, and capacity enhancements could be useful for organizations and tech developers on the continent.

While “MERL Tech” has become common around the world. Much of the focus on MERL Tech has been in English speaking contexts. I’m curious how the above challenges and concerns are being framed and addressed by evaluation communities and associations in the Americas.

• Are digital MERL tools and approaches supporting improved participation or contributing to exclusion?
• Are there similar questions about data quality, trust, and loss of contextual awareness?
• What are the main barriers and challenges in the Latin American context?

I hope to learn more from the EvalParticipativa community about what kinds of MERL Tech are being used; how well they are working, for whom and for what; and to share learning across various contexts and experiences.

---