by Gabriela Munares & Rebeca Lucas
Cities around the world, including many in Latin America, are incorporating artificial intelligence solutions into their urban services: video surveillance and facial recognition systems for public safety, traffic management, citizen services, waste collection and tourism promotion.
In many cases, this trend is advancing faster than the mechanisms for evaluating its social consequences. A good example is the case of security: cities such as Lima, Buenos Aires and Mexico City have deployed AI systems for the surveillance of public spaces without any systematic processes in place to evaluate who is disproportionately affected, what biases may be embedded in their algorithms, or how the rights of those being monitored are protected.
Against this backdrop, there is an urgent need for rigorous, participatory methodologies to evaluate the social impact of these solutions. In this context, we share a review of the article “Participatory Methodology to Assess the Impacts of Artificial Intelligence in urban contexts” (Munares, Lucas, et al., 2026), published in the Journal of Policy Evaluation and developed as part of the European CITCOM.ai project. CITCOM.ai is a European testing and experimentation initiative for AI solutions in smart cities, offering real and virtual environments to test AI and robotics models under real urban conditions.
The article proposes a methodology structured in three phases and eight steps to evaluate the social impact of AI solutions in urban services, with a participatory approach as its backbone. The methodology was validated through a workshop with 26 participants in Valencia, Spain, focusing on three use cases: mobility, tourism and waste management. Its aim is not to replace existing frameworks for the ethical evaluation of AI, but to integrate the best of them — in particular the European Commission’s ALTAI framework, and the impact evaluation methodologies of the Information Society Platform (ECP) and UNESCO — into a process that places the affected stakeholders at the centre of identifying impacts and constructing monitoring indicators.
Why no single stakeholder can see the whole picture on their own
The first challenge addressed by the methodology is fundamental: AI solutions largely function as black boxes. Most of us who use them on a daily basis do not know how they are built, what data they were trained on, or what mechanisms they trigger when we use them. Their effects span dimensions we often cannot even imagine: biases in decision-making, silent transformations in workflows, risks to vulnerable groups that fall outside the usual focus of analysis.
The technical expert understands the architectures and risk vectors in cybersecurity, but may be unaware of how a tool alters a local authority technician’s working experience or affects a community’s trust in public services. That technician, in turn, clearly perceives what has changed in their day-to-day work, but lacks the necessary concepts to articulate risks of bias or privacy. The puzzle can only be completed by bringing together different perspectives. Integrating diverse profiles into the evaluation process — technical managers, decision-makers, citizens, representatives of vulnerable groups, NGOs — is not merely one methodological option among others: it is a prerequisite for the evaluation to truly capture the complexity of the phenomenon.
The ALTAI framework as a guide for collective reflection
To ensure that this diversity of perspectives does not result in a scattered conversation, the methodology uses the European Commission’s ALTAI framework ( Assessment List for Trustworthy AI ) as a cross-cutting conceptual framework. ALTAI organises the criteria for trust and ethics in AI into seven dimensions: human agency and oversight, technical robustness and safety, privacy and data governance, transparency and accountability, diversity and non-discrimination, social well-being, and environmental well-being. This framework is not merely a bibliographic reference: it structures the identification of impacts both in the internal exploration prior to the workshop and during the participatory process with stakeholders. Its function is to compel a systematic examination of all ethical aspects that must be addressed, including those that are not immediately apparent to any particular stakeholder.
The tools supporting the process
The methodology is supported by a series of tools designed to accompany each stage: the structured description of the AI solution and its context, the identification of key stakeholders, an impact identification matrix organised according to the ALTAI principles, and a catalogue of monitoring indicators (KPIs) that enables the systematic monitoring of collectively identified impacts. These tools are not an end in themselves, but rather a framework that facilitates the participation of diverse stakeholders and ensures that results are comparable, cumulative and agreed upon.
What we learnt from the pilot — and what this implies for other contexts
The experience in Valencia confirmed that the greatest challenge is not methodological but relational: securing the participation of the right people, particularly from civil society organisations and representatives of vulnerable groups, requires a dedicated effort that goes far beyond simply sending out invitations. It involves first identifying which stakeholders have a relevant perspective on the type of impact to be evaluated, understanding what conditions they need to participate meaningfully — timing, formats, venues, languages — and designing the activity based on these realities. This challenge is probably even more pronounced in Latin American contexts, where the digital divide and inequality in access to urban services add further layers of difficulty — but where, precisely for that reason, the need for methodologies such as this is more urgent.
It is also important to be honest about what embarking on a process of this kind entails: it requires organisations adopting it to have a genuine willingness to share control of the process, to learn from perspectives that may be uncomfortable, and to maintain a commitment to participation even when time and resource pressures push in the opposite direction.
In that sense, the methodology we propose is not intended to be a rigid template, but rather a compass: it offers guidance and structure, but the exact direction each organisation takes will depend on the terrain before them — and, above all, on the genuine willingness to navigate it alongside those who have the most to say about the impact of AI on their lives.
