Franziska Sörgel, Nora Weinberger, Julia Hahn, Christine Milchram and Maria Maia
This blog post is based on the Evidence & Policy article, ‘Assessing the effectiveness of citizen participation: the development of an impact scheme’.
Citizen participation has become central to research policy, yet we rarely ask the crucial follow-up question: what difference does it actually make? In our recent Evidence & Policy article, we propose an impact scheme that helps to move participation from a well-intentioned ritual to a practice with measurable, meaningful effects.
The last decade has seen an explosion of participatory formats designed to gather citizen and stakeholder feedback on science and innovation policy. From citizens’ assemblies to co-creation workshops, public dialogue has become the new punctuation mark in research agendas and beyond. Nevertheless, a fundamental problem persists: we lack systematic ways to measure whether these processes genuinely influence research priorities or merely provide a democratic façade with little real impact. This gap matters enormously for both research institutions that invest resources in participation and for citizens who provide their time and expertise.
At the Karlsruhe Institute of Technology (KIT), we developed a structured approach to answer this question precisely. Our Citizens’ Dialogues – annual one-day events that bring scientists, citizens and university management into conversation – have long been framed as exercises in ‘listening to society’. However, listening is not the same as learning. We, therefore, designed an impact scheme to trace what happens when lay knowledge meets institutional power: what changes immediately, what evolves gradually, and what endures.
Our framework examines impact along two axes: over time (immediate, gradual, continual) and across actor groups (citizens, researchers, management). This double lens allows us to see how effects ripple differently through each layer. For example, citizens might leave an event with fresh insights or greater trust in science. At the same time, researchers begin questioning their assumptions months later, and management only adapts the strategy a year later. Impact, in this sense, is less a single outcome than an unfolding story. Developing the scheme was, in itself, an exercise in humility.
We discovered that most evaluations of participation often rely on what can be easily counted, such as attendance numbers, satisfaction ratings, and the occasional enthusiastic quote. However, the more interesting effects, such as shifts in perception, authority, or the subtle redistribution of who gets to define a ‘relevant’ research problem, require a different gaze. Our mixed-methods approach combines surveys, interviews, ethnographic observation, and, crucially, patient reflection. In short, a spreadsheet cannot capture the politics of a conversation.
Testing the scheme in KIT’s 2023 Citizens’ Dialogue, Rewarding Climate Protection – But How?! [translated by the authors], offered instructive results. We found that the dialogue attracted participants who were well educated, already interested in sustainability, predominantly male, and already connected to the university. In other words, the dialogue primarily attracted participants already engaged with sustainability issues and connected to the university, highlighting persistent challenges around inclusive recruitment. Nevertheless, the dialogues still revealed how thoughtful moderation, accessible language, and visible institutional commitment (say, a vice president who listens rather than lectures) can shift dynamics and make participation more than symbolic. Measuring impact, in this context, also means recognising its limits.
Our aim is not to impose another set of performance indicators on participation, but to support institutions in actually achieving the impact they aim for. The scheme is designed to be adaptable and flexible across various contexts, from local energy debates to transdisciplinary research projects. It encourages organisers to think beyond outputs and towards effects – to see citizen participation not as a photo opportunity but as a slow, sometimes irritating, always revealing process of mutual learning. We do not claim to have solved the puzzle of ‘impact’ as the term itself resists neat definition. Its edges are intentionally fuzzy. However, by structuring how we look for impact, we begin to see participation differently – as both a scientific and ethical practice.
By structuring how we look for impact, the scheme enables organisations to take their democratic commitments seriously – moving from aspirational statements to evidence-based assessment and continuous improvement.
Image credit: Silke Walz / KIT.
Franziska Sörgel is a Post-Doc researcher at KIT-ITAS and co-lead of the project, ‘Participatory Procedures and Processes in Research Organizations’. Her work focuses on Science and Technology Studies, Impact Assessment as well as Sociology of emotions and emotional evaluation.
Julia Hahn is a senior researcher at KIT-ITAS and co-lead of the project, ‘Participatory Procedures and Processes in Research Organizations’. Her interests include global aspects of Technology Assessment, policy engagement and participatory methods.
Nora Weinberger is a senior researcher at KIT-ITAS focusing on participatory technology assessment, including how to make public engagement more inclusive and accessible.
Christine Milchram is member of the scientific staff at KIT-ITAS and at the Office of Technology Assessment at the German Bundestag. Her work includes parliamentary technology assessment with an emphasis on energy and mobility systems and research on fairness in transdisciplinary participation processes.
Maria Maia is a senior researcher at KIT-ITAS. With a PhD in Technology Assessment and a background in Health Sciences, her research interests include citizen and stakeholders’ engagement and their perspectives on new technologies, concepts and practices of Technology Assessment, namely in health, as well as scientific developments and practices of responsible innovation.
Read the original research in Evidence & Policy:
Sörgel, F. Weinberger, N. Hahn, J. Milchram, C. & Maia, M. (2025). Assessing the effectiveness of citizen participation: the development of an impact scheme. Evidence & Policy. DOI: 10.1332/17442648Y2025D000000066. OPEN ACCESS
If you enjoyed this blog post, you may also be interested in reading:
Optimising teamworking processes in an ongoing research consortium: a qualitative study OPEN ACCESS
The influence of public policy and administration expertise on policy: an empirical study
Disclaimer: The views and opinions expressed on this blog site are solely those of the original blog post authors and other contributors. These views and opinions do not necessarily represent those of the Policy Press and/or any/all contributors to this site.