As a PhD student at the Department of Computer Science at the University of Copenhagen, Theresia Veronika Rampisela has, since 2022, researched fairness in recommender systems as part of the ADD project. Under the supervision of Christina Lioma, she has examined bias and developed evaluation methods – and concludes that several metrics suffer from serious shortcomings.
In this exit interview Theresia Veronika Rampisela provides both professional insight and personal reflections on her project and her time at ADD.
How do we actually measure fairness in algorithms?
This question has been at the core of Theresia Veronika Rampisela’s research as a PhD student in the Algorithms, Data and Democracy (ADD) project. Since joining the project in June 2022, she has worked on investigating fairness in recommender systems – the algorithms that determine what we see on platforms such as social media, streaming services, and online marketplaces.
As she ends her time in the ADD project, she is left with a clear message: fairness is not just a matter of intention, but very much a question of how we measure it.
“Some of the fairness metrics we have studied suffer from serious flaws.” – Theresia Veronica Rampisela
When the measurements become the problem
Theresia’s research has focused on examining so-called fairness metrics – the mathematical methods used to assess whether a recommender system is fair.
The starting point was that many different ways of measuring fairness already exist. But instead of taking them for granted, she and her colleagues asked a more fundamental question: do they actually work?
“We investigate documentation of existing metrics of fairness, because there are many of those. Then we look into what the flaws are, not just in the equations of these measures but also what happens to or how they react when they are used in practice.”
It quickly became clear that the problems are more extensive than one might expect.
Some fairness metrics are constructed in such a way that they can never reach their highest or lowest possible values. This means the results can be misleading:
“Let’s say the score is 0.89 out of 1, so you will think there is room for improvement because it is not 1 yet, but theoretically it is not possible to reach 1.” – Theresia Veronika Rampisela
Other metrics tend to systematically produce high fairness scores. This makes it possible, in practice, to choose a metric that makes a system appear more fair than it actually is.
Fairness for whom?
A key insight into Theresia’s research is that fairness is not a single, fixed concept.
Recommender systems are what she describes as multi-sided platforms with multiple stakeholders:
“Recommender systems are multisided platforms – there is the provider and the consumer.”
For users, fairness typically means receiving equally good recommendations – either as individuals or across groups. For providers, however, it is about visibility:
“Provider fairness usually relates more to the exposure that their content or product receives”
This means that fairness is always a matter of perspective – and of which interests are prioritized.
From critique to concrete tools
While her research begins with a critique of existing methods, it also points forward.
As part of her PhD, conducted under the supervision of Professor and ADD Co-PI, Christina Lioma, Theresia has contributed to developing concrete solutions.
“We came up with some practical guidelines on how we should use this set of fairness metrics and when to use them.”
In addition, the research group has developed new fairness metrics that address several of the limitations identified in existing approaches. Both tools and code have been made publicly available, allowing researchers and practitioners alike to apply them.
International recognition
The work has already gained recognition within the research community, where Theresia and her colleagues have received an award for their findings.
In October 2024, their article “Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study” was awarded a “Best Journal Paper” prize by Women in RecSys. The award recognises women-authored journal papers that stand out for their innovativeness and scientific rigor.
The prize was presented at the 18th ACM Conference on Recommender Systems (RecSys 2024), a leading international forum for research, systems, and methods in the field of recommender systems.
An important message for society
Although Theresia’s research is technical, it carries clear societal implications.
A central message is that we need to pay closer attention to how fairness is actually operationalized.
If the methods we use to measure fairness are flawed, we risk making decisions on the wrong basis – and accepting systems as “fair” when they are not.
“Due to the existence of different measures of fairness, you can use the metrics that is the most convenient for you.”
In this way, fairness becomes not only a technical issue, but also a matter of responsibility.
“It is not only about making systems fair, but about understanding what fairness actually means” – Theresia Veronika Rampisela
Interdisciplinarity as a strength
One of the key takeaways Theresia brings from ADD is the project’s interdisciplinary collaboration.
This is the first time she has worked closely with researchers from different fields, which has given her new perspectives on her own work:
“It is nice to see that there are different perspectives of what fairness means and what it means in different contexts.”
What’s next?
After leaving ADD, Theresia continues her research career as a postdoctoral researcher at the Department of Communication at the University of Copenhagen.
Here, she will work on generative AI in the humanities, focusing both on how humanities can use the technology and how it can be designed to support humanistic approaches.
The question of fairness, however, does not disappear.
On the contrary, her research suggests that it will only become more relevant as algorithms and AI play an increasingly prominent role in society.
“It is not only about making systems fair, but about understanding what fairness actually means,” she concludes.
About
- Name: Theresia Veronika Rampisela
- Background: PhD in Computer Science from the University of Copenhagen, specialising in fairness in recommender systems; affiliated with the Machine Learning section at the Department of Computer Science
- Role in ADD: PhD student (2022–2025) under the supervision of Professor and Co-PI Christina Lioma; worked on bias and the evaluation of fairness in recommender systems
- Fun fact: Theresia moved more than 10,000 km away from her home country to pursue a PhD in Denmark – and is a hobbyist flutist who plays in a local community orchestra in Copenhagen