De-centering the (Traditional) User: Multistakeholder Evaluation of Recommender Systems
Authors:
Robin Burke,
Gediminas Adomavicius,
Toine Bogers,
Tommaso Di Noia,
Dominik Kowald,
Julia Neidhardt,
Özlem Özgöbek,
Maria Soledad Pera,
Nava Tintarev,
Jürgen Ziegler
Abstract:
Multistakeholder recommender systems are those that account for the impacts and preferences of multiple groups of individuals, not just the end users receiving recommendations. Due to their complexity, these systems cannot be evaluated strictly by the overall utility of a single stakeholder, as is often the case of more mainstream recommender system applications. In this article, we focus our disc…
▽ More
Multistakeholder recommender systems are those that account for the impacts and preferences of multiple groups of individuals, not just the end users receiving recommendations. Due to their complexity, these systems cannot be evaluated strictly by the overall utility of a single stakeholder, as is often the case of more mainstream recommender system applications. In this article, we focus our discussion on the challenges of multistakeholder evaluation of recommender systems. We bring attention to the different aspects involved -- from the range of stakeholders involved (including but not limited to providers and consumers) to the values and specific goals of each relevant stakeholder. We discuss how to move from theoretical principles to practical implementation, providing specific use case examples. Finally, we outline open research directions for the RecSys community to explore. We aim to provide guidance to researchers and practitioners about incorporating these complex and domain-dependent issues of evaluation in the course of designing, developing, and researching applications with multistakeholder aspects.
△ Less
Submitted 22 April, 2025; v1 submitted 9 January, 2025;
originally announced January 2025.
Gender In Gender Out: A Closer Look at User Attributes in Context-Aware Recommendation
Authors:
Manel Slokom,
Özlem Özgöbek,
Martha Larson
Abstract:
This paper studies user attributes in light of current concerns in the recommender system community: diversity, coverage, calibration, and data minimization. In experiments with a conventional context-aware recommender system that leverages side information, we show that user attributes do not always improve recommendation. Then, we demonstrate that user attributes can negatively impact diversity…
▽ More
This paper studies user attributes in light of current concerns in the recommender system community: diversity, coverage, calibration, and data minimization. In experiments with a conventional context-aware recommender system that leverages side information, we show that user attributes do not always improve recommendation. Then, we demonstrate that user attributes can negatively impact diversity and coverage. Finally, we investigate the amount of information about users that ``survives'' from the training data into the recommendation lists produced by the recommender. This information is a weak signal that could in the future be exploited for calibration or studied further as a privacy leak.
△ Less
Submitted 28 July, 2022;
originally announced July 2022.
