This work-in-progress paper provides probabilistic expressions of existing measures of over-reliance on AI-based decision support systems, mostly for binary classification, facilitating direct comparison and a clearer understanding of what each measure captures. We calculate all measures on the same synthetic dataset, visualise their relationships and provide complementary measures for evaluating the unwanted effects of mitigation strategies. The aim of this short paper is to provide researchers and designers with an informed framework for selecting appropriate measures when evaluating human-AI decision-making systems.
How Do We Measure Over-Reliance? A Unified Probabilistic View
Daria Mikhaylova;Tommaso Turchi;Alessio Malizia
2026-01-01
Abstract
This work-in-progress paper provides probabilistic expressions of existing measures of over-reliance on AI-based decision support systems, mostly for binary classification, facilitating direct comparison and a clearer understanding of what each measure captures. We calculate all measures on the same synthetic dataset, visualise their relationships and provide complementary measures for evaluating the unwanted effects of mitigation strategies. The aim of this short paper is to provide researchers and designers with an informed framework for selecting appropriate measures when evaluating human-AI decision-making systems.File in questo prodotto:
Non ci sono file associati a questo prodotto.
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
