Crowd Compositions for Bias Detection and Mitigation in Predicting Recidivism

Abstract

This thesis explores an approach to predicting recidivism by leveraging crowdsourcing, contrasting traditional judicial discretion and algorithmic models. Instead of relying on judges or algorithms, participants predicted the likelihood of re-offending using the COMPAS dataset, which includes demographic and criminal record information. The study analyzed both quantitative and qualitative data to assess biases in human versus algorithmic predictions. Findings reveal that homogeneous crowds reflect the biases of their composition, leading to more pronounced gender and racial biases. In contrast, heterogeneous crowds, with equal and random distributions, present a more balanced view, though underlying biases still emerge. Both gender and racial biases influence how re-offending risk is perceived, significantly impacting risk evaluations. Specifically, crowds rated African American offenders as less likely to re-offend compared to COMPAS, which assigned them higher risk scores, while Caucasian and Hispanic offenders were perceived as more likely to re-offend by crowds. Gender differences also emerged, with males rated as less likely to re-offend and females as more likely. This study highlights crowdsourcing's potential to mitigate biases and provides insights into balancing consistency and fairness in risk assessments.