Algorithmic Arbitrariness

Algorithmic arbitrariness refers to the unpredictable and inconsistent outputs produced by seemingly similar machine learning models, even when achieving comparable performance metrics. Current research focuses on quantifying this arbitrariness across various applications, including content moderation and loan applications, and investigating its disparate impact on different social groups. This research highlights the limitations of traditional fairness and accuracy metrics, prompting the development of new evaluation frameworks and algorithms that explicitly address model multiplicity and inconsistency. Understanding and mitigating algorithmic arbitrariness is crucial for ensuring fairness, accountability, and trust in machine learning systems deployed in high-stakes decision-making.