Position Bias

Position bias studies how answer order, passage placement, list rank, or token position can change a decision even when the underlying content is unchanged. Researchers measure it with controlled permutations and counterbalanced tests in multiple-choice benchmarks, LLM judging, retrieval and recommendation, and use attention or residual-stream analyses to investigate its causes. Mitigations include randomizing or averaging over orderings, calibrating scores, training for permutation consistency, and changing positional representations. A key tension is that aggregate accuracy can hide bias, while reducing order sensitivity may cost latency or ranking quality and may not transfer across tasks or models.