Comparing Developer and LLM Biases in Code Evaluation

Aditya Mittal; Ryan Shar; Zichu Wu; Shyam Agarwal; Tongshuang Wu; Chris Donahue; Ameet Talwalkar; Wayne Chi

cs.SEMar 25, 2026

Comparing Developer and LLM Biases in Code Evaluation

Authors: Aditya Mittal, Ryan Shar, Zichu Wu, Shyam Agarwal, Tongshuang Wu, Chris Donahue, Ameet Talwalkar, Wayne Chi,

Organizations: Carnegie Mellon University

Abstract

As LLMs are increasingly used as judges in code applications, they should be evaluated in realistic interactive settings that capture partial context and ambiguous intent. We present TRACE (Tool for Rubric Analysis in Code Evaluation), a framework that evaluates LLM judges' ability to predict human preferences and automatically extracts rubric items to reveal systematic biases in how humans and models weigh each item. Across three modalities -- chat-based programming, IDE autocompletion, and instructed code editing -- we use TRACE to measure how well LLM judges align with developer preferences. Among 13 different models, the best judges underperform human annotators by 12-23%. TRACE identifies 35 significant sources of misalignment between humans and judges across interaction modalities, the majority of which correspond to existing software engineering code quality criteria. For example, in chat-based coding, judges are biased towards longer code explanations while humans prefer shorter ones. We find significant misalignment on the majority of existing code quality dimensions, showing alignment gaps between LLM judges and human preference in realistic coding applications.

Figures & tables

Figure 1: Example of developer–LLM misalignment on a code editing task. Here, the developer provides a prompt and receives two LLM code solutions. In this example, the user prefers the top response while the LLM judge selects the bottom response. We compare these responses with the extracted rubric items to see that the developer prefers less robustness and more comments, while LLM judges prefer more robustness and fewer comments.

Figure 2: Overview of TRACE. Given a set of pairwise options, TRACE follows a three-step workflow: (1) we collect LLM judgments to measure alignment with human preferences; (2) we generate rubric items capturing differences between responses (e.g., error handling ); (3) we construct feature vectors from rubric criteria and train a model to predict LLM judgments. We use the learned coefficients to identify items that misalign between LLMs and humans.

	Code Completion	Chat	Code Edit
# of Natural Languages	23	14	20
# of Programming Languages	39	57	43
Context Length - p50	2,233	1,013	3,490
Context Length - p95	13,984	12,509	28,189
Output Length - p50	108	5,491	531
Output Length - p95	613	20,156	3,053

Table 1: Coding modalities differ sharply in context length and output scale. We summarize three developer interaction settings—code completion, chat, and code edits—each with over 500 samples per dataset. Natural languages are detected using Lingua ( stahl2024lingua ) ; programming languages in chat are inferred from code block tags. Edit Distance reports the line-level Levenshtein edit distance between the two candidate responses (after newline normalization; for chat, computed on concatenated code blocks when present).

	Code Completion		Instructed Code Edits		Chat-based Coding
	$\text{Acc}_{\text{PC}}$ $\uparrow$	Acc $\uparrow$	$\text{Acc}_{\text{PC}}$ $\uparrow$	Acc $\uparrow$	$\text{Acc}_{\text{PC}}$ $\uparrow$	Acc $\uparrow$
Fine-tuned Judge
Atla Selene 1 Mini (Llama-3.1-8B)	59.01	37.67	51.8	31.6	63.76	19.00
Atla Selene 1 (Llama-3.3-70B)	61.99	46.00	54.35	30.4	67.40	24.40
Prometheus 2 (7B)	53.62	29.60	52.21	26.0	58.80	25.40
Skywork Critic (Llama-3.1-70B)	63.28	48.6	56.14	32.00	61.87	51.6

Table 2: Automated judges trail human agreement across coding modalities. We report accuracy (Acc, %) and positional accuracy (

\text{Acc}_{\text{PC}}

, %) across three modalities.

\text{Acc}_{\text{PC}}

conditions on valid, positionally consistent decisions (the judge flips its choice when the two candidates are swapped). “Fine-tuned Judge” models are trained for evaluation and output a discrete winner; “3rd Party” models are general-purpose instruction-tuned LLMs used zero-shot as judges; “Reward Models” output scalar preference scores and are inherently order-invariant. Human rows report majority–user agreement on a 30-example overlap set and the absolute-point improvement over the best model.

Figure 3: Judge misalignment highlights distinct rubric-level differences across interaction modalities. Each cell shows the signed difference between judge and human preference coefficients ( $\beta_{J,i}-\beta_{H,i}$ ) for selected rubric items within each interaction modality. Positive values (red) indicate that judges overweight a rubric item relative to humans, while negative values (blue) indicate underweighting. Rows show the highest-divergence rubric dimensions within each modality. Bolded values indicate rubric items with significant pooled judge misalignment.

Prompt (paraphrased)	Implement the stockSpan method. Given an array of daily stock prices, compute, for each day, the number of consecutive previous days whose price is less than or equal to the current price.
	Model 1	Model 2
Completion	⬇ stockSpan ( stocks ) { stack = new Stack <>(); for ( int i = 0; i < stocks . length ; i ++) { ... } }	⬇ stockSpan ( stocks ) { stack = new Stack <>({0}); out [0] = 1; for ( int i = 1; i < stocks . length ; i ++) { ... } }
Explicitness and Clarity	$\boldsymbol{\downarrow}$ Omits base case initialization, making initial logic implicit.

Table 6

Bias in LLM-as-a-Judge

Bias in large language model (LLM)-as-a-judge evaluation studies systematic distortions in automated assessments of model outputs.

See all 116 papers

Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering
Apr 18, 2026Zixiao Zhao, Amirreza Esmaeili, Fatemeh FardSoftware Engineering LLM-as-a-Judge
University of British Columbia, Canada
CriterAlign: Criterion-Centric Rationale Alignment for Code Preference Judging
May 19, 2026Zhenyu Li, Aleksandar Cvejic, Zehui Chen +1Pairwise Preference Learning Human Preference Evaluation
KAUST · USTC
From Feelings to Metrics: Understanding and Formalizing How Users Vibe-Test LLMs
Apr 15, 2026Itay Itzhak, Eliya Habba, Gabriel Stanovsky +1LLM Evaluation Human Preference Evaluation
Technion – Israel Institute of Technology · The Hebrew University of Jerusalem

Apr 18, 2026J/K move · Enter open · S save

Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering

Zixiao Zhao, Amirreza Esmaeili, Fatemeh Fard

University of British Columbia, Canada

Software Engineering LLM-as-a-Judge Language Model Bias Evaluation Bias in LLM-as-a-Judge Code Generation Evaluation

Open paper AskPDF arXiv

Large Language Models are increasingly used as judges to evaluate code artifacts when exhaustive human review or executable test coverage is unavailable. LLM-judge is increasingly relevant in agentic software engineering workflows, where it can help rank candidate solutions and guide patch selection. While attractive for scale, current practice lacks a principled account of reliability and bias: repeated evaluations of the same case can disagree; small prompt edits can swing outcomes; and seemingly semantics-preserving, human-equivalent perturbations may elicit divergent verdicts. This paper studies LLM-as-a-Judge for code through a measurement-first lens. We analyze two pointwise judging regimes across code generation, code repair task, and test generation, and we systematically probe prompt-induced biases. Our study considers difficulty levels for repeated runs and controlled prompt interventions that isolate one presentation cue at a time, and it evaluates judges using consistency and sensitivity to bias. We find that judge decisions are highly sensitive to prompt biases even when the underlying code snippet is unchanged. Across all three tasks, several biases systematically shift preferences toward the option favored by the prompt, improving accuracy when that option aligns with the gold answer but substantially reducing it otherwise. In some settings, these effects are large enough to change task-level conclusions and alter relative model rankings. These findings show that reported judge performance may reflect prompt artifacts rather than stable assessment ability, posing a direct threat to the validity and reproducibility of code evaluation. We therefore argue that LLM-as-a-Judge studies should report bias sensitivity alongside accuracy and incorporate explicit controls to support more trustworthy model comparison in software engineering.

Abstract

Bias in LLM-as-a-Judge

Explore similar work

Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering

CriterAlign: Criterion-Centric Rationale Alignment for Code Preference Judging

From Feelings to Metrics: Understanding and Formalizing How Users Vibe-Test LLMs