As LLMs are increasingly used as judges in code applications, they should be evaluated in realistic interactive settings that capture partial context and ambiguous intent. We present TRACE (Tool for Rubric Analysis in Code Evaluation), a framework that evaluates LLM judges' ability to predict human preferences and automatically extracts rubric items to reveal systematic biases in how humans and models weigh each item. Across three modalities -- chat-based programming, IDE autocompletion, and instructed code editing -- we use TRACE to measure how well LLM judges align with developer preferences. Among 13 different models, the best judges underperform human annotators by 12-23%. TRACE identifies 35 significant sources of misalignment between humans and judges across interaction modalities, the majority of which correspond to existing software engineering code quality criteria. For example, in chat-based coding, judges are biased towards longer code explanations while humans prefer shorter ones. We find significant misalignment on the majority of existing code quality dimensions, showing alignment gaps between LLM judges and human preference in realistic coding applications.
Figures & tables
Figure 1: Example of developer–LLM misalignment on a code editing task. Here, the developer provides a prompt and receives two LLM code solutions. In this example, the user prefers the top response while the LLM judge selects the bottom response. We compare these responses with the extracted rubric items to see that the developer prefers less robustness and more comments, while LLM judges prefer more robustness and fewer comments.
Figure 2: Overview of TRACE. Given a set of pairwise options, TRACE follows a three-step workflow: (1) we collect LLM judgments to measure alignment with human preferences; (2) we generate rubric items capturing differences between responses (e.g., error handling ); (3) we construct feature vectors from rubric criteria and train a model to predict LLM judgments. We use the learned coefficients to identify items that misalign between LLMs and humans.
Code Completion
Chat
Code Edit
# of Natural Languages
23
14
20
# of Programming Languages
39
57
43
Context Length - p50
2,233
1,013
3,490
Context Length - p95
13,984
12,509
28,189
Output Length - p50
108
5,491
531
Output Length - p95
613
20,156
3,053
Table 1: Coding modalities differ sharply in context length and output scale. We summarize three developer interaction settings—code completion, chat, and code edits—each with over 500 samples per dataset. Natural languages are detected using Lingua ( stahl2024lingua ) ; programming languages in chat are inferred from code block tags. Edit Distance reports the line-level Levenshtein edit distance between the two candidate responses (after newline normalization; for chat, computed on concatenated code blocks when present).
Code Completion
Instructed Code Edits
Chat-based Coding
AccPC↑
Acc ↑
AccPC↑
Acc ↑
AccPC↑
Acc ↑
Fine-tuned Judge
Atla Selene 1 Mini (Llama-3.1-8B)
59.01
37.67
51.8
31.6
63.76
19.00
Atla Selene 1 (Llama-3.3-70B)
61.99
46.00
54.35
30.4
67.40
24.40
Prometheus 2 (7B)
53.62
29.60
52.21
26.0
58.80
25.40
Skywork Critic (Llama-3.1-70B)
63.28
48.6
56.14
32.00
61.87
51.6
Table 2: Automated judges trail human agreement across coding modalities. We report accuracy (Acc, %) and positional accuracy ( AccPC , %) across three modalities. AccPC conditions on valid, positionally consistent decisions (the judge flips its choice when the two candidates are swapped). “Fine-tuned Judge” models are trained for evaluation and output a discrete winner; “3rd Party” models are general-purpose instruction-tuned LLMs used zero-shot as judges; “Reward Models” output scalar preference scores and are inherently order-invariant. Human rows report majority–user agreement on a 30-example overlap set and the absolute-point improvement over the best model.
Figure 3: Judge misalignment highlights distinct rubric-level differences across interaction modalities. Each cell shows the signed difference between judge and human preference coefficients ( βJ,i−βH,i ) for selected rubric items within each interaction modality. Positive values (red) indicate that judges overweight a rubric item relative to humans, while negative values (blue) indicate underweighting. Rows show the highest-divergence rubric dimensions within each modality. Bolded values indicate rubric items with significant pooled judge misalignment.
Prompt (paraphrased)
Implement the stockSpan method. Given an array of daily stock prices, compute, for each day, the number of consecutive previous days whose price is less than or equal to the current price.
Model 1
Model 2
Completion
⬇ stockSpan ( stocks ) { stack = new Stack <>(); for ( int i = 0; i < stocks . length ; i ++) { ... } }
⬇ stockSpan ( stocks ) { stack = new Stack <>({0}); out [0] = 1; for ( int i = 1; i < stocks . length ; i ++) { ... } }
Explicitness and Clarity
↓ Omits base case initialization, making initial logic implicit.
Large Language Models are increasingly used as judges to evaluate code artifacts when exhaustive human review or executable test coverage is unavailable. LLM-judge is increasingly relevant in agentic software engineering workflows, where it can help rank candidate solutions and guide patch selection. While attractive for scale, current practice lacks a principled account of reliability and bias: repeated evaluations of the same case can disagree; small prompt edits can swing outcomes; and seemingly semantics-preserving, human-equivalent perturbations may elicit divergent verdicts. This paper studies LLM-as-a-Judge for code through a measurement-first lens. We analyze two pointwise judging regimes across code generation, code repair task, and test generation, and we systematically probe prompt-induced biases. Our study considers difficulty levels for repeated runs and controlled prompt interventions that isolate one presentation cue at a time, and it evaluates judges using consistency and sensitivity to bias. We find that judge decisions are highly sensitive to prompt biases even when the underlying code snippet is unchanged. Across all three tasks, several biases systematically shift preferences toward the option favored by the prompt, improving accuracy when that option aligns with the gold answer but substantially reducing it otherwise. In some settings, these effects are large enough to change task-level conclusions and alter relative model rankings. These findings show that reported judge performance may reflect prompt artifacts rather than stable assessment ability, posing a direct threat to the validity and reproducibility of code evaluation. We therefore argue that LLM-as-a-Judge studies should report bias sensitivity alongside accuracy and incorporate explicit controls to support more trustworthy model comparison in software engineering.
Pairwise human preference prediction is central to evaluating code-generation systems, where quality often depends on task-specific trade-offs beyond functional correctness. While rubric-based LLM judges improve interpretability by decomposing evaluation into explicit criteria, most existing pipelines remain pointwise: they score each response independently and derive preferences by comparing aggregated scores. We show that this design is poorly matched to pairwise code preference prediction and can underperform a strong monolithic judge. We propose CriterAlign, a criterion-centric framework that adapts rubric-based judging to pairwise preference evaluation through direct criterion-level pairwise judgments, tie-driven criterion refinement, swap-consistency filtering, and final pairwise synthesis. We further introduce Human-Preference-Aligned Guidance (HPAG), synthesized offline from training examples by extracting recurring rationale gaps between human preferences and monolithic judge predictions, and injected into the criterion generator, criterion judge, and final judge. On BigCodeReward, CriterAlign improves a Qwen2.5-VL-32B monolithic judge from 60.4% to 66.3% accuracy, with ablations confirming the contributions of pairwise criterion design and HPAG.
Evaluating LLMs is challenging, as benchmark scores often fail to capture models' real-world usefulness. Instead, users often rely on ``vibe-testing'': informal experience-based evaluation, such as comparing models on coding tasks related to their own workflow. While prevalent, vibe-testing is often too ad hoc and unstructured to analyze or reproduce at scale. In this work, we study how vibe-testing works in practice and then formalize it to support systematic analysis. We first analyze two empirical resources: (1) a survey of user evaluation practices, and (2) a collection of in-the-wild model comparison reports from blogs and social media. Based on these resources, we formalize vibe-testing as a two-part process: users personalize both what they test and how they judge responses. We then introduce a proof-of-concept evaluation pipeline that follows this formulation by generating personalized prompts and comparing model outputs using user-aware subjective criteria. In experiments on coding benchmarks, we find that combining personalized prompts and user-aware evaluation can change which model is preferred, reflecting the role of vibe-testing in practice. These findings suggest that formalized vibe-testing can serve as a useful approach for bridging benchmark scores and real-world experience.
Itay Itzhak, Eliya Habba, Gabriel Stanovsky +1
Technion – Israel Institute of Technology · The Hebrew University of Jerusalem