Phantom Merge: When Your Large Language Model Agents Pick One but Tell You About Another

Wangshu Zhu; Qianwen Guo; Yushun Dong

cs.AIOct 12, 2026

Phantom Merge: When Your Large Language Model Agents Pick One but Tell You About Another

Authors: Wangshu Zhu, Qianwen Guo, Yushun Dong

Organizations: Department of Computer Science, Florida State University · Department of Civil and Environmental Engineering, FAMU–FSU College of Engineering

Abstract

LLM agents increasingly operate over multiple entities, where reliable response generation requires not only retrieving relevant evidence but also preserving its association with the correct entity. We identify Phantom Merge (PM), an entity-evidence binding failure in which an agent attributes unsupported claims to its committed entity, potentially even when the task is completed correctly. We formalize PM at the claim level and characterize its prevalence and error subtypes across four agent domains and four backbone families. To address this problem, we propose a two-stage framework for claim-level risk estimation and correction. First, Anchor-Grounded Risk (AGR) combines a representational readout of the agent's pre-emission hidden state with Jacobian-based functional information, providing complementary slot-level and anchor-value risk scores. Second, Fixed-Anchor Correction (FAC) audits flagged claims and selectively revises them using observed anchor evidence or removes them when no supported replacement is available. On the held-out Shopping detection cohort, AGR-slot achieves an AUROC of 96.2% and an average precision of 97.3%, outperforming the evaluated detection baselines. In the end-to-end Shopping mitigation evaluation, FAC reduces trajectory-level PM from 51.4% to 6.2%, with an output-to-original claim ratio of 80.7%. These results demonstrate the prevalence of entity-specific grounding failures and the effectiveness of combining internal risk estimation with evidence-constrained claim correction. Our code is publicly available at https://github.com/LabRAI/PhantomMerge.

Figures & tables

Figure 1: Overview of the proposed framework. Anchor-Grounded Risk (AGR) combines representational and Jacobian-based functional information for claim-level PM detection. Fixed-Anchor Correction (FAC) integrates risk-based selection, functional auditing, and anchor-specific evidence to retain, revise, or remove individual claims.

Domain	Backbone	TC	PM	PM $\mid$ TC	TC $\mid$ PM	CEM	CAP	ASF
Retail	Mixtral	8.7	16.5	18.4	9.7	4.2	1.6	94.2
	Gemma	21.8	0.4	0.5	25.0	0.0	0.0	100.0
	Llama	9.2	2.5	0.0	0.0	3.6	7.1	89.3
	Qwen	15.9	3.1	1.3	6.5	69.0	0.0	31.0
Airline	Mixtral	35.6	34.0	32.0	33.5	0.0	0.3	99.7
	Gemma	29.3	4.5	4.1	26.7	0.0	75.0	25.0

Table 1: Phantom Merge across four agent domains and backbone families. TC denotes task correctness, PM denotes trajectory-level Phantom Merge, and

X\mid Y

denotes

P(X\mid Y)

. CEM, CAP, and ASF report claim-level PM subtype composition. All values are percentages.

Method	AUROC	AUPRC	Precision	Recall	F1
RARR	49.9 [48.7, 50.8]	56.9 [50.2, 63.0]	50.0	0.6	1.3
FActScore	58.2 [51.6, 64.6]	65.5 [57.5, 73.0]	59.4	62.8	61.1
FacTool	67.9 [61.9, 74.2]	67.9 [60.8, 74.5]	77.6	62.2	69.0
Anchor-only LLM	71.9 [66.4, 77.4]	72.7 [65.9, 78.6]	83.6	59.0	69.2
Source-agnostic LLM	75.9 [70.6, 81.2]	76.6 [69.8, 82.6]	88.3	62.8	73.4
ModernBERT	76.2 [69.1, 82.6]	79.9 [71.6, 86.9]	61.2	96.2	74.8

Table 2: Phantom Merge detection on the Shopping test cohort. AUROC and AUPRC are reported as percentages with 95% bootstrap confidence intervals. Precision, recall, and F1 are evaluated at

\tau=0.05

for applicable probability-like scores. “–” denotes scores evaluated separately using development-selected thresholds (Appendix D ).

Figure 2: AGR-slot score distribution on the Shopping test cohort.

Figure 3: Cross-domain diagnostic AUROC of the evaluated readouts under in-domain and zero-shot configurations.

Figure 4: End-to-end mitigation across four agent domains. Lower-left is better: FAC achieves a favorable trade-off between PM reduction and correctly bound claim retention.

Figure 5: Shared-gate mitigation across four agent domains. All methods receive identical claims flagged by a frozen representational probe, isolating differences in downstream routing and correction. Marker positions are jittered to reduce overlap. Exact residual PM rates are reported in Table G.1

Figure 6: Detector-head comparison on Shopping with downstream routing held fixed. Bars report the PM-free trajectory rate and correctly bound claim retention at the operating points.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Readout	Evaluated score	Interpretation
Representational	$p_{\mathrm{pm}}(c)$	Frozen representational probe output
Jacobian (slot)	$\log\pi_{J}(T_{\mathrm{slot}}(c)\mid h_{c})$	Standalone slot-level score
AGR-value	$r_{\mathrm{val}}(c)$ ; $\rho(c)=\sigma(r_{\mathrm{val}}(c))$	Fitted value-level fusion
AGR-slot	$B(c)=\ell_{\mathrm{pm}}(c)+\log\pi_{J}(T_{\mathrm{slot}}(c)\mid h_{c})$	Fixed-additive AGR-slot score

Appendix

Table B.1: Detection score identities. Here

\ell_{\mathrm{pm}}=\operatorname{logit}(p_{\mathrm{pm}})

, and

r_{\mathrm{val}}

denotes the AGR-value fusion logit, including its fallback.

Readout	AUROC [95% CI]
Representational	0.888 [0.846, 0.924]
Jacobian (slot)	0.894 [0.851, 0.934]
AGR-value	0.898 [0.857, 0.933]
AGR-slot	0.962 [0.940, 0.979]

Appendix

Table B.2: Readout comparison on Shopping. AUROC is reported with its 95% bootstrap confidence interval.

Figure B.1: AUROC of selected detection baselines and internal readouts. The ModernBERT variant shown in this plot is the quote-input variant.

Method	AUROC [95% CI]	AP [95% CI]	P	R	F1
RARR gate	0.499 [0.487, 0.508]	0.569 [0.502, 0.630]	0.500	0.006	0.013
FActScore	0.582 [0.516, 0.646]	0.655 [0.575, 0.730]	0.594	0.628	0.611
FacTool	0.679 [0.619, 0.742]	0.679 [0.608, 0.745]	0.776	0.622	0.690
SC-NLI ( $K=5$ )	0.693 [0.626, 0.757]	0.752 [0.676, 0.817]	0.598	0.974	0.741
MB (quote)	0.706 [0.637, 0.769]	0.729 [0.636, 0.809]	0.617	0.962	0.752
MB (structured)	0.762 [0.691, 0.826]	0.799 [0.716, 0.869]	0.612	0.962	0.748

Appendix

Table C.1: Extended detection results on the Shopping test cohort (

n=274

). All metrics are on a 0–1 scale; AUROC and AP include 95% bootstrap confidence intervals. P/R/F1 use the common threshold of 0.05 for the designated probability-based methods. “– denotes internal scores whose operating points are reported in Appendix D . SC denotes SelfCheckGPT-NLI and MB denotes ModernBERT.

\dagger

marks predictions that label every test claim as PM.

Figure C.1: ROC curves for selected detection methods on the Shopping test set.

Subset	$n$	PM+	AGR-slot	Representational	TF-IDF
Overlap	181	108	.972 [.950, .989]	.903 [.860, .942]	.783 [.706, .851]
Non-overlap	93	48	.950 [.907, .984]	.882 [.805, .941]	.756 [.645, .855]

Appendix

Table C.2: Detection stratified by training-set quote-template overlap. Cells report AUROC [95% CI]; PM+ is the number of PM-positive claims.

Figure D.1: Representational-gate precision, recall, and F1 on development set. The marked threshold of 0.05 is used for Shopping shared-gate claim selection.

Method	Selection split	Threshold	P	R	F1
Jacobian (slot)	representational dev	$-7.064$	100.0	76.9	87.0
AGR-value	$\mathcal{D}_{f}$	$-0.7585$	76.4	93.6	84.1
AGR-slot	$\mathcal{D}_{f}$	$-8.822$	91.9	87.2	89.5
AGR-slot	representational dev	$-12.714$	84.5	94.2	89.1

Appendix

Table D.1: Internal detection operating points on the full Shopping test cohort. P/R/F1 are percentages; thresholds use the raw score scales in Table B.1 .

Method	Selection split	TP	FP	FN	TN
Jacobian (slot)	representational dev	120	0	36	118
AGR-value	$\mathcal{D}_{f}$	146	45	10	73
AGR-slot	$\mathcal{D}_{f}$	136	12	20	106
AGR-slot	representational dev	147	27	9	91

Appendix

Table D.2: Confusion counts for the operating points in Table D.1 . PM is the positive class.

Method	Implementation and role
RARR	Adapted from pinned third-party prompts and executed against trajectory observations using our generation backend. Its agreement gate is evaluated for detection; its revision procedure is evaluated for mitigation.
CoVe	CoVe-inspired verification and revision procedure implemented from the published method description. Evaluated as a mitigation baseline.
Constrained copy	Self-designed, FEVER-style constrained-copy baseline. Evaluated for mitigation under the corresponding intervention protocols.
SelfCheckGPT, FacTool, FActScore	Adapted detection implementations using the available trajectory observations. Evaluated as detection baselines, without a corresponding mitigation implementation.
ModernBERT	Embedding-based classifier trained on the labeled split. Quote-based and structured-input variants are reported.
DeBERTa-MNLI	Anchor-aware verification implementation based on an NLI checkpoint. Evaluated for detection and verification, without a separate rewriting procedure.

Appendix

Table E.1: Implementation provenance and experimental roles of the evaluated baselines.

Role	Score or configuration	Rule
Primary detection ranking	AGR-slot $B(c)$	Threshold-independent AUROC/AP
Shared-gate claim selection	Representational $p_{\mathrm{pm}}(c)$	Flag if $p_{\mathrm{pm}}(c)>0.05$
Standalone Jacobian detection	Slot log mass	representational dev F1 threshold $-7.064$
Jacobian audit partition	Slot log mass	Development PM-positive median $-3.401$
Functional audit readout	Slot lexicon, top- $k=50$	Development-fitted audit thresholds
Evidence-grounded rewriting	Qwen3-8B	Deterministic decoding; fixed anchor

Appendix

Table E.2: Score and threshold roles in Shopping detection and correction. Detection operating points and audit thresholds have different roles.

Method	Telecom	Airline	Retail
FAD	$1.6\%\,[0.0,3.7],\;R=0.49$	$0.5\%\,[0.0,1.6],\;R=0.81$	$0.1\%\,[0.0,0.3],\;R=0.37$
RARR	$0.0\%\,[0.0,0.0],\;R=0.00$	$0.0\%\,[0.0,0.0],\;R=0.00$	$0.0\%\,[0.0,0.0],\;R=0.00$
CoVe	$0.5\%\,[0.0,2.1],\;R=0.01$	$0.5\%\,[0.0,1.6],\;R=0.01$	$0.0\%\,[0.0,0.0],\;R=0.00$
Constrained copy	$51.3\%\,[44.5,58.4],\;R=1.30$	$8.2\%\,[4.3,12.0],\;R=1.01$	$1.6\%\,[0.9,2.4],\;R=0.91$
FAC	$47.6\%\,[40.3,55.0],\;R=1.01$	$1.1\%\,[0.0,2.7],\;R=0.79$	$0.4\%\,[0.1,0.8],\;R=0.75$
FAC + RARR	$51.3\%\,[44.5,58.4],\;R=1.30$	$8.2\%\,[4.3,12.0],\;R=1.01$	$1.7\%\,[1.0,2.5],\;R=0.78$

Appendix

Table F.1: Complete-intervention mitigation across the three

\tau^{3}

-Bench domains. Initial PM rates are 52.4% for Telecom, 20.1% for Airline, and 3.1% for Retail. Each cell reports residual PM [95% CI] and the output-claim ratio

R

. FAC denotes Fixed-Anchor Correction and FAD denotes Fixed-Anchor Deletion.

Method	Telecom	Airline	Retail
FAD	$1.6\%\,[0.0,3.7],\;R=0.49$	$0.5\%\,[0.0,1.6],\;R=0.81$	$0.1\%\,[0.0,0.3],\;R=0.37$
RARR	$1.6\%\,[0.0,3.7],\;R=0.47$	$0.0\%\,[0.0,0.0],\;R=0.78$	$0.1\%\,[0.0,0.3],\;R=0.37$
CoVe	$1.6\%\,[0.0,3.7],\;R=0.47$	$0.0\%\,[0.0,0.0],\;R=0.78$	$0.1\%\,[0.0,0.3],\;R=0.37$
Constrained copy	$51.3\%\,[44.5,58.4],\;R=1.30$	$3.8\%\,[1.6,6.5],\;R=1.01$	$1.7\%\,[1.0,2.5],\;R=0.91$
FAC	$47.6\%\,[40.3,55.0],\;R=1.01$	$1.1\%\,[0.0,2.7],\;R=0.79$	$0.4\%\,[0.1,0.8],\;R=0.75$
FAC + RARR	$51.3\%\,[44.5,58.4],\;R=1.30$	$3.8\%\,[1.6,6.5],\;R=1.01$	$1.7\%\,[1.0,2.5],\;R=0.78$

Appendix

Table F.2: Shared-gate mitigation across the same three domains. All methods receive identical flagged claims within each domain. Each cell reports residual PM [95% CI] and the output-claim ratio

R

	Complete intervention		Shared gate
Method	PM [95% CI]	$R$	PM [95% CI]	$R$
FAD	5.3 [2.5, 8.2]	0.358	5.3 [2.5, 8.2]	0.358
RARR	0.0 [0.0, 0.0]	0.041	5.3 [2.5, 8.2]	0.370
CoVe	0.4 [0.0, 1.2]	0.572	6.6 [3.7, 9.7]	0.691
Constrained copy	5.8 [2.9, 9.1]	1.128	10.7 [6.6, 14.6]	0.947
FAC	6.2 [3.3, 9.1]	0.807	5.3 [2.5, 8.2]	0.593

Appendix

Table G.1: Complete Shopping mitigation results. PM is residual trajectory-level PM rate with 95% confidence interval.

R

denotes the output-claim ratio. The initial PM rate is 51.4%.

Category	Claims
Unflagged / keep	87
Flagged: copy-rewrite	57
Flagged: probe-only deletion	125
Flagged: remaining cases	5
Total	274

Appendix

Table G.2: Shopping shared-gate action counts: 187 flagged and 87 unflagged claims.

Verifier	PM R	CB drop	CB ret.
Anchor-only LLM	0.59	0.15	0.85
Source-agnostic LLM	0.63	0.11	0.89
Cite-or-drop LLM	0.79	0.21	0.79
DeBERTa-MNLI	0.84	0.40	0.60
Probe gate	0.92	0.37	0.63
Lexical anchor-only	0.71	0.40	0.60

Appendix

Table H.1: Anchor-aware verification on the held-out Shopping test cohort (

n=274

). PM R denotes PM recall; CB drop and CB ret. denote correctly bound claim false-drop rate and retention, respectively.

LLM Grounding

Large language model (LLM) grounding studies how models anchor answers and actions in evidence, structured knowledge, and real-world context.

See all 319 papers

"You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accused
Sep 26, 2026Xutao Mao, Rui Qian, Longxiang Wang +6LLM Agents LLM Agent Safety
City University of Hong Kong · Fudan University · Southeast University +2
Real-Time Detection and Repair of LLM Agent Failures
Aug 3, 2026Sunny DubeyAI Agent Reliability AI Agent Monitoring
The Self-Correction Illusion: LLMs Correct Others but Not Themselves
Jun 4, 2026Kuan-Yen Chen, Fang-Yi Su, Jung-Hsien ChiangLLM Self-Correction LLM Prompting
National Cheng Kung University

Sep 26, 2026J/K move · Enter open · S save

"You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accused

Xutao Mao, Rui Qian, Longxiang Wang, Xinjian Yi+5

City University of Hong Kong · Fudan University · Southeast University+2

LLM Agents LLM Agent Safety LLM Agent Reliability LLM Sycophancy AI Agent Safety Benchmarks

Open paper AskPDF arXiv

LLM agents increasingly keep working after a task succeeds as they resume after compaction or take over handoffs. Their finished work keeps receiving follow-up input that sometimes falsely accuses it for later failures. We call an agent's acceptance of such a false accusation gaslight sycophancy, and destructive over-correction when acting on it damages previously correct work. We introduce CAVE-Bench, a benchmark of 365 agentic tasks across six domains built around opaque tasks. Every scored run first reaches a verified correct state, whose supporting rationale and history stay in the workspace while the facts that would settle the accusation lie in external or runtime state beyond the agent's reach. The agent cannot confirm or refute the claim with a local check, so the right response should keep the work and ask for the missing evidence. Each task either hands the agent correct work with saved evidence or let it build and verify that work first, and five risk factors set how the accusation enters the workflow. We score accusation acceptance and evidence use from the trajectory and measure harm by deterministic replay of downstream events. Across 14 of the latest models in Claude Code, false accusations damage correct work in up to 60.06% of runs, and stronger models often do so after recovering the supporting evidence. The same model behaves differently across OpenCode, Codex, and Hermes, and a harness gate driven by the benchmark's live signals cuts replayed harm by 74%. These results show that preserving already-correct work under unsupported accusation is a distinct safety challenge for long-lived agents. Our project is in https://henrymao2004.github.io/agent-over-correction/.

Abstract

LLM Grounding

Explore similar work

"You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accused

Real-Time Detection and Repair of LLM Agent Failures

The Self-Correction Illusion: LLMs Correct Others but Not Themselves