Twin Worlds: Equivariance-Based Abstention for Evidence-Grounded Reasoning

Vy Nguyen; Ziqi Xu; Jeffrey Chan; Estrid He; Feng Xia; Renqiang Luo; Erik Cambria; Xiuzhen Zhang

cs.CLAug 28, 2026

Twin Worlds: Equivariance-Based Abstention for Evidence-Grounded Reasoning

Authors: Vy Nguyen, Ziqi Xu, Jeffrey Chan, Estrid He, Feng Xia, Renqiang Luo, Erik Cambria, Xiuzhen Zhang

Organizations: RMIT University, Australia · Jilin University, China · Nanyang Technological University, Singapore

Abstract

Knowledge-intensive reasoning requires Large Language Models (LLMs) to ground answers in provided evidence. When evidence is insufficient, it is desirable that models abstain rather than confidently generating unsupported answers. Existing abstention methods rely on uncertainty estimation or evidence sufficiency checks, but neither tests whether the reasoning process for generation, driven by the interaction of provided evidence and the model's internal memory parameters, is actually grounded in the evidence. A key contributing factor is that entity mentions in context activate memorised associations, causing models to generate plausible responses ungrounded in evidence. We propose Twin Worlds (TW), a framework for improving reliability in knowledge-intensive reasoning through equivariance-based abstention: unlike invariance, which requires outputs to remain unchanged, equivariance requires outputs to transform correspondingly under entity substitutions. A model grounded in the evidence should produce answers that shift consistently when entities are substituted while their relations are preserved. TW constructs multiple worlds via typed substitutions of the original input that preserve relational structure while reducing parametric priors, and uses equivariance violations as an abstention signal. Across four benchmarks and three model backbones, TW identifies when answers are not reliably grounded in the provided evidence and outperforms uncertainty- and sufficiency-based baselines.

Figures & tables

Figure 1: Top: LLMs often answer confidently based on outdated parametric associations even when the provided evidence does not support a conclusion. Bottom: Twin Worlds applies structure-preserving transformations to test whether answers are grounded in the evidence, enabling principled abstention decisions.

Figure 2: TW framework overview. Left: Delexicalisation replaces entity mentions with typed placeholders, extracting relational structure. Middle: Typed bijective substitutions relexicalise placeholders under $k$ sampled mappings, producing $k$ twin worlds $(q^{\pi_{i}},C^{\pi_{i}})$ . Right: The equivariance score aggregates whether back-mapped answers $\pi^{-1}(\hat{y}^{\pi_{i}})$ agree with $\hat{y}$ ; high equivariance leads to answering, while low equivariance leads to abstention.

	HotpotQA				MIRAGE				FaithEval				FEVER
Method	AR	F1	Acc	RS	AR	F1	Acc	RS	AR	F1	Acc	RS	AR	F1	Acc	RS
	GPT-5.1
Zero-shot	.0010	.9325	.9320	.9989	.0060	.9157	.9130	.9935	.0510	-	.0510	.5032	.3210	.6974	.6260	.6620
AbstentionBench	.0200	.9232	.9140	.9787	.0580	.8847	.8590	.9372	.3870	-	.3870	.5255	.8430	.2625	.4000	.3618
Self-Consistency	.0110	.9346	.9280	.9876	.0180	.9276	.9250	.9661	.1640	-	.1640	.5728	.4280	.6123	.5870	.5904
RC-RAG	.2720	.8090	.6990	.7201	.2710	.7912	.6840	.7168	.4860	-	.4860	.5004	.4120	.4928	.5400	.5682

Table 1: Results of TW and baselines across benchmark datasets. Each section compares methods with the same backbone. Bold indicates the highest score and underline the second highest. A hyphen (-) denotes zero F1 on FaithEval where all questions are unanswerable; performance there is measured by RS and Acc. Higher AR indicates more frequent abstention and may reflect over-conservatism rather than genuine grounding.

Q: What was found in Los Angeles ?

E: […] Trevor Valle discusses the teeth and lower jaw of a mammoth fossil. Now, at least 10,000 years later, visitors in Los Angeles can see the remains of a mammoth […]

Twin World 1: Los Angeles

\mapsto

Nova, Trevor Valle

\mapsto

Elian Voss

Output: no information

S_{\pi_{1}}=0

Twin World 2: Los Angeles

\mapsto

Valdecor, Trevor Valle

\mapsto

Orin Vael

Output: no information

S_{\pi_{2}}=0

Twin World 3: Los Angeles

\mapsto

Mirevan, Trevor Valle

\mapsto

Casen Drel

Table 2: TW on a representative unanswerable case. Across three twin worlds, all twin answers fail to agree with the original answer “mammoth fossil”, yielding

s=0.00

and triggering abstention.

Panel of Figure 3 — Figure 3: (a) Distribution of equivariance scores on GPT-5.1. With $k=3$ , scores take the discrete values $\{0,0.33,0.67,1.00\}$ ; most queries lie away from the decision boundary at $\tau=0.60$ . (b) Human–TW agreement increases monotonically with distance from the decision boundary, indicating stronger alignment on cases with more decisive equivariance scores.

	GPT-5.1		LLaMA-4		Mistral-Small
Strategy	Faith.	Unfaith.	Faith.	Unfaith.	Faith.	Unfaith.
Typed Bijection	.804	.215	.780	.230	.761	.243
Untyped Sub.	.721	.401	.701	.385	.685	.390
Counterfactual	.687	.475	.670	.460	.655	.452
Paraphrase	.811	.800	.799	.790	.791	.780
Random Repl.	.340	. 210	.325	.305	.310	.301

Table 3: WTR across evidence conditions, substitution strategies, and model backbones. Higher WTR is desirable under faithful evidence and lower WTR under unfaithful evidence; the desired behaviour is therefore a large faithful–unfaithful separation.

Panel of Figure 4 — Figure 4: Probability of entity-associated tokens across Mistral-Small-24B layers. Synthetic substitutions consistently reduce activation relative to the corresponding real-entity inputs, with the largest activation observed for real entities under unfaithful evidence.

	GPT-5.1			LLaMA-4			Mistral-Small
Configuration	F1	Acc	RS	F1	Acc	RS	F1	Acc	RS
TW (Full, $k=3$ )	.921	.903	.971	.891	.873	.956	.867	.849	.934
$k=1$	.884	.859	.927	.853	.831	.912	.829	.808	.890
$k=6$	.922	.904	.967	.892	.874	.957	.868	.850	.935
No type constraint	.874	.843	.921	.843	.819	.908	.819	.791	.883
No bijectivity	.881	.851	.928	.851	.826	.913	.828	.801	.891

Table 4: Ablation results across all three backbones, averaged across all four datasets. Bold indicates the highest score in each column.

Method	AR
AbstentionBench	0.18
Self-Consistency	0.09
RC-RAG	0.31
Context Perturbation	0.24
Sufficient Context	0.11
Contrastive Decoding	0.22

Table 5: Abstention rate on 200 unanswerable instances drawn from all four datasets, where both Self-Consistency agreement and Sufficient Context score are high.

Q: What position did Gareth Bale play in his final season?
E: Gareth Bale announced his retirement from professional football in January 2023, having spent much of his career as a winger , citing his desire to spend more time with his family.
Gold: unanswerable
Self-Consistency: 5/5 traces answer “winger” agreement $=1.0$
Sufficient Context: passage score $=0.51$ both baselines answer
TW 1: Bale $\mapsto$ Elian Voss	Output: winger	$S_{\pi_{1}}=1$
TW 2: Bale $\mapsto$ Orin Vael	Output: no information	$S_{\pi_{2}}=0$

Table 6: A hard unanswerable case where Self-Consistency and Sufficient Context both favour answering. TW detects inconsistent back-mapped answers and abstains because the answer is not reliably determined by the provided evidence.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Backbone	Passes	Latency (s)	Access
GPT-5.1	4	3.8 $\pm$ 1.3	OpenAI API
LLaMA-4-Scout	4	3.5 $\pm$ 1.0	Together AI
Mistral-Small-24B	4	2.9 $\pm$ 1.3	Together AI

Appendix

Table 7: Wall-clock latency per query (mean

\pm

std) across all three backbones, measured over 500 queries with

k=3

twin worlds batched together with the original input.

Figure 5: Human-evaluation instructions and annotation interfaces. Top: annotators judge whether the evidence is sufficient to answer the question without relying on external knowledge. Bottom: annotators rate how well a transformed passage preserves the relational structure of the original passage on a five-point Likert scale.

		Question Type
		Answerable	Unanswerable
Answered	Correct	TP	FP U
	Incorrect	FP A
Abstained		FN	TN

Appendix

Table 8: Confusion matrix for abstention evaluation. Blue indicates desirable outcomes; red indicates errors.

$k$	Passes	Latency (s)	RS (avg)
1	2	2.1 $\pm$ 0.8	.910
2	3	2.9 $\pm$ 0.9	.942
3	4	3.8 $\pm$ 1.3	.954
4	5	4.6 $\pm$ 1.4	.955
6	7	6.3 $\pm$ 1.7	.956

Appendix

Table 9: RS and latency per query across the number of twin worlds

k

, averaged across all three backbones. Bold indicates the selected default. Gains beyond

k=3

are negligible relative to the additional inference cost.

Error Type	HotpotQA	MIRAGE	FaithEval	FEVER	Overall
Missed spans	3.8%	3.1%	4.2%	6.1%	4.3%
Type errors	1.4%	1.5%	1.9%	2.4%	1.8%
Boundary	7.9%	8.1%	9.2%	8.1%	8.3%

Appendix

Table 10: Error rates by type and dataset across the full evaluation set of 8,000 instances. Boundary cases are defined as instances with

s(q,C)\in\{0.33,0.67\}

Method	Passes	Latency (s)	RS
Zero-shot	1	1.6 $\pm$ 0.6	.923
Self-Consistency	4	3.6 $\pm$ 1.2	.910
RC-RAG	3	3.8 $\pm$ 1.4	.891
Contrastive Decoding	3	4.1 $\pm$ 1.6	.887
Sufficient Context	20+	17.9 $\pm$ 4.8	.912
TW (Ours)	4	3.8 $\pm$ 1.3	.956

Appendix

Table 11: Inference latency and RS on GPT-5.1 over 500 queries. TW uses

k+1=4

forward passes and achieves the highest RS at a latency comparable to other multi-pass baselines.

Explore similar work

Apr 2, 2026

Answering the Wrong Question: Reasoning Trace Inversion for Abstention in LLMs

For Large Language Models (LLMs) to be reliably deployed, models must effectively know when not to answer: abstain. Reasoning models, in particular, have gained attention for impressive performance on complex tasks. However, reasoning models have been shown to have worse abstention abilities. Taking the vulnerabilities of reasoning models into account, we propose our Query Misalignment Framework. Hallucinations resulting in failed abstention can be reinterpreted as LLMs answering the wrong question (rather than answering a question incorrectly). Based on this framework, we develop a new class of state-of-the-art abstention methods called Trace Inversion. First, we generate the reasoning trace of a model. Based on only the trace, we then reconstruct the most likely query that the model responded to. Finally, we compare the initial query with the reconstructed query. Low similarity score between the initial query and reconstructed query suggests that the model likely answered the question incorrectly and is flagged to abstain. Extensive experiments demonstrate that Trace Inversion effectively boosts abstention performance in four frontier LLMs across nine abstention QA datasets, beating competitive baselines in 33 out of 36 settings.

Abinitha Gourabathina, Inkit Padhi, Manish Nagireddy +2

MIT · IBM Research · MIT-IBM Watson AI Lab

LLM Abstention LLM Hallucination Mitigation LLM Reasoning

Ask PDF

Mar 20, 2026

EvidenceRL: Reinforcing Evidence Consistency for Trustworthy Language Models

Large Language Models (LLMs) are fluent but prone to hallucinations, producing answers that appear plausible yet are unsupported by available evidence. This failure is especially problematic in high-stakes domains where decisions must be justified by verifiable information. We introduce EvidenceRL, a reinforcement learning framework that enforces evidence adherence during training. EvidenceRL scores candidate responses for grounding (entailment with retrieved evidence and context) and correctness (agreement with reference answers) and optimizes the generator using Group Relative Policy Optimization (GRPO). We evaluate across two high-stakes domains, cardiac diagnosis and legal reasoning, where EvidenceRL consistently improves evidence grounding and faithfulness without sacrificing task accuracy. On cardiac diagnosis, F1@3 increases from 37.0 to 54.5 on Llama-3.2-3B while grounding (

G_{\max}@3

) rises from 47.6 to 78.2; hallucinations drop nearly 5

\times

and evidence-supported diagnoses increase from 31.8% to 61.6%. On legal reasoning, EvidenceRL raises Faithfulness from 32.8% to 67.6% on Llama-3.1-8B, demonstrating consistent behavioral change across domains. Our code is open-sourced at https://github.com/Wizaaard/EvidenceRL.git.

J. Ben Tamo, Yuxing Lu, Benoit L. Marteau +2

Georgia Institute of Technology · Pekin University

LLM Hallucination Mitigation LLM Grounding Faithfulness of Language Model Explanations RL for Language Model Reasoning

Ask PDF

Apr 18, 2026

Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL

Reinforcement fine-tuning improves the reasoning ability of large language models, but it can also encourage them to answer unanswerable queries by guessing or hallucinating missing information. Existing abstention methods either train models to produce generic refusals or encourage follow-up clarifications without verifying whether those clarifications identify the key missing information. We study queries that are clear in meaning but cannot be reliably resolved from the given information, and argue that a reliable model should not only abstain, but also explain what is missing. We propose a clarification-aware RLVR reward that, while rewarding correct answers on answerable queries, jointly optimizes explicit abstention and semantically aligned post-refusal clarification on unanswerable queries. Using this reward, we train Abstain-R1, a 3B model that improves abstention and clarification on unanswerable queries while preserving strong performance on answerable ones. Experiments on Abstain-Test, Abstain-QA, and SelfAware show that Abstain-R1 substantially improves over its base model and achieves unanswerable-query behavior competitive with larger systems including DeepSeek-R1, suggesting that calibrated abstention and clarification can be learned through verifiable rewards rather than emerging from scale alone.

Skylar Zhai, Jingcheng Liang, Dongyeop Kang

University of Minnesota

LLM Hallucination Mitigation RL for Language Model Reasoning LLM Reliability Reinforcement Learning with Verifiable Rewards Refusal Behavior in Language Models LLM Abstention

Ask PDF