R-DEIM Net: An Efficient Rationale-Augmented Dual-Expert Interaction Model for Paraphrase Detection

Pushp; Vaibhav Prajapati; Himangshu Sarma

cs.CLSep 24, 2026

R-DEIM Net: An Efficient Rationale-Augmented Dual-Expert Interaction Model for Paraphrase Detection

Authors: Pushp, Vaibhav Prajapati, Himangshu Sarma

Organizations: Indian Institute of Information Technology (IIIT), Sri City, India · University of Technology Nuremberg (UTN), Germany

Abstract

Recent advances in paraphrase detection reveal a fundamental trade-off: large language models achieve high accuracy but require high computation, while efficient Siamese-BERT variants offer practical scalability with reduced transparency in rationale generation. We present R-DEIM Net, a 76M-parameter dual-expert architecture exploring whether moderate-scale models can achieve competitive accuracy on paraphrase detection while enabling human-readable rationale generation. The architecture combines two specialized components: an Interaction Expert that captures token-level similarity patterns through multi-scale 2D convolutions and attention head allowing variable input length, and a Reasoning Expert that uses a Flan-T5-small decoder to generate rationales as auxiliary supervision. Rather than re-encoding generated text, we extract and pool decoder hidden states as complementary features for classification. On the Quora Question Pairs dataset, R-DEIM Net achieves 90.07% accuracy and 90.16% F1-score via 10-fold cross-validation. This represents competitive performance with strong transformer-based baselines (e.g., MFAE BERT: 90.54% accuracy) and recent large language model based approaches (LLaMA-70B) while using a substantially smaller parameter budget. The model generates rationales alongside predictions, providing potential for auxiliary human-readable descriptions.

Figures & tables

Figure 1: R-DEIM Net Architecture. A dual-expert multi-task framework. Branch A (Interaction) captures token-to-token patterns via multi-scale 2D-CNNs, while Branch B (Reasoning) distills latent decoder trajectories into a “reasoning vector” to ground classification in natural language rationales.

Model	Size	Train Acc.	Train $F_{1}$	Test Acc.	Test $F_{1}$
bert-base-uncased
( Devlin et al., 2018 )	110M	$90.95\pm 10.04$	$89.54\pm 14.49$	$85.99\pm 8.11$	$84.63\pm 12.64$
paraphrase-albert-base-v2	12M	$88.29\pm 6.26$	$87.87\pm 6.74$	$85.00\pm 3.99$	$84.49\pm 4.43$
paraphrase-MiniLM-L3-v2	17M	$90.55\pm 1.53$	$90.61\pm 1.53$	$86.52\pm 0.57$	$86.62\pm 0.58$
paraphrase-MiniLM-L6-v2	22M	$91.16\pm 1.69$	$91.25\pm 1.67$	$87.05\pm 0.71$	$87.20\pm 0.66$
paraphrase-MiniLM-L12-v2	33M	$\mathbf{92.97\pm 1.30}$	$\mathbf{93.05\pm 1.27}$	$\mathbf{88.42\pm 0.37}$	$\mathbf{88.56\pm 0.35}$

Table 1: Encoder benchmarking on QQP.

Model	Acc.	$F_{1}$ -Score
ABCNN ( Yin et al., 2016 )	63.59%	-
Syn-tree ( Chen et al., 2017 )	75.50%	-
Siamese-CNN ( Wang et al., 2017 )	79.60%	-
MP-CNN ( Wang et al., 2017 )	81.38%	-
XGBoost ( Ansari and Sharma, 2020 )	82.44%	80.44%
Siamese-LSTM ( Wang et al., 2017 )	82.58%	-

Table 2: Comparative Performance. R-DEIM Net obtains strong accuracy and

F_{1}

on QQP using a compact architecture (76M params).

Dataset	Acc.	$F_{1}$	Epochs	Acc.	$F_{1}$
	Zero-shot		Fine-tuning
SoDD ( Pašek et al., 2022 )	73.50	63.01	3	88.60	88.40
PIT ( Xu et al., 2015 )	68.97	70.17	3	81.26	81.04
PAWS ( Zhang et al., 2019 )	53.45	53.54	2	68.80	67.97
MRPC ( Dolan and Brockett, 2005 )	50.92	50.29	3	69.51	65.85
SciTail ( Khot et al., 2018 )	66.60	60.60	2	81.47	81.59

Table 3: Transfer learning results of R-DEIM Net.

Figure 2: Structural Interaction Distribution. ECDFs of diagonal interaction ratios. The significant rightward shift for duplicates validates that semantic identity is strongly correlated with positional interaction mass.

Interaction Regime	k5/k3	k7/k3
Low	1.14	1.15
Mid	1.11	1.09
High	1.09	1.04

Table 4: Relative activation of larger CNN kernels normalized by the

3\times 3

kernel.

Figure 3: Token-level perturbation analysis. Content words (Nouns, Verbs) exert the strongest influence on model decisions.

Figure 4: Group-level POS perturbation. The model is highly robust to the removal of syntactic function words but sensitive to the loss of semantic content.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Train ( $N$ )	Test ( $N$ )	Ratio ( $0{:}1$ )
SODD	804,576	94,514	$14{:}5$
SprintFAQ	80,800	20,200	$100{:}1$
PAWS	49,175	2,000	$63{:}50$
CQADupStack	37,924	9,482	$1{:}1$
SciTail	23,088	2,126	$17{:}10$
PIT (Twitter)	11,530	838	$15{:}8$

Appendix

Table 5: Dataset statistics and class ratios (

0{:}1

Parameter	Value
Shared Encoder	MiniLM-L12-v2 (384-dim)
SLM Decoder	Flan-T5-small (512-dim)
CNN Out Channels (per kernel)	128
CNN Kernel Sizes	[3, 5, 7]
ANN Hidden Size	512
ANN Dropout	0.3

Appendix

Table 6: Hyperparameters and configuration of R-DEIM Net.

Explore similar work

CAPS: Cascaded Adaptive Pairwise Selection for Efficient Parallel Reasoning
May 15, 2026Fangzhou Lin, Shuo Xing, Peiran Li +6Reasoning Benchmark
Texas A&M University · Georgia Institute of Technology · Tohoku University +1
ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
Nov 24, 2025Long Lian, Sida Wang, Felix Juefei-Xu +7LLM Reasoning Strategies Inference Latency
Meta Superintelligence Labs (MSL), Menlo Park, CA, United States · UC Berkeley, Berkeley, CA, United States · UCSF, San Francisco, CA, United States
Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoning
Apr 17, 2026Jiaxi Bi, Tongxu Luo, Wenyu Du +2Large Reasoning Models
The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute · DualityRL

May 15, 2026, cs.AIJ/K move · Enter open · S save

CAPS: Cascaded Adaptive Pairwise Selection for Efficient Parallel Reasoning

Fangzhou Lin, Shuo Xing, Peiran Li, Siyuan Yang+5

Texas A&M University · Georgia Institute of Technology · Tohoku University+1

Reasoning Benchmark

Open paper AskPDF arXiv

Parallel reasoning, where a generator samples many candidate solutions and an aggregator selects the best, is one of the most effective forms of test-time scaling in large language models, and pairwise self-verification has become its strongest aggregation primitive. Yet pairwise verification carries a heavy cost: each judgment reads two complete solutions in full, and existing methods perform tens of such judgments per problem regardless of whether the comparison is informative. We introduce CAPS (Cascaded Adaptive Pairwise Selection), an inference-only framework that allocates verifier compute non-uniformly along two orthogonal axes: an evidence axis that adapts how much of each candidate the judge sees, and a distribution axis that adapts how comparisons are spread across the pool. CAPS instantiates these into a four-stage cascade with an optional rescue subroutine, and admits a closed-form verifier-token cost in which the per-candidate marginal cost is roughly halved relative to uniform full-evidence schedules. On four self-verifying models (Qwen3-14B, GPT-OSS-20B, Qwen3-4B-Instruct/Thinking) and five reasoning benchmarks spanning code (LiveCodeBench-v5/v6, CodeContests) and math (AIME 2025, HMMT 2025), CAPS outperforms the leading pairwise verifier on 14 of 20 suites while using 25.4% of its verifier-token budget on code, and outperforms pointwise self-verification on all 20. The trade-off suites admit an interpretable diagnostic in terms of the verifier's accuracy at partial versus full evidence, providing a concrete pre-deployment check for cascade suitability.

Abstract

Explore similar work

CAPS: Cascaded Adaptive Pairwise Selection for Efficient Parallel Reasoning

ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models

Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoning