Retrieval shapes how language models access and ground knowledge in retrieval-augmented generation (RAG). In historical research, the target is often not an arbitrary relevant passage, but the exact record for a specific regnal month, where temporal consistency matters as much as topical relevance. This is especially challenging for Classical Chinese annals, where time is expressed through terse, implicit, non-Gregorian reign phrases that must be interpreted from surrounding context, so semantically plausible evidence can still be temporally invalid. We introduce ChunQiuTR, a time-keyed retrieval benchmark built from the Spring and Autumn Annals and its exegetical tradition. ChunQiuTR organizes records by month-level reign keys and includes chrono-near confounders that mirror realistic retrieval failures. We further propose CTD (Calendrical Temporal Dual-encoder), a time-aware dual-encoder that combines Fourier-based absolute calendrical context with relative offset biasing. Experiments show consistent gains over strong semantic dual-encoder baselines under time-keyed evaluation, supporting retrieval-time temporal consistency as a key prerequisite for faithful downstream historical RAG. Our code and datasets are available at github.com/xbdxwyh/ChunQiuTR.
Figures & tables
Figure 1: A query about a specific month can retrieve same-month commentary that repeats the date phrase, or adjacent-month near-miss events with confusable wording, so a retrieval-augmented model answers fluently but at the wrong time.
Figure 2: Overview of the ChunQiuTR construction. ( Left ) Time-key alignment yields event-level records, augmented with chrono-near counterfactual hard negatives from later histories. ( Right ) We define P/G/W temporal queries and leak-free, reign-aligned splits.
Figure 3: Overview of our Calendrical Temporal Dual-encoder (CTD). A shared Transformer dual-encoder encodes queries and records into embeddings. Temporal heads place each text on a unified regnal calendar axis (latent time scalar), supporting (a) absolute context injection and (b) relative biasing to form sijCTD . (c) Interval-overlap supervision marks in-batch multi-positives by query–record overlap and trains a multi-positive contrastive loss.
Method
Pub
Type
R@1
R@5
R@10
MRR@10
nDCG@10
Sparse retrieval
BM25
–
Sparse
0.3962
0.5209
0.5620
0.4487
0.3404
BM25+TimeKDE
–
Sparse
0.4943
0.6086
0.6709
0.5456
0.4222
SPLADE-IDF (ZS)
arXiv’24
Sparse
0.1361
0.2765
0.3569
0.1971
0.1596
SPLADE- ℓ0(ZS)
SIGIR’25
Sparse
0.0006
0.0309
0.0587
0.0132
0.0143
Fusion / late interaction
Table 1: Test-set retrieval performance on our ChunQiuTR benchmark under the official evaluation protocol.
Subset
Records
Queries
FT baseline
CTD (ours)
MRR
R@1
MRR
R@1
Qi Ji (part)
268
92
0.2081
0.1848
0.2304
0.2065
Jin Ji (part)
820
119
0.1598
0.1345
0.1751
0.1597
Table 2: Cross-corpus pilot on processed Zizhi Tongjian subsets. No target-corpus training is performed.
Method
Single-month ( ∣Q∣=1 )
Multi-month ( ∣Q∣>1 )
R@1
MRR@10
R@1
MRR@10
BM25
0.397
0.413
0.396 ↓ -0.001
0.499 ↑ +0.086
ColBERT-LFM2 (ZS)
0.298
0.312
0.386 ↑ +0.088
0.490 ↑ +0.178
mE5-Large (ZS)
0.259
0.279
0.337 ↑ +0.078
0.422 ↑ +0.143
BERT-base (FT)
0.497
0.516
0.525 ↑ +0.027
0.621 ↑ +0.106
CTD BERT-base (Ours)
0.509
0.530
0.685 ↑ +0.176
0.744 ↑ +0.214
Table 3: Impact of query span on retrieval performance on the test set. We compare single-month ( ∣Q∣=1 ) and multi-month ( ∣Q∣>1 ) queries; for multi-month queries, the numbers in parentheses give absolute changes relative to single-month queries.
Figure 4: Visualization of Qualitative Examples.
Variant
Lmulti
bijtime
cx
R@1
MRR@10
FT baseline
–
–
–
0.5771 —-
0.6044 —-
+ Lmulti
✓
–
–
0.5820 ↑ +0.0049
0.6107 ↑ +0.0063
+ Bias
✓
✓
–
0.5898 ↑ +0.0127
0.6135 ↑ +0.0091
+ Ctx
✓
–
✓
0.5850 ↑ +0.0079
0.6134 ↑ +0.0090
Full (Ours)
✓
✓
✓
0.5923 ↑ +0.0152
0.6194 ↑ +0.0150
Table 4: Ablation study on the test set under the same evaluation protocol as Table 1 . For each metric, the second line reports the change relative to the FT baseline.
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
Text (excerpt + short gloss)
Source
Label
夏,五月,郑伯克段于鄢。
Annals anchor: Zheng subdued Duan at Yan in month 5.
Guliang stresses armed revolt, conduct, and intensified censure.
《春秋谷梁传》 Guliang zhuan 传文 / zhuan
event
Appendix
Table 5: Aligned materials under the reign-based time key “鲁隐公元年五月” for “郑伯克段于鄢”. The annals give the anchor, the three zhuan provide aligned expansions, and later sources yield chrono-near non-target paraphrases.
Original chronicle snippet
Cue / update
Normalized time key
元年,春,王正月。
Initialize a new reign-year (隐公元年), month = 正月
鲁隐公元年正月
三月,公及邾仪父盟于蔑。
New month (三月), inherit current reign-year
鲁隐公元年三月
二年,春,公会戎于潜。
New year (二年) under the same ruler, reset month to 正月
鲁隐公二年正月
元年,春,王正月,公即位。
New ruler detected (桓公), reset year to 元年 and month to 正月
鲁桓公元年正月
Appendix
Table 6: Representative mappings from raw chronicle phrases to normalized reign-based time keys.
Figure 5: Example of event-level grouping under the reign-based time key 「鲁隐公元年十二月」. The model is prompted to split mixed passages and group aligned commentary snippets by event.
Figure 6: LLM-assisted reverse matching from paraphrastic event titles in Lü Zuqian’s Chunqiu ZuoZhuan Shuos (《春秋左氏传说》) to Zuo zhuan passages. We show the full text-only prompt given to a classical-Chinese LLM (DeepSeek), together with one concrete example: for ruler Yin of Lu, year 1, and the subtitle “祭仲谏郑庄封叔段” from Chunqiu ZuoZhuan Shuos , the model proposes the most likely Zuo zhuan passage. The model is required to output only the original Zuo text, or the sentinel token NONE when it cannot decide.
Figure 7: Representative temporal query types in ChunQiuTR, including point queries, past/future windows, around windows, and explicit ranges over reign-based month keys.
Template groups (index range)
Type
BASE (1–12)
point, content-oriented
BASE (13–20)
point, existence / no-event
MONTH_PAST (21–26)
window, past
MONTH_FUTURE (27–31)
window, future
MONTH_AROUND (32–36)
window, around
MONTH_RANGE (37–41)
window, range
Appendix
Table 7: Query template groups used in ChunQiuTR.
Pool
Source
# Raw units
Total chars (K)
Avg. len.
Median
Min
Max
Positive
All (annals + three zhuan )
6641
293.7
44.22
–
–
–
Positive
Chunqiu annals
1532
19.2
12.52
10
2
71
Positive
Gongyang zhuan
1776
44.5
25.05
10
2
671
Positive
Zuo zhuan
1547
189.1
122.20
48
3
2658
Positive
Guliang zhuan
1786
41.0
22.93
11
2
467
Negative
All exegetical layers
9227
1014.9
109.99
–
–
–
Appendix
Table 8: Raw record-level character-length statistics before sentence splitting. Character counts are reported in thousands (K). Positive records come from the Chunqiu annals and the three traditional zhuan , while negative records come from later exegetical layers.
Figure 8: Month-level coverage, gap counts, and gap ratios for the normalized Chunqiu timeline (overall and per Lu ruler).
Split
# months
# records
# queries
Avg. ground-truth recs/query
# event recs
# no-event recs
# neg. comments
Train
2424
16027
13053
7.3
5360
1209
9458
Validation
295
2049
1520
6.8
626
152
1271
Test
317
2096
1653
7.2
782
149
1165
Total
3036
20172
16226
7.2
6768
1510
11894
Appendix
Table 9: Final benchmark statistics and splits over month-level time keys, record-level retrieval units, and queries. The “Avg. ground-truth recs/query” column reports the average number of labeled relevant records per query in each split, and the last three columns break down records by type ( event , no_event , and neg_comment ).
Work / layer
Role in ChunQiuTR
Digital source (edition)
License note / release plan
Chunqiu (春秋)
base corpus
Chinese Wikisource ( Siku Quanshu edition; page revision oldid recorded)
Work pages: tagged PD-old . Platform text: CC BY-SA 4.0 (Wikimedia Terms of Use). We release derived metadata (time keys, alignments, queries/qrels, indices) and scripts to re-fetch raw texts from recorded oldid revisions.
Table 10: Text sources and licensing. All digitized texts are retrieved from Chinese Wikisource ( Siku Quanshu editions), with page revision IDs ( oldid ) recorded for traceability. See Appendix A for alignment and preprocessing.
Backbone
Variant
#GPU
Time / run
GPU-hours
BERT-base-chinese
FT baseline
1
≈ 15 min
≈ 0.25
BERT-base-chinese
CTD (full)
1
≈ 19 min
≈ 0.32
Qwen3-Embed-0.6B
FT baseline
2
≈ 45 min
≈ 1.50
Qwen3-Embed-0.6B
CTD (full)
2
≈ 45 min
≈ 1.50
Appendix
Table 11: Compute cost for representative fine-tuning runs on ChunQiuTR. Time is wall-clock per run; GPU-hours are computed as (#GPU) × (time in hours).
Variant
Lmulti
bijtime
cx
R@1
MRR@10
FT baseline
–
–
–
0.5088 —-
0.5597 —-
+ Lmulti
✓
–
–
0.5178 ↑ +0.0090
0.5685 ↑ +0.0088
+ Bias
✓
✓
–
0.5384 ↑ +0.0296
0.5776 ↑ +0.0179
+ Ctx
✓
–
✓
0.5620 ↑ +0.0532
0.5961 ↑ +0.0364
Full (Ours)
✓
✓
✓
0.5826 ↑ +0.0738
0.6193 ↑ +0.0596
Appendix
Table 12: Ablation on the test set with bert-base-chinese .
Method
Including pure no-event queries (fix neg =0, ne =1)
Hard-negative robustness (fix ne=1, dq=1 )
dq=1
dq=0
Δ
neg=0
neg=1
Δ
R@1
MRR@10
R@1
MRR@10
R@1
MRR@10
R@1
MRR@10
R@1
MRR@10
R@1
MRR@10
BM25
0.240
0.296
0.418
0.470
↑ +0.178
↑ +0.173
0.240
0.296
0.228
0.282
↓ -0.012
↓ -0.014
ColBERT-LFM2 (ZS)
0.220
0.277
0.333
0.386
↑ +0.113
↑ +0.109
0.220
0.277
0.222
0.278
↑ +0.002
↑ +0.001
BERT-base (FT)
0.378
0.439
0.560
0.606
↑ +0.182
↑ +0.167
0.378
0.439
0.306
0.374
↓ -0.072
↓ -0.065
CTD BERT-base (Ours)
0.407
0.456
0.583
0.619
↑ +0.175
↑ +0.163
0.407
0.456
0.407
0.456
0.000
0.000
Appendix
Table 13: No-event and hard-negative behavior under protocol variations on the test set. Left: dq=1 vs. dq=0 (fix ne =1, neg =0), where dq drops or keeps pure no_event queries. Right: neg=0 vs. neg=1 (fix ne =1, dq =1), where neg injects neg_comment passages into the gallery. Δ denotes the within-method change.
Mode
Family
Validation
Test
R@1
R@5
R@10
MRR@10
nDCG@10
R@1
R@5
R@10
MRR@10
nDCG@10
neg0_ne0_dq1
all
0.0407
0.1200
0.2232
0.0798
0.0635
0.0654
0.1466
0.2042
0.1014
0.0677
neg0_ne0_dq1
point
0.0375
0.1148
0.2459
0.0775
0.0756
0.0710
0.1696
0.2387
0.1156
0.0934
neg0_ne0_dq1
window
0.0430
0.1239
0.2065
0.0815
0.0546
0.0610
0.1283
0.1768
0.0902
0.0472
neg0_ne1_dq0
all
0.6329
0.6697
0.7191
0.6522
0.4921
0.5935
0.6497
0.6945
0.6206
0.4588
neg0_ne1_dq0
point
0.5200
0.5440
0.6057
0.5347
0.5380
0.4922
0.5307
0.5869
0.5138
0.5103
Appendix
Table 14: Full protocol grid results for CTD - Qwen3-Embed-0.6B . Mode names follow neg / ne / dq as defined in Section B.2.3 .
Mode
Family
Validation
Test
R@1
R@5
R@10
MRR@10
nDCG@10
R@1
R@5
R@10
MRR@10
nDCG@10
neg0_ne0_dq1
all
0.0159
0.0456
0.0952
0.0331
0.0205
0.0096
0.0366
0.0672
0.0234
0.0192
neg0_ne0_dq1
point
0.0258
0.0468
0.0937
0.0382
0.0265
0.0079
0.0394
0.0690
0.0226
0.0235
neg0_ne0_dq1
window
0.0086
0.0448
0.0964
0.0294
0.0161
0.0110
0.0344
0.0657
0.0240
0.0158
neg0_ne1_dq0
all
0.4474
0.5678
0.5980
0.4989
0.3794
0.4180
0.5408
0.5735
0.4696
0.3543
neg0_ne1_dq0
point
0.4434
0.4709
0.4789
0.4556
0.4612
0.4152
0.4475
0.4506
0.4284
0.4339
Appendix
Table 15: Full protocol grid results for BM25, reported in the same format as Table 14 .
Figure 9: Top-1 near-miss cases on ChunQiuTR (test set). We show the top–5 results from a baseline (left) and ours (middle), with the ground-truth set (right). ⋆ marks ground-truth passages; ✓indicates a ground-truth hit in top–5.
Figure 10: Online LLM without evidence grounding on a month-level point query from ChunQiuTR. For the query “鲁隐公元年十二月”, the gold month contains two entries ( 祭伯来 and 公子益师卒 ). Without evidence, the model either predicts an empty month or returns an incomplete answer.
Figure 11: Evidence-bounded RAG for the same query as Fig. 10 . With a small evidence pack containing the gold month records and confusable materials, the model recovers both gold entries and grounds the answer in cited evidence.
Subset
Approx. coverage
Representative reign titles
Records
Queries
Qi Ji (part)
479–489 CE
建元, 永明
268
92
Jin Ji (part)
265–279 CE
泰始, 咸宁
820
119
Appendix
Table 16: Basic statistics of the processed Zizhi Tongjian subsets used in the cross-corpus pilot.
Model / Setting
Qi Ji (part)
Jin Ji (part)
MRR
R@1
R@5
R@10
MRR
R@1
R@5
R@10
Qwen3-Embed-0.6B (ZS)
0.0692
0.0217
0.0870
0.1413
0.0691
0.0420
0.0756
0.1345
Qwen3-Embed-0.6B (FT baseline)
0.2081
0.1848
0.2174
0.2391
0.1598
0.1345
0.1849
0.1849
CTD (Ours)
0.2304
0.2065
0.2391
0.2717
0.1751
0.1597
0.1849
0.1849
Appendix
Table 17: Cross-corpus pilot results on two processed Zizhi Tongjian subsets. No target-corpus training is performed.
Item
Value
Total non-empty months
1,533
Months containing multiple events
558
Fraction multi-event
36.41%
Extra human corrections
63
Correction rate among multi-event months
11.29%
Corrections among all non-empty months
4.11%
Appendix
Table 18: Audit statistics for multi-event splitting (LLM proposals + human verification).
Source
#Candidates
Accepted
Rejected
Acceptance
Gu Donggao
899
899
0
100.00%
Kong Yingda
5,286
5,179
107
97.98%
Du Yu
5,373
5,266
107
98.01%
Lü Zuqian
360
336
24
93.33%
Appendix
Table 19: Acceptance rates for later-commentary alignments (LLM candidate proposal + human verification).
Agentic retrieval-augmented generation (RAG) enables language models to adapt retrieval based on previously retrieved evidence, but it remains unclear whether such adaptive orchestration consistently outperforms well-designed static pipelines. We conduct a controlled comparison of agentic and static RAG for Taiwanese historical question answering, sharing the same generator and hybrid retrieval backend. Despite similar aggregate performance, the two pipelines differ on 70.83% of questions, with their advantages largely canceling out when averaged. An oracle that selects the better response per question improves the composite score by 0.2417 over the better individual pipeline, revealing substantial headroom for question-level selection. We therefore introduce a post-hoc selector that compares the two responses and their cited evidence, significantly outperforming either individual pipeline and recovering 60.34% of the oracle headroom. These results show that aggregate comparisons can obscure meaningful question-level differences between retrieval strategies, suggesting that exploiting their complementarity may be more fruitful than seeking a universally superior pipeline.
Kai-Hsin Chen, Wei-Yu Chen, Xuanjun Chen +1
Graduate Institute of Networking and Multimedia, National Taiwan University · Dept. of Computer Science & Information Engineering, National Taiwan University · Graduate Institute of Communication Engineering, National Taiwan University
Modern information retrieval (IR) systems rarely represent time, yet many information needs depend on it: in clinical, journalistic, and legal search, when an event occurred can decide whether a document is relevant. Dense retrievers and Retrieval-Augmented Generation (RAG) pipelines match queries to documents well on topic but poorly on time, so they surface content that is on-topic yet temporally wrong. We introduce Temporal Textual Similarity (TTS), a task that measures how well two anchored texts align in time, independent of their topical similarity. We then present TEMPS (Temporal Embedding Model for Precise Search), a modular temporal branch that attaches to a frozen semantic retriever and trains on that signal. It resolves anchored temporal expressions to intervals and moment-matches each one to a Gaussian; the resulting ordering supervises an anchor-date-conditioned encoder, whose score we fuse with the semantic score at inference. Grounding supplies the supervision, so training uses no hand-labeled temporal data. The temporal score itself is the Gaussian-KL inclusion measure from distributional embeddings; what TEMPS adds is the grounding and the moment-matched supervision. On three temporal benchmarks, TEMPS improves MRR for every semantic backbone tested and, on TS- Retriever, lifts R@1 from 19.92 to 25.39 over the prior temporal state of the art.
Mourad Hassani, Julien Romero, Amel Bouzeghoub +1
SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, France · Aldebaran Care, France
Retrieval-Augmented Generation (RAG) has shown strong effectiveness in grounding Large Language Models (LLMs) with external knowledge. However, existing RAG and Graph RAG frameworks largely treat knowledge as static or associate time with coarse-grained timestamps or metadata, failing to capture rich temporal structures such as duration, overlap, and containment. We propose IA-RAG, a hierarchical temporal RAG framework that models knowledge as time intervals and performs retrieval under formal temporal constraints. IA-RAG represents facts as Interval Event Units (IEUs) and organizes them into a hierarchical Thematic Forest, where temporal dependencies are governed by Allen's Interval Algebra. To handle incomplete or uncertain temporal boundaries, IA-RAG further introduces a Sub-graph Time Tightening mechanism that refines fuzzy intervals through logical constraints within connected event subgraphs. In addition, IA-RAG supports implicit temporal semantic retrieval through interval-algebra-guided traversal. Experiments on multiple temporal question answering benchmarks, including TimeQA, TempReason, and ComplexTR, demonstrate that IA-RAG achieves strong temporal retrieval and reasoning performance, particularly on complex compositional temporal reasoning tasks. Our code is released at https://github.com/xiaoAugenstern/LogicalRAG_TemporalQA.
Xiaoman Wang, Yaoze Zhang, Wenzhuo Fan +7
East China Normal University · Shanghai Artificial Intelligence Laboratory · University of Shanghai for Science and Technology +1