Time as Structure: Temporal Dependency Graphs for Verifiable Deadline Computation over Legal Documents

Maryia Zhyrko; Suzan Verberne; Lifeng Han

cs.CLAug 15, 2026

Time as Structure: Temporal Dependency Graphs for Verifiable Deadline Computation over Legal Documents

Authors: Maryia Zhyrko, Suzan Verberne, Lifeng Han

Organizations: Leiden Institute of Advanced Computer Science (LIACS), Leiden University

Abstract

Miss a filing deadline by one day and the claim is barred, however strong the case. Computing that deadline is rarely simple: the period runs from a triggering event, is counted by a statutory convention, and may be suspended by a mandatory conciliation window. We ask whether a language model should answer such questions directly, or read the document and leave the arithmetic to code. We extract dated facts and their dependencies into a temporal dependency graph and compute deadlines from it with a calendar-correct engine. On UK Employment Appeal Tribunal judgments the engine reproduces six of seven timeliness rulings, and matches the judges' own dates to the day. The strongest of four language models, asked the same cases, gets the arithmetic right and the answer wrong: in two of its seven responses its stated verdict contradicts its own working, and across the three GPT models six of twenty-one responses do, each time calling a late claim timely. To test the systems at scale we move the dismissal date across the statutory boundary, generating 427 cases whose answers are computed rather than annotated. On the cases both systems answer, the pipeline is right 90.2% of the time against 61.2% for direct answering. It answers two thirds of the cases, and counting its abstentions as errors it does not exceed direct answering (60.4% against 64.6%). The limit is extraction: on contracts the errors are almost never in the arithmetic, but in choosing which event the period starts from.

Figures & tables

Panel of Figure 1 — Figure 1: System architecture. Either extractor produces a case TDG, and the statute section passes through the same pipeline to yield the rule specification. The deterministic engine binds the statutory anchor concept to a case fact, applies calendar arithmetic and the conciliation pause, and returns either a deadline with its full working or an abstention with a named cause. The dashed path is the direct baseline, which receives the same two inputs and answers in one pass.

Category	Struct.	Naive	gemma	llama
Digits period	100	50	67	63
Words period	100	70	57	40
Vague period	100	67	47	23
2 links, one doc.	100	37	93	0
2 links, cross-doc.	100	40	97	10
3 links, one doc.	100	20	97	3

Table 1: Percentage of items fully correct in the controlled benchmark. A cascade item is correct only when every linked date is correct.

	Rule-based	LLM
Facts extracted	592	288
resolving a date	486	95
usable nodes	188	50
Dependency edges	11	92
additive	9	33
ordering	2	40

Table 2: Extraction over 41 unique contracts. A usable node has a resolved calendar date, is not a bare citation year, and carries a nameable entity. Every computable edge on both sides was audited against its source sentences. We report precision on produced edges. Recall would require dependency-level annotation the corpus does not carry.

Model	Verdicts	Deadlines	Mean $\|\Delta d\|$
gemma4:e4b	1/7	0/6	87.2
gpt-5.4-nano	3/7	0/6	77.3
gpt-5.4-mini	1/7	0/6	99.0
gpt-5.4	3/7	3/6	50.5
Engine, gold facts	6/7	2/2 †	—

Table 3: Direct LLM baseline on redacted judgments, with mean absolute deadline error in days. The model deadline denominator is six, not two: models are scored against the reference deadline on every row for which one is computable, which is the six rows with complete inputs. The seventh (document-bound) row recites neither its deadline nor the conciliation dates that produced it, so no reference deadline exists for it and it is excluded. † The engine is held to the stricter standard of the judgments’ own figures, so it is scored only on the two judge-stated deadlines. It additionally reproduces the one judge-stated boundary date.

Criterion	Answers
Correct verdict	12/14
+ correct day count	9/14
+ correct anchor, rule and action date	9/14

Table 4: The 14 answered cells at three levels of strictness. The criteria are nested, so each line is a subset of the one above. The gap between the first and second lines is the three answers that are correct by margin.

Condition	Cov.	Acc ${}_{\text{all}}$	Acc ${}_{\text{ans}}$	Flips	1-flip
gemma direct	1.00	0.646	0.646	68	1/7
llama direct	1.00	0.602	0.602	117	0/7
gemma pipeline	0.67	0.604	0.902	24	3/7
llama pipeline	0.41	0.335	0.822	18	0/7

Table 5: 427 perturbed items, majority class 0.508. A system tracking the statute flips its verdict exactly once per sweep. Acc

{}_{\text{all}}

counts abstentions as errors and Acc

{}_{\text{ans}}

is over answered items. Coverage differs between conditions, so the matched comparison in the text is the fair one.

Case	$n$	pipeline	direct
Case A (s.111)	60	0.983	0.467
Case A (s.123)	60	0.983	0.567
Case B	49	1.000	0.510
Case C	55	1.000	0.982
Case D	41	0.415	0.390
Case E	14	1.000	0.857

Table 6: Matched per-case comparison, gemma extractor: the items answered by both the pipeline and the direct model (

n=286

of 427). Cases are anonymised as A–F; Case A is evaluated under both statutes. The case is the unit of analysis, so the case-level mean weights every case equally and differs from the item-level mean of Table 5 , which is dominated by the cases the pipeline answers densely.

Condition	Coverage	Incoherent	Verdicts
llama3.1:8b, separate	4/7	4/4	1/7
llama3.1:8b, single	6/7	6/6	4/6
gemma4:e4b, separate	5/7	4/5	4/6
gemma4:e4b, single	7/7	0/7	2/7
gpt-5.4, separate	7/7	4/7	5/7
gpt-5.4, single	7/7	0/7	4/7

Table 7: Consistency on the seven gold rows. Coverage counts cases where all six answers parsed and incoherence is scored over those. Verdicts are scored against the tribunal’s ruling over the rows whose verdict field parsed, which is a larger set, so the two columns are not nested.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Doc	Edge	Verdict
Rule-based (9 computable)
A	P3M from 15 Mar 1986	correct
A	P3Y , “last three years”	spurious
B	P1Y , secondment minimum	spurious
C	P6M vs stated one year	spurious
D	P1M from signature date	spurious

Appendix

Table 8: Every computable edge, audited. The three correct LLM edges share one shape: an explicit trigger phrase naming the anchor concept in the same sentence.

Approach	1st link	2nd link	3rd link
Structured	100	100	100
gemma4:e4b	97	96	93
llama3.1:8b	11	8	3

Appendix

Table 9: Per-link accuracy on the cascade task (%), the share of individual linked dates correct at each step. The structured method is exact at every step by construction.

Panel of Figure 2 — Figure 2: Mechanics of one deadline computation. The primary period runs from the anchor for the statutory period, less one day, under the beginning with convention. Early conciliation suspends the clock between Day A and Day B, and the suspended interval is added to the primary deadline, with a floor of one month after Day B. The engine also checks that Day A falls inside the primary period, since contact made after the deadline has passed grants no extension. Every quantity shown is reported alongside the verdict.

Condition	Anchor	Day A	Day B	Presented
gemma / redacted	4/7	3/4	2/3	7/7
gemma / fulltext	5/7	3/4	2/3	7/7
mini / redacted	5/7	4/4	3/3	7/7
mini / fulltext	6/7	4/4	3/3	7/7

Appendix

Table 10: Extraction recall of gold key dates per pipeline condition, deduplicated per case.

Condition	Anchor-correct	Flips	One flip
gemma direct	—	68	1/7
llama direct	—	117	0/7
gemma pipeline	0.983	24	3/7
llama pipeline	0.939	18	0/7

Appendix

Table 11: Boundary behaviour on the 427 items. Anchor-correct is accuracy on the subset where the extracted anchor equals the true shifted anchor (

n=234

and

n=132

). A system tracking the statute flips exactly once per sweep. On rows where the extracted anchor is wrong the pipeline sits at chance (0.538 and 0.452), which is the behaviour a calibrated system should show when its input is wrong.

Condition	K1	K2	K3	K5
llama, separate	5/6	3/4	3/7	1/7
llama, single	4/6	6/6	3/6	0/6
gemma, separate	3/6	4/7	1/6	1/6
gemma, single	0/7	0/7	0/7	0/7
gpt-5.4, separate	2/7	4/7	0/7	0/7
gpt-5.4, single	0/7	0/7	0/7	0/7

Appendix

Table 12: Constraint violations over evaluable cases. K1 dates, K2 arithmetic, K3 verdict, K5 paraphrase control. The frontier model’s failures are entirely K1 and K2, so its reporting layer never contradicts its own day count.

Explore similar work

Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding
Aug 9, 2026Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Fahmid Hasan Chowdhury +4LLM Evaluation Document QA
1BRAC University · 2United International University · 3North South University +1
Asking For An Old Friend: Diagnosing and Mitigating Temporal Failure Modes in LLM-based Statutory Question Answering
May 22, 2026Max Prior, Andreas Schultz, Matthias GrabmairRetrieval-Augmented Generation Temporal IR
Technical University of Munich Garching near Munich, Germany
LexKairos: Benchmarking Legal Temporal Capabilities in LLMs
Aug 10, 2026Chenyang Li, Zejia Feng, Yuqin Huang +2Legal NLP LLM Evaluation
University of Oxford · Sun Yat-Sen University · Tsinghua University

Aug 9, 2026J/K move · Enter open · S save

Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding

Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Fahmid Hasan Chowdhury, Md Adnan Arefeen+3

1BRAC University · 2United International University · 3North South University+1

LLM Evaluation Document QA Legal QA Temporal QA

Open paper AskPDF arXiv

Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time, so a question has different correct answers at different dates. In contrast to encyclopedic knowledge, where an old fact is simply overwritten, an amendment is itself an official text that states what it replaces and when it takes effect, and the earlier version stays correct for its validity period. The central challenge is therefore version resolution, that is, identifying the version in force on the queried date. Existing temporal QA datasets treat time only as an annotation, so version resolution stays untested. We present TIDE, an expert-verified benchmark of 3,050 QA pairs over 644 official customs instruments issued between 1969 and 2025 by the Government of Bangladesh, covering eight task types over deeply code-mixed documents that are heterogeneous in layout and dated in two calendars. In addition, we evaluate nine recent LLMs under a single protocol across parametric, gold-context, and retrieval access, scored by a three-judge LLM council with a hard date gate separating correct meaning from correct time. The best macro-averaged accuracy is only 68.5%. Resolving a version from an implicit date reaches 59.7%, and detecting that the supplied version does not govern the query reaches only 26.7%. Models are more likely to find correct versions than to reject incorrect ones, and they tend to follow a confident parametric answer over the supplied authoritative text. All code and data are available at https://github.com/icsetepa44/TIDE

Abstract

Explore similar work

Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding

Asking For An Old Friend: Diagnosing and Mitigating Temporal Failure Modes in LLM-based Statutory Question Answering

LexKairos: Benchmarking Legal Temporal Capabilities in LLMs