From Document Layout to Causal Topology: An End-to-End Architecture for Temporal Causal Reasoning
Authors: Zhifeng Yang, Wei Luo, Yongjian Chen, Shiqi Ou
Organizations: School of Future Technology, South China University of Technology · Guangdong Artificial Intelligence and Digital Economy Laboratory (Guangzhou)
Large language models (LLMs) generally lack deep causal reasoning capabilities in temporal reasoning. Inspired by document layout reordering, this paper proposes a paradigm shift from discrete narratives to causal topologies. It constructs two architectures: a training-free neuro-symbolic framework, temporal causal reasoning agent (TCR-Agent), which achieves interpretable causal deduction and counterfactual truncation through external causal graph instantiation and a symbolic intervention engine; and an end-to-end model, temporal causal reasoning former (TCR-Former), which internalizes these mechanisms into the Transformer attention space via a Causal-Topological Attention Mask and temporal span biases. Experiments on the TempoCausal benchmark demonstrate that TCR-Agent comprehensively surpasses existing training-free baselines. At the same time, TCR-Former achieves an accuracy 16.5% higher than that of state-of-the-art (SOTA) methods, outperforming large-scale models such as GPT-5.4 and DeepSeek-V3.2 with only approximately 8.6B parameters. Further analysis reveals a structure-reasoning disconnection phenomenon in existing methods: external symbolic constraints and chain-of-thought reasoning can improve reasoning steps, yet fail to enhance deep causal deductive capability. By internalizing temporal causal reasoning capabilities into the model, TCR-Former effectively bridges this gap, providing an efficient and reliable end-to-end foundation for complex reasoning scenarios.
Figures & tables
Figure 1: TCR-Former architecture.
Method
Algorithm
EM ( ↑ )
F1 ( ↑ )
LLM-Acc ( ↑ )
Train-Free
GPT-5.4
Standard
28.8
44.4
51.3
TG-LLM
32.0
49.3
57.1
Time-CoT
30.0
47.8
54.5
TISER
37.6
52.1
59.2
TCR-Agent (Ours)
46.9
65.6
70.4
Table 1: TempoCausal benchmark experimental results. The Bold numbers indicate the best performance among all models.
Figure 2: Experimental results on each dimension of the TempoCausal-Bench (LLM-Acc).
Component
Architecture Setting
EM ( ↑ )
F1 ( ↑ )
LLM-Acc ( ↑ )
TCR-Agent Base: GPT-5.4
Full TCR-Agent
46.9
65.6
70.4
w/o Scratchpad
43.2
60.1
66.8
w/o DAG Instantiation
40.1
55.3
63.1
w/o Counterfactual Intervention
44.0
62.7
67.5
w/o Dual-Track Reasoning
41.3
58.6
65.2
Standard
28.8
44.4
51.3
Table 2: Ablation experiment results.
Figure 3: Indicator visualization. (a): Comparison of reasoning efficiency. (b): Robustness analysis of causal graphs. (c): Error Percentage.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Method
TempReason (L3)
TGQA
EM
F1
EM
F1
T5-large [ 60 ] †
28.8
46.8
54.8
71.3
Temp-T5 [ 27 ] †
26.1
43.0
64.0
77.8
REMEMO-large [ 61 ] †
33.4
49.3
46.1
66.0
TG-LLM [ 4 ] †
35.6
46.9
79.7
85.0
TISER [ 15 ] †
91.5
94.9
84.5
94.2
Appendix
Table 3: Out-of-domain experimental results. Results with † are reported in the original papers [ 15 ] .
Component
Architecture Setting
D1
D2
D3
D4
D5
TCR-Former Base: Qwen-3-8B
Full TCR-Former
85.0
79.0
74.8
86.5
51.8
w/o Causal-Topological Attention Mask
82.1
73.5
65.7
70.3
42.5
Standard
48.3
37.6
30.8
60.4
25.1
Appendix
Table 4: Ablation of Causal-Topological Attention Mask (LLM-Acc). D1: Implicit temporal windows and conditional filtering. D2: Concurrent multi-entity state tracking. D3: Temporal lags and multi-hop dependencies. D4: Counterfactual cascading deduction. D5: Lagged amplification of minute perturbations.
D1: Implicit Temporal Windows and Conditional Filtering
Guide the model away from explicit date matching; require it to delineate boundaries through event anchors and complete conditional filtering.
"During the period from [Event A’s occurrence] to [Event B’s occurrence], how many [specific entities/states] satisfy [Condition C]?"
D2: Concurrent Multi-Entity State Tracking
Break the single-entity linear narrative structure; synchronously track the states of multiple entities and identify logical conflicts.
"Throughout the entire period when [Entity X] is in [continuous State A], what complete [State B] transitions does [Entity Y] undergo?"
D3: Temporal Lags and Multi-Hop Dependencies
Test the model’s ability to assemble scattered event fragments into a DAG, examining its topological reasoning level in reconstructing long-range causal chains across paragraphs.
"If the most critical precondition leading to [final Event Z] is [Event Y], then at the exact moment of [Event Y]’s occurrence (e.g., [time offset] prior), what action is [Entity X] performing?"
D4: Counterfactual Cascading Deduction
Examine the model’s understanding of the text’s intrinsic rules and require it to infer a counterfactual timeline based on those rules.
"Had [historical turning point Event A] not occurred, according to [the text’s established rules/values], how would [Event B], originally scheduled at [Time T], have evolved?"
D5: Lagged Amplification of Minute Perturbations
Extract asymmetric causality and long-lag effects; identify early weak triggering signals.
"What was the most fundamental local triggering event that caused [Entity X] to make [a major structural change] at [late Time T2 ]? From the first appearance of this weak signal to the eventual materialization of the macroscopic change, how long was the incubation period?"
Appendix
Table 6: Construction of the five-dimensional test for TempoCausal-Bench.
Stage 1: Question-Answer Pairs Construction Prompt Example
Please generate 5 complex questions based on the provided story, following the methods outlined below. Provide the standard answer for each question. Do NOT provide explanations.
Requirements for questions and answers: - Use the original text content as much as possible for both questions and answers. - The generated questions will be used to test other models (evaluated by EM and F1 metrics), so the answers MUST be concise. Do not ask questions that require long textual answers.
Construction Methods: Dimension 1: Implicit Temporal Windows and Conditional Filtering Core Testing Capability: Reject direct date matching. Require the model to define time boundaries via “event anchors” (Interval Bounding) and perform extraction and counting of specific entities or actions within those implicit boundaries. Construction Steps: Step 1: Select two non-simultaneous causal or sequential events from the text (Event A as the start, Event B as the end). Step 2: Specify a state or entity type that changes within this interval. Step 3: Add a distractor condition to be excluded (e.g., a person from the previous term, goods that were not successfully delivered). Question Template: “Between [the occurrence of Event A] and [the occurrence of Event B], how many [specific entities/states] satisfy [Condition C]?”
Dimension 2: Concurrent Multi-Entity State Tracking Core Testing Capability: Break away from single-entity linear tracking. Require the model to simultaneously track the independent states of two or more entities within the same time slice or interval, and find their intersections or logical conflicts. Construction Steps: Step 1: Identify two or more interacting entities in the text (e.g., a politician and their party, competitors, or a company and a regulator). Step 2: Lock onto a persistent state of one entity (e.g., “during the period of being out of office” or “while holding a certain position”). Step 3: Ask what key state transition the other entity underwent during this period. Question Template: “Throughout the entire period when [Entity X] was in [persistent state A], what complete [state B] transitions did [Entity Y] experience?”
Dimension 3: Temporal Lags and Multi-Hop Dependencies Core Testing Capability: Test the model’s ability to assemble independent events scattered across multiple paragraphs—with relative time offsets (e.g., “two years later,” “subsequently,” “upon the expiration of X’s term”)—into a complete directed acyclic graph (DAG). Construction Steps: Step 1: Locate an outcome event in the text. Step 2: Trace backward to identify multiple preceding steps leading to that event, ensuring these steps are not adjacent in the text but described with time offsets. Step 3: Ask about a hidden node somewhere in the middle of this chain. Question Template: “If the most critical prerequisite leading to [final event Z] is [event Y], then exactly [time offset, e.g., two years earlier / the previous term] when [event Y] occurred, what action was [Entity X] performing?”
Dimension 4: Counterfactual Cascading Deduction Core Testing Capability: Examine the model’s understanding of the text’s internal logical rules. Remove or invert an already occurred historical node, and require the model to deduce the collapsed timeline outcome based on rules provided in the text (e.g., electoral seat mathematics, eligibility rules for holding office). Construction Steps: Step 1: Find a historical turning point in the text that involves explicit quantitative rules or prior qualification constraints (e.g., a very close vote, a narrow win in a bid). Step 2: Assume that this turning point did NOT happen (i.e., invert the variable). Step 3: Ask how the immediate subsequent historical event on the counterfactual timeline would be rewritten, without making the question too complex. Question Template: “If [historical turning point event A] had not happened, given [the established rules/numbers in the text], what would have happened to [event B], which originally occurred at [time T]?”
Appendix
Table 7: Question-answer pair construction prompt example.
Stage 2: Question-Answer Pair Checking Prompt Example
Please evaluate the following question-answer pairs based on the provided story. For each question, check if the provided answer is correct according to the story. If the answer is incorrect, please correct it. If the question is out of the scope of the story and cannot be answered, set the corrected answer to “Unknown”.
Requirements for the corrected answers: - The generated questions will be used to test other models (evaluated by EM and F1 metrics), so the answers MUST be concise. Do not ask questions that require long textual answers.
Story: {Economic and Financial Committee (European Union). The Economic and Financial Committee (EFC) is an advisory body of the European Union, as defined by Article 134 of the Treaty on the Functioning of the European Union. Its president is also the president of the Eurogroup Working Group, which prepares dossiers for approval by the Eurogroup, whose decisions are generally ratified by ECOFIN. History: Before the third stage of the Economic and Monetary Union (EMU), which began on 1 January 1999, the Economic and Financial Committee was preceded by the Monetary Committee, which was established by Article 105 of the Treaty of Rome in 1957. In May, as part of the European Semester 2016 Spring Package, the committee was tasked with making reports on Belgium, Finland, and Italy’s government debt. Agenda and composition of the EFC…}
QA Pairs: {qa_json_str}
Please return the output in JSON format only. The root must be a JSON object with a key “checked_qa_list”, and its value is a list of objects corresponding to the input QA pairs in the same order. Each object should have the following keys: - “dimension”: The original dimension - “question”: The original question - “original_answer”: The original answer - “is_correct”: true or false, whether the original answer was correct - “answer”: The concise standard answer (use the original if it was correct, the corrected one if incorrect, or “Unknown” if out of scope)
You are a professional assistant for constructing and correcting datasets. You need to strictly follow the specified dimensions and rules to rewrite the questions that cannot be answered (Unknown) based on the given original story (Story). Please ensure to return in the legal JSON format.
Dimension Construction Rules: {dimension_rules}
Original Story: {Economic and Financial Committee (European Union). The Economic and Financial Committee (EFC) is an advisory body of the European Union, as defined by Article 134 of the Treaty on the Functioning of the European Union. Its president is also the president of the Eurogroup Working Group, which prepares dossiers for approval by the Eurogroup, whose decisions are generally ratified by ECOFIN. History: Before the third stage of the Economic and Monetary Union (EMU), which began on 1 January 1999, the Economic and Financial Committee was preceded by the Monetary Committee, which was established by Article 105 of the Treaty of Rome in 1957. In May, as part of the European Semester 2016 Spring Package, the committee was tasked with making reports on Belgium, Finland, and Italy’s government debt. Agenda and composition of the EFC…}
Pending Q&A Pairs: {qa_json_str}
Task Requirements: 1. Carefully read the 5 pairs of questions and answers provided above. 2. For the pairs where the “answer” is “Unknown”, explain that the original question cannot be answered within the story (beyond the scope of the text). Please, based on the original story and the corresponding dimension rules, rewrite a new and challenging question and provide a concise answer. 3. The rewritten question must be able to find a precise answer within the original text. It must strictly adhere to the “core testing ability” and “question template” in the corresponding dimension rules. 4. For the pairs where the “answer” is not “Unknown”, please keep the original text unchanged. Please return the result in a strict JSON format. The root node must be an array named “rewritten_qa_list”, and this array must contain 5 objects, maintaining the same order as the input question-answer pairs.
The format of each object is as follows: { “dimension”: “The name of the dimension, for example, Dimension 5”, “question”: “The original question (if not modified) or the new question after rewriting”, “answer”: “The original answer (if not modified) or the concise answer after rewriting”, “is_rewritten”: true or false (if rewriting has been done, it is true; otherwise, it is false) }
Appendix
Table 9: Sample reconstruction prompt example.
Examples of Samples From the First Stage Dataset
{ "text_chunks" : [ "Jaroslav Pelikan … was born on December 17, 1923, in Akron, Ohio … He earned a seminary degree from Concordia Seminary in 1946 … He completed his Ph.D. at the University of Chicago in 1953. Subsequently, in 1962, he joined the faculty at Yale University …" ], "prompt_texts" : [ "Extract the chronological timeline of causal events." ], "target_events" : [ "Jaroslav Pelikan earned a degree from Concordia Seminary.", "Jaroslav Pelikan completes Ph.D. at the University of Chicago.", "Jaroslav Pelikan joins the faculty at Yale University." ], "target_timestamps" : [0.35, 0.52, 0.78] }
Appendix
Table 10: Examples of samples from the first stage dataset.
Examples of Samples From the Second Stage Dataset
{ "text_chunks" : [ "Jaroslav Pelikan … was born on December 17, 1923, in Akron, Ohio … He earned a seminary degree from Concordia Seminary in 1946 … He completed his Ph.D. at the University of Chicago in 1953. Subsequently, in 1962, he joined the faculty at Yale University …" ], "prompt_texts" : [ "Please reconstruct the exact timeline of events based on the text." ], "target" : "Jaroslav Pelikan earns his degree from Concordia Seminary around 1946.\nJaroslav Pelikan completes his Ph.D. at the University of Chicago around 1953.\nJaroslav Pelikan joins the faculty at Yale University around 1962." }
Appendix
Table 11: Examples of samples from the second stage dataset.
LLM Judgment Prompt Example
You are an expert evaluator for temporal causal reasoning tasks.
Your job is to judge whether a model’s predicted answer is semantically equivalent to the ground-truth answer. Do not focus on superficial wording differences. Focus on the core facts, logical consistency, and causal implications. If the prediction conveys the same meaning as the ground truth, it should be considered correct.
Based on the above, is the Response Answer semantically correct? Please first provide a brief reasoning for your decision, then output your final verdict. Use the following format exactly:
<reasoning> [Your reasoning here] </reasoning> <verdict> Correct or Incorrect </verdict>
Appendix
Table 12: LLM judgment prompt example.
Figure 4: Confusion matrix of the LLM judgment (Qwen-3-8B).
TG-LLM Prompt Example
Example: Timeline: (Liam Mitchell was born in Harrison) starts at 1885 (Maddox Reynolds was born in Glenville) starts at 1893 (Liam Mitchell was married to Emma Scott) starts at 1916 (Emma Scott was married to Liam Mitchell) starts at 1916 (Liam Mitchell was married to Emma Scott) ends at 1918 (Emma Scott was married to Liam Mitchell) ends at 1918 (Maddox Reynolds was married to Liam Mitchell) starts at 1922 (Liam Mitchell was married to Maddox Reynolds) starts at 1922 (Maddox Reynolds was married to Liam Mitchell) ends at 1928 (Liam Mitchell was married to Maddox Reynolds) ends at 1928 (Liam Mitchell died in Boston) starts at 1941 (Emma Scott died in Oceanview) starts at 1984
Question: True or false: Event (Liam Mitchell was married to Maddox Reynolds) was longer in duration than event (Emma Scott was married to Liam Mitchell)?
Useful information: 1885 before 1893 before 1916 before 1918 before 1922 before 1928 before 1941 1918−1916=21928−1893=351928−1922=61941−1918=232<6<23<35
Answer: Let’s think step by step. To determine whether the statement is true or false, we need to compare the durations of (Liam Mitchell was married to Maddox Reynolds) and (Emma Scott was married to Liam Mitchell). The duration for each event can be calculated as follows: (Liam Mitchell was married to Maddox Reynolds): (Liam Mitchell was married to Maddox Reynolds) starts at 1922 (Liam Mitchell was married to Maddox Reynolds) ends at 1928 1928−1922=6 (Emma Scott was married to Liam Mitchell): (Emma Scott was married to Liam Mitchell) starts at 1916 (Emma Scott was married to Liam Mitchell) ends at 1918 1918−1916=2 Now, we compare the duration of the two events: 6 is greater than 2; thus, the answer is True.
Test: Timeline: (Chris Evans was born in Bristol, Connecticut) starts at 1948 (Oliver Jensen was born in Bristol, Connecticut) starts at 1948 (Chris Evans was married to Oliver Jensen) starts at 1970 (Oliver Jensen was married to Chris Evans) starts at 1970 (Chris Evans created Pearl Network) starts at 2005 (Chris Evans won prize Victory Achievement Award) starts at 2007 (Chris Evans was married to Oliver Jensen) ends at 2010 (Oliver Jensen was married to Chris Evans) ends at 2010 (Chris Evans created Pearl Network) ends at 2013
Question: True or false: Event (Chris Evans owned Pearl Network) was longer in duration than event (Oliver Jensen was married to Chris Evans)?
Appendix
Table 13: TG-LLM prompt example.
Time-CoT Prompt Example
Response Template: Let’s think step by step. First, we need to find out the <Standard Question Entity> according to the timeline. Then, we need to focus on the time <Explicit Time Parameter>. So we can obtain the answer is <Gold Answer>. Therefore, the answer(s) is/are …
Example: Question: Which movie did Zoe Cassavetes direct in 2007 ? Answer: Let’s think step by step. First, we need to find out which movie did Zoe Cassavetes direct according to timeline. 2000: Men Make Women Crazy Theory, 2007: Broken English, 2012: The Powder Room, 2015: Day Out of Days…… Then, we need to focus on the time in 2007. So we can obtain the answer in Broken English. Therefore, the answer(s) is/are Broken English.
Question: Who is the quarterback for the New York Giants as of December 2010 ? Answer: Let’s think step by step. First, we need to find out who is the quarterback for the New York giants according to timeline. 1999-2003: Kerry Collins, 2003: Jesse Palmer, 2004: Kurt Warner, 2004-2019: Eli Manning, 2017: Geno Smith, 2019-2023: Daniel Jones, 2020: Colt McCoy, etc. Then, we need to focus on the time December 2010. So we can obtain the answer is Eli Manning. Therefore, the answer(s) is/are Eli Manning.
Test: Question: Who was the president of the US when Adele Faccio was a teenager? Answer: Let’s think step by step.
Appendix
Table 14: Time-CoT prompt example.
TISER Prompt Example
You are an AI assistant that uses a Chain of Thought (CoT) approach with reflection to answer queries. Follow these steps: 1. Reason through the problem step by step within the <reasoning> tags. 2. Given your previous reasoning, identify relevant temporal events in the given context for answering the given question within <timeline> tags. Assume relations in the context are unidirectional. 3. Reflect on your reasoning and the timeline to check for any errors or improvements within the <reflection> tags. 4. Make any necessary adjustments based on your reflection. If additional reasoning is required, go back to Step 1 (reason through the problem step by step); otherwise, move to the next step (Step 5). 5. Provide your final, concise answer within the <answer> tags. If the answer is a number, just output the number, nothing else. Otherwise, output the entity or event without any additional comments.
Additional Instructions: 1. [IF during data generation] Perform your reasoning knowing that the answer is: X. 2. The <reasoning>, <reflection>, and <timeline> sections are for your internal reasoning process. All the reflection and the timeline have to be contained inside the thinking section. 3. Do not use enumerations or lists when writing; use plain text such as paragraphs. 4. The response to the query must be entirely contained within the <answer> tags.
Response Format: <reasoning> [Your step-by-step reasoning goes here. This is your internal thought process.] <timeline> [Relevant temporal events for answering the given question.] </timeline> <reflection> [Your reflection on your reasoning, checking for errors or changes required.] </reflection> [Any adjustments to your thinking based on your reflection.] </reasoning> <answer> [Your final, concise answer to the query.] </answer>
Causal reasoning in language models studies how models identify cause-and-effect structure, predict interventions, and answer counterfactual questions.
Performing causal reasoning akin to humans is fundamental to achieving strong artificial intelligence. However, existing studies on causal reasoning and large language models (LLMs) often underestimate the role of human causal cognition. Human causal reasoning is far more than recognizing causes and effects. Humans often factor in many other considerations when establishing a causal link between two events, such as norm violations, intentionality, morality, counterfactual scenarios, etc. To simulate such reasoning, it is intuitive to identify cause-effect pairs using token causality formalisms while modeling the factors that influence human causal judgment using findings from cognitive science experiments. To support the evaluation and modeling of human-like causal reasoning in LLMs, we introduce HCR-Bench, a challenging benchmark with 1,093 instances and detailed human causal reasoning step annotations, and HCR-Reasoner, an approach that systematically integrates theories of token causality and cognitive science into LLMs for human-like causal reasoning. Results suggest that HCR-Reasoner consistently improves LLMs' causal alignment with humans, and that theory-guided reasoning is highly effective.
Yanxi Zhang, Xin Cong, Zhong Zhang +3
Center for Data Science, AAIS, Peking University · Wangxuan Institute of Computer Technology, Peking University · Tsinghua University +1
Large Language Models (LLMs) still suffer from severe hallucinations and catastrophic forgetting during causal reasoning over massive, fragmented long contexts. Existing memory mechanisms typically treat retrieval as a static, single-step passive matching process, leading to severe semantic dilution and contextual fragmentation. To overcome these fundamental bottlenecks, we propose MemCoT, a test-time memory scaling framework that redefines the reasoning process by transforming long-context reasoning into an iterative, stateful information search. MemCoT introduces a multi-view long-term memory perception module that enables Zoom-In evidence localization and Zoom-Out contextual expansion, allowing the model to first identify where relevant evidence resides and then reconstruct the surrounding causal structure necessary for reasoning. In addition, MemCoT employs a task-conditioned dual short-term memory system composed of semantic state memory and episodic trajectory memory. This short-term memory records historical search decisions and dynamically guides query decomposition and pruning across iterations. Empirical evaluations demonstrate that MemCoT establishes a state-of-the-art performance. Empowered by MemCoT, several open- and closed-source models achieve SOTA performance on the LoCoMo benchmark and LongMemEval-S benchmark.
Haodong Lei, Junming Liu, Yirong Chen +2
Southeast University Nanjing, Jiangsu, China · Shanghai Artificial Intelligence Laboratory Shanghai, China
Despite significant advances, large language models (LLMs) continue to exhibit brittle performance on complex temporal reasoning tasks. This failure mode is widely attributed to inherent deficits in autoregressive logical deduction. In this paper, we challenge this prevailing narrative, demonstrating that temporal reasoning is not the fundamental bottleneck; rather, the locus of failure lies in unstructured text-to-event representation. We introduce a novel neuro-symbolic question-answering framework governed by a Probabilistic Inconsistency Signal (PIS) that explicitly isolates perceptual errors from reasoning failures. By lifting unstructured text into explicit event graphs and interval constraints, our architecture strictly decouples semantic extraction from a symbolic reasoning engine. To robustly detect structural breaks, the PIS elegantly unifies symbolic credal intervals with epistemic neural uncertainty extracted via Evidential Deep Learning on LLM hidden states. Empirical evaluations reveal a striking paradigm shift: when provided with correct structural representations, our system's explicit proof traces achieve perfect 1.0 accuracy (4000/4000) and strictly zero false positives/negatives on temporal arithmetic benchmarks. On broader, noise-injected QA settings, the framework maintains a competitive 75.1% accuracy while enabling deterministic, step-level failure localization. Ultimately, by isolating the representation bottleneck from the reasoning substrate, this work reframes temporal QA from an algorithmic reasoning challenge to a structural alignment problem, charting a verifiable path forward for reliable neuro-symbolic AI.
Tran Quang Liem
Mathematics · VNUHCM - HIGH SCHOOL FOR THE GIFTED · Ho Chi Minh City, Vietnam