cs.AIOct 25, 2026
SaveFrom Document Layout to Causal Topology: An End-to-End Architecture for Temporal Causal Reasoning
Organizations: School of Future Technology, South China University of Technology · Guangdong Artificial Intelligence and Digital Economy Laboratory (Guangzhou)
Abstract
Large language models (LLMs) generally lack deep causal reasoning capabilities in temporal reasoning. Inspired by document layout reordering, this paper proposes a paradigm shift from discrete narratives to causal topologies. It constructs two architectures: a training-free neuro-symbolic framework, temporal causal reasoning agent (TCR-Agent), which achieves interpretable causal deduction and counterfactual truncation through external causal graph instantiation and a symbolic intervention engine; and an end-to-end model, temporal causal reasoning former (TCR-Former), which internalizes these mechanisms into the Transformer attention space via a Causal-Topological Attention Mask and temporal span biases. Experiments on the TempoCausal benchmark demonstrate that TCR-Agent comprehensively surpasses existing training-free baselines. At the same time, TCR-Former achieves an accuracy 16.5% higher than that of state-of-the-art (SOTA) methods, outperforming large-scale models such as GPT-5.4 and DeepSeek-V3.2 with only approximately 8.6B parameters. Further analysis reveals a structure-reasoning disconnection phenomenon in existing methods: external symbolic constraints and chain-of-thought reasoning can improve reasoning steps, yet fail to enhance deep causal deductive capability. By internalizing temporal causal reasoning capabilities into the model, TCR-Former effectively bridges this gap, providing an efficient and reliable end-to-end foundation for complex reasoning scenarios.
Figures & tables
Figure 1: TCR-Former architecture.
| Method | Algorithm | EM ( ) | F1 ( ) | LLM-Acc ( ) |
|---|---|---|---|---|
| Train-Free | ||||
| GPT-5.4 | Standard | 28.8 | 44.4 | 51.3 |
| TG-LLM | 32.0 | 49.3 | 57.1 | |
| Time-CoT | 30.0 | 47.8 | 54.5 | |
| TISER | 37.6 | 52.1 | 59.2 | |
| TCR-Agent (Ours) | 46.9 | 65.6 | 70.4 | |
Table 1: TempoCausal benchmark experimental results. The Bold numbers indicate the best performance among all models.
Figure 2: Experimental results on each dimension of the TempoCausal-Bench (LLM-Acc).
| Component | Architecture Setting | EM ( ) | F1 ( ) | LLM-Acc ( ) |
|---|---|---|---|---|
| TCR-Agent Base: GPT-5.4 | Full TCR-Agent | 46.9 | 65.6 | 70.4 |
| w/o Scratchpad | 43.2 | 60.1 | 66.8 | |
| w/o DAG Instantiation | 40.1 | 55.3 | 63.1 | |
| w/o Counterfactual Intervention | 44.0 | 62.7 | 67.5 | |
| w/o Dual-Track Reasoning | 41.3 | 58.6 | 65.2 | |
| Standard | 28.8 | 44.4 | 51.3 |
Table 2: Ablation experiment results.
Figure 3: Indicator visualization. (a): Comparison of reasoning efficiency. (b): Robustness analysis of causal graphs. (c): Error Percentage.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | TempReason (L3) | TGQA | ||
|---|---|---|---|---|
| EM | F1 | EM | F1 | |
| T5-large [ 60 ] † | 28.8 | 46.8 | 54.8 | 71.3 |
| Temp-T5 [ 27 ] † | 26.1 | 43.0 | 64.0 | 77.8 |
| REMEMO-large [ 61 ] † | 33.4 | 49.3 | 46.1 | 66.0 |
| TG-LLM [ 4 ] † | 35.6 | 46.9 | 79.7 | 85.0 |
| TISER [ 15 ] † | 91.5 | 94.9 | 84.5 | 94.2 |
Table 3: Out-of-domain experimental results. Results with † are reported in the original papers [ 15 ] .
| Component | Architecture Setting | D1 | D2 | D3 | D4 | D5 |
|---|---|---|---|---|---|---|
| TCR-Former Base: Qwen-3-8B | Full TCR-Former | 85.0 | 79.0 | 74.8 | 86.5 | 51.8 |
| w/o Causal-Topological Attention Mask | 82.1 | 73.5 | 65.7 | 70.3 | 42.5 | |
| Standard | 48.3 | 37.6 | 30.8 | 60.4 | 25.1 |
Table 4: Ablation of Causal-Topological Attention Mask (LLM-Acc). D1: Implicit temporal windows and conditional filtering. D2: Concurrent multi-entity state tracking. D3: Temporal lags and multi-hop dependencies. D4: Counterfactual cascading deduction. D5: Lagged amplification of minute perturbations.
| Component | Architecture Setting | Error 1 | Error 2 | Error 3 |
|---|---|---|---|---|
| TCR-Former Base: Qwen-3-8B | Full TCR-Former | 3.3 | 3.1 | 2.6 |
| w/o Causal-Topological Attention Mask | 6.1 | 11.1 | 12.3 | |
| Standard | 24.3 | 22.1 | 22.3 |
Table 5: Error proportions under mask ablation (%). Error 1: Temporal Boundary Misjudgment. Error 2: Causal Chain Breakage. Error 3: Rule Inference Mismatch.
| Dimension | Core Tested Capability | Standard Question Template |
|---|---|---|
| D1: Implicit Temporal Windows and Conditional Filtering | Guide the model away from explicit date matching; require it to delineate boundaries through event anchors and complete conditional filtering. | "During the period from [Event A’s occurrence] to [Event B’s occurrence], how many [specific entities/states] satisfy [Condition C]?" |
| D2: Concurrent Multi-Entity State Tracking | Break the single-entity linear narrative structure; synchronously track the states of multiple entities and identify logical conflicts. | "Throughout the entire period when [Entity X] is in [continuous State A], what complete [State B] transitions does [Entity Y] undergo?" |
| D3: Temporal Lags and Multi-Hop Dependencies | Test the model’s ability to assemble scattered event fragments into a DAG, examining its topological reasoning level in reconstructing long-range causal chains across paragraphs. | "If the most critical precondition leading to [final Event Z] is [Event Y], then at the exact moment of [Event Y]’s occurrence (e.g., [time offset] prior), what action is [Entity X] performing?" |
| D4: Counterfactual Cascading Deduction | Examine the model’s understanding of the text’s intrinsic rules and require it to infer a counterfactual timeline based on those rules. | "Had [historical turning point Event A] not occurred, according to [the text’s established rules/values], how would [Event B], originally scheduled at [Time T], have evolved?" |
| D5: Lagged Amplification of Minute Perturbations | Extract asymmetric causality and long-lag effects; identify early weak triggering signals. | "What was the most fundamental local triggering event that caused [Entity X] to make [a major structural change] at [late Time ]? From the first appearance of this weak signal to the eventual materialization of the macroscopic change, how long was the incubation period?" |
Table 6: Construction of the five-dimensional test for TempoCausal-Bench.
| Stage 1: Question-Answer Pairs Construction Prompt Example |
|---|
| Please generate 5 complex questions based on the provided story, following the methods outlined below. Provide the standard answer for each question. Do NOT provide explanations. |
| Requirements for questions and answers: - Use the original text content as much as possible for both questions and answers. - The generated questions will be used to test other models (evaluated by EM and F1 metrics), so the answers MUST be concise. Do not ask questions that require long textual answers. |
| Construction Methods: Dimension 1: Implicit Temporal Windows and Conditional Filtering Core Testing Capability: Reject direct date matching. Require the model to define time boundaries via “event anchors” (Interval Bounding) and perform extraction and counting of specific entities or actions within those implicit boundaries. Construction Steps: Step 1: Select two non-simultaneous causal or sequential events from the text (Event A as the start, Event B as the end). Step 2: Specify a state or entity type that changes within this interval. Step 3: Add a distractor condition to be excluded (e.g., a person from the previous term, goods that were not successfully delivered). Question Template: “Between [the occurrence of Event A] and [the occurrence of Event B], how many [specific entities/states] satisfy [Condition C]?” |
| Dimension 2: Concurrent Multi-Entity State Tracking Core Testing Capability: Break away from single-entity linear tracking. Require the model to simultaneously track the independent states of two or more entities within the same time slice or interval, and find their intersections or logical conflicts. Construction Steps: Step 1: Identify two or more interacting entities in the text (e.g., a politician and their party, competitors, or a company and a regulator). Step 2: Lock onto a persistent state of one entity (e.g., “during the period of being out of office” or “while holding a certain position”). Step 3: Ask what key state transition the other entity underwent during this period. Question Template: “Throughout the entire period when [Entity X] was in [persistent state A], what complete [state B] transitions did [Entity Y] experience?” |
| Dimension 3: Temporal Lags and Multi-Hop Dependencies Core Testing Capability: Test the model’s ability to assemble independent events scattered across multiple paragraphs—with relative time offsets (e.g., “two years later,” “subsequently,” “upon the expiration of X’s term”)—into a complete directed acyclic graph (DAG). Construction Steps: Step 1: Locate an outcome event in the text. Step 2: Trace backward to identify multiple preceding steps leading to that event, ensuring these steps are not adjacent in the text but described with time offsets. Step 3: Ask about a hidden node somewhere in the middle of this chain. Question Template: “If the most critical prerequisite leading to [final event Z] is [event Y], then exactly [time offset, e.g., two years earlier / the previous term] when [event Y] occurred, what action was [Entity X] performing?” |
| Dimension 4: Counterfactual Cascading Deduction Core Testing Capability: Examine the model’s understanding of the text’s internal logical rules. Remove or invert an already occurred historical node, and require the model to deduce the collapsed timeline outcome based on rules provided in the text (e.g., electoral seat mathematics, eligibility rules for holding office). Construction Steps: Step 1: Find a historical turning point in the text that involves explicit quantitative rules or prior qualification constraints (e.g., a very close vote, a narrow win in a bid). Step 2: Assume that this turning point did NOT happen (i.e., invert the variable). Step 3: Ask how the immediate subsequent historical event on the counterfactual timeline would be rewritten, without making the question too complex. Question Template: “If [historical turning point event A] had not happened, given [the established rules/numbers in the text], what would have happened to [event B], which originally occurred at [time T]?” |
Table 7: Question-answer pair construction prompt example.
| Stage 2: Question-Answer Pair Checking Prompt Example |
|---|
| Please evaluate the following question-answer pairs based on the provided story. For each question, check if the provided answer is correct according to the story. If the answer is incorrect, please correct it. If the question is out of the scope of the story and cannot be answered, set the corrected answer to “Unknown”. |
| Requirements for the corrected answers: - The generated questions will be used to test other models (evaluated by EM and F1 metrics), so the answers MUST be concise. Do not ask questions that require long textual answers. |
| Story: {Economic and Financial Committee (European Union). The Economic and Financial Committee (EFC) is an advisory body of the European Union, as defined by Article 134 of the Treaty on the Functioning of the European Union. Its president is also the president of the Eurogroup Working Group, which prepares dossiers for approval by the Eurogroup, whose decisions are generally ratified by ECOFIN. History: Before the third stage of the Economic and Monetary Union (EMU), which began on 1 January 1999, the Economic and Financial Committee was preceded by the Monetary Committee, which was established by Article 105 of the Treaty of Rome in 1957. In May, as part of the European Semester 2016 Spring Package, the committee was tasked with making reports on Belgium, Finland, and Italy’s government debt. Agenda and composition of the EFC…} |
| QA Pairs: {qa_json_str} |
| Please return the output in JSON format only. The root must be a JSON object with a key “checked_qa_list”, and its value is a list of objects corresponding to the input QA pairs in the same order. Each object should have the following keys: - “dimension”: The original dimension - “question”: The original question - “original_answer”: The original answer - “is_correct”: true or false, whether the original answer was correct - “answer”: The concise standard answer (use the original if it was correct, the corrected one if incorrect, or “Unknown” if out of scope) |
Table 8: Question-answer pair checking prompt example.
| Stage 3: Sample Reconstruction Prompt Example |
|---|
| You are a professional assistant for constructing and correcting datasets. You need to strictly follow the specified dimensions and rules to rewrite the questions that cannot be answered (Unknown) based on the given original story (Story). Please ensure to return in the legal JSON format. |
| Dimension Construction Rules: {dimension_rules} |
| Original Story: {Economic and Financial Committee (European Union). The Economic and Financial Committee (EFC) is an advisory body of the European Union, as defined by Article 134 of the Treaty on the Functioning of the European Union. Its president is also the president of the Eurogroup Working Group, which prepares dossiers for approval by the Eurogroup, whose decisions are generally ratified by ECOFIN. History: Before the third stage of the Economic and Monetary Union (EMU), which began on 1 January 1999, the Economic and Financial Committee was preceded by the Monetary Committee, which was established by Article 105 of the Treaty of Rome in 1957. In May, as part of the European Semester 2016 Spring Package, the committee was tasked with making reports on Belgium, Finland, and Italy’s government debt. Agenda and composition of the EFC…} |
| Pending Q&A Pairs: {qa_json_str} |
| Task Requirements: 1. Carefully read the 5 pairs of questions and answers provided above. 2. For the pairs where the “answer” is “Unknown”, explain that the original question cannot be answered within the story (beyond the scope of the text). Please, based on the original story and the corresponding dimension rules, rewrite a new and challenging question and provide a concise answer. 3. The rewritten question must be able to find a precise answer within the original text. It must strictly adhere to the “core testing ability” and “question template” in the corresponding dimension rules. 4. For the pairs where the “answer” is not “Unknown”, please keep the original text unchanged. Please return the result in a strict JSON format. The root node must be an array named “rewritten_qa_list”, and this array must contain 5 objects, maintaining the same order as the input question-answer pairs. |
| The format of each object is as follows: { “dimension”: “The name of the dimension, for example, Dimension 5”, “question”: “The original question (if not modified) or the new question after rewriting”, “answer”: “The original answer (if not modified) or the concise answer after rewriting”, “is_rewritten”: true or false (if rewriting has been done, it is true; otherwise, it is false) } |
Table 9: Sample reconstruction prompt example.
| Examples of Samples From the First Stage Dataset |
|---|
| { "text_chunks" : [ "Jaroslav Pelikan … was born on December 17, 1923, in Akron, Ohio … He earned a seminary degree from Concordia Seminary in 1946 … He completed his Ph.D. at the University of Chicago in 1953. Subsequently, in 1962, he joined the faculty at Yale University …" ], "prompt_texts" : [ "Extract the chronological timeline of causal events." ], "target_events" : [ "Jaroslav Pelikan earned a degree from Concordia Seminary.", "Jaroslav Pelikan completes Ph.D. at the University of Chicago.", "Jaroslav Pelikan joins the faculty at Yale University." ], "target_timestamps" : [0.35, 0.52, 0.78] } |
Table 10: Examples of samples from the first stage dataset.
| Examples of Samples From the Second Stage Dataset |
|---|
| { "text_chunks" : [ "Jaroslav Pelikan … was born on December 17, 1923, in Akron, Ohio … He earned a seminary degree from Concordia Seminary in 1946 … He completed his Ph.D. at the University of Chicago in 1953. Subsequently, in 1962, he joined the faculty at Yale University …" ], "prompt_texts" : [ "Please reconstruct the exact timeline of events based on the text." ], "target" : "Jaroslav Pelikan earns his degree from Concordia Seminary around 1946.\nJaroslav Pelikan completes his Ph.D. at the University of Chicago around 1953.\nJaroslav Pelikan joins the faculty at Yale University around 1962." } |
Table 11: Examples of samples from the second stage dataset.
| LLM Judgment Prompt Example |
|---|
| You are an expert evaluator for temporal causal reasoning tasks. |
| Your job is to judge whether a model’s predicted answer is semantically equivalent to the ground-truth answer. Do not focus on superficial wording differences. Focus on the core facts, logical consistency, and causal implications. If the prediction conveys the same meaning as the ground truth, it should be considered correct. |
| Question: {question} Ground-Truth Answer: {ground_truth} Model’s Response Answer: {model_response} |
| Based on the above, is the Response Answer semantically correct? Please first provide a brief reasoning for your decision, then output your final verdict. Use the following format exactly: |
| <reasoning> [Your reasoning here] </reasoning> <verdict> Correct or Incorrect </verdict> |
Table 12: LLM judgment prompt example.
Figure 4: Confusion matrix of the LLM judgment (Qwen-3-8B).
| TG-LLM Prompt Example |
|---|
| Example: Timeline: (Liam Mitchell was born in Harrison) starts at 1885 (Maddox Reynolds was born in Glenville) starts at 1893 (Liam Mitchell was married to Emma Scott) starts at 1916 (Emma Scott was married to Liam Mitchell) starts at 1916 (Liam Mitchell was married to Emma Scott) ends at 1918 (Emma Scott was married to Liam Mitchell) ends at 1918 (Maddox Reynolds was married to Liam Mitchell) starts at 1922 (Liam Mitchell was married to Maddox Reynolds) starts at 1922 (Maddox Reynolds was married to Liam Mitchell) ends at 1928 (Liam Mitchell was married to Maddox Reynolds) ends at 1928 (Liam Mitchell died in Boston) starts at 1941 (Emma Scott died in Oceanview) starts at 1984 |
| Question: True or false: Event (Liam Mitchell was married to Maddox Reynolds) was longer in duration than event (Emma Scott was married to Liam Mitchell)? |
| Useful information: 1885 before 1893 before 1916 before 1918 before 1922 before 1928 before 1941 |
| Answer: Let’s think step by step. To determine whether the statement is true or false, we need to compare the durations of (Liam Mitchell was married to Maddox Reynolds) and (Emma Scott was married to Liam Mitchell). The duration for each event can be calculated as follows: (Liam Mitchell was married to Maddox Reynolds): (Liam Mitchell was married to Maddox Reynolds) starts at 1922 (Liam Mitchell was married to Maddox Reynolds) ends at 1928 (Emma Scott was married to Liam Mitchell): (Emma Scott was married to Liam Mitchell) starts at 1916 (Emma Scott was married to Liam Mitchell) ends at 1918 Now, we compare the duration of the two events: 6 is greater than 2; thus, the answer is True. |
| Test: Timeline: (Chris Evans was born in Bristol, Connecticut) starts at 1948 (Oliver Jensen was born in Bristol, Connecticut) starts at 1948 (Chris Evans was married to Oliver Jensen) starts at 1970 (Oliver Jensen was married to Chris Evans) starts at 1970 (Chris Evans created Pearl Network) starts at 2005 (Chris Evans won prize Victory Achievement Award) starts at 2007 (Chris Evans was married to Oliver Jensen) ends at 2010 (Oliver Jensen was married to Chris Evans) ends at 2010 (Chris Evans created Pearl Network) ends at 2013 |
| Question: True or false: Event (Chris Evans owned Pearl Network) was longer in duration than event (Oliver Jensen was married to Chris Evans)? |
Table 13: TG-LLM prompt example.
| Time-CoT Prompt Example |
|---|
| Response Template: Let’s think step by step. First, we need to find out the <Standard Question Entity> according to the timeline. Then, we need to focus on the time <Explicit Time Parameter>. So we can obtain the answer is <Gold Answer>. Therefore, the answer(s) is/are … |
| Example: Question: Which movie did Zoe Cassavetes direct in 2007 ? Answer: Let’s think step by step. First, we need to find out which movie did Zoe Cassavetes direct according to timeline. 2000: Men Make Women Crazy Theory, 2007: Broken English, 2012: The Powder Room, 2015: Day Out of Days…… Then, we need to focus on the time in 2007. So we can obtain the answer in Broken English. Therefore, the answer(s) is/are Broken English. |
| Question: Who is the quarterback for the New York Giants as of December 2010 ? Answer: Let’s think step by step. First, we need to find out who is the quarterback for the New York giants according to timeline. 1999-2003: Kerry Collins, 2003: Jesse Palmer, 2004: Kurt Warner, 2004-2019: Eli Manning, 2017: Geno Smith, 2019-2023: Daniel Jones, 2020: Colt McCoy, etc. Then, we need to focus on the time December 2010. So we can obtain the answer is Eli Manning. Therefore, the answer(s) is/are Eli Manning. |
| Test: Question: Who was the president of the US when Adele Faccio was a teenager? Answer: Let’s think step by step. |
Table 14: Time-CoT prompt example.
| TISER Prompt Example |
|---|
| You are an AI assistant that uses a Chain of Thought (CoT) approach with reflection to answer queries. Follow these steps: 1. Reason through the problem step by step within the <reasoning> tags. 2. Given your previous reasoning, identify relevant temporal events in the given context for answering the given question within <timeline> tags. Assume relations in the context are unidirectional. 3. Reflect on your reasoning and the timeline to check for any errors or improvements within the <reflection> tags. 4. Make any necessary adjustments based on your reflection. If additional reasoning is required, go back to Step 1 (reason through the problem step by step); otherwise, move to the next step (Step 5). 5. Provide your final, concise answer within the <answer> tags. If the answer is a number, just output the number, nothing else. Otherwise, output the entity or event without any additional comments. |
| Additional Instructions: 1. [IF during data generation] Perform your reasoning knowing that the answer is: X. 2. The <reasoning>, <reflection>, and <timeline> sections are for your internal reasoning process. All the reflection and the timeline have to be contained inside the thinking section. 3. Do not use enumerations or lists when writing; use plain text such as paragraphs. 4. The response to the query must be entirely contained within the <answer> tags. |
| Response Format: <reasoning> [Your step-by-step reasoning goes here. This is your internal thought process.] <timeline> [Relevant temporal events for answering the given question.] </timeline> <reflection> [Your reflection on your reasoning, checking for errors or changes required.] </reflection> [Any adjustments to your thinking based on your reflection.] </reasoning> <answer> [Your final, concise answer to the query.] </answer> |
Table 15: TISER prompt example.
More in this topic
Causal Reasoning in Language Models
Causal reasoning in language models studies how models identify cause-and-effect structure, predict interventions, and answer counterfactual questions.
See all 102 papers