SemEval-2026 Task 12: Abductive Event Reasoning: Towards Real-World Event Causal Inference for Large Language Models
Authors: Pengfei Cao, Mingxuan Yang, Yubo Chen, Chenlong Zhang, Mingxuan Liu, Kang Liu, Jun Zhao
Organizations: The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
Understanding why real-world events occur is important for both natural language processing and practical decision-making, yet direct-cause inference remains underexplored in evidence-rich settings. To address this gap, we organized SemEval-2026 Task 12: Abductive Event Reasoning (AER).\footnote{The task data is available at https://github.com/sooo66/semeval2026-task12-dataset.git} The task asks systems to identify the most plausible direct cause of a target event from supporting evidence. We formulate AER as an evidence-grounded multiple-choice benchmark that captures key challenges of real-world causal reasoning, including distributed evidence, indirect background factors, and semantically related but non-causal distractors. The shared task attracted 122 participants and received 518 submissions. This paper presents the task formulation, dataset construction pipeline, evaluation setup, and system results. AER provides a focused benchmark for abductive reasoning over real-world events and highlights challenges for future work on causal reasoning and multi-document understanding.
Figures & tables
Figure 1: Overview of the Abductive Event Reasoning (AER) task. Given a noisy multi-document evidence collection, the goal is to identify the most plausible direct cause of a target event via evidence-grounded abductive inference. The task is challenging because the supporting evidence is distributed and noisy, systems must focus on direct triggers rather than background conditions, and multiple answer options may be correct.
Target Event
Videos of the assassination circulated on social media.
Context
A collection of documents containing mixed relevant and irrelevant evidence
Options
A ✗
The shooter used a handmade gun.
B ✗
Security arrested the suspected gunman, Tetsuya Yamagami.
C ✗
Shinzo Abe became the deputy chief cabinet secretary in the early 2000s.
D ✓
A man fired twice at Shinzo Abe.
Table 1: An illustrative example from the AER dataset. The target event concerns the circulation of assassination videos on social media, and the candidate causes range from peripheral background information to the most immediate precipitating event. This example highlights the task’s focus on selecting the most plausible direct cause rather than merely related context.
Figure 2: Construction pipeline of the AER benchmark.
Statistic
Value
Number of topics
60
Total number of questions
2,831
Training questions
1,819
Development questions
400
Test questions
612
Time span
2016-06 to 2025-11
Table 2: Core statistics of the AER benchmark. Agreement statistics will be added after the final annotation analysis is completed.
Figure 3: Distributional overview of the dataset: (A) topic composition across six categories, (B) frequency of correct labels at options A–D, (C) topic representative-time distribution with the GPT-4 knowledge cutoff (2023-12-01) marked by a dashed line, and (D) document length distribution in tokens, with mean and median indicated.
Model
Ori_T
Sum_T
GPT-4
68.66
70.35
Qwen-2.5-72B-Instruct
53.65
60.72
GLM-4
58.12
60.36
Table 3: Pilot results on a 200-instance sample under original-document input (Ori_T) and summarized input (Sum_T).