Temporal Data Leakage

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 19

All topics

Frontier Autolab: Organizational Memory, Adversarial Dissent and Temporal Leakage in Multi-Agent LLM Firms Across Fifty Years of Technological Change
Sep 29, 2026Bravish GhoshMulti-Agent LLM Systems LLM Agent Memory
Independent researcher
Time-Series Foundation Models That Understand Data Revisions
Sep 23, 2026Taimoor AhmadTime Series Forecasting Temporal Data Leakage
Superior University Lahore, Pakistan
Temporal Leakage in LLM Backtesting: Measurement, Validation, and Adjusted Scores
Aug 4, 2026Zeyu Zhang, Bradly C. StadieLLM Evaluation LLM Auditing
Department of Statistics and Data Science Northwestern University
Distilling Temporal Search and Reasoning: Evolving LLMs for Future Prediction via Harness-Assisted Efficient Data Synthesis
Jul 28, 2026Wanxu Cai, Zhengyu Chen, Huaisheng Zhu +3Data Leakage Temporal Reasoning
1The Chinese University of Hong Kong · 2Meituan LongCat Team
HindsightBench: A Black-Box Behavioral Audit Protocol for Parametric Hindsight in Time-Indexed LLM Decision Tasks
Jul 21, 2026Haozhe JiaLLM Auditing Temporal Data Leakage
University College Dublin
Fenced Citation-Context Retrieval for Case Law: Temporal Leakage and Degree Control Across Two Jurisdictions
Jul 19, 2026Yao Liu, Tien-Ping Tan, Zhilan LiuData Leakage Legal IR
The Engineering and Technology College, Chengdu University of Technology, Leshan, China · School of Computer Sciences, Universiti Sains Malaysia, Penang, Malaysia · Department of Art and Design, The Engineering and Technology College, Chengdu University of Technology, Leshan, China
Hindcast: Replaying Prediction Markets to Evaluate LLM Forecasters
Jul 15, 2026Xiao Ye, Jacob Dineen, Evan Zhu +3LLM Evaluation Forecasting Benchmarks
School of Computing and Augmented Intelligence, Arizona State University
Pitfalls of Administrative Censoring in Survival Models with Time-Indexed Inputs
Jul 11, 2026Yanqi Xu, Hui Dai, Carlos Fernandez-Granda +2Data Leakage Survival Analysis
Center for Data Science, New York University · University of Chicagoy · 3NYU Grossman School of Medicine +1
MACROCAST: A Vintage-Consistent Time Series Foundation Model for Real-Time Macroeconomic Forecasting
Jun 27, 2026Andrea Carriero, Davide Pettenuzzo, Shubhranshu ShekharData Leakage Time Series Forecasting
School of Economics and Finance, Queen Mary University of London. · School of Business and Economics, Brandeis University.
Leakage-Aware Benchmarking of LLM Forecasting: Real-Time Nowcasts as the Decision-Time Input for Macro Factor Ranking
Jun 21, 2026Mao Guan, Qian ChenLearning to Rank Financial Forecasting
Independent Researcher.
WorldReasoner: Evaluating Whether Language Model Agents Forecast Events with Valid Reasoning
Jun 10, 2026Yizhou Chi, Eric Chamoun, Zifeng Ding +1Event Time Prediction LLM Agent Evaluation
Department of Computer Science and Technology University of Cambridge
NumLeak: Public Numeric Benchmarks as Latent Labels in Foundation Models
May 28, 2026Anany KotawalaData Leakage LLM Evaluation
Princeton University, Princeton, NJ, USA.
When Can We Trust Early Warnings? Leakage-Excluded Early Outcome Prediction from LMS Interaction Logs
May 25, 2026Ngoc Luyen Le, Marie-Hélène Abel, Bertrand LaforgeData Leakage AI in Education
Gamaizer, 93340 Le Raincy, France · Université de technologie de Compiègne, CNRS, Heudiasyc (Heuristics and Diagnosis of Complex Systems), CS 60319 - 60203 Compiègne Cedex, France · Sorbonne Université, CNRS UMR 7585, LPMHE (Laboratoire de Physique Nucléaire et des Hautes Énergies), 75252 Paris cedex 05, France
Teaching Large Language Models When Not to Know: Learning Temporal Critique for Ex-Ante Reasoning
May 14, 2026Chenlu Ding, Jiancan Wu, Yanchen Luo +3LLM Prompting LLM Fine-Tuning
University of Science and Technology of China · The Hong Kong Polytechnic University · University of Notre Dame
TEMPO: Temporal Enforcement via Mode-Separated Policy Optimization for Trustworthy LLM Backtesting
May 13, 2026Zeyu Zhang, Bradly C. StadieLLM Evaluation RL for Language Model Reasoning
Department of Statistic and Data Science Northwestern University
Ensuring Reliability in Programming Knowledge Tracing: A Re-evaluation of Attention-augmented Models and Experimental Protocols
May 6, 2026Jaewook Kim, Hyeoncheol KimProgramming Education Temporal Data Leakage
Korea University, Seoul, Republic of Korea
OracleProto: A Reproducible Framework for Benchmarking LLM Native Forecasting via Knowledge Cutoff and Temporal Masking
May 5, 2026Yiding Ma, Chengyun Ruan, Kaibo Huang +2LLM Evaluation Forecasting Benchmarks
Beijing University of Posts and Telecommunications
Scaling Point-in-Time Language Models
Apr 24, 2026Bryan Kelly, Semyon Malamud, Johannes Schwab +1Decoder-Only Language Models Temporal Data Leakage
Yale School of Management · AQR Capital Management · NBER +3
From Leakage to Fidelity: Reliable Benchmarking for Temporal Cascade Prediction
Oct 29, 2025Jie Peng, Rui Wang, Qiang Wang +4Forecasting Benchmarks Temporal Data Leakage
Renmin University of China · Alibaba

Sep 29, 2026, cs.CEJ/K move · Enter open · S save

Frontier Autolab: Organizational Memory, Adversarial Dissent and Temporal Leakage in Multi-Agent LLM Firms Across Fifty Years of Technological Change

Bravish Ghosh

Independent researcher

Multi-Agent LLM Systems LLM Agent Memory Long-Horizon Agent Evaluation Multi-Agent Collaboration Temporal Data Leakage

Open paper AskPDF arXiv

Multi-agent LLM systems are increasingly structured like organizations, with roles, critics and shared memory, yet they are evaluated on tasks that last minutes. We ask how such an organization behaves when the ground it stands on keeps moving. Frontier Autolab is a long-horizon testbed in which one simulated firm, voiced by sixteen role personas and a dedicated Red Team, must re-found itself in nine technology eras from 1990 to 2040. Each era is temporally gated: the firm decides from a dated briefing, a historian-judge then reveals what happened and scores the decision on a five-dimension rubric, and lessons enter a persistent Playbook. Six eras are scored against history, one against the live market and two are open forecasts. Across four trajectories (36 era decisions, 180 subscores) we find a consistent foresight-commitment gap: in all 24 historically scored eras the judge rated the firm's recognition of the coming shift above its choice of where to build (mean gap 1.9 points on a 10-point scale), because boards chose the layer their existing assets could reach. Organizational design shaped long-run character. A Red Team armed with numeric kill gates produced fifty years of gated pilots and no product, and the rubric rated this firm highest; firms whose memory stored market-structure lessons pivoted every era, while a firm whose memory stored only validation procedure kept one method throughout. We also show why such results are hard to trust. Scores rise across eras in every run while the judge's own hindsight subscore falls (within-run r = -0.58), so apparent learning is confounded with recall of history, and we trace further distortions to self-judging, briefing selection and score aggregation. We release all records and an API harness, and specify fictional and post-cutoff eras that would turn the testbed into a benchmark.

Momentum

Related topics

Latest papers 19