Long-Horizon Agent Evaluation

Ask about this topic

Momentum

47 papers in the last four weeks, up 176% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 228

All topics
CardsList

  1. Does an Agent's History Tell You When Compaction Will Hurt? A Modest, Bounded Effect on the TRACE Paired-Replay Corpus

    Oct 6, 2026Egor Pakhomov, Erik NijkampLong-Horizon Agent EvaluationAgent Memory Management

    Salesforce AI Research

  2. Stateless Language Agents: Scaling Long-Horizon Automated Research

    Oct 6, 2026Qizheng Zhang, Changxiu Ji, Isaac Sun +15Software Engineering AgentsLLM Agent Orchestration

    Stanford University

  3. What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents

    Oct 5, 2026Zhongxiang Sun, Jiahao Yan, Hongkang Zhao +5Software Engineering AgentsHuman-in-the-Loop Evaluation

    Gaoling School of Artificial Intelligence, Renmin University of China · Kuaishou Technology

  4. LexiHorizon: Stabilizing Reinforcement Learning for Long-Horizon Deep Search

    Oct 4, 2026Zhiqing Nong, Liang Wen, Chao-Hsuan LiuLong-Horizon Agent EvaluationRL for IR

  5. EMBER-Bench: Benchmarking Cross-Event Causal Memory in Long-Horizon Embodied Tasks

    Oct 4, 2026Aoyang Cai, Boning Zhao, Shaoxuan Xie +5Next-Action PredictionEmbodied QA

    Tsinghua University · Beijing Academy of Artificial Intelligence · The University of Hong Kong

  6. Kepler: Auditable World Models for ARC-AGI-3

    Sep 30, 2026Wensen WuLong-Horizon Agent EvaluationARC-AGI

    Independent Researcher

  7. ReLiveGym: Evaluating Long-Lived Agents over Weeks of Replayed Reality

    Sep 30, 2026Xisen Jin, Jingheng Li, Zhenglun Chen +2Continual Learning for LLM AgentsLong-Horizon Agent Evaluation

    Sahara AI · University of Southern California

  8. EurekaBench: Measuring Agentic Ability to Discover New Scientific Insights

    Sep 30, 2026Jiayi Geng, Zhengxuan Wu, Kevin S. Chen +12Scientific DiscoveryLong-Horizon Agent Evaluation

    Carnegie Mellon University, Language Technologies Institute · Stanford University, Department of Computer Science · Yale University, Quantitative Biology Institute +8

  9. EngramBench: A Capability-Grounded Benchmark for Skill-Evolution Harnesses

    Sep 30, 2026Zhixuan Tan, Pengjie Gu, Zhao Li +4Software Engineering BenchmarksLong-Horizon Agent Evaluation

    The Chinese University of Hong Kong, Shenzhen · MemoraX AI

  10. When Harnesses Lose the Signal: Causal Evaluation of Recovery in LLM Agents

    Sep 30, 2026Shuyao Xiao, Shengling Wang, Xuan Chen +7Long-Horizon Agent EvaluationCausal Interventions in Language Models

    School of Artificial Intelligence, Beijing Normal University · Ke Holdings

  11. Staying on Task: Testing the Foundations of Long-Horizon Agent Reliability

    Sep 30, 2026Jeffrey Willette, Krishna C. Puvvada, Boris GinsburgLong-Context Language Model EvaluationLong-Horizon Agent Evaluation

    NVIDIA

  12. EnterpriseBench: Benchmarking LLM Agents on Enterprise-Level Strategic Reasoning and Decision-Making

    Sep 29, 2026Min Yang, Yichen Pan, Jinghua Piao +3AI-Assisted Decision MakingLLM Agent Evaluation

    Shandong University · Zhongguancun Academy · Beijing Institute of Technology +1

  13. Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym

    Sep 29, 2026Jio Oh, Seunghyun Do, Young-Jun Lee +2LLM Agent EvaluationLong-Horizon Agent Evaluation

    KAIST · University of Minnesota

  14. LoLBench: Evaluating Coding Agents with Long-Horizon Proposals on Large Software Systems

    Sep 29, 2026Yun Peng, Zihan Wu, Zeyang Zhuang +6Software EngineeringAI Coding Agents

    Fudan University, China · City University of Hong Kong, Hong Kong · Chinese University of Hong Kong, Hong Kong +5

  15. Traverse: Learning When to Remember, Reset, and Redirect for Long-Horizon Web Search

    Sep 29, 2026Jingyuan Ma, Lynx Aster, He Zhang +5Long-Horizon Agent EvaluationWeb Search Agents

    State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · ByteDance

  16. PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval

    Sep 29, 2026Bin Kang, Jiarui Ouyang, Li Jiang +2GUI AgentsLong-Horizon Agent Evaluation

    University of Chinese Academy of Sciences · Harbin Institute of Technology · Shenzhen Loop Area Institute +2

  17. Frontier Autolab: Organizational Memory, Adversarial Dissent and Temporal Leakage in Multi-Agent LLM Firms Across Fifty Years of Technological Change

    Sep 29, 2026Bravish GhoshMulti-Agent LLM SystemsLLM Agent Memory

    Independent researcher

  18. StateTape: Action-Conditioned Evidence Lifecycle Modeling for Long-Horizon Coding Agents

    Sep 28, 2026Ziyang Yu, Liang Zhao, Bowen Zhu +1AI Coding AgentsLLM Agent Memory

    Causal Dynamics Lab · Emory University

  19. ARISE: Adapting to Evolving Capability Gaps in Agentic Reinforcement Learning

    Sep 28, 2026Kun Feng, Yuchen Fang, Yiyang Tan +6Agentic RLLong-Horizon Agent Evaluation

    ShanghaiTech University · Ant Group

  20. Long-Horizon Scaling: How Model Capabilities Shape the Returns to Computation

    Sep 28, 2026Haoyu Zheng, Zhengyu Chen, Huaisheng Zhu +5Cost-Aware InferenceInference-Time Scaling

    Meituan LongCat Team · Zhejiang University · Westlake University +2

  21. PDEU-Bench: Benchmarking the Personalized Planning Lifecycle of Tool-Calling LLM Agents

    Sep 28, 2026Huayi Lai, Shichao Song, Qingchen Yu +4Long-Horizon Agent EvaluationAI Agent Benchmarks

    Renmin University of China · Beihang University

  22. CEO Arena: Evaluating Long-Horizon Multi-Agent Decision-Making in Competitive Markets

    Sep 28, 2026An Yan, Yu Huo, Zhiwei Shang +2Game TheoryLLM Agent Evaluation

    DeepWisdom · Fudan University · The Chinese University of Hong Kong +2

  23. LongPuzzleBench: Evaluating GUI Agents on Long-Horizon Visual Puzzles

    Sep 28, 2026Bingo Zhang, Haochuan Lu, Zongjie Li +3GUI AgentsLong-Horizon Agent Evaluation

    Vera Praxis · Tencent · The Hong Kong University of Science and Technology +2

  24. FromPitch2Board: Benchmarking LLM Agents in Long-Horizon Football Management

    Sep 28, 2026Peiyu ZangLong-Horizon Agent EvaluationAI Agent Benchmarks

    School of Mathematical Sciences, Beijing Normal University · CoRe Lab, Institute for Artificial Intelligence, Peking University

  25. Opera: A Verbal Critic Framework for Long-horizon Coding Agents

    Sep 27, 2026Kai Mei, Zhiyuan Hu, Yutong Dai +7AI Coding AgentsAI Agent Auditing

    Salesforce AI Research · Rutgers University · Lehigh University

  26. When Successful Strategies Fail: Adaptation to Environmental Novelty in Terminal Agents

    Sep 27, 2026Janvijay Singh, Vaishnavi Shrivastava, Dilek Hakkani-Tur +2Long-Horizon Agent EvaluationAI Agent Benchmarks

    University of Illinois Urbana-Champaign · Microsoft Research AI Frontiers

  27. Self-Designed Evaluators and Warm Memory for Long-Horizon Agents

    Sep 27, 2026Saeid Asgari, Emre Kiciman, Leonardo de Oliveira Nunes +1Language Model Generation EvaluationLong-Horizon Agent Evaluation

    Microsoft

  28. EverMine: Dissecting the Self-Evolution of Research Capabilities in Long-Horizon Alpha Research

    Sep 27, 2026Siyuan Li, Jiangfeng Zhang, Rui Yao +3Quantitative FinanceLong-Horizon Agent Evaluation

    The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong Polytechnic University · Singapore Management University

  29. Long-Horizon Analog Design Bench: Benchmarking Agents on Hours-Long Analog and Mixed-Signal Circuit Design Tasks

    Sep 27, 2026Analog Design Bench TeamBenchmark DesignElectronic Design Automation

  30. Adaptive Consistency Graph for Long-Horizon Agents

    Sep 26, 2026Jiecong Wang, Hao Peng, Zhanyi WangLong-Horizon Agent EvaluationGraph-Based Agent Memory

    Beihang University · QI-ANXIN GROUP

  31. Dude, Where's My State? Execution Information Requirements for Stateful Agents

    Sep 26, 2026Nikita Mehrotra, Ashish Tiwari, Priyanshu Gupta +1Software Engineering AgentsLong-Horizon Agent Evaluation

  32. SCLATE: a Substrate for Continual-Learning Agent Training and Evaluation

    Sep 26, 2026Youngmok Jung, Sirajul Salekin, Henry Tran +3Continual Learning for LLM AgentsLong-Horizon Agent Evaluation

    Apple

  33. Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems

    Sep 24, 2026Shuang Yang, Zijie Zhuang, Changxin Lao +20AI Agents for Scientific DiscoveryLong-Horizon Agent Evaluation

  34. The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks

    Sep 22, 2026Wenbo Pan, Zhichao Liu, Shujie Liu +6Software Engineering BenchmarksLong-Horizon Agent Evaluation

    City University of Hong Kong · Microsoft · Independent Researcher

  35. GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay

    Sep 21, 2026Yiran Wang, Xingyilang Yin, Junfu Pu +12Game-Playing AgentsLong-Horizon Agent Evaluation

    ARC Lab, Tencent · GVC Lab, Great Bay University · University of Macau +3

  36. DolphinBench: Mapping the Pareto Frontier of Agent Memory

    Sep 21, 2026Soumil Rathi, Deshraj Yadav, Taranjeet SinghAgent MemoryPersistent Memory for Language Models

    Mem0

  37. DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculum

    Sep 17, 2026Haoqiang Kang, Yiming Zhang, Yiyang Guo +5Long-Horizon PlanningLong-Horizon Agent Evaluation

    UC San Diego · Independent · Johns Hopkins University +1

  38. Reach or Solve? Attributing Agentic RL Gains with Checkpoint Handoffs

    Sep 17, 2026Xuan Liu, Jingbin QianAgentic RLLong-Horizon Agent Evaluation

    Shanghai Jiao Tong University · Rice University

  39. SIMLIFE: Pattern Understanding for Long-Horizon Human-Agent Partnership

    Sep 17, 2026Run Peng, Zinnia Nie, Jing Ding +7Human Behavior SimulationLong-Horizon Agent Evaluation

    University of Michigan · ByteDance · Osaka University +1

  40. Locating Hidden Failures Makes Long-Horizon Agents More Reliable

    Sep 15, 2026Salman Rahman, Yubin Kim, Mihir Parmar +15Agent Failure AnalysisAI Agent Reliability

    University of California, Los Angeles · New York University · Google Research +1

  41. Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems

    Sep 15, 2026Deepak Akkil, Tamer Abuelsaad, Karthik Vikram +5Multi-Agent LLM SystemsLong-Horizon Agent Evaluation

    Emergence AI

  42. BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents

    Sep 14, 2026Sadia Asif, Mohammad Mohammadi Amiri, Momin Abbas +2Long-Horizon Agent EvaluationAI Agent Safety Benchmarks

    Rensselaer Polytechnic Institute · IBM Research

  43. When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis

    Sep 14, 2026Kaiyuan Liu, Qiuyang Mang, Bo Peng +6LLM Agent EvaluationTest-Time Scaling

    UC Berkeley · University of Washington · Princeton University +1

  44. MemRiskBench: Trace-Aware Risk-Preserving Evaluation for Long-Horizon LLM Agents

    Sep 14, 2026Jianhua Jiang, Dongbo Yuan, Weihua LiLong-Horizon Agent EvaluationAI Agent Safety Benchmarks

    School of Artificial Intelligence and Computer Science, Jilin University of Finance and Economics, 130117 Changchun, China · Jilin Province Key Laboratory of Fintech, Jilin University of Finance and Economics, 130117 Changchun, China · School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology, Auckland, New Zealand

  45. DynSTEER: Dynamic Stage-wise Trajectory Evaluation and Execution-time Review for Agents

    Sep 13, 2026Zhichao Shi, Xuhui Jiang, Wenjie Zhang +5LLM Agent EvaluationAI Agent Evaluation

    School of Advanced Interdisciplinary Sciences, UCAS · State Key Lab of AI Safety, Institute of Computing Technology, CAS · DataArc Tech Ltd. +3

  46. Safety Signals to Verify NetOps Agents with Action-Level Granularity

    Sep 13, 2026Tobias Labarta, Frederik Pahde, Novak Boškov +5Long-Horizon Agent EvaluationAI Agent Safety

    Fraunhofer Heinrich-Hertz-Institut, Berlin, Germany · Nokia Bell Labs, Stuttgart, Germany · Nokia Bell Labs, Murray Hill, NJ 07974, USA +2

  47. T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks

    Sep 12, 2026Junyao Yang, Yucheng Shi, Zhongzhi Li +4Terminal AgentsLong-Horizon Agent Evaluation

    Tencent Hy Foundation Model · National University of Singapore · University of Georgia +2

  48. Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures

    Sep 11, 2026Harsh Raj, David Lee, Anas Mahmoud +7Agent Failure AnalysisAgentic Search

    Scale AI · Scale AI Research

  49. JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition

    Sep 9, 2026Zixiang Chen, Yuheng Lu, Zihao Cheng +8Computer-Use Agent BenchmarksGUI Agents

    School of Computer Science and Engineering, Beihang University, Beijing, China · Beijing Institute of Technology, Beijing, China · Baidu Inc., Beijing, China

  50. EvoNav-Bench: Benchmarking Lifelong Navigation in Evolving Environments

    Sep 8, 2026Xilin Wang, Guoxi Zhang, Hongming Xu +3Embodied NavigationRobot Navigation

    State Key Laboratory of General Artificial Intelligence, BIGAI · 2Tsinghua University

  51. Linguistic Trajectory Encoding for Efficient Long-Horizon Spatial Memory in Embodied Agents

    Sep 4, 2026Tianyidan Xie, Shenyi Wang, Qiang Tang +7Long-Horizon Agent EvaluationTrajectory Representation Learning

    Nanjing University · University of British Columbia · Zhejiang Sci-Tech University +2

  52. Parsing the Stream: A Live Trace Model for Long-Horizon Agents and Their Observers

    Sep 1, 2026Egor Pakhomov, Erik NijkampLong-Horizon Agent EvaluationAI Agent Monitoring

    Salesforce AI Research

  53. Measuring the Behavioral Fidelity of Long-Horizon Human Activity Simulations

    Sep 1, 2026Yi Fei Cheng, Fan Yang, Iremsu Bas +3Human Behavior SimulationLong-Horizon Agent Evaluation

    Human-Computer Interaction Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA · Fujitsu Ltd., Kawasaki, Japan · Fujitsu Research of America, Pittsburgh, PA, USA

  54. Deploying and Evaluating a Smart-Agriculture Agentic Engine for Full-Season Soybean Farm Operations

    Aug 31, 2026Ao Qu, Panagiotis Michelakis, Linyuan Han +8Software EngineeringMulti-Agent Orchestration

    Faculty of Computing Harbin Institute of Technology State Key Laboratory of Smart Farm Technologies and Systems Harbin, China · School of Electrical and Computer Engineering National Technical University of Athens Athens, Greece

  55. E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation

    Aug 31, 2026Wei Fan, Xinjie Shen, Xudong Guo +8Long-Horizon Agent EvaluationAI Agent Benchmarks

    1Qwen Team, Alibaba Group · Department of Computer Science and Engineering, HKUST · 2Taobao & Tmall Group, Alibaba Group

  56. ATLAS: Dual-Horizon Diagnostic Evaluation for Industrial Tool-Use Agents

    Aug 31, 2026Wei Chen, Peilun Zhou, Zhaoyu Hu +8Tool-Use EvaluationLLM Agent Evaluation

    University of Science and Technology of China, Meituan · Meituan

  57. Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents

    Aug 30, 2026Zongyue Li, Chengyue Yu, Lei Zang +3Deep Research AgentsLong-Horizon Agent Evaluation

    Zhejiang University · Ant Group

  58. UrbanGround: From Local Perception to Spatial Agency in a Real-Scale City

    Aug 27, 2026Tianjie Ju, Zheng Wu, Yueqing Sun +15Spatial Reasoning BenchmarksRobot Navigation

    Shanghai Jiao Tong University · National University of Singapore · Meituan +3

  59. TraceML: What Auto-Research Agents Miss in Long-Horizon ML Development

    Aug 26, 2026Jiarui Yan, Weiwei Sun, Sijie Li +2Long-Horizon Agent EvaluationAI Agent Benchmarks

    Carnegie Mellon University

  60. PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives

    Aug 13, 2026Kaixin Ding, Xi Chen, Minghong Cai +9Long-Horizon Agent EvaluationWorld Model Evaluation

    The University of Hong Kong · Kling Team, Kuaishou Technology · The Chinese University of Hong Kong +1