Trajectory-Level Credit

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, up 120% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 83

All topics

Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis
Oct 1, 2026Qijia He, Ruinan Jin, Jun Luo +2Convergence Trajectory-Level Credit
The Ohio State University · Arizona State University
PRICE the Action Chunks: Physical Relational Credit Assignment for Embodied Reinforcement Learning
Sep 30, 2026Yangang Zou, Jiajun Lu, Weitao Zhou +6Offline Reinforcement Learning Credit Assignment
School of Data Science, Fudan University · Shanghai Innovation Institute · Beihang University +4
Credit-Guided Policy Improvement for Test-time Adaptive Vision-Language Navigation
Sep 29, 2026Yang Li, Sijia Zhang, Yihan Li +4Vision-Language Navigation Trajectory-Level Credit
School of Artificial Intelligence, Tianjin University, China · The University of Hong Kong, Hong Kong SAR, China · School of Computer Science and Information Engineering, Hefei University of Technology, China
VLALight: A Vision-Language-Action Model for Traffic Signal Control
Sep 29, 2026Pan Zhang, Siqi Lai, Kemu Dong +1Traffic Signal Control Traffic
The Hong Kong University of Science and Technology (Guangzhou) · Dalian University of Technology
Trajectory-Level Mode Guidance for Controllable Diffusion-Based Multi-Robot Motion Planning
Sep 29, 2026Tianyou Yu, Shengze Cai, Chao XuDiffusion Planning Multi-Robot Motion Planning
Institute of Cyber-System and Control, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China
Before the Token Commits: Trajectory-Level Benchmarking of Visual Hallucinations in Diffusion VLMs
Sep 28, 2026Yadong Wang, Siping Yue, Yu Tian +2Large Language Model Hallucination Visual Hallucinations
Nanjing University of Aeronautics and Astronautics · State Key Laboratory of Ocean Sensing, ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou, 311215, China · Security Capability Center
MarsLab: A Martian Rover Simulator for Planetary Rover Autonomous Navigation
Sep 28, 2026Hoyun Kim, Beomsu Kim, Giseop KimAutonomous Surface Vehicles Terrain
Department of Robotics and Mechatronics Engineering, DGIST, Daegu, Republic of Korea
Vision--Language Signals in Constrained RL: Safety Gains Without Anticipation
Sep 28, 2026Samuel Tetteh, Cody FlemingFrozen Vision-Language Models Safety Alignment
Iowa State University Ames, Iowa, USA
VehDyn: A Driving World Model Benchmark for Vehicle Dynamics
Sep 27, 2026Tianyi Wang, Wangsheng Du, Jiazhou Chen +12Vehicle Dynamics Video World Models
VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control
Sep 17, 2026Zhongbo Zhang, Jiayi Jin, Yifan Wang +4Embodied Artificial Intelligence Trajectory-Level Credit
Dalian University of Technology · Nanyang Technological University
The Mirage of Calibrated Confidence: Trajectory-Independence of Verbalized Confidence in Vision-Language Models
Sep 16, 2026Jisoo Yang, Jaeho Han, Trung X. Pham +1Confidence Estimation Trajectory-Level Credit
Department of Artificial Intelligence, Chung-Ang University, Republic of Korea · Van Lang University, Ho Chi Minh City, Vietnam
AeroCopilotBench: Safety-Gated Evaluation of LLM Agents on Aircraft Emergency Procedures in an Executable Cockpit
Aug 17, 2026Yuchen Yuan, Zhenghuang Wu, Yuangan Li +2Pilot Large Language Model Agents
School of Aeronautic Science and Engineering, Beihang University
Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learning
Aug 13, 2026Yao Zhou, Hang Gao, Fengge Wu +2Offline Reinforcement Learning Trajectory-Level Credit
Institute of Software Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
Redistribution-based Cost Inference Improves Sparse Safe Offline RL
Aug 12, 2026Ebenezer Gelo, Geraud Nangue Tasse, Steven James +1Model-Based Reinforcement Learning Trajectory-Level Credit
University of the Witwatersrand · University of the Witwatersrand Johannesburg, South Africa
Language-Structured Relational Q-Learning for Threat-Aware Control in Safety-Critical Driving
Aug 11, 2026Aditya Humnabadkar, Huaizhong Zhang, Ardhendu BeheraQ-Learning Trajectory-Level Credit
Department of Computer Sciences, Edge Hill University, United Kingdom
DreOPD: Degraded-Reference Extrapolative On-Policy Distillation for Flow-matching Models
Aug 10, 2026Mingfeng Lin, Chengfei Cai, Lin Xu +2Trajectory-Level Credit Degradation
Harbin Institute of Technology (Shenzhen) · Zhejiang University · Harbin Institute of Technology
How Much, Then Where: Credit-Conserving Action-to-Token Allocation for Multi-Turn Agent Reinforcement Learning
Aug 7, 2026Lichao Ma, Yang Sun, Shuaitao Zhao +9Credit Assignment Step-Level Credit
1Peking University · 2Meituan LongCat Interaction Team · 3Fudan University +4
Routing Is Least Learnable Where It Is Most Valuable: Bounds on Representation Routing for Web Agents
Aug 6, 2026Jiaming Wei, Zekun Wu, Adriano Koshiyama +1Web Agents Strategic Agents
University College London / Holistic AI · Holistic AI / University College London · Holistic AI +1
When Shared Rollouts Fail in Defensive Driving Evaluation: A NAVSIM Score Basis Audit
Aug 5, 2026Ziang Wei, Minjun Yu, Zheyuan Lai +2Rollout Trajectory-Level Credit
Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs
Aug 3, 2026Zixuan Huang, Yang Zhou, Kaixuan Wang +7Autonomous Driving Trajectory-Level Credit
1Beihang University · 2Zhuoyu Technology, Shenzhen, China · 3Zhejiang University +1
HindSearch: Trajectory-Level Hindsight Critique for Search-Augmented Reinforcement Learning
Aug 3, 2026Haowei Liu, Jiamian Wang, Hsin-Tai Wu +2Hindsight Search Agents
Santa Clara University Santa Clara, CA · Rochester Institute of Technology Rochester, NY · Independent Researcher Sunnyvale, CA
Outcome-Confounded Local Supervision in On-Policy Distillation
Jul 26, 2026Guoqing MaTeacher Trajectory-Level Credit
The Chinese University of Hong Kong, Shenzhen
Rewarding Better Thinking for LLM Preference Alignment
Jul 22, 2026Xubo Liu, Wenya Guo, Ruxue Yan +2Preference Alignment Trajectory-Level Credit
VCIP, DISSec, College of Computer Science, Nankai University, Tianjin, China
A Systematic Evaluation of Trajectory Data Curation for LoRA Fine-Tuning of Code Agents
Jul 19, 2026Yunze HanCoding Agents Supervised Finetuning
Vdynamics GmbH, 80935 Munich, Germany
HiLLTS: Zero-Shot Hierarchical LLM-Guided Traffic Signal Control for Sustainable Transportation
Jul 17, 2026Yue Ding, Tendai Mukande, Mingming LiuTraffic Signal Control Congestion
Research Ireland Centre for Researching Training in Machine Learning at Dublin City University
AdvNav: Behavior-Guided Black-Box Adversarial Attacks on Vision-Language Navigation
Jul 13, 2026Chenyang Li, Kaige Li, Zeyu Jiang +1Vision-Language Navigation Black-Box Adversarial Attacks
The Hong Kong University of Science and Technology (Guangzhou) · Sun Yat-sen University
Adaptive Compute in Latent World Models: When Depth Helps, Hurts, or Doesn't Matter
Jul 11, 2026Achyuthan SivasankarAutoregressive Rollout World Model Planning
New York University
Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions
Jul 8, 2026Zavier LiShape Optimization Adaptive Optimizers
Xidian University Xi’an, China
EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agents
Jul 7, 2026Mianqiu Huang, Taofeng Xue, Chong Peng +12Computer-Use Agents Offline Reinforcement Learning
1Meituan · 2Fudan University · 3Shanghai Jiao Tong University +1
TurnOPD: Making On-Policy Distillation Turn-Aware for Efficient Long-Horizon Agent Training
Jul 7, 2026Yuhang Zhou, Kai Zheng, Haoling Li +3Efficient On-Policy Distillation Online-Policy Distillation
Fudan University · Tencent Hunyuan
A Large-Language-Model Supported Personalized Driving Framework for Lane Change in Highway Scenarios
Jun 30, 2026Dong Bi, Yongqi Zhao, Paul Kovacevic +4Human Driving Automated Vehicles
School of Intelligent Connected Vehicle, Hubei University of Automotive Technology, 442002 Shiyan · Institute of Automotive Engineering, Graz University of Technology, Graz 8010, Austria
SWE-Router: Routing in Multi-turn Agentic Software Engineering Tasks
Jun 30, 2026Seongho Son, Sangwoong Yoon, Jiahua Tang +3Large Language Model Routing Software Engineering
University College London, United Kingdom · Ulsan National Institute of Science and Technology, South Korea · PSL Research University, France +1
PS-PPO: Prefix-Sampling PPO for Critic-Free RLHF
Jun 29, 2026Doo Hwan Hwang, Kee-Eung KimReinforcement Learning From Human Feedback Critic-Free Reinforcement Learning
Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea.
HumanMoveVQA: Can Video MLLMs reason about human movement in videos?
Jun 26, 2026Pulkit Gera, Faegheh Sardari, Asmar Nadeem +4Fine-Grained Video Understanding Human Motion
CVSSP, University of Surrey, Guildford, UK · Tesco, UK
RouterVLA: Turning Smoke Tests into Supervision for Heterogeneous VLA Selection
Jun 25, 2026Xingyu Ren, Chugang Yi, Ge Ma +1Trajectory-Level Credit Destination-Only Scoring
1The Chinese University of Hong Kong · University of Maryland, College Park · 3Tsinghua University
ExTra: Exploratory Trajectory Optimization for Language Model Reinforcement Learning
Jun 23, 2026Wenyang Hu, Junxiang Jia, Zhen Shu +3Verifiable Rewards Trajectory-Level Credit
1National University of Singapore · 2SAP
Drowning in Routine: Signal Dilution in Multi-Turn Agent Training
Jun 20, 2026Yann Pernot, Vi RetaultTrajectory-Level Credit
Mila - Québec AI Institute · Polytechnique Montréal
Towards Understanding the Power and Limits of the Muon Optimizer: A River-Valley Perspective
Jun 19, 2026Tianqi Shen, Jinji Yang, Runze Shi +3Muon Adam
Department of Computer Science City University of Hong Kong (CityUHK) · Department of Statistics and Data Science University of Pennsylvania · School of Statistics and Data Science Shanghai University of Finance and Economics
When Does Trajectory-Level Supervision Permit Efficient Offline Reinforcement Learning?
Jun 16, 2026Xuanfei Ren, Tengyang XieModel-Based Reinforcement Learning Offline Reinforcement Learning
University of Wisconsin-Madison · Madison, WI
VLADriveBench: Evaluating CoT-Action Relationship in VLA for Autonomous Driving
Jun 10, 2026Thach Nguyen, Danhua Guo, Tom Lampo +2Diffusion-Based Vision-Language-Actions Trajectory-Level Credit
Uber AV Labs
PAWS: Preference Learning with Advantage-Weighted Segments
Jun 10, 2026Aleksandar Taranovic, Onur Celik, Niklas Freymuth +6Preference Learning Trajectory-Level Credit
Autonomous Learning Robots, Karlsruhe Institute of Technology, Karlsruhe, Germany · Bosch Center for Artificial Intelligence, Renningen, Germany · Institute for Material Handling and Logistics (IFL), Karlsruhe Institute of Technology, Karlsruhe, Germany
Seeing Before Colliding: Anticipatory Safe RL with Frozen Vision-Language Models
Jun 9, 2026Samuel Tetteh, Cody FlemingSafety Constraints Trajectory-Level Credit
Iowa State University Ames, Iowa, USA
DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Driving
Jun 7, 2026Qimao Chen, Fang Li, Yuechen Luo +11Autonomous Driving Drives
1Tsinghua University · 2Xiaomi EV
TraRA: Trajectory-level Recognition Aggregation for Video Text Spotting in Urban Surveillance
Jun 5, 2026Duc Tri Tran, Trung Thanh Nguyen, Vijay John +2Video Surveillance Multi-Object Tracking
RIKEN, Japan · Hanoi University of Science and Technology, Vietnam · Nagoya University, Japan +2
TRACE: Trajectory Reasoning through Adaptive Cross-Step Evidence Aggregation for LLM Agents
Jun 5, 2026Vijitha Mittapalli, Shreyaa Jayant Dani, Satya Srujana Pilli +7Traces Trajectory-Level Credit
University of Massachusetts at Amherst · Adobe Research · Dolby Labs +2
StainFlow: Entity-Stain Tracking and Evidence Linking for Process Rewards in GUI Agents
Jun 5, 2026Haojie Hao, Longkun Hao, Yihang Lou +8Process Reward Model Graphical User Interface Agents
Beihang University · Peking University · Renmin University of China +4
Fix the Mind, Not the Move: Interpretable AI Assistance via Knowledge-Gap Localization
Jun 4, 2026Ayano Hiranaka, Ya-Chuan Hsu, Stefanos Nikolaidis +2Student Misconceptions Trajectory-Level Credit
Thomas Lord Department of Computer Science, University of Southern California, Los Angeles, USA.
MMG2Skill: Can Agents Distill In-the-Wild Guides into Self-Evolving Skills?
Jun 1, 2026Xinyu Che, Junqi Xiong, Yunfei Ge +10Skill Evolution Skills
Nanjing University · Kuaishou Technology
CARVE: Certified Affordable Repair of Vetoed Maneuvers via Envelopes for Interactive Driving
May 31, 2026Yifan WangHuman Driving Reachability
Department of Mechanical Engineering, McGill University, QC, H3A 2T7, Canada
BraveGuard: From Open-World Threats to Safer Computer-Use Agents
May 31, 2026Yunhao Feng, Xiaohu Du, Xinhao Deng +13LLM Defense Mechanisms Computer-Use Agents
1Fudan University · Ant Group · 8ShanghaiUniversity +5
DriveMA: Driving Vision-Language-Action Models with verifiable Meta-Actions
May 29, 2026Weicheng Zheng, Yixin Huang, Qiao Sun +2Diffusion-Based Vision-Language-Actions Drives
1Shanghai Qi Zhi Institute · 3Tongji University · 2IIIS Tsinghua University
DiffCrossGait: Trajectory-Level Alignment for 2D-3D Cross-Modal Gait Recognition via Latent Diffusion
May 29, 2026Zhiyang Lu, Ming ChengGait Recognition Cross-Modal
1Fujian Key Laboratory of Urban Intelligent Sensing and Com · 2Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Min
On Distributional Reinforcement Learning in Chaotic Dynamical Systems
May 28, 2026James Rudd-Jones, Mirco Musolesi, María Pérez-OrtizDistributional Reinforcement Learning Chaos
Centre for Artificial Intelligence Department of Computer Science University College London London, UK · Department of Computer Science and Engineering, University of Bologna Bologna, Italy
Beyond Trajectory Rewards: Step-level Credit Assignment for Agentic Search via Graph Modeling
May 28, 2026Yuchen Liu, Yingjie Feng, Lixiong Qin +5Credit Assignment Agentic Reinforcement Learning
Beijing University of Posts and Telecommunications · Li Auto Inc.
LiteCoder-Terminal: Scaling Long-Horizon Terminal Environments for Learning Language Agents
May 28, 2026Xiaoxuan Peng, Kaiqi Zhang, Xinyu Lu +5Synthetic Environments Environment
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences
Honest Lying: Understanding Memory Confabulation in Reflexive Agents
May 28, 2026Prakhar Dixit, Sadia Kamal, Tim OatesReflections Confabulation
Department of Computer Science, University of Maryland Baltimore County, Baltimore, MD, USA.
Knowing When to Ask: Segment-Level Credit Assignment for LLM Tool Use
May 27, 2026Abhijit Kumar, Zoey Wu, Mohit SuleyLarge Language Model Tool Use Credit Assignment
Microsoft AI Redmond, WA
RECTOR: Priority-Aware Rule-Based Reranking for Compliance-Aware Autonomous Driving Trajectory Selection
May 24, 2026Hadi Hajieghrary, Benedikt Walter, Chaitanya Shinde +2Waymo Open Motion Dataset Trajectory-Level Credit
TORC Robotics LLC, an independent subsidiary of Daimler Truck AG. · and MassRobotics.
PANDO: Efficient Multimodal AI Agents via Online Skill Distillation
May 24, 2026Yubo Li, Yidi Miao, Yuntian Shen +1Multimodal Agents Web Agents
SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent
May 23, 2026Yuyang Hu, Hongjin Qian, Shuting Wang +5Long-Horizon Agents Long-Horizon Task Planning
GSAI, Renmin University of China · Beijing Academy of Artificial Intelligence

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis

Qijia He, Ruinan Jin, Jun Luo, Shaofeng Zou+1

The Ohio State University · Arizona State University

Convergence Trajectory-Level Credit

Open paper AskPDF arXiv

Asynchronous reinforcement learning (RL) improves the efficiency of large language model post-training but introduces stale rollouts generated by earlier policies. Theoretical understanding of how this staleness affects convergence and how to mitigate its impact remains limited. We derive a convergence bound for GRPO-style algorithms that explicitly characterizes the tradeoff between the gradient estimator's second moment and bias. For trajectory-level importance-weighted estimators, our analysis shows that once the second moment is uniformly controlled, delay enters the bound through the bias introduced by clipping or rescaling. Guided by this insight, we propose a novel group mass capping GRPO (GMC-GRPO) method, which minimizes a ratio-based bias bound within a class of weighted estimators sharing a common second-moment guarantee. We establish convergence guarantees for asynchronous GMC-GRPO and show that, compared with TIC-GRPO, it improves the threshold dependence of the fourth-order delay term from

O(ε^{-4})

O(ε^{-2})

ε\to0

, where

1+ε

is the ratio threshold. Under local policy overlap, the delay-dependent term decreases as

G^{-2/5}

after tuning the step size, where

G

is the group size. For fixed behavior and current policies, the bias introduced by group rescaling also vanishes as

G\to\infty

, whereas the bias from trajectory-wise clipping can persist. Experiments across Qwen3 models and reasoning benchmarks demonstrate improved robustness to stale rollouts, with GMC-GRPO achieving the best performance among stable baselines under large rollout delays.

Momentum

Related topics

Latest papers 83