REINS: A Self-Evolving Agent Harness for Real-Time Trajectory Planning
Organizations: Department of Informatics, University of Oslo, Norway · Department of Computer Science, Aalborg University, Denmark · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China · Shandong Key Laboratory of Foundational Software, School of Software, Shandong University, China ·
Abstract
Agent harnesses -- runtime scaffolds that wrap an LLM's reasoning loop with tool dispatch, verification, and persistent memory -- have become the standard substrate for deploying LLMs as autonomous agents, as demonstrated by Claude Code and Codex in software engineering. In trajectory planning, however, this paradigm has not taken hold: existing LLM-based trajectory planning methods either keep the LLM in the decision loop, exceeding the millisecond-level control budget, or offload it to offline rules or training-time supervision -- leaving the runtime planner non-evolving. Intermediate hybrids still lack the verification-and-consolidation loop that makes harnesses reliable. We ask: what runtime harness would let an LLM planner meet real-time constraints while self-evolving into a deployable one? We propose REINS, a harness built on three LLM-planner couplings: LLM outputs are (i) grounded to a learned variable-level dynamics model, (ii) verified via calibrated predicates and forward rollout, and (iii) consolidated into an indexed rule memory that serves future scenes in O(log n) without re-invoking the LLM. The harness closes a self-evolution loop: familiar scenes resolve by memory lookup; novel scenes invoke the LLM, whose verified outputs enrich the memory. Without modifying LLM weights, REINS attains 98%+ compliance, nearly 50% collision reduction, and 1.9-4.6 ms latency on simulated and real-world benchmarks at the tactical decision layer, with LLM invocation rate monotonically declining as memory matures. Code: https://github.com/ZhihongCui/REINS
Figures & tables
| Dataset | Scenario | Metric | DriveSpeak | LLaDA | MPC | MCTS | PDDL | DriReg | SymAgent | Plan-R1 | REINS |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Simulation | Hig. | Compl. ( ) | - | - | - | - | 0.9123 | 0.9452 | 0.9561 | 0.9717 | 0.9956 |
| RE ( ) | -6.0360 | -5.8537 | -5.5613 | -5.2311 | -4.7868 | -4.1282 | -3.8662 | -3.5623 | 0.5228 | ||
| CR ( ,%) | 38.28 | 35.51 | 14.27 | 16.76 | 25.93 | 21.76 | 16.52 | 12.83 | 5.67 | ||
| PE ( , ms) | 5.8921 | 5.2995 | 2.4742 | 2.3591 | 4.9521 | 4.3127 | 3.9621 | 3.7391 | 2.4998 | ||
| Mer. | Compl. ( ) | - | - | - | - | 0.9011 | 0.9382 | 0.9572 | 0.9641 | 0.9924 | |
| RE ( ) | -5.5730 | -5.2621 | -5.3891 | -5.1890 | -4.7129 | -4.1067 | -3.5622 | -3.1023 | -0.0400 |
| Metric | w/o | w/o LLM | w/o Rule | w/o | Full | |
|---|---|---|---|---|---|---|
| World | Prior | Generation | Retrieval | |||
| Hig. | Compl. ( ) | 0.9521 | 0.9628 | 0.9312 | 0.9845 | 0.9956 |
| RE ( ) | -1.9234 | -1.5390 | -2.4156 | -0.8921 | 0.5228 | |
| CR ( ,%) | 8.47 | 8.26 | 11.23 | 8.12 | 5.67 | |
| PE ( , ms) | 2.9532 | 2.6831 | 2.5124 | 3.8921 | 2.4998 | |
| Mer. | Compl. ( ) | 0.9687 | 0.9738 | 0.9198 | 0.9812 | 0.9924 |
| Checkpoint | LLM Call Rate (%) | Compl. ( ) | CR (%) ( ) | |
|---|---|---|---|---|
| ep. 250 | 113 | 70.0 | 0.9885 | 7.05 |
| ep. 500 | 127 | 48.44 | 0.9920 | 6.11 |
| ep. 750 | 138 | 25.0 | 0.9942 | 5.83 |
| ep. 1000 (final) | 145 | 12.81 | 0.9956 | 5.67 |
| Frozen (1000-ep eval) | 88 | — | 0.9800 | 9.63 |
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
| Macro Variable | Micro Variable | Description |
| Environment | weather_type | Weather condition (clear/rain/fog/snow) |
| visibility_distance | Visibility range (m) | |
| road_friction | Road surface friction coefficient | |
| road_type | Scenario type (highway/merge/intersection/roundabout) | |
| time_of_day | Time period (day/night/dawn/dusk) | |
| speed_limit | Posted speed limit (km/h) |
| Category | Setting |
|---|---|
| Simulation Environment | |
| Simulator | highway-env v1.8.2 |
| Scenarios | Highway, Merge, Roundabout, |
| Intersection | |
| Episode length | 100 steps (20s at 5 Hz) |
| Observation | 15 nearest vehicles |
| Scen. | Metric | LLaDA | MPC | SymAgent | REINS |
|---|---|---|---|---|---|
| Hig. | ADE | 26.5 | 20.1 | 18.1 | 15.9 |
| FDE | 29.5 | 25.1 | 24.5 | 20.5 | |
| Reward | 14.7 | 21.6 | 23.1 | 26.3 | |
| Mer. | ADE | 17.6 | 13.3 | 12.7 | 9.9 |
| FDE | 21.8 | 20.0 | 18.5 | 15.2 | |
| Reward | 25.2 | 50.7 | 54.5 | 58.4 |
| Dataset | Scen. | Metric | =2 | =4 | = 6 | =8 | =10 |
|---|---|---|---|---|---|---|---|
| Sim. | Hig. | Compl. | 0.9934 | 0.9948 | 0.9956 | 0.9952 | 0.9947 |
| RE | -0.42 | 0.31 | 0.52 | 0.38 | 0.15 | ||
| CR (%) | 9.34 | 6.89 | 5.67 | 6.72 | 7.98 | ||
| PE (ms) | 2.21 | 2.35 | 2.50 | 2.68 | 2.89 | ||
| Rou. | Compl. | 0.9898 | 0.9915 | 0.9925 | 0.9918 | 0.9908 | |
| RE | -1.23 | -0.78 | -0.54 | -0.72 | -0.95 |
| Template | Simulation | Real-world | Total | ||||
|---|---|---|---|---|---|---|---|
| Hig. | Mer. | Rou. | Int. | highD | rounD | ||
| IF-THEN | 42 | 48 | 50 | 55 | 38 | 43 | 276 |
| AND-THEN | 38 | 45 | 47 | 52 | 32 | 37 | 251 |
| THRESHOLD | 32 | 35 | 36 | 40 | 30 | 32 | 205 |
| PRIORITY | 12 | 20 | 22 | 28 | 15 | 21 | 118 |
| MULTI-ACTION | 10 | 13 | 14 | 17 | 11 | 12 | 77 |
| Scenario | Checkpoint | LLM Call (%) | Compl. ( ) | CR (%) ( ) | |
|---|---|---|---|---|---|
| Merge | ep. 250 | 136 | 71.36 | 0.9858 | 8.20 |
| ep. 500 | 153 | 49.12 | 0.9893 | 7.35 | |
| ep. 750 | 167 | 26.48 | 0.9912 | 7.05 | |
| ep. 1000 (final) | 175 | 13.92 | 0.9924 | 6.86 | |
| Frozen (1000-ep eval) | 105 | — | 0.9770 | 10.65 | |
| Roundabout | ep. 250 | 144 | 73.84 | 0.9852 | 10.55 |
| ID | Guard Condition | Action | Template |
|---|---|---|---|
| R01 | emergency brake | THRESHOLD | |
| R02 | DEC to limit | THRESHOLD | |
| R03 | DEC / maintain gap | IF-THEN | |
| R04 | DEC (moderate) | AND-THEN | |
| R05 | DEC (preemptive) | AND-THEN | |
| R06 | reduce speed 20% | THRESHOLD |
| Component | Driving (this paper) | Manipulation (sketch) | Transfer |
|---|---|---|---|
| Variable space and groups | 44 variables in Env, Traffic, Ego, Action, Outcome | joint angles and torques, end-effector pose, contact forces, object poses | redesign |
| Action space | 5 driving meta-actions | skill primitives (reach, grasp, lift, place, retreat) | redesign |
| Predicate grammar and thresholds | TTC and THW guards | force-margin, slip, and grasp-stability guards | grammar unchanged, thresholds recalibrated |
| Dynamics learning | LLM prior, group-weighted ridge, per-variable Transformer | same, retrained on domain data | unchanged |
| Verification | collision and violation terms in | contact-force and stability terms in | protocol unchanged |
| Indexed memory and retrieval | variable-keyed graph schema | same | unchanged |
| Condition | PDDL | DriReg | LLaDA |
|---|---|---|---|
| Observation space, action space, episodes | shared | shared | shared |
| GPT-4o backbone (2024-08-06), temperature 0 for rule generation | shared | shared | shared |
| 44-variable vocabulary (Appendix A.1 ) in rule-generation prompts | own vocabulary | own vocabulary | n/a (linguistic rules) |
| Threshold calibration | uncalibrated | uncalibrated | n/a (no thresholds) |
| CR (%) | PDDL | PDDL matched | DriReg | DriReg matched | REINS |
|---|---|---|---|---|---|
| Highway | 25.93 | 21.37 | 21.76 | 18.92 | 5.67 |
| Intersection | 32.12 | 27.64 | 29.89 | 25.81 | 9.08 |
| Method | Own rules | Common reference (REINS set) | ||
|---|---|---|---|---|
| Highway | Intersection | Highway | Intersection | |
| MPC | n/a | n/a | 0.8712 | 0.8423 |
| MCTS | n/a | n/a | 0.8831 | 0.8517 |
| DriveSpeak | n/a | n/a | 0.8246 | 0.7912 |
| LLaDA | n/a | n/a | 0.8412 | 0.8104 |
| PDDL | 0.9123 | 0.8789 | 0.8867 | 0.8621 |
| Variation (Highway) | Rule overlap with GPT-4o | Threshold shift after calibration | Compl. | CR (%) |
|---|---|---|---|---|
| GPT-4o (paper) | n/a | n/a | 0.9956 | 5.67 |
| Claude Sonnet 4.5 | 87.3% | 3.2% | 0.9948 | 5.89 |
| Qwen2.5-72B-Instruct | 81.6% | 3.8% | 0.9931 | 6.24 |
| Three prompt paraphrases, GPT-4o | 91.2–94.6% | — | — | 5.58–5.91 |
| CR (%) on Highway | no noise | 2.5% noise | 5% noise | 10% noise | 5% frame drops |
|---|---|---|---|---|---|
| REINS | 5.67 | 5.92 | 6.51 | 8.14 | 6.88 |
| Plan-R1 | 12.83 | 13.71 | 15.42 | 19.87 | 16.50 |
| Scenario | Checkpoint | LLM Call (%) | Compl. | CR (%) | PE (ms) | |
|---|---|---|---|---|---|---|
| Setting A: from highD and rounD, deployed on highway-env | ||||||
| Highway | ep. 250 | 265 | 86.2 | 0.9612 | 11.84 | 2.71 |
| ep. 1000 | 349 | 14.9 | 0.9914 | 6.21 | 2.54 | |
| ep. 1000, acquisition frozen | — | — | — | 11.20 | — | |
| native, ep. 1000 | 145 | 12.81 | 0.9956 | 5.67 | 2.50 | |
| Intersection | ep. 250 | 287 | 91.3 | 0.9487 | 15.62 | 4.87 |
| Stage | Mean | p95 | Deployment scheduling |
|---|---|---|---|
| Fallback action for the current step | 2.6 ms | 3.9 ms | control path, within budget |
| Prompt assembly and state serialization | 0.9 ms | 1.4 ms | background |
| LLM rule proposal (GPT-4o API) | 1.58 s | 2.94 s | background |
| Parsing and schema validation | 2.1 ms | 3.0 ms | background |
| Threshold calibration on the sliding window | 1.3 ms | 1.9 ms | background |
| Rollout verification at | 1.6 ms | 2.2 ms | background |
| Memory size | 1,023 (paper) | 10,000 | 50,000 | 100,000 |
|---|---|---|---|---|
| Cypher retrieval (ms) | 0.41 | 0.55 | 0.68 | 0.79 |
| End-to-end PE (ms) | 2.50 | 2.64 | 2.77 | 2.88 |
LLM Agent Planning
Large language model (LLM) agent planning studies how agents decompose goals, choose actions and tools, and adapt plans from feedback.
See all 239 papers