Correctness Boundaries of Typed Plans for Natural-Language Robot Fleet Control
Organizations: Innopolis University, Innopolis, Russia.
Abstract
We examine what plan validation and execution reports establish in natural-language robot fleet control. Commands are traced through typed plans, deterministic corrections, fleet coordination and outcome checks in a three-level Robot Operating System 2 implementation with twenty simulated robots. An offline benchmark of 44 cases checks plan structure and selected command requirements. Across five local configurations without a response schema, 70.0-87.3% of responses pass. All runs in a selected 18-run simulation series meet their pose-based task conditions. This series includes two successful repeats that replaced incomplete runs after a software change. Saved plans and logs document duplicate pickup assignments and goal adjustments within the original task zones. Separate formation traces show a structurally accepted plan with the wrong formation shape and a rejected route, and an arrival report that omitted a requested robot after replanning. Carry completion records pickup and subsequent drop-off zone presence for the same robot. The cases distinguish changes to executable actions from changes to the requirements covered by a completion report. They motivate checking the effective assignments, goals and reported outcome against the original command.
Figures & tables
| Property | Check in this implementation | Evidence and limit |
| Plan representation | Recursive node and field checks, with selected type coercions | E1 also checks case-specific requirements. Acceptance leaves other mission properties open. |
| Robot assignments | Active-formation membership and single-carrier rules | Single-carrier rule fired in two selected M2 runs. |
| Occupied goals | Chat-level goal adjustment and planner-level blocked-cell substitution | Both mechanisms appear in M4 run 20260818_140918 . No chat rewrites in the 13 selected language runs. |
| Shared drop-off | Spread coincident goals for distinct carry tasks | Logged in three M3 runs. The design does not isolate its effect. |
| Formation assembly | Stage followers and check pose tolerances | M4 includes an assembled formation and a separate staging failure. |
| Formation route | Static path check with the planner’s footprint approximation | LNS2 rejected the leader’s route in M4 run 20260819_073009 . No pre-assembly check of the full oriented formation route. |
| Series | Runs | Inclusion and evidence |
| Selected M2/M3 | 18 | Four cells. Session IDs and code-version labels are recorded. All 18 worktrees were dirty. Thirteen runs have a language journal, of which eight have per-phase checkpoints. |
| Superseded B/M2 | 2 | Two runs at 4/5 tasks replaced after a software change. The three original successes remain in the selected cell. |
| M3 before/after | 2 additional | The pre-guard runs have logs and derived metrics but no decision journal. The three post-guard runs are already counted in the B-M3 cell above. |
| M4 formation | 4 recorded attempts | Two retained qualitative cases ( 20260818_140918 , 20260819_073009 ). A pilot ( 20260818_135527 ) stopped early and a 42 s recording ( 20260819_072231 ) with no motion was discarded. M4 is outside the task-completion scoring for M2/M3. |
| Schema off | Schema on | ||||
| Model | Size | Call (%) | Case (%) | Call (%) | Case (%) |
| Local | |||||
| Qwen3.5-4B | 4B | 70.0 | 93.2 | 77.7 | 95.5 |
| Gemma 3 12B | 12B | 71.4 | 72.7 | — | — |
| Qwen2.5-14B | 14B | 75.0 | 81.8 | 75.5 | 81.8 |
| Qwen3.5-9B | 9B | 87.3 | 95.5 | 19.1 | 22.7 |
| Category | Failed responses |
| Formation fields or staging | 82 |
| Plan structure | 81 |
| Goal checks | 34 |
| Unparseable JSON | 34 |
| Other validator messages | 22 |
| Escalation reason | 5 |
| Cell | Mission | Full runs | Tasks per run | Task interval (sim s) mean SD | Language loop (wall s) |
| no-LLM | M2 | 5/5 | 5/5 | 0 | |
| model-9b | M2 | 5/5 | 5/5 | 9.5 | |
| B | M2 | 5/5 | 5/5 | 31.6 | |
| B-M3 | M3 | 3/3 | 9/9 | 51.2 |
Robot Task Planning
Robot task planning turns goals and observations into executable actions while coordinating skills, motion, uncertainty, and safety.
See all 255 papers