We examine what plan validation and execution reports establish in natural-language robot fleet control. Commands are traced through typed plans, deterministic corrections, fleet coordination and outcome checks in a three-level Robot Operating System 2 implementation with twenty simulated robots. An offline benchmark of 44 cases checks plan structure and selected command requirements. Across five local configurations without a response schema, 70.0-87.3% of responses pass. All runs in a selected 18-run simulation series meet their pose-based task conditions. This series includes two successful repeats that replaced incomplete runs after a software change. Saved plans and logs document duplicate pickup assignments and goal adjustments within the original task zones. Separate formation traces show a structurally accepted plan with the wrong formation shape and a rejected route, and an arrival report that omitted a requested robot after replanning. Carry completion records pickup and subsequent drop-off zone presence for the same robot. The cases distinguish changes to executable actions from changes to the requirements covered by a completion report. They motivate checking the effective assignments, goals and reported outcome against the original command.
Figures & tables
Figure 1: Illustrative duplicate-carrier correction at the language–coordination boundary. An additional robot is routed to the same pickup. The guard retains the robot with the drop-off leg and removes the extra pickup assignment
Figure 2: Operator-chat path from a typed fleet plan through interface checks, fleet coordination, and per-robot execution
Property
Check in this implementation
Evidence and limit
Plan representation
Recursive node and field checks, with selected type coercions
E1 also checks case-specific requirements. Acceptance leaves other mission properties open.
Robot assignments
Active-formation membership and single-carrier rules
Single-carrier rule fired in two selected M2 runs.
Occupied goals
Chat-level goal adjustment and planner-level blocked-cell substitution
Both mechanisms appear in M4 run 20260818_140918 . No chat rewrites in the 13 selected language runs.
Shared drop-off
Spread coincident goals for distinct carry tasks
Logged in three M3 runs. The design does not isolate its effect.
Formation assembly
Stage followers and check pose tolerances
M4 includes an assembled formation and a separate staging failure.
Formation route
Static path check with the planner’s footprint approximation
LNS2 rejected the leader’s route in M4 run 20260819_073009 . No pre-assembly check of the full oriented formation route.
Table 1: Implemented checks and their observed scope
Series
Runs
Inclusion and evidence
Selected M2/M3
18
Four cells. Session IDs and code-version labels are recorded. All 18 worktrees were dirty. Thirteen runs have a language journal, of which eight have per-phase checkpoints.
Superseded B/M2
2
Two runs at 4/5 tasks replaced after a software change. The three original successes remain in the selected cell.
M3 before/after
2 additional
The pre-guard runs have logs and derived metrics but no decision journal. The three post-guard runs are already counted in the B-M3 cell above.
M4 formation
4 recorded attempts
Two retained qualitative cases ( 20260818_140918 , 20260819_073009 ). A pilot ( 20260818_135527 ) stopped early and a 42 s recording ( 20260819_072231 ) with no motion was discarded. M4 is outside the task-completion scoring for M2/M3.
Table 2: Provenance and analysis populations for the simulation evidence
Figure 3: The amongus map ( 66.6×37.75 m, 0.05 m cells) and the eleven distinct zones of radius 1.5 m used by the nine M3 tasks. Circles show the zone boundaries and center markers identify visit, pickup, and drop-off zones. Arrows link pickup and drop-off locations. Stars mark the M2 subset. Squares mark the home locations of the five groups of four robots
Schema off
Schema on
Model
Size
Call (%)
Case (%)
Call (%)
Case (%)
Local
Qwen3.5-4B
4B
70.0
93.2
77.7
95.5
Gemma 3 12B
12B
71.4
72.7
—
—
Qwen2.5-14B
14B
75.0
81.8
75.5
81.8
Qwen3.5-9B
9B
87.3
95.5
19.1
22.7
Table 3: Offline validation on 44 cases with five scored responses per case and condition. Call rate is the percentage of 220 responses that passed. Case rate is the percentage of 44 cases with at least one pass in five attempts. Model sizes are nominal. For mixture-of-experts models they are total/active parameters. n/d denotes an undisclosed size. A dash denotes an unreported condition
Category
Failed responses
Formation fields or staging
82
Plan structure
81
Goal checks
34
Unparseable JSON
34
Other validator messages
22
Escalation reason
5
Table 4: Failure categories for local schema-off responses in E1. The scoring script assigns one category per failed response using a fixed priority order over error messages
Cell
Mission
Full runs
Tasks per run
Task interval (sim s) mean ± SD
Language loop (wall s)
no-LLM
M2
5/5
5/5
203.5±10.6
0
model-9b
M2
5/5
5/5
252.2±26.6
9.5
B
M2
5/5
5/5
260.0±16.5
31.6
B-M3
M3
3/3
9/9
587.7±67.0
51.2
Table 5: Selected M2/M3 runs. Task counts are pose-based outcomes per run. The task interval uses simulation seconds and is reported as mean and sample standard deviation (SD). Language-loop time is the mean wall-clock duration, including auxiliary calls and retries. The version log reports that 3/5 original B/M2 runs completed all tasks. The two runs that stopped at 4/5 tasks were replaced after a software change
CREDIT Center & Department of Electrical and Computer Engineering, Prairie View A&M University, Texas A&M University Systems, Prairie View, TX 77446, USA