NSDL: Neuro-Symbolic Division of Labor for Multi-Robot Task Planning
Organizations: Zhejiang University, Hangzhou, China · The Hong Kong Polytechnic University, Hong Kong SAR, China
Abstract
Planning for a heterogeneous robot team from a scene image and an instruction involves two kinds of decisions. Some must be judged from the scene: which goals are already met, how the task splits, and which robot does what. Others follow from these and can be computed: prerequisite actions, handoffs between robots, and timing. We argue that each kind should go to the component suited to it: a small vision-language model learns the first, and a deterministic solver computes the second. In NSDL (Neuro-Symbolic Division of Labor), the trained model writes its judgments as a plan sketch, which lists the goals that still need action and the intended steps in order, each with a robot. The solver expands the sketch into an executable joint plan. The model calls two tools, a detector for evidence and the solver, and is trained with the solver in the loop. We evaluate on the public VIKI-L2 benchmark and on LOCUS-Bench, a new benchmark that grades the two kinds of difficulty separately. NSDL outperforms end-to-end planners and formalize-then-plan baselines on both benchmarks and with three backbones from 3B to 8B; the 7B model reaches 97.0% out-of-distribution accuracy on VIKI-L2. Ordering and scheduling errors nearly disappear once the solver writes the structure; the remaining failures are misjudged states, and most goals the model misses are ones it never lists. Code, model weights, and LOCUS-Bench are publicly available at https://github.com/Xincheng-Tan/NSDL.
Figures & tables
| Method | Judgment | ID | OOD | rej.% ID/OOD |
|---|---|---|---|---|
| End-to-end | ||||
| Qwen2.5-VL-7B zero-shot (rep.) | untrained | .004 | .000 | – |
| Qwen2.5-VL-72B zero-shot (rep.) | untrained | .084 | .012 | – |
| GPT-4o zero-shot (rep.) | untrained | .226 | .100 | – |
| VIKI-R 3B (rep., SFT+GRPO) [ 2 ] | model | .936 | .321 | – |
| Ans-SFT 7B (rep.) [ 2 ] | model | .969 | .256 | – |
| Method | Judgment | ID | OOD-S | OOD-M | OOD-X | mean | rej.% | |
|---|---|---|---|---|---|---|---|---|
| A: no training on LOCUS | ||||||||
| Qwen2.5-VL-7B zero-shot (our tools) | untrained | ✓ | .283 | .121 | .135 | .175 | .179 | 22.0 |
| gpt-5.6-terra (our tools) | untrained | ✓ | .317 | .186 | .120 | .050 | .168 | 42.9 |
| VIKI-R 7B [ 2 ] , op-level protocol | model | – | .072 | .014 | .040 | .000 | .032 | 71.3 |
| VIKI-R 7B [ 2 ] , native protocol | model | – | .467 | .229 | .255 | .125 | .269 | – |
| Formalize-then-plan, prompt-only | untrained | ✓ | .150 | .064 | .100 | .225 | .135 | 31.4 |
| Configuration | ID | OOD-S | OOD-M | OOD-X | mean | verdict | V-ID | V-OOD |
|---|---|---|---|---|---|---|---|---|
| Block 1: evidence | ||||||||
| detect off | .667 | .207 | .365 | .225 | .366 | .683 | .843 | .327 |
| YOLO, crops (NSDL-SFT) | .794 | .350 | .420 | .325 | .472 | .747 | .980 | .970 |
| YOLO, crops disabled | .794 | .321 | .420 | .350 | .471 | – | .978 | .970 |
| GT rel., query scope | .722 | .307 | .405 | .350 | .446 | .707 | .938 | .758 |
| GT rel., all entities | .728 | .343 | .425 | .425 | .480 | .728 | .937 | .753 |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Configuration | Value | Paired baseline, , |
|---|---|---|
| NSDL-SFT, crops disabled (LOCUS) | .511 | NSDL-SFT with crops, , |
| GT rel., detector’s forms (LOCUS) | .541 | crops disabled, , |
| GT rel., detector’s forms, all entities (VIKI) | same as query scope | on every row |
| GT rel., detector’s forms (VIKI) | .985 / 1.000 | crops disabled, / , |
| YOLO-World replacing the detector, 7B | no detector | |
| YOLO-World replacing the detector, 8B | no detector |
| Split | family-cells | S | M | empty | mean | non-empty | max | |
|---|---|---|---|---|---|---|---|---|
| train | 6300 | 30 | 0–2 | 0–2 | 972 | 5.1 | 6.0 | 11 |
| ID test | 180 | 30 | 0–2 | 0–2 | 31 | 5.0 | 6.1 | 11 |
| OOD-S | 140 | 7 | 3–4 | 0–2 | 12 | 6.5 | 7.1 | 15 |
| OOD-M | 200 | 10 | 0–2 | 3–4 | 27 | 6.3 | 7.3 | 10 |
| OOD-X | 40 | 2 | 3–4 | 3 | 3 | 7.5 | 8.1 | 10 |