Structure for Reading, Prose for Writing: Asymmetric Structural Conditioning in Multi-Agent Document Authoring
Organizations: ML Research Labs*, Canberra, Australia
Abstract
Multi-agent pipelines that author formal documents must both read a requester's forms and write against them. We report a deployed tender-response system, running an open-weights model under sovereignty constraints, and evaluate it against human-written bids the same organisation actually submitted. On a blind comparison where the system had no worked example available, an LLM judge rated its answers at least as good as the human-submitted answer on of ground-truth sections, better on , missing on none, and flagged one unsupported claim in total. Classifying every gap the judge identified shows that were content absent from the system's own sources -- knowledge the human author held and the pipeline was never given -- so only of the adverse verdicts involve a deficiency the system could have avoided. A divergence from ground truth is more often an information-availability result than a writing-quality one, and evaluations that do not separate the two understate such systems. Against this backdrop we report a conditioning asymmetry. It is well established that rendering documents as structural markup rather than flat prose improves extraction, and we reproduce that on three reading tasks. The benefit does not transfer to conditioning: converting a bid's instruction material from prose to nested XML dropped answer quality from to under a paired comparison. We further find that naming a forbidden construction concentrates rather than removes it -- of surviving defects fall in the two forms the prompt explicitly names -- and that coupling a stochastic annotation to a deterministic windowing function moves the extracted requirement count from to on a byte-identical file. Structure belongs where the model reads; prose and self-applied tests belong where it writes.
Figures & tables
| Verdict | Share | |
|---|---|---|
| better | 4 | 7% |
| as good | 36 | 65% |
| weaker | 15 | 27% |
| missing | 0 | 0% |
| at least as good | 40 | 73% |
| sections w/ unsupported claim | 1 | 2% |
| Cause | Share | |
|---|---|---|
| Information unavailable to the system | 48 | 68% |
| Additional, non-required detail | 7 | 10% |
| Avoidable – available but unused | 15 | 21% |
| Task | Flat / join | Markup |
|---|---|---|
| Annotation routing | 61% | 97.8% |
| Workbook slots ( ) | 404/280/340 | 310/310/310 |
| Past-bid segmentation | varied | identical ( ) |
| Past-bid pair floor | 3 | 47 |
| Metric | Prose | XML |
| answering verdict | 23/31 | 15/31 |
| Mean restated share | 17.6% | 25.0% |
| Verbatim fidelity | 19/24 | 13/24 |
| Requester-as-subject openings | 4 | 10 |
| Surface form | Named? | |
| Nominalised passive | 14 | yes |
| Requester-as-subject | 11 | yes |
| Product-name opener | 1 | yes |
| Modal must / shall | 0 | yes |
| Suppressed by the strip test | — | no |
| Quantity | Run 1 | Run 2 |
|---|---|---|
| Elements parsed | 349 | 349 |
| Elements echoed back | 349/349 | 348/349 |
| Answer slots found | 54 | 56 |
| Window 2 span | E0104–E0273 | E0107–E0275 |
| Questions extracted | 68 | 51 |
Multi-Agent LLM Systems
Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.
See all 1,040 papers