We describe the RMIT-ADM+S team's participation in the TREC 2026 Retrieval-Augmented Generation (RAG) track using the ClimbMix corpus. We submitted seven research-agent runs using proprietary OpenAI and open-weight models, exploring different retrieval strategies, writer models, and agent workflows. Pre-submission ablations identified writer selection and adjacent-chunk expansion as the main factors affecting answer quality. In the official NIST evaluation, six of seven runs exceeded the track median across all three rubric/nugget collections. Our best-performing run, brief-hybrid, achieved the highest all-item coverage among our submissions and tied the track maximum (0.892) for Batch-1 vital coverage. Human side-by-side preferences differed across evaluation batches: the open-weight owa-glm5 led Batch 1 despite lower automated coverage, while sol-research-first led Batch 2. These findings highlight differences between automated coverage measures and human preferences, as well as the potential of lower-cost open-weight models for research-agent systems.
Figures & tables
Figure 1. Control flows for three submissions: our leading coverage run ( brief-hybrid ) and two SxS priority runs ( owa-glm5 , sol-research-first ). Purple blocks are expanded in the panel below. Solid arrows show execution order, not prompt inputs; each scout sees only the request. Unboxed notes mark absent stages; k is the typical requested passages per search. Dashed blue arrows are conditional: review feedback resumes the writer loop, which may search again; the revised draft bypasses the once-only review. For staged evidence see Section 3.1 . Three separate vertical pipelines for brief-hybrid, owa-glm5, and sol-research-first. Brief-hybrid uses GPT-5.6-Sol in all roles, with brief and review stages, hybrid RRF retrieval at k=10, and one review pass. Owa-glm5 uses GLM-5 across all stages and adds a blind obligation scout whose items are excluded from the review hook. Sol-research-first uses GPT-5.6-Sol in all roles with a prose coverage plan, blind scout, semantic plus keyword search at k=10, and a final answer from the same 40-round research loop. No blind scout for brief-hybrid and no post-draft review for sol-research-first are unboxed absence notes, not processing stages. Solid arrows show execution order; scouts receive only the request, not the preceding stage's output. Brief, scout, and reviewer stages are separate LLM calls; uncited scans and export are code. Conditional arrows return review feedback to the writer, which may search again, and send revised drafts directly to export without another review. Purple fill links the three research-and-draft blocks to a lower panel expanding their shared staged-evidence loop with adjacent-chunk expansion, commit_context retention, 500k context limit, and code-enforced citation and word limits.
System
Models (main / support)
Retrieval ( k )
Planning / Review
Adj.
sol-default
Sol / —
Sem+Kw (10)
None
No
sol-research-first ⋆
Sol / Sol
Sem+Kw (10)
Plan+Scout
Yes
brief-base
Sol / Sol
Sem+Kw (10)
Brief+Rev
Yes
brief-hybrid
Sol / Sol
Hyb (10)
Brief+Rev
Yes
facets
Luna / —
Sem+Kw (10), Hyb (15)
Facets+coverage check
No
facetrag
GPT-OSS-120B / Qwen3-Next-80B
Sem+Kw+SSR+Bool (10), Hyb (15)
Facets+multi-stage review
—
Table 1. Configurations of the submitted systems. Runs marked ⋆ entered NIST human SxS assessment. Models denote the main and supporting LLMs (orchestrator and analyzer for facetrag ). Sem: semantic; Kw: keyword; Hyb: hybrid RRF; SSR: shortest-substring ranking; Bool: Boolean search; Brief: requirements brief; Scout: blind obligation scout; Rev: post-draft review; Adj.: automatic adjacent-chunk expansion. Retrieval k denotes the most frequently requested number of passages per search call.
System
Srch
Words
Refs
In (K)
Cache (K)
Out (K)
$/topic
sol-default
23.9
882.5
27.9
182.2
202.4
6.1
1.566
sol-research-first ⋆
31.4
936.6
27.7
330.5
369.3
10.5
2.823
brief-base
26.4
830.2
25.8
319.6
388.5
8.0
2.356
brief-hybrid
22.6
834.3
24.7
293.1
322.4
7.5
2.156
facets
11.7
613.0
13.6
138.9
71.4
5.9
0.040
facetrag
33.7
388.9
13.9
411.7
0
49.7
0.106
Table 2. Average resource usage and estimated cost per report across all 119 test topics. Srch: search calls; Refs: cited references; In: uncached input tokens; Cache: cached input tokens; Out: output tokens. Token counts are in thousands (K). Runs marked ⋆ entered NIST human SxS assessment.
System
W
L
T
Win rate
sol-research-first
108
10
1
91.2%
sol-default
107
11
1
90.3%
brief-hybrid
101
18
0
84.9%
brief-base
99
19
1
83.6%
facets
13
105
1
11.3%
facetrag
0
119
0
0.0%
Table 3. Internal both-order RAGDoll arena comparisons against the organizer’s shipped agentic BM25 baseline over all 119 narratives (GPT-5.6-Luna judge, seed 13; not official TREC judgments). Win rate is (W+0.5T)/119 . owa-glm5 was finalized after this sweep.
B1 Human Rubric ( n=17 )
B2 Manual Nuggets ( n=25 )
B2 Auto Nuggets ( n=25 )
Type
System
Vital / All
≥ Med (V/A)
Vital / All
≥ Med (V/A)
Vital / All
≥ Med (V/A)
Proprietary
brief-hybrid
0.892 ⋆ / 0.909
17 / 17
0.691 / 0.723
24 / 25
0.644 / 0.565
25 / 25
sol-research-first
0.892 ⋆ / 0.887
17 / 17
0.684 / 0.702
25 / 24
0.649 / 0.560
25 / 25
facets
0.831 / 0.841
17 / 16
0.627 / 0.642
23 / 23
0.390 / 0.333
25 / 24
brief-base
0.824 / 0.862
17 / 16
0.691 / 0.721
24 / 25
0.648 / 0.555
25 / 25
sol-default
0.790 / 0.861
17 / 17
0.679 / 0.705
25 / 25
0.630 / 0.563
25 / 25
Table 4. Official NIST TREC 2026 RAG coverage ( vital_strict / all_strict ) and numbers of narratives scoring at or above the global track median ( ≥ Med, V/A) across our seven submitted runs. Underlining marks our highest run per metric; ⋆ denotes a tie with the global track maximum before rounding. Two B2 manual narratives ( rag2026-89 , rag2026-114 ) contain no vital nuggets, capping vital_strict at 23/25=0.920 .
Figure 2. Official human side-by-side Bradley–Terry ratings with published 95% confidence intervals and track medians and maxima. Batch 1 ( n=17 ) and Batch 2 ( n=25 ) were fitted separately and used different topic sets and assessment protocols. Two stacked panels showing Bradley–Terry ratings for owa-glm5 and sol-research-first. Owa-glm5 leads in Batch 1, while sol-research-first leads in Batch 2.
Figure 3. Per-topic official all_strict coverage difference ( Δ ) relative to the TREC 2026 RAG track median across Batch 1 ( n=17 , left) and Batch 2 manual nuggets ( n=25 , right), sorted within each panel by brief-hybrid Δ . Shaded bars show the difference between the track maximum and median. Two-panel per-topic plot showing substantial variation in coverage relative to the track median, including the coverage– preference divergence on rag2026-68 and the owa-glm5 failure on rag2026-65.
Measure
owa-glm5
sol-research-first
Words
805.3
951.2
Unique references
16.6
28.9
Flesch–Kincaid Grade Level ↓
15.7
22.2
Topics with shorter report
15/17
2/17
Table 5. Report characteristics on the 17 matched Batch-1 topics (means, except topic counts).
Autonomous research agents use language models and tools to search literature, design and run experiments, analyze results, and support scientific discovery.