Retrieved-Span Training for Efficient Query-Focused Meeting Summarization on QMSum

Edward Xi Yang

cs.CLAug 18, 2026

Retrieved-Span Training for Efficient Query-Focused Meeting Summarization on QMSum

Authors: Edward Xi Yang

Organizations: Ertas AI · Ertas AI Melbourne, Australia

Abstract

QMSum provides no scorer, making query-focused meeting summarization results difficult to compare. We rescore or generate 15 systems under one implementation. Through a common inference port, a released 406M Fusion-in-Decoder specialist loses 6.30 ROUGE-1 when moved from capped long input to 2,000-word retrieved spans. Fine-tuning it on this span regime recovers the loss. On test it scores 36.33 ROUGE-1 versus 35.41 for our 1.2B system; the meeting-cluster 95% interval for the difference is [-0.27, +2.22], so QMSum does not statistically separate them. The smaller system uses about one-third as many total parameters and less than half the peak inference memory. Within the fixed 1.2B base, span-regime fine-tuning adds 5.29 [+4.02, +6.56], while replacing the first 4,500 transcript words with 2,000 retrieved words adds 1.55 on test and 0.29 on validation. Separately, under one concise prompt and reference-overlap scorer, a released 406M specialist exceeds five proprietary hosted models by at least 6.2 ROUGE-1, but output length and absent human or factuality evaluation limit this ordering. Conclusions are limited to QMSum and automatic metrics.

Figures & tables

	protocol-exact	promoted
window width	900 words	375 words
span budget	3,000 words	2,000 words
ranking cross-encoder	ms-marco-MiniLM-L-6-v2 , 6 layers, 22.7M	ms-marco-MiniLM-L-12-v2 , 12 layers, 33M
training pairs	13,738 (16.6% positive)	33,655 (11.9% positive)
utterance recall at its own budget (validation)	0.627	0.670
test ROUGE-1 of the full system	33.39	35.41

Table 1

System	Params	R1	R2	R-L	R-Lsum	BERTScore
Socratic-SegEnc (released outputs)	406M	38.60	13.91	24.98	33.71	0.8737
Span-trained SegEnc	406M	36.33	12.72	23.69	32.17	0.8710
Ours (promoted)	1.2B + 33M	35.41	12.28	24.63	31.36	0.8733
Socratic-SegEnc (our port)	406M	35.30	11.87	23.00	30.56	0.8695
Ours (protocol-exact)	1.2B + 22.7M	33.39	10.65	22.83	29.30	0.8680
GPT-5.6 Luna (zero-shot)	undisclosed	32.43	7.89	19.05	27.49	0.8608

Table 1: The single-protocol scale. Full official test split (n=281), one frozen scorer, at most one test touch per system. Every row generated and scored by us, except Socratic-SegEnc (released outputs), scored from the authors’ predictions (Section 5.4).

Figure 1: The runnable-protocol scale. Every bar uses predictions generated through our evaluation path on the official test split (n=281) and one frozen scorer. The Socratic-SegEnc bar is our 35.30 port result, matching the separately labelled port row in Table 1 . Systems are sorted by ROUGE-1, with parameter counts written on the bars.

system, test split	R1 precision	R1 recall	R1 F	median words	x reference
ours, protocol-exact	0.359	0.348	0.334	56	0.95
gpt-5.6-luna	0.288	0.444	0.324	86	1.46
gpt-5.6-sol	0.279	0.417	0.309	81	1.37
claude-opus-5	0.195	0.629	0.289	223	3.78
claude-haiku-4-5	0.200	0.574	0.287	190	3.22
claude-sonnet-5	0.192	0.592	0.279	214	3.63

Table 4

comparison	delta ROUGE-1	95% interval
vs claude-sonnet-5	+7.52	[+6.24, +8.79]
vs distilbart (best community checkpoint)	+6.76	[+5.24, +8.30]
vs claude-haiku-4-5	+6.74	[+5.49, +7.97]
vs claude-opus-5	+6.54	[+5.23, +7.80]
vs gpt-5.6-sol	+4.49	[+3.24, +5.77]
vs gpt-5.6-luna	+2.98	[+1.75, +4.24]

Table 5

Figure 2: Every margin against our promoted system, with both sources of uncertainty. Bars are 95% paired bootstrap intervals over the 281 test queries. The shaded band is the 1.59-point reseeding range of Section 8.4 , which the paired bootstrap does not capture.

configuration (test split)	R1	R2	R-L	R-Lsum	BERTScore
protocol-exact	33.39	10.65	22.83	29.30	0.868
promoted (the deviating configuration)	35.41	12.28	24.63	31.36	0.873

Table 7

comparison (ours minus theirs)	delta BERTScore	95% interval
vs led-base	+0.0915	[+0.0866, +0.0964]
vs pegasus	+0.0389	[+0.0352, +0.0424]
vs claude-haiku-4-5	+0.0302	[+0.0280, +0.0323]
vs claude-sonnet-5	+0.0255	[+0.0232, +0.0277]
vs bart-large-cnn	+0.0220	[+0.0190, +0.0249]
vs claude-opus-5	+0.0211	[+0.0189, +0.0233]

Table 8

locator	window	utterance recall @2000w	@3000w
MiniLM-L6 (original)	900	0.5035	0.6268
MiniLM-L12	900	0.5108	0.6319
MiniLM-L6	375	0.6420	0.7468
MiniLM-L12	375	0.6704	0.7650

Table 9

span budget	utterance recall	val ROUGE-1
1,500	0.610	34.70
2,000	0.670	35.39
3,000	0.765	34.50

Table 10

configuration	R1
our pipeline	35.61
our gold-span oracle	36.71
stock Socratic SegEnc (authors’ predictions)	38.10

Table 11

cell, official test split	R1	R2	R-L	R-Lsum	BERTScore
base, truncated transcript	28.57	5.55	17.79	24.55	0.8607
base, our located spans	30.12	6.70	19.29	25.04	0.8687
fine-tuned, our located spans (promoted)	35.41	12.28	24.63	31.36	0.8733

Table 12

Figure 3: The two levers, de-confounded. One base model, one adapter, three conditions: only the input regime changes between the first two bars, and only the fine-tune between the second and third.

checkpoint	source regime	val R1	change
stock Socratic SegEnc	capped long input, about 11,800 words	35.30	baseline
stock Socratic SegEnc	our located spans, 2,000 words	29.00	-6.30 vs stock long-input
span-trained SegEnc, reported checkpoint	the same located spans	37.05	+8.05 vs stock-on-spans; +1.75 net

Table 14

effective epoch, matched-chunk lineage	val R1
4	37.13	end of the initial fine-tune, 22 minutes
5	38.31	peak
6	36.59
7	36.97
8	37.05	last checkpoint, the figure we report

Table 15

Figure 4: What one model’s training epochs are worth on this benchmark. Five checkpoints of the span-trained specialist differing only in training epoch, on validation. Quoting the peak rather than the plateau would overstate the model by 1.1 ROUGE-1.

basis	R1	margin over our system
last checkpoint, without checkpoint selection	37.05	+1.67, [+0.58, +2.76]
plateau mean, epochs 5 to 8	37.23	about +1.8
worst checkpoint after the peak	36.59	+1.20
peak, reported for completeness only	38.31	+2.93, [+1.88, +3.99]

Table 17

metric	ours	span-trained SegEnc	diff	95% interval
ROUGE-1	35.41	36.33	+0.93	[-0.42, +2.24]
ROUGE-2	12.28	12.72	+0.43	[-0.77, +1.64]
ROUGE-L	24.63	23.69	-0.94	[-2.06, +0.18]
ROUGE-Lsum	31.36	32.17	+0.81	[-0.47, +2.05]
BERTScore	0.8733	0.8710	-0.0022	[-0.0046, +0.0001]

Table 18

	span-trained SegEnc	ours	ratio
parameters, incl. the shared 33M locator	439M	1.2B + 33M	2.8x
peak inference VRAM, max over the validation split	2.66 GB	5.73 GB	2.16x
peak inference VRAM, median	2.66 GB	4.71 GB	1.77x
mean latency per query, two validation runs each, locate step included	1.85s and 2.00s	2.35s and 2.76s	1.3x to 1.4x
training to this checkpoint	about 45 minutes (eight matched-chunk epochs; the first four measured at 22 minutes)	4.7 to 5.6 hours	large
test ROUGE-1	36.33	35.41	difference unresolved

Table 2: Resource comparison at an unresolved test difference. Both systems include the shared locator; ratios divide our measured value by the span-trained SegEnc value. Crossing intervals do not establish formal equivalence.

seed	ROUGE-1	ROUGE-2	ROUGE-Lsum	BERTScore
20260723 (promoted)	35.39	11.39	31.09	0.8724
101	33.80	10.37	29.86	0.8701
range	1.59	1.02	1.23	0.002

Table 20

Artifact	Repository	Revision
Summarizer adapter, promoted configuration	ErtasAI/qmsum-summarizer-lfm2.5-1.2b-lora	7d73ab6535ea
Locator, promoted, 12 layers at a 375-word window	ErtasAI/qmsum-locator-minilm-l12-w375	f54ed53f816b
Locator, protocol-exact, 6 layers at a 900-word window	ErtasAI/qmsum-locator-minilm-l6-w900	a1b9e64d82ca
Span-trained SegEnc of Section 7.5, 406M	ErtasAI/qmsum-summarizer-segenc-406m-spans	26abdfc1b037

Table 21

Text Summarization

Text summarization develops methods for condensing documents while preserving the information, perspective, and evidence readers need.

See all 142 papers

Segment-level Tree Search for Long Meeting Document Summarization
Jun 7, 2026Sangwon Ryu, Heejin Do, Jun Seo +4Monte Carlo Tree Search Text Summarization
GSAI, POSTECH · ETH Zurich, ETH AI Center · Agentic AI Lab, KT +2
Preserving What Matters: Semantic Scaffolds Beyond Saturation in Summarization Evaluation
Sep 18, 2026Nikhil Reddy Pottanigari, Ramin Fahimi, Noah Bolger +2LLM Evaluation Text Summarization
ServiceNow Canada
Generating Query-Focused Summarization Datasets from Query-Free Summarization Datasets
May 6, 2026Yllias Chali, Deen AbdullahSynthetic Data Generation Text Summarization
University of Lethbridge Alberta, Canada

Jun 7, 2026J/K move · Enter open · S save

Segment-level Tree Search for Long Meeting Document Summarization

Sangwon Ryu, Heejin Do, Jun Seo, Daehui Kim+3

GSAI, POSTECH · ETH Zurich, ETH AI Center · Agentic AI Lab, KT+2

Monte Carlo Tree Search Text Summarization

Open paper AskPDF arXiv

Meeting documents are challenging to summarize due to their length and complex conversational structure. Existing approaches typically adopt multi-stage pipelines that extract information prior to summarization; however, these approaches often suffer from cumulative error propagation without intermediate validation, a limitation further amplified by short and low-quality reference summaries. We propose segment-level summarization via Monte Carlo Tree Search (S3), a training-free framework that constructs a final summary by composing segment-level summary candidates. S3 partitions a long document into segments and generates multiple summary candidates per segment, forming nodes of a search tree. The best-scoring combination is selected via self-reward-guided tree search and refined into the final output. Despite using a 7B model, S3 achieves performance comparable to larger 72B models while producing length-appropriate summaries.

Abstract

Text Summarization

Explore similar work

Segment-level Tree Search for Long Meeting Document Summarization

Preserving What Matters: Semantic Scaffolds Beyond Saturation in Summarization Evaluation

Generating Query-Focused Summarization Datasets from Query-Free Summarization Datasets