Retrieval-augmented systems often retrieve passages that are accurate and relevant but state the answer at a coarser level of granularity than the question requires. For example, a question may ask which specific disease a drug treats (familial hyperlipidemia), while the retrieved passage mentions the broader disease family (hyperlipidemia). Prior work on context sufficiency treats such coarse passages simply as insufficient. We argue that they form a distinct case, which we call resolvable insufficiency: text alone cannot settle the fine-grained answer, but the broader category it mentions maps to a short list of candidate answers in a domain ontology. When language models read these coarse, family-only passages, they exhibit a phenomenon we call Clade (Category-Level Answering from Deficient Evidence): they answer instead of abstaining or searching further. On biomedical questions from PrimeKG, family-only passages reduce abstention by 13.8 points compared to uninformative passages, and the resulting guesses are wrong about half the time. A counterfactual test confirms the cause: replacing an exact disease name with its broader family name lowers abstention by 17.4 points compared to deleting the sentence. Simple fixes like confidence filtering, scaling to a larger model, and explicitly instructing the model to abstain cannot resolve Clade, and this replicates on ecological species interactions in GloBI. Finally, we test whether an ontology can turn coarse passages into exact answers. Given the correct family, an ontology list includes the true answer in 98.5% of cases, but string matching finds that family on the page for only 19.8%. Letting the model choose the family raises this to 30.0% and end-to-end recovery from 10.0% to 26.8%. Overall, our results show when language models should stop guessing and consult an ontology, and highlight which steps of that process remain unsolved.
Figures & tables
Figure 1: Resolvable insufficiency as an agent workflow, with the bottleneck we measure at each stage. The retrieved passage is accurate and relevant but names only the answer’s family. A page-type check should send it to an ontology, which expands the family into a short candidate list for a chooser. The dashed red path is Clade : the model guesses from the family-only passage instead. In current practice, only the ontology step works reliably.
PrimeKG
GloBI
Source records
8,100,498
7,822,670
Edges
indication 9,388
eats 235,960
contraind. 30,675
preys on 87,678
off-label 2,568
parasite of 117,894
pollinates 15,854
Hierarchy
MONDO
GloBI taxonomy
Table 1: Data for both domains. PrimeKG links drugs to diseases, and MONDO groups diseases into families. GloBI links species to species, and its taxonomy groups species into genera. A family is a parent with two or more children. Facts measured are edges we check for a mention of the answer.
Text names it
Text does not
Relation
n
knows
not
knows
not
Indication
600
9.7
7.2
21.8
61.3
Contraind.
600
1.5
5.2
20.7
72.7
Off-label
600
4.3
6.2
24.2
65.3
All
1,800
5.2
6.2
22.2
66.4
Table 2: Share of PrimeKG facts (%) by whether the drug’s Wikipedia article names the disease and whether Qwen2.5 0.5B picks it from five options closed book (“knows”).
Relation
Text
n
Named
Family
Silent
Indication
Wikipedia
1,722
20.3
15.5
64.2
PubMed
1,722
26.3
18.6
55.1
both
1,722
29.0
19.7
51.3
Contraind.
Wikipedia
1,722
9.8
4.1
86.1
PubMed
1,722
11.4
5.4
83.2
both
1,722
15.9
6.9
77.2
Table 3: Share of knowledge-graph facts whose text names the answer, names only its family, or names neither. PrimeKG uses 574 drugs with three facts of each relation.
Model
Passage Type
Correct
Wrong
Declines
Prec.
Qwen2.5 7B
named
71.7
3.8
24.5
95.0
family-only
26.5
18.1
55.3
59.4
silent
13.8
2.5
83.8
84.6
OLMo 2 7B
named
77.4
15.1
7.5
83.7
family-only
37.2
44.7
18.1
45.5
silent
30.6
24.4
45.0
55.7
Table 4: Open-book outcomes (%) on PrimeKG questions where all four distractors come from the answer’s MONDO neighborhood (106 named, 215 family-only, and 160 silent questions). Precision is the percentage of correct answers among questions where the model did not abstain.
Figure 2: Effect of a family-only page relative to a silent page. Each point is Δ from Eq. 1 for one outcome, with its 95% bootstrap CI; points left of zero mean the family-only page lowers that outcome. The pooled row averages the three 7-8B models over 481 questions. Faded rows are models outside the pooled set.
Figure 3: Page edits on the same questions: PrimeKG (130) and GloBI (255). The family edit replaces the answer with its parent; the removed edit deletes the sentence. Bars show outcomes with the page (%), averaged over the 7-8B models.
Figure 4: Three candidate fixes. (a) Confidence separates correct from wrong answers poorly on family-only pages. (b) An instruction to decline on broad categories raises abstention on every page type, including named pages (pooled 7-8B models, 95% CIs). (c) The drop in abstention is largest when the family has 20 or fewer members.
Metric
PrimeKG
GloBI
Questions in pooled set (all-neighbor)
481
833
Abstention page effect, family − silent
−13.8
−14.3
Wrong-answer page effect, family − silent
+7.7
+11.6
Precision on named passages (%)
84–95
84–95
Precision on family-only passages (%)
45–59
22–28
Confidence AUROC on named passages
.73–.87
.82–.86
Table 5: Comparison of biomedical drug–disease facts (PrimeKG with MONDO) and ecological species interactions (GloBI with biological taxonomy) across the three 7–8B models. Effects are reported in percentage points; ranges span individual models. Every difference reported has a 95% CI excluding zero.
Figure 5: Accuracy of choosing the exact disease for 410 family-only facts. Blue and gray bars choose from the given-family list; hatched bars do not receive the true family.
Metric
Given
Page-built
Model-chosen
Usable lists (of 410)
410
166
375
Holds sampled answer
98.5%
19.8%
30.0%
Holds any true answer
99.5%
24.6%
44.9%
Median list size
14
32
12
End-to-end recovery
–
10.0%
26.8%
Table 6: Candidate lists for the 410 family-only facts: given family (oracle), all families on the page (page-built), and the one family Qwen2.5-7B picks from the page (model-chosen). Of the 244 unusable page-built lists, 209 exceed 64-candidate cap and 35 match no multi-child MONDO family. End-to-end recovery uses the best chooser for each list.
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Our four experiments. (1) We sort each page as named, family-only or silent, and ask each question closed book and with the page. (2) We edit named pages: the answer becomes its family name, or we delete the sentence. (3) We test three fixes. (4) We try to recover the exact answer from the ontology.
Main paper
Supplement
Data, page types, Tables 1 and 3
Appendix B
Odds ratios, CIs, p -values, Eq. 1
Appendix C
Questions, prompts, Table 4 , Figure 2
Appendix D
Page edits, Figure 3
Appendix E
Confidence, scale, instruction, Figure 4
Appendix F
GloBI, Table 5
Appendix G
Appendix
Table 7: Where each part of the main paper is explained in this supplement.
True disease
Unrelated diseases
Relation
named
+ family
named
+ family
Indication
19.3
34.7
1.4
2.7
Contraindication
6.7
9.3
0.9
1.8
Off-label use
11.8
23.7
0.9
3.3
Appendix
Table 8: Share of facts (%) whose Wikipedia text matches the true disease or one of three unrelated diseases from the same relation (600 facts per relation; 1800 unrelated draws each). “+ family” also counts family names.
Relation
Named
Family only
Silent
Indication
84.2 (101)
44.7 (94)
44.0 (405)
Contraind.
32.5 (40)
18.8 (16)
23.4 (544)
Off-label
57.1 (63)
37.5 (72)
31.6 (465)
All
65.7 (204)
39.6 (182)
32.0 (1,414)
Appendix
Table 9: Closed-book accuracy of Qwen2.5-7B by what the drug’s Wikipedia page says about the answer. Chance is 20%.
Page type
Relation
n
All-neighbor (%)
Named
both
300
35.3
Family-only
contraindication
106
83.0
Family-only
indication
150
84.7
Silent
contraindication
150
45.3
Silent
indication
150
61.3
Appendix
Table 10: Questions per group and the share whose four wrong options are all siblings or cousins of the answer in MONDO.
Model
Group
Correct
Wrong
Declines
Qwen2.5 7B
named
60.4
17.9
21.7
family-only
35.3
32.6
32.1
silent
25.0
15.6
59.4
OLMo 2 7B
named
65.1
31.1
3.8
family-only
36.7
56.7
6.5
silent
37.5
51.2
11.2
Appendix
Table 11: Closed-book outcomes (%) on the 481 neighborhood-matched questions (106 named, 215 family-only, and 160 silent) when no passage is provided.
Model
Passage
Corr.
Wrong
Decl.
Prec.
Conf.
Qwen2.5 7B
named
64.7
3.3
32.0
95.1
0.99
family-only
29.3
17.6
53.1
62.5
1.00
silent
13.0
2.7
84.3
83.0
0.99
OLMo 2 7B
named
71.3
20.7
8.0
77.5
0.80
family-only
39.1
42.6
18.4
47.8
0.84
silent
32.0
28.0
40.0
53.3
0.76
Appendix
Table 12: Open-book outcomes across all 856 PrimeKG questions (%; 300 named, 256 family-only, and 300 silent). Precision is the share of correct answers among non-abstained questions; Conf. is the median probability assigned to wrong answers.
Declines (Abstention)
Correct Answers
Wrong Answers
Model
Fam.
Sil.
Diff. [95% CI], p
Fam.
Sil.
Diff. [95% CI], p
Fam.
Sil.
Diff. [95% CI], p
Qwen2.5 7B
+23.3
+24.4
−1.1 [ −10.5 , 8.4], .82
−8.8
−11.2
+2.4 [ −5.7 , 10.5], .57
−14.4
−13.1
−1.3 [ −9.4 , 7.1], .76
OLMo 2 7B
+11.6
+33.8
−22.1 [ −31.8 , −12.6 ], 6e-6
+0.5
−6.9
+7.3 [ −2.6 , 17.0], .15
−12.1
−26.9
+14.8 [3.6, 26.1], .009
Granite 3.1 8B
+9.3
+27.5
−18.2 [ −28.5 , −8.1 ], .0006
−2.8
−11.2
+8.5 [ −0.9 , 17.8], .064
−6.5
−16.2
+9.7 [ −0.1 , 19.9], .056
Qwen2.5 0.5B
+9.8
+18.1
−8.4 [ −15.7 , −1.1 ], .027
+4.2
0.0
+4.2 [ −6.1 , 14.4], .42
−14.0
−18.1
+4.2 [ −6.4 , 14.6], .44
Pooled 7–8B
+14.7
+28.5
−13.8 [ −20.3 , −7.5 ], 4e-5
−3.7
−9.8
+6.1 [ −0.1 , 12.4], .058
−11.0
−18.8
+7.7 [1.2, 14.0], .019
Appendix
Table 13: Within-question page effects (in percentage points): the change in each outcome when adding a passage relative to closed-book answering on family-only (Fam., n=215 ) and silent (Sil., n=160 ) neighborhood-matched questions, along with the difference-in-differences estimate.
Indication
Contraindication
Model
Family-only
Silent
Family-only
Silent
Qwen2.5 7B
26.0
2.2
6.8
2.9
OLMo 2 7B
38.6
21.7
53.4
27.9
Granite 3.1 8B
37.8
16.3
10.2
1.5
Qwen2.5 0.5B
46.5
46.7
69.3
55.9
Appendix
Table 14: Open-book wrong-answer rates (%) broken down by clinical relation on neighborhood-matched questions (indication: 127 family-only, 92 silent; contraindication: 88 family-only, 68 silent).
Family size
n
Median
Decline effect [95% CI]
p
≤10
71
7
−17.3 [ −26.2 , −8.2 ]
.001
11–20
69
16
−18.4 [ −26.6 , −10.3 ]
< .001
>20
68
44
−6.5 [ −15.7 , +3.0 ]
.174
Appendix
Table 15: Decline effect ( Δ for declines, points) by the number of members in the named family, pooled over the three 7–8B models.
Model
Version
Decl.
Corr.
Wrong
Prec.
PrimeKG (130 questions)
Qwen2.5-7B
original
27.7
66.9
5.4
92.6
family
43.8
46.2
10.0
82.2
removed
70.0
26.9
3.1
89.7
OLMo 2 7B
original
6.2
74.6
19.2
79.5
family
7.7
58.5
33.8
63.3
Appendix
Table 16: Outcomes (%) with each version of the page. In every model and domain, the family version draws fewer declines than the removed version.
Outcome
Family − removed
Family − original
PrimeKG
Declines
−17.4 [ −23.6 , −11.5 ]
+11.5 [ +7.4 , +15.9 ]
Wrong
+5.1 [ +0.5 , +10.3 ]
+9.5 [ +5.9 , +13.6 ]
Correct
+12.3 [ +6.9 , +17.9 ]
−21.0 [ −26.9 , −15.6 ]
GloBI
Declines
−27.7 [ −32.2 , −23.5 ]
+19.7 [ +15.4 , +23.9 ]
Wrong
+6.7 [ +3.4 , +10.2 ]
+11.1 [ +8.2 , +14.1 ]
Correct
+21.0 [ +16.9 , +25.4 ]
−30.8 [ −36.2 , −25.5 ]
Appendix
Table 17: Pooled edit effects over the three 7–8B models (points, 95% bootstrap CI).
Model
Page
Ans.
Prec.
Conf. R
Conf. W
AUROC
PrimeKG
Qwen2.5-0.5B
named
97
84.5
.83
.49
.85
family
194
38.1
.71
.60
.64
Qwen2.5-7B
named
80
95.0
.98
1.00
.73
family
96
59.4
.98
.90
.72
OLMo 2 7B
named
98
83.7
.97
.75
.86
Appendix
Table 18: Confidence of answers with the page. Ans. is the number of questions answered (not declined); Conf. R and Conf. W are the mean confidence of right and wrong answers.
Declines
Precision
Model
Page
base
instr.
base
instr.
Qwen2.5-7B
named
24.5
29.2
95.0
94.7
family
55.4
64.2
59.4
64.9
silent
83.8
88.1
84.6
89.5
OLMo 2 7B
named
7.5
27.4
83.7
89.6
family
18.1
56.3
45.5
47.9
Appendix
Table 19: Declines and precision (%) with the page, without (base) and with (instr.) the instruction to decline on a broad category.
Page type
eats
parasite of
pollinates
preys on
Named
93.3
94.7
94.7
89.3
Family-only
100.0
98.7
100.0
93.3
Silent
90.7
85.3
92.0
78.7
Appendix
Table 20: Share of GloBI questions (%, 75 per cell) whose four wrong options are all from the partner’s genus or family.
Correct
Wrong
Declines
Model
Page
cl.
page
cl.
page
cl.
page
Qwen2.5-7B
named
7.9
56.6
4.3
3.2
87.8
40.1
family
1.7
1.7
2.7
4.4
95.6
93.9
silent
3.5
0.4
1.2
1.2
95.4
98.5
OLMo 2 7B
named
20.8
69.9
24.7
12.9
54.5
17.2
family
10.9
12.6
22.8
34.4
66.3
53.1
Appendix
Table 21: GloBI outcomes (%) closed book (cl.) and with the page, on questions whose four wrong options all come from the neighborhood (279 named, 294 family-only, 260 silent).
Table 23: Page-built lists from Wikipedia alone and from Wikipedia plus PubMed, over all 410 family-only facts. The remaining unusable lists match no family with two or more children.
Signal
Picks
Ties
None
Acc./pick
Acc./all
Wikipedia lookup
82
80
248
50.0
10.0
PubMed co-mention
324
12
74
45.4
35.9
Combined evidence
327
14
69
44.6
35.6
Popularity prior
409
0
1
46.9
46.8
Normalized PubMed
336
0
74
31.0
25.4
Appendix
Table 24: Signals inside the given-family list (410 facts, 8448 candidates). Accuracy (%) counts a pick as correct if it matches any true disease of the drug.
Condition
Sampled
Any true
Model, given-family list
18.3
35.9
Model + page, given-family list
17.8
32.4
Model, random list
30.5
34.9
Model + page, random list
29.0
32.7
Appendix
Table 25: Qwen2.5-7B recovery accuracy (%) on the 410 family-only facts, scoring only the sampled answer (Sampled) or any true answer of the drug (Any true).
Drug
PrimeKG Target
Evidence Pick
Match?
Lovastatin
familial hyperlipidemia
familial hyperlipidemia
yes
Tetrabenazine
juvenile Huntington disease
juvenile Huntington disease
yes
Moxifloxacin
Klebsiella pneumonia
Legionnaires’ disease
no
Carboplatin
malignant germ cell tumor of ovary
ovarian carcinoma
no
Appendix
Table 26: Qualitative examples of candidate diseases chosen by combined literature evidence within the given-family list on indication facts.
Rule
Sampled answer
Any true answer
Qwen2.5-7B picks the family
30.0
44.9
First mentioned
25.1
37.3
Random
13.9
22.0
Smallest family
7.8
13.2
Page-built list (all families)
19.8
24.6
Any shown family (ceiling)
50.0
64.6
Appendix
Table 27: Share of the 410 family-only facts (%) whose chosen list holds the sampled answer, or any true answer of the drug. At most ten families are shown per page.
Model
Page
Corr.
Abst.
Family
Sibl.
Other
Qwen2.5-7B
named
20.0
47.5
0.0
0.0
32.5
family
0.0
42.5
17.5
0.0
40.0
silent
0.0
52.5
0.0
0.0
47.5
OLMo 2 7B
named
25.0
37.5
0.0
0.0
37.5
family
0.0
32.5
20.0
2.5
45.0
silent
0.0
52.5
0.0
0.0
47.5
Appendix
Table 28: Free-text outcomes with the page (%, 40 questions per row).