Revisiting What Makes a Good Context Subset: A Sparse Latent Basis Perspective
Organizations: Kahlert School of Computing, University of Utah, Salt Lake City, UT 84112, USA
Abstract
Selecting context subsets is increasingly important in modern LLM inference, yet what makes a subset effective remains unclear. We study this question in the controlled setting of few-shot demonstration selection. We examine a hypothesis that demonstrations should collectively provide task-relevant information needed by a target query. Assuming that the query representation provides a useful proxy for these requirements, we formulate selection as sparse reconstruction and instantiate it with Residual Explanation Search (RES), a Matching-Pursuit-style diagnostic procedure. Across seven benchmarks and eight open-weight LLMs, no retrieval objective is universally best. RES outperforms MMR in 34 of 56 model--dataset pairs, with its clearest advantage on sentiment tasks, suggesting that query-conditioned collective reconstruction may capture useful complementarity that pairwise dissimilarity misses. Relevance-based retrieval remains strong on template-heavy reasoning benchmarks, while all retrieval methods underperform zero-shot prediction on average on StrategyQA, where a solution-reconstruction analysis is consistent with the query not exposing the required external knowledge. Target-model hidden states further show that retrieval objectives interact differently with representation spaces: middle-layer representations generally favor similarity retrieval, whereas last-layer representations are often stronger for residual- and diversity-based selection. These findings support a conditional, representation-dependent account of context subset quality rather than a universally superior selector.
Figures & tables
| Method | AQUA | MathQA | QNLI | SST-2 | SST-5 | StrategyQA | SUBJ |
|---|---|---|---|---|---|---|---|
| random | 8/8 (100.0%) | 8/8 (100.0%) | 5/8 (62.5%) | 7/8 (87.5%) | 8/8 (100.0%) | 5/8 (62.5%) | 8/8 (100.0%) |
| bm25 | 1/8 (12.5%) | 0/8 (0.0%) | 2/8 (25.0%) | 5/8 (62.5%) | 5/8 (62.5%) | 4/8 (50.0%) | 2/8 (25.0%) |
| top- | 1/8 (12.5%) | 0/8 (0.0%) | 3/8 (37.5%) | 6/8 (75.0%) | 4/8 (50.0%) | 1/8 (12.5%) | 1/8 (12.5%) |
| mmr | 3/8 (37.5%) | 5/8 (62.5%) | 4/8 (50.0%) | 6/8 (75.0%) | 7/8 (87.5%) | 3/8 (37.5%) | 6/8 (75.0%) |
| task | AQUA | MathQA | QNLI | SST-2 | SST-5 | SUBJ | StrategyQA |
|---|---|---|---|---|---|---|---|
| method | |||||||
| bm25 | |||||||
| mmr | |||||||
| random | |||||||
| res | |||||||
| top- |
| Dataset | RES | Top- | MMR | Random |
|---|---|---|---|---|
| MathQA | ||||
| AQUA | ||||
| StrategyQA |
| Dataset | Target query | Comparison | Retrieved-context pattern |
|---|---|---|---|
| StrategyQA | Was ship that recovered Apollo 13 named after a World War II battle? | Top- vs. random | Top- retrieves space-related demonstrations involving Apollo missions , the moon , and lunar landing . Random retrieves unrelated facts. Neither provides the missing facts about the Apollo 13 recovery ship or its name origin. |
| SST-2 | A fairly by-the-books blend of action and romance with sprinklings of intentional and unintentional comedy. | RES vs. Top- /MMR | Top- focuses on romantic comedy examples, and MMR adds mostly topical or stylistic variation. RES retrieves more varied sentiment cues, such as odd , haphazard , serious suspense , and uniformly good . |
| MathQA | A man bought 20 shares of Rs. 50 at 5 discount, the rate of dividend being 13. The rate of interest obtained is: | RES vs. MMR | Both methods retrieve a near-template match involving shares , discount , dividend , and interest rate . The remaining demonstrations differ but are less clearly decisive. |
| AQUA | Let A and B be independent events with and . Find . | RES vs. MMR | MMR retrieves several probability -related demonstrations, while RES retrieves broader algebra, ratio, business, and logarithm examples. |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Precompute | Run Time | Space |
|---|---|---|---|
| Top- | - | ||
| MMR | - | ||
| MMR † | |||
| RES | - |
| Dataset | Prompt Template |
|---|---|
| SST-2 | Review: <review> |
| Sentiment: <positive|negative> | |
| SST-5 | Review: <review> |
| Sentiment: <terrible|bad|okay|good|great> | |
| SUBJ | Input: <text> |
| Type: <objective|subjective> |
| Dataset | Prompt template |
|---|---|
| StrategyQA | Exemplar: Question: <question> Answer: <answer> Reasoning: <facts> Query: Question: <question> Answer: |
| MathQA AQUA | Exemplar: Question: <question> Options: <options> Answer: <choice> Explanation: <rationale> Query: Question: <question> Options: <options> Answer: |
In-Context Example Selection
In-context example selection chooses demonstrations that help language and multimodal models perform a query-specific task without changing model weights.
See all 32 papers