CB-Agent: Concept-Based In-Context Learning for Sequential Action Selection

Qingyue Zhang; Huaze Tang; Minyu Zhang; Yuxiang Yan; Wenbo Ding; Shao-Lun Huang

cs.AIOct 10, 2026

CB-Agent: Concept-Based In-Context Learning for Sequential Action Selection

Authors: Qingyue Zhang, Huaze Tang, Minyu Zhang, Yuxiang Yan, Wenbo Ding, Shao-Lun Huang

Organizations: Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China

Abstract

In-context learning (ICL) allows large language models to use demonstrations without updating their parameters. For interactive agents, demonstrations are trajectories of reasoning, actions, and observations that illustrate how to complete a task. Reusing demonstrations in new situations is challenging: directly inputting full trajectories consumes limited prompt space and constrains the overall effectiveness, while natural-language summaries require the agent to translate general guidance into specific action choices. Therefore, we study whether expert action choices can directly supervise a reusable scoring rule. We propose CB-Agent, which extends concept-based ICL to sequential action selection. We extract environment-action decisions from expert trajectories and fit a concept vector by maximum likelihood to model the conditional distribution over candidate actions given the context. This model combines the concept with frozen language-model features and priors. During ReAct interaction, the agent uses this distribution to select environment actions without including demonstrations in the action-scoring context. Across TextWorld, ALFWorld, and ScienceWorld, CB-Agent achieves the highest cross-benchmark average among the evaluated methods at all three tested model sizes and ranks first on TextWorld and ScienceWorld at each size. ALFWorld experiments support the learned concept's contribution beyond prior-only scoring and show further gains when combined with Reflexion. Code is available at https://anonymous.4open.science/r/CB-Agent-DFF9/.

Figures & tables

Figure 1: Overview of CB-Agent. (a) Only environment-action steps in expert trajectories provide decision examples comprising a pre-action context, candidate actions, and the expert action. Preceding thoughts remain in the context. (b) A frozen language model supplies action features and length-adjusted prior weights; a shared concept is estimated by regularized maximum likelihood. (c) A ReAct response determines whether to think or act. A Think response is recorded in the interaction history. On the action branch, the agent constructs candidate actions and selects one using the estimated concept and prior weights. The interaction history is updated after either branch, while the concept and language model remain fixed.

Model	Method	TextWorld	ALFWorld	ScienceWorld	Avg.
		Fixed action space		Open action space
Qwen3-4B	ReAct	61.8	76.9	53.9	64.2
	Act-only	52.0	54.5	48.6	51.7
	Self-Consistency	57.4	69.4	35.4	54.1
	StateAct	43.3	79.1	28.7	50.4
	ExpeL	37.0	67.2	58.7	54.3

Table 1: Results across three benchmarks and model sizes. All metrics use a 0–100 scale: normalized scores for TextWorld and ScienceWorld, and success rate for ALFWorld. Avg. is the unweighted mean across the three benchmarks. Bold marks the best result within each model block.

Panel of Figure 2 — Figure 2: t-SNE visualizations of concept vectors on ALFWorld for (a) Qwen3-4B, (b) Qwen3-8B, and (c) Qwen3-14B. Each point represents a concept estimated from five randomly sampled expert trajectories of a single task type, with colors denoting the six task types. Concepts from the same task type tend to cluster across all three backbones. Each panel is projected independently.

Table 4

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
Prediction and interaction
$x$ , $y$ , $\mathcal{Y}$	Input text, candidate label, and finite nonempty label set in CB-ICL.
$Q_{0}(y)$	Positive base weight for label $y$ in the CB-ICL model.
$\mathrm{inst}$	Task instruction specifying the goal of an interaction.
$\tau$ , $T$	Complete interaction trajectory and its number of steps, including both environment and Think actions.
$t$ , $a_{t}$ , $o_{t+1}$	Interaction-step index, environment or Think action, and recorded feedback.

Appendix

Table 4: Principal notation for CB-ICL and CB-Agent.

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 681 papers

ActICL: Post-Training for Agentic Multimodal In-Context Learning
Oct 14, 2026Yanshu Li, Ruixiang Tang, Junyi Jessy Li +1Multimodal ICL Retrieval-Augmented ICL
University of Texas at Austin · 2Rutgers University
Downstream-Aware Context Selection for Online In-Context Reinforcement Learning
Sep 27, 2026Ruihan A. Li, Shangtong Zhang, Rohan ChandraIn-Context RL In-Context Learning
University of Illinois Urbana-Champaign · University of Virginia
Do LLMs Learn from Rewards in Context? : Rethinking the role of reward in In-Context RL
Oct 8, 2026Minchan Kwon, Seunghee Koh, Sunghyun Baek +2In-Context RL Contextual RL
Korea Advanced Institute of Science and Technology (KAIST)

Oct 14, 2026J/K move · Enter open · S save

ActICL: Post-Training for Agentic Multimodal In-Context Learning

Yanshu Li, Ruixiang Tang, Junyi Jessy Li, Elias Stengel-Eskin

University of Texas at Austin · 2Rutgers University

Multimodal ICL Retrieval-Augmented ICL Multimodal Large Language Models

Open paper AskPDF arXiv

Multimodal in-context learning (ICL) is an important capability of multimodal large language models (MLLMs). Its performance is closely tied to the demonstrations it uses, making demonstration retrieval a core problem for ICL. Common methods use fixed scoring methods to retrieve all the demonstrations at once, which may introduce redundancy and noise. Context-aware retrieval methods have emerged to address this problem by allowing incremental retrieval from a growing context. However, none of them lets the MLLM learn how to judge what new demonstrations it needs and then carry out progressive retrieval on its own. To this end, we introduce ActICL, which trains an MLLM agent to acquire and use an ICL context via tool-augmented reasoning. An ActICL agent analyzes its own information gaps during iterative reasoning and tool-use actions. Based on these gaps, it performs targeted retrieval to acquire a set of initial candidates. It can then use the existing context to further filter the candidates and refine the context, or it can iteratively perform a new retrieval. When the agent judges that the context is sufficient for the task, or when the resource limit is reached with the user request met, it sends the acquired context to an answering model, either the agent itself or another MLLM. We use a two-stage post-training procedure that jointly optimizes the agent's acquisition and answering. With an adaptive number of demonstrations, ActICL improves average performance by 11.6 percentage points over standard similarity-based retrieval across seven benchmarks and three MLLMs, and by 4.4 points compared to the strongest prior baseline. The contexts acquired by ActICL agents can also be used effectively by frontier models. When GPT-5.6-Sol performs 4-shot ICL, ActICL's contexts improve its performance by 3.5 points over standard retrieval, averaged over three benchmarks.

Abstract

LLM Agents

Explore similar work

ActICL: Post-Training for Agentic Multimodal In-Context Learning

Downstream-Aware Context Selection for Online In-Context Reinforcement Learning

Do LLMs Learn from Rewards in Context? : Rethinking the role of reward in In-Context RL