ActICL: Post-Training for Agentic Multimodal In-Context Learning

Yanshu Li; Ruixiang Tang; Junyi Jessy Li; Elias Stengel-Eskin

cs.CVOct 14, 2026

ActICL: Post-Training for Agentic Multimodal In-Context Learning

Authors: Yanshu Li, Ruixiang Tang, Junyi Jessy Li, Elias Stengel-Eskin

Organizations: University of Texas at Austin · 2Rutgers University

Abstract

Multimodal in-context learning (ICL) is an important capability of multimodal large language models (MLLMs). Its performance is closely tied to the demonstrations it uses, making demonstration retrieval a core problem for ICL. Common methods use fixed scoring methods to retrieve all the demonstrations at once, which may introduce redundancy and noise. Context-aware retrieval methods have emerged to address this problem by allowing incremental retrieval from a growing context. However, none of them lets the MLLM learn how to judge what new demonstrations it needs and then carry out progressive retrieval on its own. To this end, we introduce ActICL, which trains an MLLM agent to acquire and use an ICL context via tool-augmented reasoning. An ActICL agent analyzes its own information gaps during iterative reasoning and tool-use actions. Based on these gaps, it performs targeted retrieval to acquire a set of initial candidates. It can then use the existing context to further filter the candidates and refine the context, or it can iteratively perform a new retrieval. When the agent judges that the context is sufficient for the task, or when the resource limit is reached with the user request met, it sends the acquired context to an answering model, either the agent itself or another MLLM. We use a two-stage post-training procedure that jointly optimizes the agent's acquisition and answering. With an adaptive number of demonstrations, ActICL improves average performance by 11.6 percentage points over standard similarity-based retrieval across seven benchmarks and three MLLMs, and by 4.4 points compared to the strongest prior baseline. The contexts acquired by ActICL agents can also be used effectively by frontier models. When GPT-5.6-Sol performs 4-shot ICL, ActICL's contexts improve its performance by 3.5 points over standard retrieval, averaged over three benchmarks.

Figures & tables

Figure 1: Given a multimodal query, we seek to construct a context with demonstrations that helps an MLLM answer the query correctly. Prior work has largely (a) used off-the-shelf similarity-based retrievers, which often return redundant demonstrations, or (b) trained context-aware retrievers, which can fail to generalize at test time. Our approach (c), ActICL, treats ICL context construction as an iterative agentic task, training an agent to formulate complex search queries, process and filter retrieved evidence, and decide when to submit a context to an answering model.

Figure 2: The overall illustration of ActICL. Given a query and acquisition instructions, the agent reasons at every step to choose among four actions: search submits a goal whose enabled channels retrieve previews from the demonstration bank, read reveals full demonstrations, and curate adds or removes read demonstrations in the retained context, until solve commits an admissible context. The agent learns on-policy which contexts make ICL effective and how to acquire them.

Methods	TextVQA	MME-R	CharXiv	GVL	MathVision	MDK12	VL-ICL	Avg.
Qwen3-VL-8B	0.735	0.238	0.473	0.554	0.528	0.462	0.293	0.469
+Rand. Sample	0.728	0.259	0.482	0.527	0.540	0.449	0.562	0.507
+MMICES	0.743	0.278	0.495	0.583	0.543	0.485	0.695	0.546
+ContextNav	0.764	0.295	0.527	0.605	0.558	0.510	0.761	0.574
+LSD	0.779	0.308	0.519	0.629	0.552	0.518	0.784	0.584
+ProMSA	0.819	0.326	0.546	0.684	0.575	0.561	0.873	0.626

Table 1: Comparison of ICL performance with baseline methods. Bold indicates the best accuracy, and underlining indicates the best accuracy excluding ActICL. The first row for each model reports zero-shot performance. All other results use four shots, except ActICL in the adaptive setting.

Method	MME-R		MathVision		MMStar		MuirBench	BLINK
Method	4-shot	8-shot	4-shot	8-shot	4-shot	8-shot	MuirBench	BLINK
Qwen3-VL-8B	0.278	0.316	0.543	0.557	0.746	0.753	0.653	0.702
+LoRA	0.295	0.324	0.536	0.569	0.762	0.784	0.709	0.735
+DARA	0.306	0.305	0.529	0.568	0.770	0.769	0.684	0.719
+ActICL	0.344	0.375	0.591	0.584	0.783	0.797	0.717	0.744
Qwen3-VL-30B-A3B	0.308	0.332	0.597	0.609	0.751	0.746	0.629	0.683

Table 2: Performance comparison with baseline methods on three multimodal ICL tasks and two multi-image understanding tasks. All few-shot prompts are constructed using MMICES.

Method	PerceptionBench	MindCube	MMMU-Pro
Qwen3.5-397B-A17B	0.458	0.477	0.757
+MMICES	0.472	0.492	0.761
+ContextNav	0.487	0.524	0.772
+ActICL	0.500	0.545	0.780
GPT-5.6-Luna (max)	0.559	0.638	0.785
+MMICES	0.568	0.634	0.802

Table 3: 4-shot performance of different retrievers with larger-scale open-source and frontier closed-source models. ContextNav uses Gemini3.5-Flash as its policy, while ActICL uses Qwen3.5-9B.

Figure 4: Multimodal retrieval ablation on Qwen3.5-9B under the adaptive setting.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Tier	Calls $H$	$B_{\mathrm{acq}}$	$B_{\mathrm{ctx}}$
Tight	14	98,304	8,192
Standard	20	196,608	16,384
Wide	28	393,216	32,768

Appendix

Table 5: Resource tiers.

H

bounds the number of acquisition calls,

B_{\mathrm{acq}}

bounds the cumulative input and generated tokens of all acquisition calls, and

B_{\mathrm{ctx}}

bounds the retained demonstrations.

	Query mixture			Pool
Source	RL (%)	RL	SFT	Bank
ReasonVQA	10	2,000	400	8,000
A-OKVQA	8	1,600	320	5,000
VCR (Q $\to$ A)	8	1,600	320	7,500
DocVQA	10	2,000	400	5,500
ChartQA	8	1,600	320	5,000

Appendix

Table 6: Training mixture and demonstration-pool targets. RL counts are instances, and SFT counts are accepted trajectory–answer pairs per backbone. Bank counts are distinct records and are allocated independently of the query mixture.

	Core routes (%)
Profile	Source	Similarity	Cross	Prob. (%)
Source-heavy	50	25	25	20
Mixed	25	50	25	50
Similarity-heavy	12.5	62.5	25	30

Appendix

Table 7: Sampling profiles for the 256-record core. Route percentages apply within the core, and the last column gives the probability of selecting each profile.

		Fixed count $n$
Tier	Adaptive	1	2	4	8	Total
Tight	10	2.5	4	2.5	1	20
Standard	25	1.5	7	11	5.5	50
Wide	15	1	4	6.5	3.5	30
Total	50	5	15	20	10	100

Appendix

Table 8: Joint distribution of count requests and resource tiers, as percentages of all training instances. A configuration is admitted only after a feasible acquisition path under its limits has been validated.

Setting	RS-SFT	Joint RL
Training volume and batching
Training volume	4,000 pairs, 2 epochs	20,000 instances, 1 pass
Global batch	16 pairs	32 groups per rollout batch
Optimizer minibatch	16 pairs	8 complete groups
Optimizer updates	500	2,500
Optimizer and numerics

Appendix

Table 9: Training settings per backbone. A joint-RL group contains all

K

acquisition trajectories of an instance and the answers sampled from its valid contexts.

Parameter	Symbol	Value
Acquisition trajectories per instance	$K$	4
Answers per valid context	$M$	2
Clipping width	$\epsilon$	0.2
Answer baseline mixture	$\mu$	0.1
Answer reference baseline	$b_{\mathrm{ref}}$	0.5
Fixed loss normalizer	$L_{\mathrm{ref}}$	8,192

Appendix

Table 10: Joint RL hyperparameters. The cost schedule depends on the fraction of instances sampled and is independent of the learning-rate schedule.

Benchmark	Available pool (bank / query)	Stratification
Released splits preserved
TextVQA	train / validation	native boundary
GVL	train / test	native boundary, per graph task
VL-ICL Bench	released image-to-text support / query construction	per subtask
New 6:4 partition
MME-Reasoning	public labeled release	reasoning category and subtype, difficulty where available

Appendix

Table 11: Evaluation pools and stratification. Benchmarks with released splits keep their native bank / query boundaries. The others receive a new partition that targets a 6:4 bank / query ratio within each stratum.

Method	MME-R	CharXiv	MathVision	Avg.
Base	0.343	0.675	0.728	0.582
SFT only	0.431	0.743	0.854	0.676
Joint RL only	0.423	0.748	0.850	0.674
w/o acquisition	0.402	0.740	0.838	0.660
w/o answering	0.415	0.745	0.847	0.669
ActICL	0.427	0.751	0.862	0.680

Appendix

Table 12: Ablation on training stages with Qwen3.5-9B under four-shot requests. “w/o acquisition” and “w/o answering” run joint RL after SFT without the corresponding part.

Variant	MME-R	CharXiv	MathVision	Avg.
w/o read	0.445	0.758	0.852	0.685
w/o curate	0.448	0.771	0.859	0.693
Add-only curate	0.457	0.782	0.862	0.700
ActICL	0.468	0.788	0.870	0.709

Appendix

Table 13: Ablation on read and curate with Qwen3.5-9B under the adaptive setting.

Variant	MME-R	CharXiv	MathVision	Avg.	#Shots	Acq. (k)
w/o resource costs	0.461	0.779	0.868	0.703	2.94	32.7
w/o cost schedule	0.459	0.764	0.857	0.693	2.78	30.2
w/o fixed reference	0.442	0.779	0.857	0.693	2.10	28.9
ActICL	0.468	0.788	0.870	0.709	2.62	26.3

Appendix

Table 14: Reward ablation with Qwen3.5-9B under the adaptive setting. #Shots is the mean committed count and Acq. is the mean number of acquisition tokens, both averaged over the three benchmarks.

Training bank	MME-R	CharXiv	MathVision	Avg.
Own bank
Source-only	0.459	0.767	0.862	0.696
Random	0.448	0.759	0.858	0.688
ActICL	0.468	0.788	0.870	0.709
Mixed bank
Source-only	0.422	0.743	0.844	0.670

Appendix

Table 15: Ablation on training-bank construction with Qwen3.5-9B under the adaptive setting. The top block evaluates each benchmark on its own support bank, and the bottom block on a mixed bank that merges the support banks of the three benchmarks.

Multimodal ICL

Multimodal in-context learning (ICL) studies how models use examples and other context to solve tasks involving images, text, audio, and video.

See all 46 papers

Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context Learning
Sep 9, 2026Mingbo Yang, Wenqiang Wang, Zhaolu Kang +4Contrastive Learning Multimodal ICL
Sun Yat-Sen University Shenzhen, Guangdong, China · Peking University Beijing, Beijing, China · Sun Yat-Sen University Shenzhen, Guangdong, China Pengcheng Laboratory +1
Hyper-ICL: Attention Calibration with Hyperbolic Anchor Distillation for Multimodal In-Context Learning
Jun 3, 2026Niloufar Alipour Talemi, Hossein Kashiani, Fatemeh AfghahEfficient Multimodal Inference Multimodal ICL
Holcombe Department of Electrical and Computer Engineering, Clemson University, Clemson, United States.
TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals
Sep 14, 2026Zihan Xue, Po-Yi Lu, Serhii Honcharenko +5Multimodal ICL Modality Gap in VLMs
University of California, Los Angeles · 2National Taiwan University · 3Texas A&M University +2

Sep 9, 2026J/K move · Enter open · S save

Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context Learning

Mingbo Yang, Wenqiang Wang, Zhaolu Kang, Peng Chen+3

Sun Yat-Sen University Shenzhen, Guangdong, China · Peking University Beijing, Beijing, China · Sun Yat-Sen University Shenzhen, Guangdong, China Pengcheng Laboratory+1

Contrastive Learning Multimodal ICL Multimodal Reasoning Retrieval-Augmented ICL

Open paper AskPDF arXiv

In-context learning (ICL) is widely used in multimodal large language models (MLLMs) and achieves strong performance across a wide range of multimodal tasks. However, existing multimodal ICL methods often rely on surface level imitation of in-context demonstrations, making it difficult for MLLMs to align their responses with the reasoning path required by the given multimodal input. This limitation becomes more pronounced in complex multimodal tasks, thereby restricting further improvements in MLLM performance. To address this issue, we propose a new multimodal ICL framework that combines contrastive demonstration modeling with the self-refinement capability of MLLMs. Specifically, our framework reformulates each demonstration by explicitly contrasting a suboptimal response with a better response under the same input, together with a reasoning path that reveals how the response should be refined. This contrastive formulation makes the reasoning path toward the desired response more explicit and guides the MLLM beyond superficial imitation. Furthermore, because effective refinement depends on the current response, we introduce a response-conditioned retrieval mechanism to select demonstrations whose reasoning paths are more relevant to the current response. In addition, we use a lightweight alignment controller to predict response quality and determine whether further refinement is needed. Experiments on three types of multimodal tasks show that the proposed framework consistently improves MLLM performance, with particularly notable gains on visual question answering (VQA).

Abstract

Multimodal ICL

Explore similar work

Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context Learning

Hyper-ICL: Attention Calibration with Hyperbolic Anchor Distillation for Multimodal In-Context Learning

TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals