cs.AINov 18, 2026

Continual Tool-Space Extension Learning for Multimodal Agents

Authors: Jinbae Seo, Jungin Park, Hyunsung Jang, Namkoo Ha, Jiyoung Lee, Kwanghoon Sohn

Organizations: Yonsei University · Daegu Gyeongbuk Institute of Science and Technology (DGIST) · LIG Defense & Aerospace · School of AI and Software, Ewha Womans University · 

Abstract

Multimodal agents largely rely on generic tools such as cropping, search, and code execution, leaving specialist vision models outside their tool space. However, bringing such a model into an already trained agent is not straightforward. The challenge is to learn each new tool efficiently while retaining the ability to use existing ones. This work proposes COTEX, continual tool-space extension learning, that adds new tools at efficient adapter-level cost. COTEX locates two sets of parameter directions; one triggers a tool call and thereby opens a multi-turn reasoning trajectory, while the other determines which tool is chosen. The adapter of a new tool is restricted to the calling directions that leave earlier tool choices unaffected by orthogonal constraint. We further introduce a protocol that converts specialist models' existing task-specific annotations into verified tool-use trajectories. Experiments on two representative agents and three specialist tools show gains on ten vision-language benchmarks, with at most a 0.3-point drop on existing tools.

Figures & tables

Explore similar work

CardsList
  1. MetaForge: A Self-Evolving Multimodal Agent that Retrieves, Adapts, and Forges Tools On Demand

    Jun 1, 2026Shouang Wei, Houcheng Min, Xinpeng Dong +8LLM Agent Skill LearningMultimodal Agents

  2. With Tools or Because of Tools? Tracing Capability Gains in Multimodal Agents

    Jun 1, 2026Jiawei Guo, Donglei Yu, Yu Chen +6Tool-Use EvaluationLLM Agent Evaluation