cs.CVNov 10, 2026

TAMEing the Open-World Personalization: Towards Open-Set Personalized MLLM Assistant

Authors: Rongpei Hong, Jian Lang, Ting Zhong, Fan Zhou

Organizations: University of Electronic Science and Technology of China · Intelligent Digital Media Technology Key Laboratory of Sichuan Province

Abstract

Personalizing Multimodal Large Language Models (MLLMs) as intelligent assistants has recently emerged as an active research topic, where an MLLM is expected to go beyond generic, one-size-fits-all replies, and instead generate responses grounded in user-specific objects, entities, and concepts encountered during interactions. Nevertheless, existing studies mainly focus on personalizing MLLMs on a static, predefined set of personalized concepts and require users to manually and frequently extend this set to accommodate newly emerging concepts, which is infeasible in dynamic-world scenarios. To address this limitation, we recast conventional MLLM personalization as an Open-Set problem, and propose TAME-O, the first long-context open-set personalized MLLM assistant. Specifically, TAME-O introduces a training-free, plug-and-play Concept Reconciliation Skill (CR Skill). It enables the assistant to automatically unlock novel concepts during prolonged human-MLLM interactions while preventing confusion with previously unlocked ones, allowing the MLLM to continually grasp more and more concepts and progressively refine the modeling of known ones. Extensive experiments under the open-set personalization setting underscore the efficacy of our method.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. PersonaVLM: Long-Term Personalized Multimodal LLMs

    Mar 20, 2026Chang Nie, Chaoyou Fu, Yifan Zhang +2Personalized Language ModelsMultimodal Agents

  2. Personalizing MLLMs via Reinforced Multimodal Reference Game

    Jun 27, 2026Deepayan Das, Davide Talon, Yiming Wang +2Multimodal Large Language ModelsPersonalized Language Models

  3. POPI: Personalizing LLMs via Optimized Natural Language Preference Inference

    Oct 17, 2025Yizhuo Chen, Xin Liu, Ruijie Wang +7LLM PersonalizationPersonalized Language Models