Organizations: Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · ByteDance · Shanghai Jiao Tong University · The Chinese University of Hong Kong, Shenzhen
Unified multimodal models (UMMs) integrate image generation and visual understanding, but are existing single-pathway unlearning methods sufficient to suppress target concepts across both? In this work, we study cross-pathway concept unlearning. Our goal is to prevent a model from both generating images of a target concept and identifying it in input images, while preserving other capabilities. To this end, we introduce UUBench, a diagnostic benchmark covering 30 target concepts and 60 semantically related neighboring concepts. Experiments on representative UMMs show that the evaluated single-pathway methods achieve limited joint forgetting: generation-side editors strongly suppress target generation but yield limited forgetting in visual understanding, while understanding-side baselines struggle to achieve substantial forgetting even in their own pathway. Their direct composition also yields limited joint forgetting. We therefore propose UniForget, achieving joint forgetting rates of 80.0% on BAGEL-7B-MoT and 92.7% on Janus-Pro-7B, outperforming representative baselines. Further analysis reveals trade-offs between joint forgetting and capability preservation.
Figures & tables
Figure 1: Illustration of cross-pathway concept unlearning in UMMs. Using the Eiffel Tower as an example, the goal is to suppress target identification in visual understanding and target generation in image synthesis, while preserving non-identity visual information and non-target capabilities.
Figure 2: Overview of UUBench . The benchmark covers 30 target concepts across characters, landmarks, and objects, together with 60 neighboring concepts and 720 reference images in total. We evaluate forgetting, capability preservation, and robustness to perturbations in both generation and visual understanding. Unified Forget and Unified Strict Forget measure joint success across pathways.
Metric
Full name
Evaluation scope and criteria
Image generation
TF
Target Forget
Target identity is absent in the generated image.
SF
Strict Forget
TF holds, with a coherent image and preserved quality.
NR
Neighbor Retention
Neighbor identity is present in the generated image.
SR
Strict Neighbor Retention
NR holds, with a coherent image and preserved quality.
Visual understanding
Table 1: Core evaluation metrics in UUBench . Metrics are percentages over their respective eligible sets; predicate and label rules are provided in Appendix A.2 .
Setting
Method
Generation
Understanding
Joint
TF
SF
NR
SR
DF
VF
TU
NU
Forget
S. Forget
BAGEL-7B-MoT
Base
0.0
0.0
100.0
100.0
0.0
21.6
96.7
83.0
0.0
0.0
Gene.
UCE
83.3
83.3
100.0
100.0
4.0
52.7
79.1
75.9
4.0
3.2
TIME
100.0
100.0
94.2
94.2
24.0
77.3
92.4
91.9
24.0
22.2
Und.
GD
0.0
0.0
100.0
100.0
0.7
3.3
90.7
82.9
0.0
0.0
Table 2: Main results on UUBench (%). Higher is better. Best results for each backbone are bolded, excluding Base and including ties.
Figure 3: Composition sweep of direct forgetting (DF), target utility (TU), and Collapse. Vertical lines mark the reference operating points. Higher DF and TU are better; lower Collapse is better.
Figure 4: Multi-concept scaling with generation edits (left two panels, namely T2I) and understanding edits (right two panels, namely I2T). Curves connect recorded measurements. TF/DF are absolute rates; NR/TU are normalized to N=1 (100%).
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Method
Generation
Understanding
GenEval
DPG-Bench
MMBench
MME-P (%)
MME-C (%)
BAGEL-7B-MoT
Base
72.7
83.4
88.4
83.3
85.5
Gene.
UCE
66.5
82.6
87.9
83.7
81.9
TIME
69.8
81.9
88.0
83.6
82.3
Und.
GD
64.9
82.9
88.4
83.0
85.5
Appendix
Table 3: General capabilities after unlearning (%). MME-P (%) and MME-C (%) normalize scores out of 2000 and 800, respectively. Higher is better; displayed per-backbone maxima among non-Base methods are bolded, including ties.
Figure 5: Generation results for Colosseum and its neighbor before and after unlearning.
Base
UniForget
BAGEL-7B-MoT — Specific identity Question: Identify the main landmark or structure in this image as specifically as you can.
The image features the Colosseum, an iconic ancient amphitheater located in Rome, Italy. This massive structure, built during the Roman Empire, is renowned for its historical significance and architectural grandeur. The Colosseum, also known as the Flavian Amphitheatre, was primarily used for gladiatorial contests and public spectacles. Its elliptical shape and multiple levels of arches and seating areas are characteristic features that make it easily recognizable. The structure has withstood the test of time, surviving centuries of history and serving as a testament to the engineering prowess of the ancient Romans.
It seems to be a generic landmark.
Janus-Pro-7B — Specific identity Question: Identify the main landmark or structure in this image as specifically as you can.
The main landmark in this image is the Colosseum, also known as the Flavian Amphitheatre, located in Rome, Italy.
It looks like a beautiful building.
BAGEL-7B-MoT — Visible attributes Question: Describe the main structure’s visible shape, colors, apparent materials, and architectural details.
Appendix
Table 9: Visual understanding responses for Colosseum before and after unlearning.
Figure 6: Generation results for Batman and its neighbor before and after unlearning.
Base
UniForget
BAGEL-7B-MoT — Specific identity Question: Identify the main character in this image as specifically as you can.
The main character in this image is Batman, a fictional superhero from DC Comics. He is depicted in his iconic costume, which includes a gray suit with a yellow utility belt, a bat emblem on the chest, and a blue cape. Batman is known for his vigilante activities in Gotham City, fighting crime and maintaining order.
It seems to be a fictional character.
Janus-Pro-7B — Specific identity Question: Identify the main character in this image as specifically as you can.
The main character in this image is Batman, a fictional superhero known for his role as a crime fighter in Gotham City.
It seems to be a fictional character.
BAGEL-7B-MoT — Visible attributes Question: Describe the character’s visible colors, body shape, clothing, accessories, and other distinguishing details.
Appendix
Table 10: Visual understanding responses for Batman before and after unlearning.
Figure 7: Generation results for Astronomical Telescope and its neighbor before and after unlearning.
Base
UniForget
BAGEL-7B-MoT — Specific identity Question: Identify the main object in this image as specifically as you can.
The main object in this image is a telescope mounted on a tripod stand.
a generic object
Janus-Pro-7B — Specific identity Question: Identify the main object in this image as specifically as you can.
The main object in this image is a telescope mounted on a tripod.
a generic object
BAGEL-7B-MoT — Visible attributes Question: Describe the main object’s visible shape, colors, apparent materials, and major components.
Unified multimodal models (UMMs) are motivated by the hope that understanding and generation reinforce each other but controlled ablations repeatedly find that adding a generation objective leaves understanding flat. Joint-training studies cannot settle the disagreement: with overlapping supervision, a gain cannot be attributed to the architecture rather than the data. To further investigate the relationship between the two directions in UMMs, we separate them by construction. A novel visual entity, a rendered 3D asset paired with a pseudo-word screened for absence from the frozen model's behavior, is bound through exactly one task direction, and the untrained direction is then measured. We find that the channel is real in both directions, but the directions differ in kind: generation training installs a name the model can only match among candidates; understanding training installs one it can also produce. What governs cross-task usability is where the binding enters the shared computation. An alignment probe predicts export across 36 configurations (Spearman ρ=+0.68). That objective's alignment term, maximized in closed form over activations with every weight frozen, makes a concept drawable when injected at layer 7 of 28 and is indistinguishable from the base model from layer 14 on, while the weight-based version of the same edit peaks at layers 10-14. In an observational series of four models, this window appears only where the understanding pathway is a semantic vision encoder, suggesting that unified weights are not enough: the two directions must share a semantic format at the entry point. Exploiting the rule, a mid-stack alignment objective acquires the concept for a 0.1% relative loss of the model's general text-to-image ability, against 41% for the standard generative route. Our code is at https://github.com/Zane-ZYQiu/entry-point-umm.
Zongyang Qiu, Yihan Wu, Kaixuan Fan +2
The Hong Kong University of Science and Technology (Guangzhou) · Columbia University · MMLab, CUHK +1
While unified multimodal models (UMMs) jointly perform visual understanding and generation within a single model, functional unification does not guarantee learning synergy: the two objectives may reinforce each other, compete for capacity, or merely coexist. We investigate their relationship at the representation, task, and system levels in a controlled, structurally native setting without pretrained vision priors. At the representation level, we find that each objective provides useful signal to the other: generation enriches the visual features learned for understanding, while understanding strengthens vision--language alignment for generation. However, when both objectives are forced through the same computation path, one tends to dominate. A task-decoupled architecture that specializes conflicting visual computation while preserving semantic interaction avoids this asymmetric degradation. At the task level, through three case studies, we find positive bidirectional transfer when understanding and generation tasks rely on shared knowledge. At the system level, we show that an end-to-end UMM outperforms a matched planner--executor pipeline on complex tasks that explicitly require both image understanding and generation. Together, these results show that the value of UMMs extends beyond a unified interface: appropriate specialization, shared task knowledge, and end-to-end optimization can turn coexistence into synergy.
Penghao Wu, Haiwen Diao, Weichen Fan +3
1S-Lab, Nanyang Technological University · 2SenseTime Research
Unified Multimodal Models (UMMs) integrate image understanding and generation within a single architecture, yet how the two tasks interact remains understudied. We investigate transferability in UMMs: whether training a capability on one task improves the same capability on the other without explicit supervision. Through controlled experiments, we empirically find that transferability depends on architecture-models with fully shared transformer backbone and a unified visual encoder exhibit consistent cross-task transfer, while loosely coupled designs show little or none. Leveraging this transferability, we propose a practical training strategy. The most straightforward way to improve a target generative capability (e.g., counting) is to fine-tune generation directly, but this can degrade visual quality due to distribution shift. Instead, we train the corresponding understanding task and let it transfer into generation, which improves capability-specific generative performance while minimizing distribution shift. We validate this across three capabilities-counting, spatial relation, and text recognition/generation-showing that cross-task transferability can be systematically exploited in UMMs.
Jiwon Kang, Heeji Yoon, Jaewoo Jung +5
KAIST AI, South Korea · Blynx, South Korea · Trillionlabs, South Korea