Intern Lumina U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding and Image Generation

Qian Yu; Zongyang Qiu; Juncheng Yan; Hongliang Li; Qi Qin; Keqi Wang; Siqi Luo; Junkui Ju

cs.CVOct 14, 2026

Intern Lumina U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding and Image Generation

Authors: Qian Yu, Zongyang Qiu, Juncheng Yan, Hongliang Li, Qi Qin, Keqi Wang, Siqi Luo, Junkui Ju, Yapeng Mi, Tiantian Lu, Tian Liu, Pengyu Xie, Chenhao Ji, Minghui Zhang, Yiang Zhao, Siyu Yan, Haiying Xu, Yanbo Wang, Jianghan Shen, Yuandong Pu, Jiayi Lei, Borui Zhang, Jinxin Shi, Bo Zhang, Yuhang Zang, Shenglong Ye, Lixin Gu, Hongjie Zhang, Linghan Liu, Xinyuan Chen, Yuewen Cao, Lihe Zhang, Xiaohong Liu, Junjun He, Qiangwei Liu, Qi Zhang, Yu Cheng, Tianfan Xue, Hui Xiong, Chang Xu, Yu Qiao, Yihao Liu, Bin Fu

Abstract

Unified multimodal generation covers discrete, continuous, and hybrid paradigms. Discrete approaches with single-codebook visual tokenizers align naturally with text modeling but have limited representational capacity. Continuous and hybrid approaches retain richer visual representations, but they model text and images in heterogeneous spaces with different training objectives, potentially introducing optimization conflicts between modalities with misaligned representation spaces. These choices expose a central question: how to efficiently model high-capacity visual representations alongside textual content in a shared optimization space within a single backbone. We present Intern Lumina U2, to our knowledge the first diffusion language model to represent multi-codebook visual tokens in a fully discrete space. Each visual position is represented by a group of codes from complementary codebooks, increasing its representational capacity while preserving compatibility with discrete text modeling. At the input, the embeddings are concatenated at each position; at the output, a lightweight multi-codebook autoregressive head predicts the codes within each position sequentially, while the diffusion backbone processes all positions in parallel. Built on the unified 4D representation of AToken, Intern Lumina U2 supports image, video, and 3D inputs within a single backbone and achieves leading performance among unified models in visual understanding, image generation, and image editing. We further build a full-stack training, inference, and evaluation system on both Ascend and NVIDIA platforms to verify the cross-platform effectiveness of our model. We also improve the inference efficiency through a stability-aware sampling strategy, which combines confidence-plateau-guided unmasking approach with drift-triggered cache refreshing mechanisms, accelerating image generation by 4.45x.

Figures & tables

	Supported modalities				Representation and decoding
Model	Image und.	Image gen.	Video und.	3D und.	Discrete tokens	Masked-diff. decoding	Multi- codebook	Diffusion text	No continuous generator
Autoregressive
Emu3	✓	✓	✓	✗	✓	✗	✗	✗	✓
Emu3.5	✓	✓	✗	✗	✓	✗	✗	✗	✓
Janus-Pro	✓	✓	✗	✗	✓	✗	✗	✗	✓
VILA-U	✓	✓	✓	✗	✓	✗	✓	✗	✓

Table 1: Capability and design checklist for representative unified multimodal models, grouped by backbone paradigm. Discrete tokens refers to the visual state inside the backbone; Masked-diff. decoding marks models that generate visual tokens by iterative masked denoising rather than left-to-right prediction; Multi-codebook denotes multiple codes per visual position with their dependence modeled explicitly; Diffusion text marks models that also generate text through the masked-diffusion process rather than autoregressively; No continuous generator marks systems that decode images through backbone tokens, without a separate continuous (diffusion or flow) generator. Systems: Emu3 ( Wang et al., 2026a ) , Emu3.5 ( Cui and others, 2025 ) , Janus-Pro ( Chen et al., 2025 ) , VILA-U ( Wu et al., 2025d ) , UniTok ( Ma et al., 2025a ) , BAGEL ( Deng et al., 2025 ) , InternVL-U ( Tian and others, 2026 ) , SenseNova-U1 ( SenseNova Team, 2026 ) , Show-o2 ( Xie et al., 2025b ) , Show-o ( Xie et al., 2025a ) , MMaDA ( Yang et al., 2025 ) , Lavida-O ( Li et al., 2026b ) , Lumina-DiMOO ( Xin et al., 2025 ) , LLaDA2.0-Uni ( Bie and others, 2026 ) .

Figure 3: Overall architecture of Intern Lumina U2. Visual inputs are encoded into visual tokens by AToken and text into text tokens; the MoE diffusion LLM recovers masked text tokens through the text head, with text answers generated by block diffusion, and masked visual tokens through the multi-codebook AR head, which predicts the $K$ codes of each visual token sequentially.

Figure 4: Three candidate input interfaces for multi-codebook visual tokens. We adopt the third design: independent embeddings for the eight codebooks, concatenation at each spatial position, and projection into one backbone token.

Figure 5: Training and inference of the multi-codebook AR head: one teacher-forced causal pass yields all $K$ logits at training; at inference the prefix grows for $K$ steps, each sampled code $\hat{c}_{k}$ is embedded into $e_{k}$ and appended. Positions are processed in parallel, codebooks autoregressively.

Figure 6: Task-aware attention mask at block level. (a) Image answer: one noisy visual answer

\tilde{V}

after the clean context

C

, bidirectional over all positions. (b) Text answer: block diffusion over the noisy blocks

\tilde{b}_{1},\tilde{b}_{2},\tilde{b}_{3}

and their clean copies

b_{1},b_{2},b_{3}

, with

M_{\mathrm{BD}}

M_{\mathrm{OBC}}

, and

M_{\mathrm{BC}}

. (c) A text answer followed by an image answer, as in thinking before generation: the image reads the clean copy of the preceding text blocks. (d) Multi-turn text with conditions

C_{1},C_{2}

, noisy answers

\tilde{x}_{1},\tilde{x}_{2}

, and clean copies

x_{1},x_{2}

: a later turn reads the clean copies of earlier answers.

Figure 7: Data construction pipeline. Six data categories are built with category-specific rules, converted to one record format, encoded offline, and re-weighted at each training stage. Bars show backbone-position shares (Table 2 ).

Stage	Position share (%)
	Und.	T2I	Text	Edit	Video	3D
Vision–Language alignment	30	50	20	–	–	–
Perception pretraining	30	50	20	–	–	–
Omni-visual mid-training, Stage 1	30	60	10	–	–	–
Omni-visual mid-training, Stage 2	40	40	10	5	4	1
Instruction SFT	25	50	5	10	8	2

Table 2: Data mixture at each training stage. Shares are fractions of estimated backbone positions per pass over the mixture; the corresponding resolution and optimization settings are given in Table 3 .

Configuration	Vision- Language alignment	Perception pretraining	Omni-visual mid-training		Instruction SFT
			Stage 1	Stage 2
Training setup
Generation resolution	$256$	$512$	$1024$	$1024$	$1024$
Sequence length	8,192	8,192	16,384	16,384	16,384
Text length limit	2,048	2,048	2,048	2,048	2,048
Text block length	32	32	32	32	32

Table 3: Training configurations for the primary Ascend run and the NVIDIA reference line.

Method	Steps	BB fwd	Lat. (s)	Speedup	GenEval
Cumulative system optimizations (fixed 96-step schedule)
Production path	96.0	192	159.9	1.00 $\times$	86.6
+ CFG batching	96.0	192	117.2	1.36 $\times$	89.0
+ AR-head KV cache	96.0	192	84.2	1.90 $\times$	88.6
+ hidden-state cache	96.0	192 (65%)	64.1	2.49 $\times$	89.2
+ V-verify refresh	96.0	192 (63%)	64.5	2.48 $\times$	89.3

Table 4: Inference acceleration under a matched

1024\times 1024

text-to-image protocol with the same checkpoint, 553 GenEval prompts, one image per prompt, CFG scale

4.0

, temperature

0.9

, and seed

42

. “Steps” is the realized number of outer diffusion steps, prompt-dependent for the plateau rows and fixed elsewhere. “BB fwd” counts conditional plus unconditional backbone forwards, with the cache-hit ratio in parentheses where a cache is active. Latency is wall-clock per image after a discarded kernel preheat and includes cache warm-up. The selected PHV setting is

\delta_{p}=0.18

f_{p}=0.005

, with the remaining cache and V-verify controls fixed across rows. GenEval uses one image per prompt and is therefore not comparable to the four-seed protocol of Table 7 ; no configuration scores below the production path.

Figure 8: Confidence calibration for masked visual positions under the fixed $96$ -step, $1024\times 1024$ protocol. The visual confidence remains below the text-scale commit threshold, and its trajectory provides a more useful signal for adaptive sampling than its absolute level.

Figure 9: Offline temporal-stability test for PHV on $200$ prompts. Each plateau-eligible position is paired with a still-masked matched-control token from the same image that does not satisfy the plateau criterion while exhibiting comparable confidence and EMA values. Both are held masked for $2$ , $4$ , or $8$ additional refinement steps. (a) The difference in top-1 flip rate between the plateau token and its matched-control token grows with the hold duration. (b) Per-prompt flip rates at $H=8$ ; $157$ of $200$ prompts fall below the diagonal. (c) Posterior max-probability trajectories for three representative plateau–matched-control pairs.

	Understanding-only		Unified					Intern Lumina U2
Benchmark	Qwen2.5- VL	LLaDA-V	BAGEL	Intern VL-U	Lumina- DiMOO	LLaDA-o	LLaDA 2.0-Uni	Ascend	NVIDIA
#Params	7B	8B	7B+7B	2B+1.7B	8B	8B	16B-A1.4B	16.9B-A1.4B
Documents and charts
ChartQA	84.1	78.3	74.3	76.6	8.3	87.9	80.1	86.2	81.2
CharXiv (DQ)	73.9	47.0	70.6	53.3	27.8	69.8	68.4	85.9	84.0
Perception and hallucination

Table 5: Image understanding and reasoning, grouped into document and chart understanding, perception and hallucination, and knowledge and mathematical reasoning. Benchmarks: ChartQA ( Masry et al., 2022 ) , CharXiv ( Wang et al., 2024b ) , HallusionBench ( Guan et al., 2024 ) , MME ( Fu et al., 2025a ) , RealWorldQA ( xAI, 2024 ) , MMMU ( Yue et al., 2024 ) , MMMU-Pro ( Yue et al., 2025 ) , MathVista ( Lu et al., 2024 ) , MathVision ( Wang et al., 2024a ) , and DynaMath ( Zou et al., 2025 ) . Baseline numbers are taken from Bie and others (2026) . Bold marks the best score in each row.

Model	Video-MME	MLVU	MVBench
Video-LLaVA-7B	39.9	–	43.1
LLaVA-Mini	–	42.8	44.5
VideoLLaMA2-7B	46.6	–	54.6
LLaDA-V	56.1	59.5	–
Ours (Ascend)	52.6	62.1	61.8
Ours (NVIDIA)	56.7	64.9	57.2

Table 6: Video and 3D understanding from the same checkpoint. Left: video benchmarks (Video-MME ( Fu et al., 2025b ) without subtitles, MLVU ( Zhou et al., 2025b ) m-avg, MVBench ( Li et al., 2024 ) ); baseline numbers are taken from Lin et al. (2024) ; Zhang et al. (2025) ; Cheng et al. (2024) ; You et al. (2026a) ; Fu et al. (2025b) . Right: 3D MM-Vet ( Qi et al., 2024 ) , with baseline numbers for ShapeLLM and PointLLM ( Xu et al., 2024 ) taken from Qi et al. (2024) . Bold marks the best score in each column.

Model	#Params	GenEval	DPG-Bench	TIIF	WISE
Generation-only models
FLUX.1 [Dev]	12B	66	83.84	71.8	0.50
Qwen-Image	20B	87	88.32	86.8	0.62
Unified understanding-and-generation models
Janus-Pro	7B	80	84.19	65.0	0.35
Show-o2	7B	76	86.14	–	0.39

Table 7: Text-to-image generation. GenEval ( Ghosh et al., 2023 ) , DPG-Bench ( Hu et al., 2024 ) , TIIF-Bench ( Wei et al., 2026 ) , and WISE ( Niu et al., 2026 ) overall scores (GenEval on a 0–100 scale); higher is better. Baseline numbers are taken from Bie and others (2026) , Tian and others (2026) , and Xie et al. (2025b) ; TIIF reports the overall score on long prompts. UniGenBench results are reported per dimension in Table 8 . Bold marks the best unified-model score in each column

Model	Sty	Wor	Att	Act	Rel	Log	Gra	Com	Lay	Txt	Overall
Generation-only models
FLUX.1 [Dev]	83.90	88.92	67.84	62.17	67.26	30.91	60.96	47.04	71.83	32.18	61.30
Qwen-Image	94.70	94.15	87.93	82.60	80.08	51.59	60.96	72.94	86.57	72.13	78.36
Unified understanding-and-generation models
Janus-Pro	90.80	86.71	67.74	64.26	68.40	37.05	64.44	62.11	72.01	2.59	61.61
BAGEL	90.20	85.60	67.74	61.98	70.69	30.23	66.44	58.12	76.49	7.76	61.53

Table 8: UniGenBench ( Wang et al., 2025b ) (EN) results on the ten first-level dimensions; Overall is their mean. Baseline numbers are taken from Bie and others (2026) ; bold marks the best unified-model score in each column. Dimensions: Sty(le), Wor(ld knowledge), Att(ribute), Act(ion), Rel(ationship), Log(ical reasoning), Gra(mmar), Com(pound), Lay(out), Txt (text generation).

Model	Short prompts				Long prompts
Model	Basic	Adv.	Real	Overall	Basic	Adv.	Real	Overall
Generation-only models
FLUX.1 [Dev]	83.1	65.8	70.7	71.1	78.7	68.5	71.5	71.8
Qwen-Image	86.2	79.3	90.3	86.1	87.2	80.9	91.4	86.8
Unified understanding-and-generation models
Janus-Pro	79.3	59.7	65.8	65.5	78.3	58.8	60.3	65.0

Table 9: TIIF-Bench ( Wei et al., 2026 ) results by instruction category for short and long prompts. Basic and Advanced are the average of their sub-categories; Real World is the Designer real-world score. Baseline numbers are taken from Tian and others (2026) ; bold marks the best unified-model score in each column.

Model	Alignment	Text	Reasoning	Style	Diversity	Overall
Generation-only models
FLUX.1 [Dev]	0.786	0.523	0.253	0.368	0.238	0.434
Qwen-Image	0.882	0.891	0.306	0.418	0.197	0.539
Unified understanding-and-generation models
Janus-Pro	0.553	0.001	0.139	0.276	0.365	0.267
BAGEL	0.769	0.244	0.173	0.367	0.251	0.361

Table 10: OneIG-Bench ( Chang et al., 2025 ) (EN). Baseline numbers are taken from Bie and others (2026) ; Overall is the mean of the five dimensions. Bold marks the best unified-model score in each column.

Model	GenExam Physics		VeriphyT2IBench
Model	Strict	Relaxed	Local	Global
Reference and generation-only models
Seedream 4.0	3.5	49.0	64.80	65.07
FLUX.1 Kontext max	0.0	25.6	–	–
Gemini 2.5 Flash Image	–	–	69.98	66.14
GPT-Image-1	–	–	68.32	68.56

Table 11: Physics-faithful scientific-diagram generation on the physics subset of GenExam and on VeriphyT2IBench. GenExam reports aggregate strict and relaxed MLLM-judge scores; VeriphyT2IBench reports question-level accuracy on its local attribute bank and global whole-diagram bank. Baseline values and protocol definitions follow Zhang et al. (2026) ; Princigram uses physics-specific structured supervision and prompting, whereas the other rows are general-purpose generators. Bold marks the best score in each column.

Local scoring	Seedream	Gemini	GPT-Image	SenseNova	Princigram-D	Princigram-B	Intern Lumina U2
E&M	65.55	67.75	69.58	53.46	76.00	78.59	83.30
Mechanics	56.19	61.63	60.25	52.16	68.24	71.59	71.34
Optics	77.81	82.33	79.54	80.96	75.64	78.92	79.23
Quantum mechanics	69.67	73.27	71.35	68.93	73.41	77.59	76.32
Thermodynamics	42.91	58.90	49.24	46.24	60.40	66.95	63.73
Acoustics	62.42	68.44	66.88	65.26	68.39	73.01	72.17

Table 12: VeriphyT2IBench domain-level breakdown for representative models. Each panel reports question-level accuracy (percent; higher is better) for the local attribute bank or the global whole-diagram bank. The overall rows are omitted because the corresponding aggregate scores are already reported in Table 11 .

Easy prompts	Seedream	Gemini	GPT-Image	SenseNova	Princigram-D	Princigram-B	Intern Lumina U2
E&M	75.32	80.56	80.41	75.45	82.11	84.74	83.78
Mechanics	66.08	67.84	68.19	64.96	74.41	76.77	75.40
Optics	80.81	83.21	81.79	82.98	78.05	79.01	81.37
Quantum mechanics	70.97	74.13	75.41	73.39	75.58	81.34	81.47
Thermodynamics	64.75	66.77	67.66	64.61	68.93	72.13	69.30
Acoustics	64.88	69.86	69.19	69.72	67.84	71.13	71.94

Table 13: VeriphyT2IBench breakdown by prompt difficulty. Scores are percentages and higher is better. The panels report the six physics domains at easy, medium, and hard difficulty for the same representative models as Table 12 .

Model	#Params	ImgEdit	GEdit-EN	GEdit-CN
Editing-specialist models
Step1X-Edit	12B	3.06	6.97	6.86
Qwen-Image-Edit	20B	4.35	7.56	7.52
Unified understanding-and-generation models
BAGEL	7B+7B	3.20	6.52	6.50
OmniGen2	3B+4B	3.44	6.41	–

Table 14: Instruction-based image editing. ImgEdit ( Ye et al., 2025b ) overall score (1–5) and GEdit-Bench ( Liu et al., 2025b ) overall score

G_{O}

(0–10) for English and Chinese prompts. Baseline numbers are taken from Bie and others (2026) and Tian and others (2026) . Bold marks the best unified-model score in each column.

Design	Table organization	Softmax width	Params	Peak memory (GB)	Step time (s)	Next-code accuracy
D1 universal	$1\times 189{,}968$	$189{,}968$	778.11M	43.79	7.1	$0.0426\pm 0.0002$
D2 offset-shared	$1\times 32{,}784$	$32{,}784$	778.11M	18.68	6.2	$0.0415\pm 0.0002$
D3 independent (ours)	$8\times 4{,}096$	4,096	793.02M	18.62	6.6	$\mathbf{0.0459\pm 0.0002}$

Table 15: Ablation of the visual input interface. Each design determines both the lookup-table organization and the corresponding classifier width. Accuracy is next-code top-1 on held-out supervised codes at the final checkpoint (

\approx 5

M codes per arm;

\pm

one standard error). Memory and step time are measured at sequence length

4608

on identical hardware. Bold marks the best value in each column.

Figure 10: Input-interface ablation. (a) Final next-code top-1 accuracy for each of the eight codebooks, normalized by D3; the blue annotations give D3’s absolute accuracy at the first and last codebooks. (b) Image loss over the two training epochs, shown as a rolling median with an inset for the converged tail. Higher is better in (a), whereas lower is better in (b).

Output head	Codebook dependence	GenEval	DPG-Bench	TIIF	UniGenBench
Independent parallel	no	68.94	83.22	70.1	59.6
Q-Former	no	69.67	78.92	63.9	56.7
Multi-codebook AR (ours)	yes	82.78	83.27	73.2	62.1

Table 16: Ablation of the visual output head: three training runs at 512-pixel generation resolution with comparable token budgets (about 197B–220B tokens each). All scores are on a 0–100 scale; TIIF is the mean accuracy and UniGenBench the EN ten-dimension average. Bold marks the best score in each column.

Figure 11: Inference-sampling ablation over the plateau change threshold $\delta_{p}$ and confidence floor $f_{p}$ , $553$ prompts per cell. (a) GenEval over the $6\times 6$ grid; the black box marks the selected quality-first setting $\delta_{p}{=}0.18$ , $f_{p}{=}0.005$ . (b) Speed–quality frontier over all $36$ cells, i.e. the configurations not improved on in both latency and quality by any other cell; the selected setting (red) reaches GenEval $89.3$ at $35.9$ s per image ( $4.45\times$ ).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Value	Field	Value
Transformer layers	20	First dense layers	1
Hidden size	2048	FFN size (dense)	5120
Attention heads	16	KV heads	4
Head dimension	128	RoPE dimension / fraction	64 / 0.5
RoPE base	600000	Max position embeddings	16384
RMSNorm / eps	yes / $10^{-6}$	Activation	silu

Appendix

Table 17: Complete model configuration. Upper part: backbone settings. Lower part: visual interface modules.

Role	Token	Id
Text mask (corruption target)	<\|mask\|>	156895
Padding	<\|pad\|>	156892
Answer boundary	<BOA> / <EOA>	157165 / 157166
Image boundary	<BOI> / <EOI>	157167 / 157168
3D boundary	<BO3D> / <EO3D>	157169 / 157170
Video boundary	<BOV> / <EOV>	157171 / 157172

Appendix

Table 18: Special tokens. Ids in the lower block index the per-codebook visual embedding tables; all other ids index the text vocabulary.

idx	token	type	turn	is_clean	block	position id
Turn 0: clean condition (media + question)
0	<BOI>	1	0	1	$-1$	0
1–2	$v_{00}\;v_{01}$	2	0	1	$-1$	1–2
3	$\lrcorner$	2	0	1	$-1$	3
4–5	$v_{10}\;v_{11}$	2	0	1	$-1$	4–5
6	$\lrcorner$	2	0	1	$-1$	6

Appendix

Table 19: A packed image-understanding sample with per-position metadata. In this example, type 1 denotes text or structural tokens, type 2 a media-condition token, type 3 a text-answer token, and type 4 a visual-answer token. The clean user-provided condition and the assistant answer form one turn, and the clean copy of the answer belongs to the same turn. The is_clean flag is 1 for clean tokens and 0 for noisy tokens, the block id is

-1

for conditions and 0 for this answer block, and a clean copy reuses the position ids of its noisy counterpart. Here

v_{ij}

is the visual token (

K

codes) at grid row

i

, column

j

;

\lrcorner

is the image-row delimiter (code 5001);

q_{i}

and

a_{i}

are question and answer tokens; and M is <|mask|> . Labels are

-100

at all positions that do not participate in supervision; only masked answer positions contribute to the loss.

Modality	Grid	Flattening order	Delimiter tokens
Image $H\times W$	$(H/16)\times(W/16)$	row-major $(y,x)$	5001 per row
Video $T\times H\times W$	$\lceil T/4\rceil\times(H/16)\times(W/16)$	frame-major $(t,y,x)$	5004 per row, 5005 per frame
3D asset	sparse voxels, coords $(x,y,z)$	$(z,y,x)$	5002 per row, 5003 per layer

Appendix

Table 20: Visual token layouts. Patch size is

(t,h,w)=(4,16,16)

; every grid position carries

K{=}8

codes.

Category	Type	Sources	Encoding resolution	Stages
Image understanding	Public	LLaVA-OneVision ( Li et al., 2025 ) and LLaVA-OneVision-1.5 ( An et al., 2025 ) instruction sets	256 / 512 / native	1–5
	Private	Image subsets of the InternVL ( Wang et al., 2025a ) instruction data, covering captioning; document, chart, table, and equation data; OCR data built from web image–text pairs; grounding, counting, and GUI or web screenshot data; and structured-reasoning data and samples with <think> reasoning traces
Video understanding	Public	Caption and QA sets over WebVid ( Bain et al., 2021 ) , Kinetics-400/600/700 ( Kay et al., 2017 ; Carreira et al., 2018 ; Carreira et al., 2019 ) , ActivityNet ( Caba Heilbron et al., 2015 ) , YouCook2 ( Zhou et al., 2018 ) , LSMDC ( Rohrbach et al., 2017 ) , TextVR ( Wu et al., 2025c ) , NTU RGB+D ( Shahroudy et al., 2016 ) , NExT-QA ( Xiao et al., 2021 ) , CLEVRER ( Yi et al., 2020 ) , STAR ( Wu et al., 2021 ) , Perception Test ( Pătrăucean et al., 2023 ) , EgoSchema ( Mangalam et al., 2023 ) , and MovieChat ( Song et al., 2024 )	64 frames, $448^{2}$ area	4–5
	Private	Video subsets of the InternVL instruction data; temporal-reasoning and counting sets
3D understanding	Public	Objaverse ( Deitke et al., 2023 ) assets with Cap3D-style captions ( Luo et al., 2023 ) ; GAPartNet ( Geng et al., 2023 ) part-level annotations; Sketchfab ( Sketchfab, 2026 ) assets	sparse voxels	4–5
	Private	–

Appendix

Table 21: Sources by data category, with the public and private sources of each category listed separately. Encoding resolution is the long side (images and editing) or the frame area budget (video); “native” means the original resolution clamped to a

[256,1024]

long side. Stages are numbered 1–5 as in Section 5 .

Source	Scored	Text rule	Following $<60$	Aesthetic top 70%	Kept
English	22.69	$-$ 4.33	$-$ 4.75	$-$ 3.09	10.51 (46%)
Chinese	31.35	$-$ 6.92	$-$ 6.26	$-$ 4.15	14.02 (45%)

Appendix

Table 22: Quality filtering of the curated text-to-image sources (counts in millions).

Figure 12: Qualitative examples for text-to-image generation and omni-visual understanding.

Figure 13: Qualitative examples for image editing.

Unified Multimodal Models

Unified multimodal models combine multiple data types in shared systems for cross-modal understanding, generation, reasoning, and interaction.

See all 287 papers

LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model
Apr 22, 2026Inclusion AI, Tiwei Bie, Haoxing Chen +15Unified Multimodal Models Multimodal Generation
AGI Research Center, Inclusion AI
TextLDM: Language Modeling with Continuous Latent Diffusion
May 8, 2026Jiaxiu Jiang, Jingjing Ren, Wenbo Li +10Diffusion Transformer Latent Diffusion Models
Joy Future Academy · HIT · HKUST(GZ)
Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens
Mar 19, 2026Yuqing Wang, Chuofan Ma, Zhijie Lin +7Image Generation Diffusion-Based Image Generation
University of Hong Kong · ByteDance Seed · Carnegie Mellon University +2

Apr 22, 2026J/K move · Enter open · S save

LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model

Inclusion AI, Tiwei Bie, Haoxing Chen, Tieyuan Chen+14

AGI Research Center, Inclusion AI

Unified Multimodal Models Multimodal Generation Diffusion-Based Image Generation Multimodal Foundation Models Diffusion Language Models Image Editing+1

Open paper AskPDF arXiv

We present LLaDA2.0-Uni, a unified discrete diffusion large language model (dLLM) that supports multimodal understanding and generation within a natively integrated framework. Its architecture combines a fully semantic discrete tokenizer, a MoE-based dLLM backbone, and a diffusion decoder. By discretizing continuous visual inputs via SigLIP-VQ, the model enables block-level masked diffusion for both text and vision inputs within the backbone, while the decoder reconstructs visual tokens into high-fidelity images. Inference efficiency is enhanced beyond parallel decoding through prefix-aware optimizations in the backbone and few-step distillation in the decoder. Supported by carefully curated large-scale data and a tailored multi-stage training pipeline, LLaDA2.0-Uni matches specialized VLMs in multimodal understanding while delivering strong performance in image generation and editing. Its native support for interleaved generation and reasoning establishes a promising and scalable paradigm for next-generation unified foundation models. Codes and models are available at https://github.com/inclusionAI/LLaDA2.0-Uni.

Abstract

Unified Multimodal Models

Explore similar work

LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model

TextLDM: Language Modeling with Continuous Latent Diffusion

Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens