Subject-driven image generation has shown great success in creating personalized content, but its capabilities are largely confined to single subjects in common poses. Current approaches face a fundamental conflict when handling multiple subjects with complex, distinct actions: preserving individual identities while enforcing precise pose structures. This challenge often leads to identity fusion and pose distortion, as appearance and structure signals become entangled within the model's architecture. To resolve this conflict, we introduce ASTRA(Adaptive Synthesis through Targeted Retrieval Augmentation), a novel framework that architecturally disentangles subject appearance from pose structure within a unified Diffusion Transformer. ASTRA achieves this through a dual-pronged strategy. It first employs a Retrieval-Augmented Pose (RAG-Pose) pipeline to provide a clean, explicit structural prior from a curated database. Then, its core generative model learns to process these dual visual conditions using our Enhanced Universal Rotary Position Embedding (EURoPE), an asymmetric encoding mechanism that decouples identity tokens from spatial locations while binding pose tokens to the canvas. Concurrently, a Disentangled Semantic Modulation (DSM) adapter offloads the identity preservation task into the text conditioning stream. Extensive experiments demonstrate that our integrated approach achieves superior disentanglement. On our designed COCO-based complex pose benchmark, ASTRA achieves a new state-of-the-art in pose adherence, while maintaining high identity fidelity and text alignment in DreamBench.
Figures & tables
Figure 2: The ASTRA framework for multi-subject pose-controllable generation. The left panel shows the overall architecture, which uniquely conditions a DiT on reference identities ( Iref ), a text prompt, and an explicit pose structure ( Cpose ). Its effectiveness stems from our Enhanced Universal Rotary Position Embedding (EURoPE), detailed on the right. EURoPE employs an asymmetric strategy: it decouples identity from spatial location using universal rotary position embedding (UnoPE) for Iref tokens, while enforcing strict structural constraints by binding Cpose tokens to the latent canvas with native RoPE. Concurrently, Disentangled Semantic Modulation (DSM) offloads identity preservation by modulating text embeddings with visual features.
Figure 3: The inference pipeline of ASTRA. A user prompt is first refined into a canonical query by a VLM-based optimizer to isolate pose-defining semantics. This query facilitates a vector search on our curated database to find an optimal pose map ( Cpose ). The retrieved pose, acting as explicit structural guidance, is then passed to the ASTRA model along with the subject identities ( Iref ) and original text to synthesize the final image.
Figure 4: Qualitative comparison with different methods on single and multi subject driven generation
Method
CLIP-I↑
DINO↑
CLIP-T↑
Oracle (reference images)
0.885
0.774
N/A
Textual Inversion [ 8 ]
0.780
0.569
0.255
DreamBooth [ 30 ]
0.803
0.668
0.305
BLIP-Diffusion [ 18 ]
0.805
0.670
0.302
ELITE [ 41 ]
0.772
0.647
0.296
Re-Imagen [ 4 ]
0.740
0.600
0.270
Table 1: Quantitative results for single-subject driven generation on DreamBench. We present the oracle results in the first row and compare both tuning and tuning-free methods. The best results are shown in bold and the second-best result is underlined.
Method
CLIP-I↑
DINO↑
CLIP-T↑
DreamBooth [ 30 ]
0.695
0.430
0.308
BLIP-Diffusion [ 18 ]
0.698
0.464
0.300
SubjectDiffusion [ 21 ]
0.696
0.506
0.310
MIP-Adapter [ 13 ]
0.726
0.482
0.311
MS-Diffusion [ 40 ]
0.726
0.525
0.319
OmniGen [ 45 ]
0.722
0.511
0.331
Table 2: Quantitative results for multi-subject driven generation based on DreamBench. Our method achieves state-of-the-art performance among both tuning and tuning-free methods.
Method
OKS ↑
CLIP-I ↑
DINO ↑
CLIP-T ↑
DreamO [ 23 ]
0.0015
0.6736
0.4665
0.3084
OmniGen [ 45 ]
0.0219
0.6865
0.4284
0.2999
OmniGen2 [ 42 ]
0.0270
0.6938
0.4670
0.3075
UNO [ 43 ]
0.0277
0.6857
0.4283
0.2970
IP-Adapter* [ 38 ]
0.0314
0.6942
0.4310
0.3050
PhotoMaker* [ 19 ]
0.0308
0.6920
0.4305
0.3042
Table 3: Evaluation on the COCO-based Complex Pose Benchmark. All compared methods receive identical multi-person pose maps and identity crops. Methods marked with * use ControlNet and OpenPose for explicit pose conditioning. ASTRA achieves the highest OKS while maintaining strong identity fidelity.
Configuration
OKS ↑
CLIP-I ↑
DINO ↑
CLIP-T ↑
(a) w/o RAG-Pose
0.0095
0.6815
0.4173
0.3061
(b) w/ Symmetric RoPE
0.0041
0.3813
0.2255
0.1705
(c) w/ Symmetric UNOPE
0.0076
0.6028
0.3591
0.2876
(d) w/o DSM
0.0345
0.6754
0.4208
0.3072
(e) ASTRA (Ours)
0.0452
0.7087
0.4457
0.3194
Table 4: Ablation study of ASTRA’s key components on the COCO-based complex pose benchmark. The performance of the full model (e) is the main result, while rows (a-d) show the impact of removing or altering each component.
Pose-guided text-to-image generation often suffers from limb distortions and feature crosstalk in complex multi-person scenarios. While existing UNet-based adapters struggle with long-range spatial dependencies, emerging Multimodal Diffusion Transformers (MM-DiTs) offer superior global modeling. However, naive signal concatenation in MM-DiTs severely disrupts pre-trained latent distributions. To address this, we propose TrioPose, a native pose-driven framework built upon the SD3.5M architecture. Specifically, we introduce a Triple-Stream Pose-Aware DiT (TSPA-DiT) that treats pose as an independent modality. It employs layer-wise activation and zero-initialized dual-residual injection to smoothly enforce geometric constraints while preserving pre-trained latent stability. To resolve severe multi-instance occlusions, we design a Learnable Relational Bias Mask that categorizes topological connectivity into fine-grained physical states, mapping them into continuous attention soft constraints to effectively decouple inter-instance interference. Furthermore, a Pose-Guided Spatial Loss Weighting strategy modulates the native diffusion objective using heatmap-derived error maps, focusing anatomical supervision strictly on distortion-prone regions. Extensive experiments demonstrate that TrioPose achieves state-of-the-art performance across challenging benchmarks, including Human-Art, CrowdPose, and OCHuman. Notably, it attains an AP of 64.33 on Human-Art, representing a 30% improvement over prior arts, while setting new standards for visual fidelity and text-image semantic alignment in complex multi-human generation.
Dian Gu, Zhengyi Yang
Institute of Automation, Chinese Academy of Sciences Beijing, China
Multi-subject image generation aims to synthesize images that faithfully preserve the identities of multiple reference subjects while following textual instructions. However, existing methods often suffer from identity inconsistency and limited compositional control, as they rely on diffusion models to implicitly associate text prompts with reference images. In this work, we propose Hierarchical Concept-to-Appearance Guidance (CAG), a framework that provides explicit, structured supervision from high-level concepts to fine-grained appearances. At the conceptual level, we introduce a VAE dropout training strategy that randomly omits reference VAE features, encouraging the model to rely more on robust semantic signals from a Visual Language Model (VLM) and thereby promoting consistent concept-level generation in the absence of complete appearance cues. At the appearance level, we integrate the VLM-derived correspondences into a correspondence-aware masked attention module within the Diffusion Transformer (DiT). This module restricts each text token to attend only to its matched reference regions, ensuring precise attribute binding and reliable multi-subject composition. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the multi-subject image generation, substantially improving prompt following and subject consistency.
Yijia Xu, Zihao Wang, Haokun Gui +1
Peking University · Hong Kong University of Science and Technology
Subject Customization is a foundational task in modern image generation. By providing a few reference images and a text prompt, users can generate images of a specific object in any desired scene. However, existing methods still struggle to achieve effective pose control for customized subjects. In practice, they often exhibit inaccurate poses or inconsistent cross-pose appearances. These limitations suggest that understanding objects in a volumetric manner remains a significant challenge for 2D-native backbones. To address this challenge, we propose Pose-ICL, a tuning-free framework that leverages 3D-aware In-Context Learning (ICL) to directly adapt to new subjects through multiple paired image-pose references. Its core mechanism,Surface-Anchored Position Embedding (SAPE), equips the model with explicit 3D awareness by anchoring image tokens to the surface coordinates of a volumetric bounding box. Dedicated refinements ensure its seamless compatibility with existing DiT models. Extensive evaluations on both 3D assets and real-world subjects demonstrate that Pose-ICL significantly outperforms current methods in both pose accuracy and identity consistency.
Xuan Han, Yihao Zhao, Mingyu You
College of Electronic and Information Engineering, Tongji University, Shanghai, China · State Key Laboratory of Autonomous Intelligent Unmanned Systems, Frontiers Science Center for Intelligent Autonomous Systems, Ministry of Education, Shanghai, China.