While Text-to-Image (T2I) diffusion models have achieved remarkable success, precise spatial and orientational control in multi-object scenes remains a persistent challenge. Existing methods either rely on computationally expensive dense 3D maps or suffer from severe attribute leakage and "cut-and-paste" artifacts. To address these limitations, we propose PoseAdapter, a lightweight framework for high-fidelity 2.5D controllable image generation. Instead of dense spatial maps, it establishes precise spatial-angular anchors using an efficient condition layout: individual object captions, 2D bounding boxes, and 3D angles. To resolve the generative trade-off between strict instance isolation and global coherence, we introduce a Context-Aware Dual-Stream Representation. By injecting local object tokens and relation-enriched scene tokens into the visual stream of modern MM-DiT architectures via parallel masked and unmasked pathways, PoseAdapter eliminates attribute leakage while preserving natural inter-object relationships and scene-level coherence. To support this paradigm, we construct OrientLayout, a high-quality dataset featuring standardized 2.5D annotations and instance-level decoupled semantics. Extensive experiments demonstrate that PoseAdapter outperforms state-of-the-art baselines in spatial accuracy, orientational precision, and multi-object visual fidelity. Code and dataset will be available at https://github.com/cyf23/PoseAdapter.
Figures & tables
Figure 1. High-fidelity 2.5D controllable generation by PoseAdapter . (a) Single-object Setting : Precise adjustment of 3D orientations (azimuth, polar, rotation) and spatial bounding boxes. (b) Multi-object Setting : Strict attribute decoupling in complex scenes, assigning specific identities and colors to distinct entities while eliminating attribute leakage. (a) Single-object generation controlled by 3D angles and 2D bounding boxes. (b) Multi-object scenes demonstrating precise attribute assignment without semantic blending.
Method
Arbitrary 3D Orient.
Lightweight Cond.
Real-World Domain
Arch. Agnostic
Zero-1-to-3 ( Liu et al., 2023a )
✓
×
×
✓
CompassControl ( Parihar et al., 2025 )
×
✓
×
✓
ORIGEN ( Min et al., 2025 )
✓
×
✓
×
LOOSECONTROL ( Bhat et al., 2024 )
×
×
✓
✓
SceneDesigner ( Qin et al., 2025 )
✓
×
✓
✓
PoseAdapter (Ours)
✓
✓
✓
✓
Table 1. Comparison of orientation-aware image generation paradigms. “Arch. Agnostic” denotes seamless integration with standard T2I architectures. “Lightweight Cond.” indicates that the method does not require dense spatial maps (e.g., depth or CNOCS maps) as input conditions.
Figure 2. Overall pipeline of the proposed PoseAdapter. Given a set of instance conditions (captions, 2D bounding boxes, and 3D angles), we first encode them into unified object tokens To . A scene transformer then models their interactions to produce relation-enriched scene tokens Ts . Finally, these dual-stream representations are exclusively injected into the pre-trained MM-DiT visual stream via a masked local stream (enforcing precise boundaries) and an unmasked global stream (preserving inter-object coherence). The overall architecture diagram of the proposed PoseAdapter pipeline. On the left, Condition Encoding processes object captions, 2D bounding boxes, and angles through respective encoders and a fusion MLP to generate object tokens. In the middle, a Shared Object Encoder (Transformer) converts these local tokens into relation-enriched scene tokens, establishing a Dual-Stream Representation. On the right, within an MM-DiT block, the visual hidden states interact with these tokens via two parallel cross-attention branches: a Local Stream (using layout masks) for the object tokens, and an unmasked Global Stream for the scene tokens. The outputs are aggregated and residually added to the visual stream.
Method
Overall Quality
OrientLayout-Single-Front
OrientLayout-Single-Back
OrientLayout-Multi
FID ↓
CLIP ↑
O-CLIP ↑
Accls↑
mIoU ↑
Acc22.5∘↑
AccBOV↑
Accls↑
mIoU ↑
Acc22.5∘↑
AccBOV↑
Accls↑
mIoU ↑
Acc22.5∘↑
AccBOV↑
LOOSECONTROL
36.78
0.287
0.232
34.72
32.94
10.18
12.28
35.23
34.87
6.47
8.82
24.20
29.45
2.43
6.40
SceneDesigner
25.10
0.347
0.288
75.79
69.79
73.68
86.67
74.71
66.45
62.35
80.58
47.48
46.38
56.00
80.00
PoseAdapter (Ours)
24.60
0.344
0.310
97.84
87.96
87.02
95.44
99.40
88.89
71.76
88.82
91.67
81.72
72.70
93.22
Table 2. Quantitative comparison on OrientLayout. Acc22.5∘ : automated accuracy (OrientAnything); AccBOV : human verification.
Figure 3. Qualitative comparison of controllable image generation. The proposed PoseAdapter demonstrates precise layout and orientation control across diverse settings. The Condition column displays the respective inputs: 2.5D visualizations for the proposed approach, CNOCS maps for SceneDesigner, and depth maps for LOOSECONTROL. A large grid of images comparing 2.5D controllable image generation methods. The top rows display the input spatial conditions, showing our lightweight 2D bounding boxes with orientation arrows versus the dense CNOCS and depth maps used by baselines. The grid is divided into columns representing different full prompts, such as a dog, a lion, a locomotive, an airplane, and multi-object scenes like a room with cameras or chairs. The rows show the generated results from LOOSECONTROL, SceneDesigner, and our PoseAdapter. Visually, our results align strictly with the colored bounding boxes and full prompts, while baselines show obvious misalignments, missing objects, or color blending in multi-object columns.
Figure 4. Qualitative ablation results of relying solely on the global stream. A qualitative comparison matrix with three columns: Condition, Global Stream (Scene Token), and Dual Stream (Ours). It displays generated images for three prompts: a pink flamingo and a white rooster, a yellow bulldozer and a silver helicopter, and a red oven and a silver coffee machine. The Global Stream shows attribute leakage, while the Dual Stream cleanly separates the attributes.
Method
Split
OrientLayout
Overall
Accls (%) ↑
mIoU (%) ↑
AccBOV (%) ↑
CLIP ↑
O-CLIP ↑
w/o Box Mask Loss
Single
97.56
87.32
90.55
0.342
0.309
Multi
86.84
79.06
90.61
w/o t -Sampling Shift
Single
96.40
86.21
89.45
0.341
0.307
Multi
87.86
78.44
89.91
w/o Both Strategies
Single
94.79
82.66
86.81
0.323
0.300
Table 5. Quantitative ablation study of PoseAdapter.
Figure 5. Qualitative ablation results of relying solely on the local stream. A qualitative comparison matrix with three columns: Condition, Local Stream (Object Token), and Dual Stream (Ours). It displays generated images for three prompts: a golden retriever on a wooden boat, a grey wolf and a bald eagle, and a bedroom scene with a sofa. The Local Stream shows disjointed composition and unnatural clipping (highlighted by a red box on the sofa), while the Dual Stream achieves photorealistic harmony.
Pose-guided text-to-image generation often suffers from limb distortions and feature crosstalk in complex multi-person scenarios. While existing UNet-based adapters struggle with long-range spatial dependencies, emerging Multimodal Diffusion Transformers (MM-DiTs) offer superior global modeling. However, naive signal concatenation in MM-DiTs severely disrupts pre-trained latent distributions. To address this, we propose TrioPose, a native pose-driven framework built upon the SD3.5M architecture. Specifically, we introduce a Triple-Stream Pose-Aware DiT (TSPA-DiT) that treats pose as an independent modality. It employs layer-wise activation and zero-initialized dual-residual injection to smoothly enforce geometric constraints while preserving pre-trained latent stability. To resolve severe multi-instance occlusions, we design a Learnable Relational Bias Mask that categorizes topological connectivity into fine-grained physical states, mapping them into continuous attention soft constraints to effectively decouple inter-instance interference. Furthermore, a Pose-Guided Spatial Loss Weighting strategy modulates the native diffusion objective using heatmap-derived error maps, focusing anatomical supervision strictly on distortion-prone regions. Extensive experiments demonstrate that TrioPose achieves state-of-the-art performance across challenging benchmarks, including Human-Art, CrowdPose, and OCHuman. Notably, it attains an AP of 64.33 on Human-Art, representing a 30% improvement over prior arts, while setting new standards for visual fidelity and text-image semantic alignment in complex multi-human generation.
Dian Gu, Zhengyi Yang
Institute of Automation, Chinese Academy of Sciences Beijing, China
Subject-driven image generation has shown great success in creating personalized content, but its capabilities are largely confined to single subjects in common poses. Current approaches face a fundamental conflict when handling multiple subjects with complex, distinct actions: preserving individual identities while enforcing precise pose structures. This challenge often leads to identity fusion and pose distortion, as appearance and structure signals become entangled within the model's architecture. To resolve this conflict, we introduce ASTRA(Adaptive Synthesis through Targeted Retrieval Augmentation), a novel framework that architecturally disentangles subject appearance from pose structure within a unified Diffusion Transformer. ASTRA achieves this through a dual-pronged strategy. It first employs a Retrieval-Augmented Pose (RAG-Pose) pipeline to provide a clean, explicit structural prior from a curated database. Then, its core generative model learns to process these dual visual conditions using our Enhanced Universal Rotary Position Embedding (EURoPE), an asymmetric encoding mechanism that decouples identity tokens from spatial locations while binding pose tokens to the canvas. Concurrently, a Disentangled Semantic Modulation (DSM) adapter offloads the identity preservation task into the text conditioning stream. Extensive experiments demonstrate that our integrated approach achieves superior disentanglement. On our designed COCO-based complex pose benchmark, ASTRA achieves a new state-of-the-art in pose adherence, while maintaining high identity fidelity and text alignment in DreamBench.
Current text-to-image models struggle to provide precise camera control using natural language alone. In this work, we present a framework for precise camera control with global scene understanding in text-to-image generation by learning parametric camera tokens. We fine-tune image generation models for viewpoint-conditioned text-to-image generation on a curated dataset that combines 3D-rendered images for geometric supervision and photorealistic augmentations for appearance and background diversity. Qualitative and quantitative experiments demonstrate that our method achieves state-of-the-art accuracy while preserving image quality and prompt fidelity. Unlike prior methods that overfit to object-specific appearance correlations, our viewpoint tokens learn factorized geometric representations that transfer to unseen object categories. Our work shows that text-vision latent spaces can be endowed with explicit 3D camera structure, offering a pathway toward geometrically-aware prompts for text-to-image generation. Project page: https://randdl.github.io/viewtoken_control/