Bridging Language and Spherical Space: Object-Centric Control for Text-to-Panorama Generation

Derui Li; Qian Qiao; Yuhao Sun; Wenhao Guo; Peng Lu

cs.CVAug 21, 2026

Bridging Language and Spherical Space: Object-Centric Control for Text-to-Panorama Generation

Authors: Derui Li, Qian Qiao, Yuhao Sun, Wenhao Guo, Peng Lu

Organizations: Beijing University of Posts and Telecommunications Beijing, China

Abstract

Panoramic image generation is increasingly important for immersive applications such as virtual reality, augmented reality, and 3D content creation. Unlike perspective images, panoramic images represent a viewer-centered $360^\circ$ surrounding space, where directional expressions such as left, right, front, and behind play a central role in spatial understanding. However, existing text-to-panorama methods largely rely on implicit spatial reasoning and often fail to faithfully ground object-level directional descriptions in spherical panoramic scenes. A straightforward alternative is to introduce explicit layouts, but requiring manually specified spatial conditions reduces the flexibility of language-based interaction and does not directly resolve the misalignment between egocentric directional language and panoramic image space. To address this issue, we propose PanoCtrl, an object-centric framework for controllable text-to-panorama generation. Our method explicitly bridges natural language and spherical panoramic space by converting textual descriptions into structured object-level spherical conditions and integrating them into the diffusion process. Specifically, we introduce PanoParse, a text-conditioned parser that predicts object semantics and spherical bounding field-of-view (BFoV) parameters, and PanoControl, which injects object-level semantic and spatial guidance into the diffusion transformer through object-aware attention and spatial residual enhancement. To support this task, we construct PanoGround, a dataset with object-level spherical annotations and diverse directional descriptions for controllable panoramic generation. Extensive experiments demonstrate that PanoCtrl achieves state-of-the-art performance in both spatial alignment and image quality.

Figures & tables

Figure 1. Motivation examples for controllable text-to-panorama generation. (a) Illustration of the alignment gap between egocentric directional space and ERP space. (b) Qualitative comparison with representative methods, including LayerPano3D ( Yang et al., 2025a ) and DiT360 ( Feng et al., 2025 ) , under directional descriptions. Red boxes indicate incorrect or spatially inconsistent object placements, while green boxes indicate correctly aligned ones.

Figure 2. Overall framework of PanoCtrl . PanoParse converts the input prompt into object-level spherical conditions, including semantic categories and BFoV parameters, and PanoControl integrates them into the diffusion transformer through object-aware attention and spatial residual enhancement for controllable panoramic image generation.

	Spatial Alignment			Image Quality
Method	OPR $\uparrow$	RTA $\uparrow$	SLE $\downarrow$	FID $\downarrow$	FAED $\downarrow$	IS $\uparrow$	CS $\uparrow$
PanFusion	76.61	19.87	52.30	86.31	9.46	2.59	23.21
SMGD	80.08	22.63	62.27	77.48	5.10	2.97	25.24
PAR	73.94	23.02	58.88	70.40	6.86	3.19	27.86
WorldGen	77.07	23.01	48.96	60.71	4.78	3.31	29.42
Matrix-3D	78.01	24.19	53.76	51.78	3.05	3.16	30.53

Table 1. Quantitative comparison on PanoGround.

Figure 3. Qualitative comparison on challenging panoramic scenes with multi-object directional descriptions. Captions are derived from the ground-truth panoramas, and colored phrases correspond to the colored boxes in the images. Compared with prior methods, Ours achieves more accurate directional grounding and more coherent layouts.

Model	Parse	Attn	Spa	OPR $\uparrow$	RTA $\uparrow$	SLE $\downarrow$	FID $\downarrow$	CS $\uparrow$
Baseline				82.47	24.35	44.82	50.94	27.83
I	✓			88.94	27.12	42.63	49.85	29.41
II	✓		✓	91.32	29.76	38.18	50.82	28.58
III	✓	✓		93.87	33.41	30.12	47.96	31.67
Ours	✓	✓	✓	98.59	36.91	25.34	46.86	32.54

Table 2. Ablation study of the main components in PanoCtrl .

Setting	OPR $\uparrow$	RTA $\uparrow$	SLE $\downarrow$	FID $\downarrow$	CS $\uparrow$
w/o $\mathcal{L}_{\text{parse}}$	90.26	32.84	31.92	47.28	31.42
w/o $\mathcal{L}_{\text{obj}}$	96.85	35.96	27.43	47.02	32.06
Ours	98.59	36.91	25.34	46.86	32.54

Table 3. Ablation study of the training objectives.

$\alpha$	OPR $\uparrow$	RTA $\uparrow$	SLE $\downarrow$	FID $\downarrow$	CS $\uparrow$
0.3	95.12	34.04	33.82	47.91	31.66
0.5	96.67	34.78	29.96	47.08	31.84
0.7	98.59	36.91	25.34	46.86	32.54
0.9	96.88	35.12	27.41	47.52	31.97

Table 4. Ablation study of the spatial blending coefficient

\alpha

Queries	OPR $\uparrow$	RTA $\uparrow$	SLE $\downarrow$	FID $\downarrow$	CS $\uparrow$
2	94.12	33.21	37.84	47.86	31.02
4	96.43	34.98	31.62	47.22	31.74
8	98.59	36.91	25.34	46.86	32.54
16	98.48	36.02	27.91	46.65	32.81

Table 5. Ablation study of the number of object queries.

Controllable Image Generation

Controllable image generation steers images using spatial, visual, semantic, stylistic, or physical constraints beyond ordinary text prompts.

See all 216 papers

Enhancing In-context Panoramic Generation via Geometric-aware Pretraining
Jul 9, 2026Haoran Feng, Ruiyang Zhang, Longyi Zhang +2Conditional Image Generation Image Generation
Insta360 Research · Tsinghua University · Beihang University +1
PanoWorld: Towards Spatial Supersensing in 360 $^\circ$ Panorama World
May 13, 2026Changpeng Wang, Xin Lin, Junhan Liu +53D Spatial Reasoning 3D Scene Understanding
1Zhejiang University · University of California, San Diego · University of California, Irvine +1
Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation
Mar 14, 2026Stefan Ainetter, Thomas Deixelberger, Edoardo A. Dominici +3Diffusion Model Guidance 3D Scene Generation
Huawei Technologies, Austria · Graz University of Technology, Austria · Huawei Technologies, Switzerland

Jul 9, 2026J/K move · Enter open · S save

Enhancing In-context Panoramic Generation via Geometric-aware Pretraining

Haoran Feng, Ruiyang Zhang, Longyi Zhang, Dizhe Zhang+1

Insta360 Research · Tsinghua University · Beihang University+1

Conditional Image Generation Image Generation Image Editing

Open paper AskPDF arXiv

In this work, we present Canvas360, a two-stage framework for in-context panoramic generation that combines geometry-aware pretraining with downstream task-specific fine-tuning. To address the lack of large-scale, high-quality training data tailored to in-context panoramic tasks, we propose Canvas360Dataset, a collection of 1M high-quality paired panoramic samples for style transfer, inpainting, outpainting, and editing, enabling effective supervision across diverse in-context generation scenarios. On the modeling side, Canvas360 enhances text-to-panorama generation through parallel depth generation, velocity circular padding, and similarity loss regularization, enabling the model to learn geometry-aware representations, capture object distortion details, and improve geometric consistency and global coherence. Furthermore, empowered by strong panoramic priors, Canvas360 enables a unified in-context panoramic generation framework that supports diverse downstream tasks via token-level concatenation, surpassing prior methods in both task coverage and modeling flexibility. Extensive experiments show that Canvas360 improves panoramic image fidelity, achieving particularly strong performance on the panorama-specific FAED metric and competitive or leading results across the reported quantitative evaluations. More information can be found on our project page: https://zry000.github.io/Canvas360/

Abstract

Controllable Image Generation

Explore similar work

Enhancing In-context Panoramic Generation via Geometric-aware Pretraining

PanoWorld: Towards Spatial Supersensing in 360∘^\circ∘ Panorama World

Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation

PanoWorld: Towards Spatial Supersensing in 360 $^\circ$ Panorama World