TextSphere: Efficient Text-to-Image Generation via Shared Hyperspherical Alignment and Uniformity
Authors: Yixia Zhang, Girish A. Koushik, Xilu Wang, Wenwu Wang
Organizations: Department of Computer Science, University of Surrey, Guildford, United Kingdom · CVSSP, University of Surrey, Guildford, United Kingdom
Recent advances in text-to-image generation have substantially improved image quality, particularly through diffusion-based methods that require multiple inference steps. In contrast, Sphere Encoder offers an efficient generative framework capable of producing images with fewer forward steps. However, extending it to free-form text is nontrivial, because its class-conditioned design lacks cross-modal alignment. To address this challenge, we propose TextSphere, an efficient text-to-image generative framework that extends Sphere Encoder with language conditioning and cross-modal alignment. We incorporate pretrained CLIP text embeddings through adaptive layer normalization (AdaLN) to guide image generation while retaining the original spherical sampling mechanism. To bridge the cross-modal representation gap, we introduce a Spherical Semantic Alignment and Uniformity (SSAU) objective that jointly promotes semantic alignment and representation uniformity on the hypersphere. Experiments on CUB-200-2011, MS-COCO-2014, and Flickr30k demonstrate that TextSphere consistently reduces generation FID (gFID) over the text-conditioned baseline while achieving reconstruction FID (rFID) below 1.0. With only 4 and 6 sampling steps, TextSphere achieves lower gFID than the compared diffusion- and GAN-based models trained from scratch on CUB-200-2011 and MS-COCO-2014, respectively. We further observe that improved reconstruction fidelity does not necessarily yield better generation quality. In addition, we use Partial Least Squares Singular Value Decomposition (PLS-SVD) to analyze cross-modal alignment and modality gap, offering insight into the effectiveness of our approach.
Figures & tables
Figure 2: TextSphere Framework Overview. It consists of a frozen pretrained CLIP text encoder, an image encoder, and an image decoder. During training, the image encoder maps ground-truth images onto a spherical latent space via RMS normalization (Spherify), as defined in Eq. ( 1 ), while the CLIP text encoder extracts semantic embeddings t from the corresponding captions. The training objectives jointly optimize (1) contrastive alignment, (2) uniformity regularization, and (3) visual fidelity, with the decoder sampling random latent representations from the hypersphere to generate images.
Figure 3: Inference pipeline of TextSphere.
CUB-200-2011
MS-COCO-2014
Flickr30k
rFID ↓
rIS ↑
rFID ↓
rIS ↑
rFID ↓
rIS ↑
0.92
5.86
0.78
36.25
0.82
24.17
Table 1: Quantitative reconstruction performance across three datasets.
Dataset
CFG
Steps
gFID ↓
gIS ↑
CLIPScore ↑
CUB-200-2011
9.0
16
6.82
5.34
24.74
MS-COCO-2014
2.0
6
7.11
28.67
24.11
Flickr30k
3.0
8
13.86
17.20
23.65
Table 2: Best-performing configurations for each dataset in terms of gFID. More details are provided in Fig. 7 in the Supplement.
Methods
CUB-200-2011
MS-COCO-2014
Steps
gFID ↓
gIS ↑
Steps
gFID ↓
gIS ↑
Improved VQ-Diffusion [ 44 ]
100
10.21
–
100
8.44
–
VQ-Diffusion [ 7 ]
100
10.32
–
100
13.86
–
TCTS+FAS [ 18 ]
16
12.35
–
16
14.45
–
UNIFIED DISCRETE [ 14 ]
500
16.19
6.02
500
25.11
21.03
Ours
4/16
8.03/6.82
5.38/5.34
6
7.11
28.67
Table 3: Comparison with diffusion-based methods trained from scratch.
Figure 4: Qualitative comparison with prior methods using publicly available checkpoints on CUB-200-2011 (left) and MS-COCO-2014 (right).
Methods
CUB-200-2011
MSCOCO2014
gFID ↓
gIS ↑
gFID ↓
gIS ↑
HyperCGAN [ 9 ]
11.00
–
18.58
–
ALR-GAN [ 43 ]
15.14
4.96
29.04
34.70
DFGAN [ 46 ]
14.81
5.10
19.32
–
LAFITE [ 58 ]
10.48
5.97
8.12
32.34
RAT-GAN [ 52 ]
13.91
5.36
14.60
–
Table 4: Comparison with GAN-based methods trained from scratch.
Figure 5: PLS-SVD analysis of the text-conditioned Sphere Encoder with the proposed alignment loss LSSAU on the CUB-200-2011 dataset.
Figure 6: Reconstructed images across three datasets.
Figure 7: Effect of CFG scale and iterative refinement steps on text-conditioned Sphere Encoder across three datasets. Left: gFID ( ↓ ) and gIS ( ↑ ) under varying CFG scales with the number of forward steps fixed at 4. Right: gFID and gIS across different refinement steps (4 − 16, interval 2), using the dataset-specific CFG scales that achieve the lowest gFID, i.e. , best CFG=9.0 on CUB, CFG=2.0 on COCO, CFG=3.0 on Flickr30k.
λu
CUB
COCO
Flickr
rFID
rIS
rFID
rIS
rFID
rIS
0.2
0.92
5.86
0.78
36.25
0.82
24.17
0.1
2.11
5.80
0.89
36.29
0.98
24.12
Table 5: Reconstruction results of rFID and rIS using different uniformity weights across three datasets.
Figure 8: Generation results of gFID and gIS using different uniformity weights across three datasets. w0.2 and w0.1 use uniformity weights of 0.2 and 0.1, respectively. Left: gFID and gIS comparisons under varying CFG scales with the number of forward steps at 4. Right: gFID and gIS across different refinement steps (4-16, interval 2), using the dataset-specific CFG scales that achieve the lowest gFID, i.e. , best CFG=9.0 on CUB, CFG=2.0 on COCO, CFG=3.0 on Flickr30K.
Reconstruction Results (CFG/Step Independent)
Metric
CA
AdaLN
rFID ↓
1.76
0.77
rIS ↑
5.77
5.89
Generation Results (fixed steps=4)
CFG
gFID ↓
gIS ↑
CLIPScore ↑
CA
AdaLN
CA
AdaLN
CA
AdaLN
Table 6: Quantitative comparisons using Cross-Attention (CA) vs. AdaLN for text conditioning on CUB-200-2011 dataset. Top: reconstruction results. Middle: varying CFG scales with fixed forward steps=4. Bottom: varying forward steps with best CFG scale=10.0.
Reconstruction Results (CFG/Step Independent)
Metric
LSphere+λSSAULSSAU (Ours)
LSphere (Original)
rFID ↓
0.92
0.77
rIS ↑
5.86
5.89
Generation Results (fixed steps=4)
CFG
gFID ↓
gIS ↑
CLIPScore ↑
LSphere+λSSAULSSAU
LSphere
LSphere+λSSAULSSAU
LSphere
LSphere+λSSAULSSAU
LSphere
Table 7: Ablation study of the proposed SSAU objective on the CUB-200-2011 dataset.
Reconstruction Results (CFG/Step Independent)
Metric
LSphere+λSSAULSSAU (Ours)
LSphere (Original)
rFID ↓
0.78
0.78
rIS ↑
36.25
36.24
Generation Results (fixed steps=4)
CFG
gFID ↓
gIS ↑
CLIPScore ↑
LSphere+λSSAULSSAU
LSphere
LSphere+λSSAULSSAU
LSphere
LSphere+λSSAULSSAU
LSphere
Table 8: Ablation study of the proposed SSAU objective on the MS-COCO-2014 dataset.
Reconstruction Results (CFG/Step Independent)
Metric
LSphere+λSSAULSSAU (Ours)
LSphere (Original)
rFID ↓
0.82
0.73
rIS ↑
24.17
24.38
Generation Results (fixed steps=4)
CFG
gFID ↓
gIS ↑
CLIPScore ↑
LSphere+λSSAULSSAU
LSphere
LSphere+λSSAULSSAU
LSphere
LSphere+λSSAULSSAU
LSphere
Table 9: Ablation study of the proposed SSAU objective on the Flickr30k dataset.
Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.