Image Generation with a Sphere Encoder

Kaiyu Yue; Menglin Jia; Ji Hou; Tom Goldstein

cs.CVFeb 16, 2026

Image Generation with a Sphere Encoder

Authors: Kaiyu Yue, Menglin Jia, Ji Hou, Tom Goldstein

Organizations: Meta · University of Maryland.

Abstract

We introduce the Sphere Encoder, an efficient generative framework capable of producing images in a single forward pass and competing with many-step diffusion models using fewer than five steps. Our approach works by learning an encoder that maps natural images uniformly onto a spherical latent space, and a decoder that maps random latent vectors back to the image space. Trained solely through image reconstruction losses, the model generates an image by simply decoding a random point on the sphere. Our architecture naturally supports conditional generation, and looping the encoder/decoder a few times can further enhance image quality. Across several datasets, the sphere encoder approach yields performance competitive with state of the art diffusions, but with a small fraction of the inference cost. Project page is available at https://sphere-encoder.github.io .

Figures & tables

Figure 1: Selected images generated by the Sphere Encoder in one-step for CIFAR-10 ( $32\times 32$ ) and Animal-Faces, two-steps for Oxford-Flowers, and four-steps for ImageNet ( $256\times 256$ ).

Figure 2: A sphere encoder $E$ maps the natural image distribution uniformly onto a global sphere $S$ . The decoder $D$ then generates a realistic image by decoding a random point on the sphere.

Figure 3: Posterior hole problem in VAEs . Columns: (1) Input images; (2) Autoencoder reconstructions; (3) Samples from standard Gaussian prior; and (4) Samples from estimated Gaussian posterior on Animal-Faces training set. Unlike modern FLUX.1/2 ( Labs et al., 2025 ) and SD-VAE ( Podell et al., 2024 ) , our sphere encoder produces realistic images by decoding random points sampled from the sphere.

Figure 4: Spherifying latent with noise . Encoder $E$ maps image $\mathbf{x}$ to a latent, which $f$ projects to $\mathbf{v}$ on sphere $S$ . During training, random Gaussian noise $\sigma\cdot\mathbf{e}$ is added to $\mathbf{v}$ , where $\sigma$ is jittered magnitude. Decoder $D$ reconstructs the image $\hat{\mathbf{x}}$ from the re-projected noisy latent $f(\mathbf{v}+\sigma\cdot\mathbf{e})$ .

method	steps	rFID $\downarrow$	gFID $\downarrow$	IS $\uparrow$
conditional generation w/o cfg
sphere-l	1	0.59	18.68	9.1
	2	-	8.28	9.9
	4	-	2.72	10.5
	6	-	1.65	10.7
stylegan 2 ( Karras et al., 2020 )	1	-	6.96	9.53

Table 1: Few-step generation results on CIFAR-10.

Figure 5: Uncurated CIFAR-10 conditional generation with different sampling steps and with/without CFG. Convincing images can be formed with a single forward pass, with reliability and gFID improving with up to 4 steps.

sphere-l model $\diagdown$ steps	1	2	4	6
of w/ cfg=1.6	25.12	14.08	11.25	10.63
of	27.12	16.60	12.98	12.26
af	21.70	19.29	18.23	17.97
stylegan 2 ( Karras et al., 2020 )
af - cat	5.13	-	-	-
af - dog	19.37	-	-	-

Table 2: Few-step generation results (gFID

\downarrow

) on Animal-Faces (AF) and Oxford-Flowers (OF).

Figure 6: Latent interpolation on Animal-Faces and Oxford-Flowers. Images are generated in 4 steps without CFG. (left) Interpolation in a 2D space that spans 4 synthetic images. (right) Each row interpolates between a random vector $\mathbf{e}$ on the sphere and a class conditional vector $\mathbf{y}$ . Note that our model exhibits fast/sudden transitions between image classes rather than producing “hybrid” images that unrealistically merge properties of different object types. This property is necessary for a model to reliably convert random samples from the sphere into realistic images, as it makes the probability of observing a hybrid image small.

	method	params	steps	gFID $\downarrow$	IS $\uparrow$
gans	biggan-deep ( 4 )	$56$ M	1	6.90	171.4
	stylegan-xl ( 67 )	$166$ M	1	2.30	265.1
	gigagan ( 34 )	$569$ M	1	3.45	225.5
diffusions	adm-g ( 13 )	$554$ M	250	4.59	186.7
	adm-g	$554$ M	25	5.44	-
	sid ( 29 )	$2$ B	1000	2.44	256.3

Table 3: Few-step generation results on ImageNet

256\times 256

Figure 7: Latent space visualization using random projection on CIFAR-10 training set. Each sphere shows the latent vectors of a different class. The conditional latent distributions appear approximately uniform.

Figure 8: Conditional manipulation via iterative encoding and decoding on ImageNet model. We demonstrate the model’s expressivity using an out-of-domain input (a “woolly panda” top-left). Each row shows the result of conditioning the iterative process on different ImageNet classes without CFG. For example, the first row is conditioned on class 580 (greenhouse, nursery, glasshouse).

Figure 9: Image crossover using the sphere encoder trained on Animal-Faces. A composite of two images (A and B) is iteratively processed through the encoder-decoder pipeline until it converges to a coherent sample on the learned image manifold.

Panel of Figure 10 — Figure 10: Quantitative impact of the angle $\alpha$ on ImageNet.

loss $\mathcal{L}$	steps	rFID $\downarrow$	gFID $\downarrow$	IS $\uparrow$	gFID $\downarrow$	IS $\uparrow$
			w/o cfg		w/ cfg $=1.6$
$\mathcal{L}_{\text{pix-recon}}$	1	1.70	25.35	168.8	20.68	204.4
	4	-	13.58	162.9	11.76	231.7
+ $\mathcal{L}_{\text{pix-con}}$	1	0.89	17.19	183.8	14.74	218.3
	4	-	10.12	189.9	10.47	247.1
+ $\mathcal{L}_{\text{lat-con}}$	1	1.32	15.40	188.3	13.14	225.1

Table 4: Ablation of loss terms on ImageNet. Loss terms are added incrementally from top to bottom.

Figure 12: Consistency optimization path from noisy latent to clean latent on the sphere, pushing the decoder to generate consistent and diverse images from right to left.

volume compression ratio $=1.5$
			w/o cfg		w/ cfg $=1.2$
latent $L$	steps	rFID $\downarrow$	gFID $\downarrow$	IS $\uparrow$	gFID $\downarrow$	IS $\uparrow$
$16^{2}\times 512$	1	0.64	16.06	169.8	14.25	187.6
	4	-	10.03	153.0	8.71	180.8
$32^{2}\times 128$	1	0.60	20.63	145.1	17.69	164.9
	4	-	15.45	127.0	12.75	154.6

Table 5: Ablation on latent spatial resolution with two optimal compression ratios on ImageNet.

$\gamma$	share $\mathbf{e}$	1	2	4	6	8	10
0	-	16.57	7.99	7.68	7.49	7.61	7.71
	✓	16.66	7.97	5.99	7.78	10.11	12.25
1	-	16.73	8.12	9.02	12.17	15.08	18.09
	✓	16.81	8.02	8.53	12.32	17.50	22.78

Table 6: Ablation on sampling schemes with few-step generation and no CFG. Results are reported with gFID

\downarrow

on ImageNet.

Figure 13: Qualitative impact of sampling schemes on generation without CFG for CIFAR-10 (top) and ImageNet (bottom). The $\checkmark$ indicates sharing the same noise $\mathbf{e}$ across steps. Shared noise with $\gamma=1$ yields superior coherence and a sharp “paper art” aesthetic on ImageNet as sampling steps increase.

Figure 14: Randomly selected images generated by the Sphere encoder for ImageNet ( $256\times 256$ ). Results are generated using Sphere-XL with 4-step sampling and CFG $=1.4$ .

Figure 15: Uncurated results on ImageNet ( $256\times 256$ ). Results are generated using Sphere-XL with 4-step sampling and CFG $=1.4$ .

Figure 16: Uncurated results on ImageNet ( $256\times 256$ ). Results are generated using Sphere-XL with 4-step sampling and CFG $=1.4$ .

Figure 17: Uncurated qualitative results on CIFAR-10 ( $32\times 32$ ), Oxford-Flowers and Animal-Faces ( $256\times 256$ ). Results are 2-step generation without CFG.

Figure 18: Uncurated qualitative results on CIFAR-10 ( $32\times 32$ ), Oxford-Flowers and Animal-Faces ( $256\times 256$ ). Results are 4-step generation without CFG.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 19: Memorization risk with longer $10$ K training epochs on CIFAR-10. Each row is a different sampling run showing near-duplicate birds of the real training image (in red box).

without CFG
method	steps	rFID $\downarrow$	gFID $\downarrow$	IS $\uparrow$
sphere-l	1	0.68	16.02	9.5
	2	-	4.92	10.3
	4	-	1.24	10.7
	6	-	1.01	10.9
	8	-	1.01	10.8

Appendix

Table 7: Few-step conditional generation results on CIFAR-10 with longer

10

K training epochs.

method	cfg	steps	rFID $\downarrow$	gFID $\downarrow$	IS $\uparrow$
sphere-l	1.2	1	0.59	18.82	9.1
		2	-	8.41	10.0
		4	-	2.53	10.6
		6	-	1.41	10.8
		8	-	1.18	10.8
		10	-	1.12	10.9

Appendix

Table 8: Few-step generation results on CIFAR-10 with CFG.

	steps w/ cfg $=1.0$					steps w/ cfg $=1.2$					steps w/ cfg $=1.4$					steps w/ cfg $=1.6$
cfg position	1	2	4	6	8	1	2	4	6	8	1	2	4	6	8	1	2	4	6	8
enc	16.7	7.9	6.0	7.9	10.0	16.7	8.0	6.1	7.8	10.3	16.8	7.9	6.1	7.9	10.5	16.8	7.9	6.11	8.1	10.7
dec	16.7	8.0	5.9	7.8	10.1	15.0	8.3	4.7	5.1	6.2	15.4	9.3	5.0	4.5	5.0	16.4	10.2	5.7	4.7	4.8
combo	16.7	7.9	6.0	7.8	10.2	15.3	8.0	5.1	6.1	7.9	15.1	8.2	4.7	5.2	6.5	15.0	8.6	4.7	4.7	5.7

Appendix

Table 9: Ablation on CFG position . Results are reported with gFID

\downarrow

on ImageNet. The sampling scheme is the fixed strength with sharing noise

\mathbf{e}

in spheriying process across all steps.

			w/o cfg		w/ cfg $=1.6$
base max $\alpha$	mix range	steps	gFID $\downarrow$	IS $\uparrow$	gFID $\downarrow$	IS $\uparrow$
$80^{\circ}$	-	1	21.17	8.8	18.93	9.3
	-	4	8.48	10.0	7.47	10.1
$80^{\circ}$	$[80^{\circ},85^{\circ}]$	1	19.82	8.7	18.39	9.1
		4	7.98	10.1	7.05	10.2
$80^{\circ}$	$[80^{\circ},89^{\circ}]$	1	29.60	8.8	30.23	9.2

Appendix

Table 10: Impact of mixing a range of larger angles during training on CIFAR-10. All settings yield rFID scores in the range of

0.46\text{--}0.48

add-ons	steps	rFID $\downarrow$	gFID $\downarrow$	IS $\uparrow$	gFID $\downarrow$	IS $\uparrow$
			w/o cfg		w/ cfg $=1.6$
-	1	1.20	17.56	182.5	14.88	217.8
	4	-	8.73	177.6	8.05	245.0
+ BN	1	0.94	17.28	181.6	14.71	216.0
	4	-	8.39	171.3	7.65	240.3
+ $\mathcal{L}_{\text{SWD}}$	1	1.01	18.95	164.4	15.47	202.1

Appendix

Table 11: Ablation on explicit uniform regularization on the spherical latent space.

calib by	steps	rFID $\downarrow$	gFID $\downarrow$	IS $\uparrow$	gFID $\downarrow$	IS $\uparrow$
			w/o cfg		w/ cfg $=1.6$
on-the-fly	1	0.99	14.21	214.2	13.02	246.0
	4	-	6.85	202.3	7.39	268.2
generated	1	0.99	14.21	214.2	13.02	246.0
images	4	-	7.30	211.1	8.08	273.9
reference	1	1.00	14.21	214.2	13.02	246.0

Appendix

Table 12: Recalibrating BN stats for inference on ImageNet.

Figure 20: Quantitative impact of volume compression ratio on ImageNet. Details in Table 13 .

compression ratio	latent $L$	steps	rFID $\downarrow$	gFID $\downarrow$	IS $\uparrow$
24	$16^{2}\times 32$	1	1.61	27.99	195.6
		4	-	15.34	245.1
12	$16^{2}\times 64$	1	1.38	20.82	233.8
		4	-	10.28	259.0
6.0	$16^{2}\times 128$	1	1.15	17.26	225.0
		4	-	7.72	229.4

Appendix

Table 13: Ablation on pixel/latent volume compression ratio on ImageNet without CFG.

trunc $\alpha$	steps	gFID $\downarrow$	IS $\uparrow$	gFID $\downarrow$	IS $\uparrow$
		w/o cfg		w/ cfg $=1.2$
-	1	17.81	233.9	16.30	261.0
	4	5.92	192.3	5.02	229.7
0.05	1	17.89	231.7	15.81	277.0
	4	5.88	192.2	5.08	228.2
0.5	1	17.96	231.4	15.73	275.8

Appendix

Table 14: Ablation of noise prior distribution on ImageNet.

				w/o cfg		w/ cfg $=1.6$
angle $\alpha$	$\sigma$	steps	rFID $\downarrow$	gFID $\downarrow$	IS $\uparrow$	gFID $\downarrow$	IS $\uparrow$
$45^{\circ}$	1.0	1	0.69	69.57	31.3	59.09	39.9
		4	-	80.32	18.7	73.04	21.9
$80^{\circ}$	5.7	1	0.87	60.14	33.4	51.73	41.3
		4	-	42.72	39.3	34.14	54.3
$81^{\circ}$	6.3	1	0.94	47.68	40.4	40.11	51.1

Appendix

Table 15: Detailed quantitative results for the impact of the angle

\alpha

on ImageNet for Figure 11 .

hyperparameter $\diagdown$ dataset	cifar-10	animal-faces	oxford-flowers	imagenet
image size	$32$	$256$	$256$	$256$
batch size	$256$	$256$	$256$	$256$
learning rate (lr)	$1\times 10^{-4}$	$1\times 10^{-4}$	$1\times 10^{-4}$	$1\times 10^{-4}$
lr decay schedule	cosine	cosine	cosine	cosine
min lr	$1\times 10^{-6}$	$1\times 10^{-6}$	$1\times 10^{-6}$	$1\times 10^{-6}$
weight decay	$0.0$	$0.0$	$0.0$	$0.0$

Appendix

Table 16: Training hyperparameters and model details for main experiments.

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 463 papers

Sphere Encoder 2
Oct 1, 2026Kaiyu Yue, Sean McLeish, Ruchit Rawal +3Autoencoders Image Generation
University of Maryland · Lawrence Livermore National Laboratory · Cornell University
Efficient Image Synthesis with Sphere Latent Encoder
May 15, 2026Tung Do, Thuan Hoang Nguyen, Hao LiLatent Diffusion Models Few-Step Diffusion Sampling
Mohamed Bin Zayed University of Artificial Intelligence, UAE
Dancing Diffusion: Learning Continuous Programs for Image Generation and Editing
Oct 20, 2026Cui Enhe, Jinyue Yang, Bojian Yin +1Diffusion-Based Image Generation Diffusion-Based Image Editing
Renmin University of China · Institute of Automation, Chinese Academy of Sciences

Oct 1, 2026J/K move · Enter open · S save

Sphere Encoder 2

Kaiyu Yue, Sean McLeish, Ruchit Rawal, Brian Bartoldson+2

University of Maryland · Lawrence Livermore National Laboratory · Cornell University

Autoencoders Image Generation Hyperspherical Representation Learning One-Step Generative Modeling

Open paper AskPDF arXiv

Sphere Encoder is an autoencoder that generates images by decoding random points from a high-dimensional latent sphere. We identify two limitations of the original formulation that reduce its generation quality. First, random points concentrate near the equator relative to the pole on an encoded latent, but the training rotation never reaches this region, leaving a gap that limits one-step generation. Second, training for generation with pixel-wise reconstruction loss encourages the decoder to average over plausible images, producing blurry images that lack high-frequency details. We present Sphere Encoder 2 to address both limitations, substantially improving image generation quality while maintaining the speed and simplicity of a autoencoder. Models are released at https://github.com/kaiyuyue/sphere2.

Abstract

Image Generation

Explore similar work

Sphere Encoder 2

Efficient Image Synthesis with Sphere Latent Encoder

Dancing Diffusion: Learning Continuous Programs for Image Generation and Editing