DSS-GAN: Directional State Space GAN with Mamba backbone for Class-Conditional Image Synthesis

Aleksander Ogonowski; Konrad Klimaszewski; Przemysław Rokita

cs.LGMar 18, 2026

DSS-GAN: Directional State Space GAN with Mamba backbone for Class-Conditional Image Synthesis

Authors: Aleksander Ogonowski, Konrad Klimaszewski, Przemysław Rokita

Organizations: Warsaw University of Technology · National Centre for Nuclear Research

Abstract

We present DSS-GAN, the first generative adversarial network to employ Mamba as a hierarchical generator backbone for noise-to-image synthesis. The central contribution is Directional Latent Routing (DLR), a novel conditioning mechanism that decomposes the latent vector into direction-specific subvectors, each jointly projected with a class embedding to produce a feature-wise affine modulation of the corresponding Mamba scan. Unlike conventional class conditioning that injects a global signal, DLR couples class identity and latent structure along distinct spatial axes of the feature map, applied consistently across all generative scales. DSS-GAN achieves improved FID, KID, and precision-recall scores compared to StyleGAN2-ADA across multiple tested datasets. Analysis of the latent space reveals that directional subvectors exhibit measurable specialization: perturbations along individual components produce structured, direction-correlated changes in the synthesized image.

Figures & tables

Figure 1: DSS-GAN generator architecture (Only two scan directions shown for clarity). $Z_{base}$ is base global latent vector, $z^{k_{K}}_{dir}$ are directional latent vectors. For $256\times 256$ resolution $N$ is $5$ and $C_{0}$ - feature map channels in DLR blocks is $148$ .

Block	Resolution	$\mathbf{z}_{\text{base}}$	$\mathbf{z}_{\text{dir}}$	$y$
Tokenizer	$8{\times}8$	seed	—	additive bias
DLR	$8{\times}8\to 256{\times}256$ (×6)	—	shared, per-direction	shared, per-direction
Conv	$512{\times}512$	—	—	global
toRGB	$512{\times}512$	—	—	—

Table 1: Generator routing summary for the DSS-GAN blocks: Tokenizer ( 3.1 ), DLR ( 3.2 ), Conv and toRGB ( 3.2.4 ). The

\mathbf{z}_{\text{base}}

seeds the tokenizer only;

\mathbf{z}_{\text{dir}}

and

y

are re-injected unchanged at every DLR stage.

	Global						Mean Class
Stylegan2
refinement
from	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$
$None$	21.75	5.26	.815	.242	1.078	.577	27.35	13.86	.814	.188	1.031	.563
$128{\times}128$	11.66	2.71	.822	.407	1.144	.748	12.93	5.67	.826	.394	1.125	.744
$64{\times}64$	18.48	8.27	.574	.264	0.357	.388	20.75	13.45	.541	.264	0.322	.364

Table 2: Ablation: effect of Stylegan2 CNN refinement block boundary resolution on AFHQ ( cat, dog, wild )

128{\times}128

. All variants share the same training configuration.

Figure 2: DLR block. For each scan direction $k$ , the directional latent $\mathbf{z}_{\text{dir}}^{k}$ and class embedding $\mathbf{e}_{k}$ are projected to affine parameters $(\boldsymbol{\gamma}_{k},\boldsymbol{\beta}_{k})$ that modulate the token sequence before the Mamba SSM. Both the class and the directional latent determine the contribution of each scan direction to the generated feature map, with directional weights $w^{\text{dir}}_{k}=f(y,z^{\text{dir}}_{k})$ satisfying $\sum_{k=1}^{K}w_{\text{dir}}^{k}=1$ . A random $180^{\circ}$ rotation is applied before the block and inverted after unscan.

Mamba Block	mean norm	CV	mean norm	CV	ratio $\uparrow$
	None		$\{0^{\circ},180^{\circ}\}$ (ours)
Tokenizer	0.02947	0.121	0.02713	0.118	0.921 $\times$
$8{\times}8$	0.01911	0.156	0.01862	0.205	0.974 $\times$
$16{\times}16$	0.00876	0.231	0.00955	0.254	1.091 $\times$
$32{\times}32$	0.00346	0.239	0.00427	0.230	1.233 $\times$
$64{\times}64$	0.00130	0.193	0.00160	0.215	1.232 $\times$

Table 3: Ablation: effect of

180^{\circ}

rotation on AFHQ

128{\times}128

(top) and mean gradient norm per Mamba block averaged over all epochs (bottom).

Figure 3: Per-direction feature maps (mean absolute activation, averaged over 10 channels) at each resolution stage, for a representative sample from the LSUN tower class (from bridge, church, tower classes ) in Mamba blocks. Each row corresponds to one scan direction; each column to one resolution stage. At low resolutions directions capture complementary aspects of global structure; at higher resolutions the directional geometry becomes explicit, with row, column, and diagonal activations forming spatially oriented patterns consistent with their respective scan axes.

Figure 4: Evolution of per-direction routing weights across resolution stages during training (AFHQ $256{\times}256$ , 3-direction model). Each line corresponds to a training snapshot; colour indicates epoch (light = early, dark = late). The dashed red line marks the uniform weight $1/K$ . Arrows indicate the direction of change between the first and last snapshot. Weights diverge from uniform and specialise by resolution, with the column scan dominating at $16{\times}16$ and the diagonal scan peaking at $32{\times}32$ .

	General			Mamba Stages					Mamba Directions
Factor	All dir	Base	CNN	$8{\times}8$	$16{\times}16$	$32{\times}32$	$64{\times}64$	$128{\times}128$	Col	Row	Diag
Class	0.676	0.044	0.047	0.612	0.464	0.190	0.149	0.144	0.484	0.455	0.485
Latent	0.682	0.621	—	0.628	0.496	0.091	0.322	0.290	0.608	0.591	0.573

Table 4: Sensitivity analysis: Perceptual impact (LPIPS

\uparrow

) of Class vs. Latent variations across model components.

Panel of Figure 5 — Figure 5: The same latent vector $\mathbf{z}$ generated with three different class labels. The spatial layout is preserved across classes, confirming that $\mathbf{z}_{\text{base}}$ controls composition independently of class identity. Left: AFHQ $256\times 256$ , right : LSUN $256\times 256$ .

Panel of Figure 6 — Figure 6: Partial class override experiments on AFHQ $256\times 256$ . Top: base class cat , target class dog . Bottom: base class cat , target class wild . Each column replaces a different subset of per-direction classes with the target class, keeping $\mathbf{z}$ fixed. The row direction shows little influence on class conversion; the majority of class information is modulated by the blocks at low resolutions.

Figure 7: Swapping directional subvectors $\mathbf{z}_{\text{dir}}^{k}$ between samples. Each column replaces the subvector of one direction while keeping all other components of $\mathbf{z}$ fixed.

Panel of Figure 8 — Figure 8: Effect of noise perturbation on latent space components. Left: perturbation of the column direction ( $\mathbf{z}_{\mathrm{dir}}^{\mathrm{col}}$ ). Right: perturbation of the tokenizer component ( $\mathbf{z}_{\mathrm{base}}$ ).

	Global						Bedroom						Kitchen
Model	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$
StyleGAN2-ADA	20.31	10.10	.59	.25	.94	.65	26.06	11.51	.58	.24	.85	.61	26.64	12.70	.62	.27	.97	.58
DSS-GAN 3-dir	23.22	13.23	.52	.28	.62	.67	25.16	13.45	.52	.28	.62	.67	31.29	20.28	.52	.28	.62	.67
DSS-GAN 2-dir (row + column)	17.79	8.79	.64	.20	.88	.75	21.56	9.61	.59	.26	.78	.76	23.32	12.30	.71	.18	.97	.78
DSS-GAN 1-dir (row)	20.28	11.93	.55	.30	.66	.73	21.39	10.06	.55	.35	.69	.79	28.78	19.35	.59	.27	.62	.72
DSS-GAN 1-dir (column)	20.30	10.73	.58	.23	.65	.68	24.97	11.95	.58	.24	.69	.69	25.83	14.43	.60	.18	.70	.75

Table 5: Results on LSUN Rooms

128{\times}128

(classes: Bedroom, Kitchen).

	Global						Mean class
Model	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$
StyleGAN2-ADA	15.63	10.91	.56	.38	.55	.68	14.30	10.28	.59	.33	.53	.70
DSS-GAN 1-dir	12.42	6.17	.69	.56	.85	.83	11.05	7.39	.76	.57	.90	.89
DSS-GAN 3-dir	13.40	7.43	.74	.52	.88	.83	12.19	8.91	.79	.54	.91	.92

Table 6: Results on CelebA

128{\times}128

. Classes: Bald, Black Hair, Blond Hair, Eyeglasses, Goatee, Male, Mustache, Smiling, Wearing Hat, Young (10k samples per class).

	Global						Cat						Dog						Wild
Model	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$
StyleGAN2-ADA	13.16	4.13	.75	.21	.84	.64	11.04	4.70	.78	.14	1.15	.78	33.27	18.49	.73	.25	.55	.43	7.56	1.96	.77	.12	.82	.64
DSS-GAN 1-dir	13.58	3.37	.85	.35	1.14	.67	9.74	4.46	.88	.26	1.41	.87	27.34	14.60	.85	.55	.70	.53	6.07	1.96	.77	.23	1.02	.62
DSS-GAN 2-dir (row+col)	22.50	5.85	.87	.16	1.14	.53	20.09	10.04	.95	.08	1.91	.74	41.37	23.01	.86	.32	.78	.44	19.61	7.94	.73	.06	.66	.38
DSS-GAN 2-dir (row+diag)	22.76	5.17	.89	.17	1.33	.53	20.88	9.35	.94	.10	1.97	.72	40.03	19.03	.91	.25	.90	.45	21.90	8.67	.77	.13	1.02	.41
DSS-GAN 3-dir	10.29	2.39	.88	.36	1.33	.74	8.87	2.65	.91	.18	1.87	.89	26.51	13.05	.83	.46	.73	.51	6.73	1.51	.87	.09	1.74	.77

Table 7: Results on AFHQ

256{\times}256

	Global						Bridge						Church						Tower
Model	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$
StyleGAN2-ADA	10.85	5.39	.72	.39	0.86	0.81	19.23	9.02	.62	.30	0.63	0.69	11.99	5.75	.74	.29	1.25	0.81	15.40	6.50	.68	.33	0.70	0.69
DSS-GAN 3-dir	12.66	3.45	.79	.31	1.08	0.75	22.75	6.65	.70	.20	0.85	0.55	12.11	6.35	.81	.25	1.25	0.75	18.40	5.44	.76	.27	0.89	0.66
DSS-GAN 2-dir	21.79	10.30	.71	.19	0.84	0.65	35.49	20.98	.69	.16	0.70	0.56	19.54	11.00	.78	.14	1.11	0.75	25.60	12.76	.68	.22	0.76	0.65
DSS-GAN 1-dir	22.05	7.83	.72	.25	0.92	0.68	40.10	17.17	.66	.19	0.58	0.54	20.52	8.21	.80	.21	1.35	0.81	28.70	11.31	.71	.29	0.81	0.67

Table 8: Results on LSUN

256{\times}256

(15k samples per class).

	Global
Model	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$
StyleGAN2-ADA	7.64	2.68	.79	.54	1.25	.92
DSS-GAN 3-dir	8.27	2.53	.83	.53	1.39	.93
DSS-GAN 1-dir	8.69	3.06	.80	.52	1.25	.90

Table 9: Results on FFHQ

256{\times}256

	Global						Cat						Dog						Wild
Model	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$
StyleGAN2	11.74	3.06	.86	.22	1.35	.73	9.38	2.55	.86	.17	1.69	.85	27.49	10.40	.85	.25	.80	.51	9.13	2.22	.82	.11	1.30	.76
DSS-GAN 1-dir	17.77	4.91	.80	.28	1.03	.59	12.64	5.14	.90	.16	1.53	.83	39.46	25.34	.77	.43	.59	.41	12.65	3.15	.71	.23	.91	.55
DSS-GAN 3-dir	14.44	4.11	.86	.25	1.18	.64	12.63	4.65	.91	.16	1.56	.83	36.06	22.70	.87	.43	.80	.48	8.81	2.50	.75	.19	1.22	.59

Table 10: Results on AFHQ

512{\times}512

Model	Parameters	Weight size (MB)	Latency (batch 1) $\downarrow$	Peak throughput $\uparrow$
StyleGAN2-ADA	25.0 M	24.7	9.1 ms	1451.6 img/s ( $b{=}128$ )
DSS-GAN (3-dir)	7.3 M	21.3	15.1 ms	280.6 img/s ( $b{=}32$ )
DSS-GAN (1-dir)	4.4 M	13.6	8.1 ms	441.6 img/s ( $b{=}32$ )

Table 11: Computational efficiency and memory footprint at

256{\times}256

resolution, measured on a single NVIDIA H100 GPU.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Generated samples, FFHQ 256 $\times$ 256.

Figure 12: Class embedding override: bridge $\rightarrow$ church. Starting from a base cat image, DLR class embeddings are progressively replaced with church embeddings across individual directions (row, column, diagonal), combinations, early stages only (8 $\times$ 8, 16 $\times$ 16), all DLR directions, and all DLR with CNN refinement block.

Figure 13: Class embedding override: bridge $\rightarrow$ tower with same protocol as Fig. 12 .

Figure 14: Swapping directional subvectors $z^{k}_{dir}$ between samples in LSUN dataset. Each column replaces the values of one direction or block stage while keeping all other components of latent fixed. From left: reference, row, column, diagonal, all directions combined, all directions in DLR early-stage only (8×8, 16×16), and tokenizer $\mathbf{z}_{\mathrm{base}}$ swap.

Panel of Figure 15 — Figure 15: Latent space perturbation analysis (bridge). (a) Tokenizer $\mathbf{z}_{\mathrm{tok}}$ ; (b) column direction $\mathbf{z}_{\mathrm{dir}}^{\mathrm{col}}$ ; (c) diagonal direction $\mathbf{z}_{\mathrm{dir}}^{\mathrm{diag}}$ ; (d) combined column and diagonal. Each panel shows the effect of scaling perturbation magnitude $\gamma$ .

Panel of Figure 16 — Figure 16: Latent space perturbation analysis (church outdoor). (a) Tokenizer $\mathbf{z}_{\mathrm{tok}}$ ; (b) column direction $\mathbf{z}_{\mathrm{dir}}^{\mathrm{col}}$ ; (c) diagonal direction $\mathbf{z}_{\mathrm{dir}}^{\mathrm{diag}}$ ; (d) combined column and diagonal. Each panel shows the effect of scaling perturbation magnitude $\gamma$ .

Panel of Figure 17 — Figure 17: Latent space perturbation analysis (tower). (a) Tokenizer $\mathbf{z}_{\mathrm{tok}}$ ; (b) column direction $\mathbf{z}_{\mathrm{dir}}^{\mathrm{col}}$ ; (c) diagonal direction $\mathbf{z}_{\mathrm{dir}}^{\mathrm{diag}}$ ; (d) combined column and diagonal. Each panel shows the effect of scaling perturbation magnitude $\gamma$ .

	Global						Cat						Dog						Wild
Model	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$	FID $\downarrow$	KID $\downarrow$	P $\uparrow$	R $\uparrow$	D $\uparrow$	C $\uparrow$
DSS-GAN $3{\times}$ same dir	19.20	4.16	.894	.217	1.457	.569	13.97	5.58	.959	.074	1.750	.820	38.26	21.33	.899	.315	.907	.463	20.35	8.20	.814	.133	1.090	.435
DSS-GAN 3-dir	10.29	2.39	.879	.356	1.327	.743	7.88	2.74	.903	.293	1.652	.872	25.15	12.86	.868	.548	.805	.601	6.08	1.35	.877	.150	1.624	.765

Appendix

Table 12: Generation quality comparison between identical and distinct scan direction configurations (AFHQ

256{\times}256

). Best checkpoint per run reported.

Figure 18: Per-direction routing weights across DLR blocks during training (AFHQ $256{\times}256$ ), for the same-direction ( $3{\times}$ row) configuration. Each line corresponds to a training snapshot; colour indicates epoch (light = early, dark = late). The dashed red line marks the uniform weight $1/K{=}0.33$ . Arrows indicate the direction of change from the first to the last snapshot. With identical directions, routing weights remain noisy throughout training and show no convergence toward a stable resolution-dependent pattern, in contrast to the distinct-direction case (Figure 4 ).

Figure 19: Routing diagnostics for

3{\times}

same (red) vs.

3{\times}

distinct (blue) directions (AFHQ

256{\times}256

). From left to right: class-to-routing scale

|\alpha|

with EMA decay switch marked (dotted line); global FID over training epochs with FID at the best checkpoint annotated; R1 penalty with maximum values annotated; L1 loss (EMA) with EMA decay switch marked.

Optimiser and training
Hyperparameter	128	256	512
Adam $(\beta_{1},\beta_{2})$	$(0.0,\ 0.99)$	′′	′′
LR $G$	$9{\times}10^{-5}$	′′	′′
LR $D$	$3{\times}10^{-5}$	′′	′′
Batch size	128	96	48
Regularisation

Appendix

Table 13: Training configuration across resolutions. ′′ denotes identical value to the 128

\times

128 baseline.

Generative Adversarial Networks

Generative Adversarial Networks (GANs) train competing generators and discriminators to synthesize data, translate images, and model complex distributions.

See all 177 papers

Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Model
Jul 11, 2025Fang Chen, Alex Villa, Gongbo Liang +3Diffusion Model Alignment Conditional Image Generation
University of California Merced · Texas A&M University-San Antonio · Oregon State University +1
Coarse to Fine: Iterative Adversarial Neural Cellular Automata for Medical Image Synthesis
Aug 28, 2026Anh Thi Luu, Nick Lemke, Anirban MukhopadhyayMedical Image Synthesis Generative Adversarial Networks
Technical University of Darmstadt, Darmstadt, Germany · ImFusion GmbH, Munich, Germany
UNITY: Attention Flow Networks for Adaptive Conditioning in Diffusion
Jun 18, 2026Aryan Das, Koushik Biswas, Moloud Abdar +1Compositional Image Generation Conditional Diffusion Models
VIT Bhopal, India · IIIT Delhi, India · The University of Queensland, Australia +1

Jul 11, 2025J/K move · Enter open · S save

Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Model

Fang Chen, Alex Villa, Gongbo Liang, Fuxing Li+2

University of California Merced · Texas A&M University-San Antonio · Oregon State University+1

Diffusion Model Alignment Conditional Image Generation Image Generation Diffusion-Based Image Generation

Open paper AskPDF arXiv

Training data for class-conditional image synthesis often exhibit a long-tailed distribution with limited amount of images for tail classes. Such an imbalance causes mode collapse and reduces the diversity of synthesized images for tail classes. For class-conditional diffusion models trained with imbalanced data, we aim to improve the diversity and fidelity of tail class images without compromising the quality of head class images. We propose contrastive conditional-unconditional alignment (CCUA), which comprises two synergistic loss functions. Our first loss is an Alignment Loss (AL) that aligns class-conditional generation with unconditional generation at large timesteps. Alignment loss makes the denoising process insensitive to class conditions for the initial steps, which enriches tail classes through knowledge sharing from head classes. Secondly, we diversify unconditional generation via an Unsupervised Contrastive Loss (UCL) to increase the distance/dissimilarity among synthetic images. We combine the two losses to implicitly diversify conditional generation. Our framework is easy to implement as demonstrated on both U-Net based architecture and Diffusion Transformer. Our method outperforms vanilla denoising diffusion probabilistic models, score-based diffusion model, and alternative contrastive methods for class-imbalanced image generation across various datasets, in particular ImageNet-LT with 256

\times

256 resolution.

Abstract

Generative Adversarial Networks

Explore similar work

Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Model

Coarse to Fine: Iterative Adversarial Neural Cellular Automata for Medical Image Synthesis

UNITY: Attention Flow Networks for Adaptive Conditioning in Diffusion