We present DSS-GAN, the first generative adversarial network to employ Mamba as a hierarchical generator backbone for noise-to-image synthesis. The central contribution is Directional Latent Routing (DLR), a novel conditioning mechanism that decomposes the latent vector into direction-specific subvectors, each jointly projected with a class embedding to produce a feature-wise affine modulation of the corresponding Mamba scan. Unlike conventional class conditioning that injects a global signal, DLR couples class identity and latent structure along distinct spatial axes of the feature map, applied consistently across all generative scales. DSS-GAN achieves improved FID, KID, and precision-recall scores compared to StyleGAN2-ADA across multiple tested datasets. Analysis of the latent space reveals that directional subvectors exhibit measurable specialization: perturbations along individual components produce structured, direction-correlated changes in the synthesized image.
Figures & tables
Figure 1: DSS-GAN generator architecture (Only two scan directions shown for clarity). Zbase is base global latent vector, zdirkK are directional latent vectors. For 256×256 resolution N is 5 and C0 - feature map channels in DLR blocks is 148 .
Block
Resolution
zbase
zdir
y
Tokenizer
8×8
seed
—
additive bias
DLR
8×8→256×256 (×6)
—
shared, per-direction
shared, per-direction
Conv
512×512
—
—
global
toRGB
512×512
—
—
—
Table 1: Generator routing summary for the DSS-GAN blocks: Tokenizer ( 3.1 ), DLR ( 3.2 ), Conv and toRGB ( 3.2.4 ). The zbase seeds the tokenizer only; zdir and y are re-injected unchanged at every DLR stage.
Global
Mean Class
Stylegan2
refinement
from
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
None
21.75
5.26
.815
.242
1.078
.577
27.35
13.86
.814
.188
1.031
.563
128×128
11.66
2.71
.822
.407
1.144
.748
12.93
5.67
.826
.394
1.125
.744
64×64
18.48
8.27
.574
.264
0.357
.388
20.75
13.45
.541
.264
0.322
.364
Table 2: Ablation: effect of Stylegan2 CNN refinement block boundary resolution on AFHQ ( cat, dog, wild ) 128×128 . All variants share the same training configuration.
Figure 2: DLR block. For each scan direction k , the directional latent zdirk and class embedding ek are projected to affine parameters (γk,βk) that modulate the token sequence before the Mamba SSM. Both the class and the directional latent determine the contribution of each scan direction to the generated feature map, with directional weights wkdir=f(y,zkdir) satisfying ∑k=1Kwdirk=1 . A random 180∘ rotation is applied before the block and inverted after unscan.
None
{0∘,180∘} (ours)
Mamba Block
mean norm
CV
mean norm
CV
ratio ↑
Tokenizer
0.02947
0.121
0.02713
0.118
0.921 ×
8×8
0.01911
0.156
0.01862
0.205
0.974 ×
16×16
0.00876
0.231
0.00955
0.254
1.091 ×
32×32
0.00346
0.239
0.00427
0.230
1.233 ×
64×64
0.00130
0.193
0.00160
0.215
1.232 ×
Table 3: Ablation: effect of 180∘ rotation on AFHQ 128×128 (top) and mean gradient norm per Mamba block averaged over all epochs (bottom).
Figure 3: Per-direction feature maps (mean absolute activation, averaged over 10 channels) at each resolution stage, for a representative sample from the LSUN tower class (from bridge, church, tower classes ) in Mamba blocks. Each row corresponds to one scan direction; each column to one resolution stage. At low resolutions directions capture complementary aspects of global structure; at higher resolutions the directional geometry becomes explicit, with row, column, and diagonal activations forming spatially oriented patterns consistent with their respective scan axes.
Figure 4: Evolution of per-direction routing weights across resolution stages during training (AFHQ 256×256 , 3-direction model). Each line corresponds to a training snapshot; colour indicates epoch (light = early, dark = late). The dashed red line marks the uniform weight 1/K . Arrows indicate the direction of change between the first and last snapshot. Weights diverge from uniform and specialise by resolution, with the column scan dominating at 16×16 and the diagonal scan peaking at 32×32 .
General
Mamba Stages
Mamba Directions
Factor
All dir
Base
CNN
8×8
16×16
32×32
64×64
128×128
Col
Row
Diag
Class
0.676
0.044
0.047
0.612
0.464
0.190
0.149
0.144
0.484
0.455
0.485
Latent
0.682
0.621
—
0.628
0.496
0.091
0.322
0.290
0.608
0.591
0.573
Table 4: Sensitivity analysis: Perceptual impact (LPIPS ↑ ) of Class vs. Latent variations across model components.
Figure 5: The same latent vector z generated with three different class labels. The spatial layout is preserved across classes, confirming that zbase controls composition independently of class identity. Left: AFHQ 256×256 , right : LSUN 256×256 .
Figure 6: Partial class override experiments on AFHQ 256×256 . Top: base class cat , target class dog . Bottom: base class cat , target class wild . Each column replaces a different subset of per-direction classes with the target class, keeping z fixed. The row direction shows little influence on class conversion; the majority of class information is modulated by the blocks at low resolutions.
Figure 7: Swapping directional subvectors zdirk between samples. Each column replaces the subvector of one direction while keeping all other components of z fixed.
Figure 8: Effect of noise perturbation on latent space components. Left: perturbation of the column direction ( zdircol ). Right: perturbation of the tokenizer component ( zbase ).
Global
Bedroom
Kitchen
Model
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
StyleGAN2-ADA
20.31
10.10
.59
.25
.94
.65
26.06
11.51
.58
.24
.85
.61
26.64
12.70
.62
.27
.97
.58
DSS-GAN 3-dir
23.22
13.23
.52
.28
.62
.67
25.16
13.45
.52
.28
.62
.67
31.29
20.28
.52
.28
.62
.67
DSS-GAN 2-dir (row + column)
17.79
8.79
.64
.20
.88
.75
21.56
9.61
.59
.26
.78
.76
23.32
12.30
.71
.18
.97
.78
DSS-GAN 1-dir (row)
20.28
11.93
.55
.30
.66
.73
21.39
10.06
.55
.35
.69
.79
28.78
19.35
.59
.27
.62
.72
DSS-GAN 1-dir (column)
20.30
10.73
.58
.23
.65
.68
24.97
11.95
.58
.24
.69
.69
25.83
14.43
.60
.18
.70
.75
Table 5: Results on LSUN Rooms 128×128 (classes: Bedroom, Kitchen).
Global
Mean class
Model
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
StyleGAN2-ADA
15.63
10.91
.56
.38
.55
.68
14.30
10.28
.59
.33
.53
.70
DSS-GAN 1-dir
12.42
6.17
.69
.56
.85
.83
11.05
7.39
.76
.57
.90
.89
DSS-GAN 3-dir
13.40
7.43
.74
.52
.88
.83
12.19
8.91
.79
.54
.91
.92
Table 6: Results on CelebA 128×128 . Classes: Bald, Black Hair, Blond Hair, Eyeglasses, Goatee, Male, Mustache, Smiling, Wearing Hat, Young (10k samples per class).
Global
Cat
Dog
Wild
Model
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
StyleGAN2-ADA
13.16
4.13
.75
.21
.84
.64
11.04
4.70
.78
.14
1.15
.78
33.27
18.49
.73
.25
.55
.43
7.56
1.96
.77
.12
.82
.64
DSS-GAN 1-dir
13.58
3.37
.85
.35
1.14
.67
9.74
4.46
.88
.26
1.41
.87
27.34
14.60
.85
.55
.70
.53
6.07
1.96
.77
.23
1.02
.62
DSS-GAN 2-dir (row+col)
22.50
5.85
.87
.16
1.14
.53
20.09
10.04
.95
.08
1.91
.74
41.37
23.01
.86
.32
.78
.44
19.61
7.94
.73
.06
.66
.38
DSS-GAN 2-dir (row+diag)
22.76
5.17
.89
.17
1.33
.53
20.88
9.35
.94
.10
1.97
.72
40.03
19.03
.91
.25
.90
.45
21.90
8.67
.77
.13
1.02
.41
DSS-GAN 3-dir
10.29
2.39
.88
.36
1.33
.74
8.87
2.65
.91
.18
1.87
.89
26.51
13.05
.83
.46
.73
.51
6.73
1.51
.87
.09
1.74
.77
Table 7: Results on AFHQ 256×256 .
Global
Bridge
Church
Tower
Model
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
StyleGAN2-ADA
10.85
5.39
.72
.39
0.86
0.81
19.23
9.02
.62
.30
0.63
0.69
11.99
5.75
.74
.29
1.25
0.81
15.40
6.50
.68
.33
0.70
0.69
DSS-GAN 3-dir
12.66
3.45
.79
.31
1.08
0.75
22.75
6.65
.70
.20
0.85
0.55
12.11
6.35
.81
.25
1.25
0.75
18.40
5.44
.76
.27
0.89
0.66
DSS-GAN 2-dir
21.79
10.30
.71
.19
0.84
0.65
35.49
20.98
.69
.16
0.70
0.56
19.54
11.00
.78
.14
1.11
0.75
25.60
12.76
.68
.22
0.76
0.65
DSS-GAN 1-dir
22.05
7.83
.72
.25
0.92
0.68
40.10
17.17
.66
.19
0.58
0.54
20.52
8.21
.80
.21
1.35
0.81
28.70
11.31
.71
.29
0.81
0.67
Table 8: Results on LSUN 256×256 (15k samples per class).
Global
Model
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
StyleGAN2-ADA
7.64
2.68
.79
.54
1.25
.92
DSS-GAN 3-dir
8.27
2.53
.83
.53
1.39
.93
DSS-GAN 1-dir
8.69
3.06
.80
.52
1.25
.90
Table 9: Results on FFHQ 256×256 .
Global
Cat
Dog
Wild
Model
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
StyleGAN2
11.74
3.06
.86
.22
1.35
.73
9.38
2.55
.86
.17
1.69
.85
27.49
10.40
.85
.25
.80
.51
9.13
2.22
.82
.11
1.30
.76
DSS-GAN 1-dir
17.77
4.91
.80
.28
1.03
.59
12.64
5.14
.90
.16
1.53
.83
39.46
25.34
.77
.43
.59
.41
12.65
3.15
.71
.23
.91
.55
DSS-GAN 3-dir
14.44
4.11
.86
.25
1.18
.64
12.63
4.65
.91
.16
1.56
.83
36.06
22.70
.87
.43
.80
.48
8.81
2.50
.75
.19
1.22
.59
Table 10: Results on AFHQ 512×512 .
Model
Parameters
Weight size (MB)
Latency (batch 1) ↓
Peak throughput ↑
StyleGAN2-ADA
25.0 M
24.7
9.1 ms
1451.6 img/s ( b=128 )
DSS-GAN (3-dir)
7.3 M
21.3
15.1 ms
280.6 img/s ( b=32 )
DSS-GAN (1-dir)
4.4 M
13.6
8.1 ms
441.6 img/s ( b=32 )
Table 11: Computational efficiency and memory footprint at 256×256 resolution, measured on a single NVIDIA H100 GPU.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Generated samples, FFHQ 256 × 256.
Figure 21
Figure 12: Class embedding override: bridge → church. Starting from a base cat image, DLR class embeddings are progressively replaced with church embeddings across individual directions (row, column, diagonal), combinations, early stages only (8 × 8, 16 × 16), all DLR directions, and all DLR with CNN refinement block.
Figure 13: Class embedding override: bridge → tower with same protocol as Fig. 12 .
Figure 14: Swapping directional subvectors zdirk between samples in LSUN dataset. Each column replaces the values of one direction or block stage while keeping all other components of latent fixed. From left: reference, row, column, diagonal, all directions combined, all directions in DLR early-stage only (8×8, 16×16), and tokenizer zbase swap.
Figure 15: Latent space perturbation analysis (bridge). (a) Tokenizer ztok ; (b) column direction zdircol ; (c) diagonal direction zdirdiag ; (d) combined column and diagonal. Each panel shows the effect of scaling perturbation magnitude γ .
Figure 16: Latent space perturbation analysis (church outdoor). (a) Tokenizer ztok ; (b) column direction zdircol ; (c) diagonal direction zdirdiag ; (d) combined column and diagonal. Each panel shows the effect of scaling perturbation magnitude γ .
Figure 17: Latent space perturbation analysis (tower). (a) Tokenizer ztok ; (b) column direction zdircol ; (c) diagonal direction zdirdiag ; (d) combined column and diagonal. Each panel shows the effect of scaling perturbation magnitude γ .
Global
Cat
Dog
Wild
Model
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
FID ↓
KID ↓
P ↑
R ↑
D ↑
C ↑
DSS-GAN 3× same dir
19.20
4.16
.894
.217
1.457
.569
13.97
5.58
.959
.074
1.750
.820
38.26
21.33
.899
.315
.907
.463
20.35
8.20
.814
.133
1.090
.435
DSS-GAN 3-dir
10.29
2.39
.879
.356
1.327
.743
7.88
2.74
.903
.293
1.652
.872
25.15
12.86
.868
.548
.805
.601
6.08
1.35
.877
.150
1.624
.765
Appendix
Table 12: Generation quality comparison between identical and distinct scan direction configurations (AFHQ 256×256 ). Best checkpoint per run reported.
Figure 18: Per-direction routing weights across DLR blocks during training (AFHQ 256×256 ), for the same-direction ( 3× row) configuration. Each line corresponds to a training snapshot; colour indicates epoch (light = early, dark = late). The dashed red line marks the uniform weight 1/K=0.33 . Arrows indicate the direction of change from the first to the last snapshot. With identical directions, routing weights remain noisy throughout training and show no convergence toward a stable resolution-dependent pattern, in contrast to the distinct-direction case (Figure 4 ).
Figure 19: Routing diagnostics for 3× same (red) vs. 3× distinct (blue) directions (AFHQ 256×256 ). From left to right: class-to-routing scale ∣α∣ with EMA decay switch marked (dotted line); global FID over training epochs with FID at the best checkpoint annotated; R1 penalty with maximum values annotated; L1 loss (EMA) with EMA decay switch marked.
Hyperparameter
128
256
512
Optimiser and training
Adam (β1,β2)
(0.0,0.99)
′′
′′
LR G
9×10−5
′′
′′
LR D
3×10−5
′′
′′
Batch size
128
96
48
Regularisation
Appendix
Table 13: Training configuration across resolutions. ′′ denotes identical value to the 128 × 128 baseline.
Generative Adversarial Networks (GANs) train competing generators and discriminators to synthesize data, translate images, and model complex distributions.
Training data for class-conditional image synthesis often exhibit a long-tailed distribution with limited amount of images for tail classes. Such an imbalance causes mode collapse and reduces the diversity of synthesized images for tail classes. For class-conditional diffusion models trained with imbalanced data, we aim to improve the diversity and fidelity of tail class images without compromising the quality of head class images. We propose contrastive conditional-unconditional alignment (CCUA), which comprises two synergistic loss functions. Our first loss is an Alignment Loss (AL) that aligns class-conditional generation with unconditional generation at large timesteps. Alignment loss makes the denoising process insensitive to class conditions for the initial steps, which enriches tail classes through knowledge sharing from head classes. Secondly, we diversify unconditional generation via an Unsupervised Contrastive Loss (UCL) to increase the distance/dissimilarity among synthetic images. We combine the two losses to implicitly diversify conditional generation. Our framework is easy to implement as demonstrated on both U-Net based architecture and Diffusion Transformer. Our method outperforms vanilla denoising diffusion probabilistic models, score-based diffusion model, and alternative contrastive methods for class-imbalanced image generation across various datasets, in particular ImageNet-LT with 256×256 resolution.
Fang Chen, Alex Villa, Gongbo Liang +3
University of California Merced · Texas A&M University-San Antonio · Oregon State University +1
Large-scale, publicly available datasets have driven advances in deep learning, but privacy and legal restrictions often limit data sharing in medical imaging. Synthetic data generation offers a privacy-friendly alternative to enable the training of high-performance models on health data. While most state-of-the-art generative models produce high-quality images, they remain computationally expensive, which limits their applicability on resource-constrained hardware. We propose StyleGANCA, the first lightweight general-purpose NCA-based generative adversarial network. The architecture integrates a StyleGAN-inspired mapping network and adaptive style modulation into a multi-scale NCA synthesis process, enabling latent-controlled image generation through iterative local interactions. We evaluate StyleGANCA on BloodMNIST and PathMNIST against adversarial, variational, diffusion, and NCA-based baselines. Experimental results demonstrate that StyleGANCA achieves competitive image quality with substantially fewer parameters than baseline architectures, achieving the best FID and KID scores on PathMNIST with only 617k parameters. Furthermore, downstream experiments show that the generated images preserve class-specific information and effectively support the training of multi-class classifiers. Our code is publicly available at: https://github.com/MECLabTUDA/StyleGANCA
Anh Thi Luu, Nick Lemke, Anirban Mukhopadhyay
Technical University of Darmstadt, Darmstadt, Germany · ImFusion GmbH, Munich, Germany
We introduce UNITY, a Universal-to-Specialized adapter for efficient and scalable composite conditioning in diffusion based image generation. Unlike prior methods that train separate adapters for each conditioning modality, UNITY jointly learns shared semantics across multiple conditioning types and subsequently specializes without modifying the underlying architecture. The proposed two stage training paradigm consists of a Universal Stage that captures cross modal representations across all conditioning modalities using half of the total training steps, followed by a Specialization Stage that refines modality specific features using the remaining training budget. At the core of UNITY are the Morphable Attention Flow (MAF) Network and Morph Wrapper modules, which enable channel aware and spatially adaptive feature alignment through learnable flow fields and attention based fusion. This constant complexity formulation supports flexible operation under both single and composite conditioning settings while significantly reducing inference latency and memory consumption. Extensive experiments across multiple datasets demonstrate that UNITY achieves state of the art image fidelity while maintaining superior memory efficiency. Code: https://github.com/arya-domain/UNITY
Aryan Das, Koushik Biswas, Moloud Abdar +1
VIT Bhopal, India · IIIT Delhi, India · The University of Queensland, Australia +1