Generative Model via Quantile Assignment

Georgi Hrusanov; Oliver Y. Chén; Julien S. Bodelet

cs.LGFeb 20, 2026

Generative Model via Quantile Assignment

Authors: Georgi Hrusanov, Oliver Y. Chén, Julien S. Bodelet

Organizations: University of Lausanne, Lausanne, Switzerland · Lausanne University Hospital (CHUV), Lausanne, Switzerland · Department of Statistics, Stanford University, Stanford, USA

Abstract

Deep Generative models (DGMs) play two key roles in modern machine learning: (i) producing new information (e.g., image synthesis) and (ii) reducing dimensionality. However, traditional architectures often rely on auxiliary networks such as encoders in Variational Autoencoders (VAEs) or discriminators in Generative Adversarial Networks (GANs), which introduce training instability, computational overhead, and risks like mode collapse. We present NeuroSQL, a new generative paradigm that eliminates the need for auxiliary networks by learning low-dimensional latent representations implicitly. NeuroSQL leverages an asymptotic approximation that expresses the latent variables as the solution to an optimal transportation problem. Specifically, NeuroSQL learns the latent variables by solving a linear assignment problem and then passes the latent information to a standalone generator. We benchmark its performance against GANs, VAEs, and a budget-matched diffusion baseline on four datasets: handwritten digits (MNIST), faces (CelebA), animal faces (AFHQ), and brain images (OASIS). Compared to VAEs, GANs, and diffusion models: (1) in terms of image quality, NeuroSQL achieves overall lower mean pixel distance between synthetic and authentic images and stronger perceptual/structural fidelity; (2) computationally, NeuroSQL requires the least training time; and (3) practically, NeuroSQL provides an effective solution for generating synthetic data with limited training samples. By embracing quantile assignment rather than an encoder, NeuroSQL provides a fast, stable, and robust way to generate synthetic data with minimal information loss.

Figures & tables

Figure 1: A schematic representation of the neuroSQL architecture. Left: Algorithm for optimizing latent embeddings and generator parameters $\theta$ . Right: Data flow in neuroSQL for data synthesis. Here, “cost” refers to cost matrix entries: $C_{i,k}=\ell\big(X_{i},G_{\theta}(Q_{k})\big)$ solved by the linear assignment problem. “Momentum update” indicates: $\hat{Z}_{i}^{(t)}\leftarrow\rho\,Q_{\pi^{(t)}(i)}+(1-\rho)\,\hat{Z}_{i}^{(t-1)}$ .

Panel of Figure 2 — Figure 2: A Comparison of Latent Space of neuroSQL and VAE. Visualization of the two-dimensional latent space obtained from MNIST using neuroSQL (left) and VAE (right). NeuroSQL’s latent space forms more distinct, better-separated clusters than those of the VAE, whose clusters show more overlaps.

Figure 3: 2D brain images generated from neuroSQL (top row), VAE (middle row) and GAN (bottom row) with U-Net generator.

Dataset (Res.)	Model	pFID $\downarrow$	SSIM $\uparrow$	P
MNIST (28 $\times$ 28)	neuroSQL	0.61 $\pm$ 0.12	0.616 $\pm$ 0.074	2.79
	VAE	1.26 $\pm$ 0.26	0.179 $\pm$ 0.030	4.17
	GAN	2.00 $\pm$ 0.22	0.233 $\pm$ 0.018	5.56
	Diffusion	0.65 $\pm$ 0.34	0.492 $\pm$ 0.055	147.91
CelebA (64 $\times$ 64)	neuroSQL	5.81 $\pm$ 4.05	0.262 $\pm$ 0.036	7.54
	VAE	10.75 $\pm$ 7.92	0.196 $\pm$ 0.047	14.03

Table 1: Performance comparison across datasets (mean

\pm

std). Lower is better for proxy FID; higher is better for SSIM.

Method	Prec.	Rec.	Acc.
NeuroSQL	0.8788	0.8763	0.8771
VAE	0.7236	0.6705	0.6749

Table 2: Downstream classification performance on MNIST. A classifier was trained solely on synthetic data (10k samples) generated by neuroSQL and VAE and evaluated on real test data.

Panel of Figure 4 — Figure 4: Qualitative comparison of 36 randomly generated images for models trained on MNIST.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Computational scalability: mean epoch time versus sample size $N$ for VAE, GAN, and neuroSQL with different local assignment sizes $m$ .

Latent Dim	Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
8	Greedy	10.764394	0.376178	0.296806
	Hungarian	11.121147	0.378670	0.274640
16	Greedy	11.632382	0.398841	0.241317
	Hungarian	11.798410	0.390295	0.240504
24	Greedy	13.222514	0.384961	0.272349
	Hungarian	11.338643	0.379329	0.271508

Appendix

Table 3: Ablation Study: U-Net Latent Dimensions

Latent Dim	Assign. Method	Method	Peak VRAM (MB)	RAM Used (MB)	Mean Epoch Time (s)	Mean Epoch GPU (MB)	Mean Epoch $\Delta$ RAM (MB)	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
2	greedy	NeuroSQL	81.5	2764	1.40	116	11.2	0.628	0.045	0.580
	hungarian	NeuroSQL	284.6	3078	1.37	293	0.39	0.576	0.041	0.616
		VAE	319.8	3091	1.33	320	0.10	1.259	0.050	0.249
		GAN	405.6	3115	1.56	406	0.17	2.060	0.064	0.223
3	greedy	NeuroSQL	284.8	3206	1.39	293	0.14	0.567	0.040	0.574
	hungarian	NeuroSQL	284.9	3298	1.38	293	0.00	0.556	0.037	0.607

Appendix

Table 4: Ablation: Assignment Methods and Resource Usage on MNIST (seed=11). Metrics averaged over epochs where applicable.

Latent dimension	Method	FID (Proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
2	NeuroSQL	7.914	0.390	0.259
	VAE	8.198	0.410	0.246
	GAN	9.936	0.450	0.218
4	NeuroSQL	9.241	0.403	0.257
	VAE	8.720	0.463	0.212
	GAN	19.420	0.509	0.206

Appendix

Table 5: Results on OASIS, a brain imaging dataset, using ConvNet across latent dimensions. Lower is better for FID (proxy) and LPIPS; higher is better for SSIM.

Latent dimension	Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM
16	NeuroSQL	25.410	0.249	0.301
	VAE	51.139	0.309	0.171
	GAN	168.993	0.727	0.008
32	NeuroSQL	31.957	0.257	0.242
	VAE	66.045	0.362	0.128
	GAN	158.103	0.687	0.008

Appendix

Table 6: Results on OASIS, a brain imaging dataset, using ResNet across latent dimensions. Lower is better for FID (proxy) and LPIPS; higher is better for SSIM.

Latent dimension	Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
4	NeuroSQL	8.519902	0.386861	0.254752
	VAE	7.858056	0.377805	0.265082
	GAN	26.234322	0.597633	0.206676
8	NeuroSQL	8.579974	0.401890	0.238080
	VAE	5.976874	0.384821	0.264277
	GAN	35.444614	0.609803	0.172225

Appendix

Table 7: Results on OASIS, a brain imaging dataset, using a U-Net across latent dimensions. Lower is better for FID (proxy) and LPIPS; higher is better for SSIM.

Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
NeuroSQL	8.559346	0.387453	0.258766
VAE	6.772966	0.396473	0.254754
GAN	28.591262	0.605914	0.193429

Appendix

Table 8: Results on OASIS, a brain imaging dataset, using a U-Net — with results averaged across latent dimensions

\{4,8,16,32,64,128\}

Latent dimension	Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
2	NeuroSQL	9.64453	0.22094	0.31376
	VAE	11.49087	0.24068	0.27396
	GAN	26.56548	0.47277	0.10216
4	NeuroSQL	6.99789	0.22845	0.31645
	VAE	8.57467	0.23091	0.26471
	GAN	29.68144	0.49874	0.06955

Appendix

Table 9: Results on CelebA, a face attributes dataset, using ConvNet across latent dimensions. Lower is better for FID (proxy) and LPIPS; higher is better for SSIM.

Latent dimension	Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
2	NeuroSQL	4.40599	0.18367	0.27506
	VAE	8.01552	0.19720	0.24972
	GAN	19.01699	0.20240	0.13535
4	NeuroSQL	4.47511	0.16968	0.26754
	VAE	6.03891	0.20652	0.20760
	GAN	14.21587	0.26898	0.14515

Appendix

Table 10: Results on CelebA, a face attributes dataset, using ResNet across latent dimensions. Lower is better for FID (proxy) and LPIPS; higher is better for SSIM.

Latent dimension	Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
2	NeuroSQL	4.96169	0.18253	0.27537
	VAE	7.33227	0.19269	0.25696
	GAN	14.09393	0.22041	0.12822
4	NeuroSQL	3.54758	0.18932	0.27993
	VAE	4.99991	0.19061	0.22841
	GAN	7.66398	0.19251	0.14899

Appendix

Table 11: Results on CelebA, a face attributes dataset, using U-Net (unconditional) across latent dimensions. Lower is better for FID (proxy) and LPIPS; higher is better for SSIM.

Latent dimension	Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
2	NeuroSQL	22.023664	0.563209	0.339032
	VAE	44.276279	0.531442	0.284219
	GAN	74.178604	0.693773	0.080698
4	NeuroSQL	35.913769	0.538813	0.357857
	VAE	35.430927	0.527985	0.285893
	GAN	48.706165	0.669665	0.129869

Appendix

Table 12: Results on AFHQ, an animal faces dataset, using ConvNet across latent dimensions. Lower is better for FID (proxy) and LPIPS; higher is better for SSIM.

Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
NeuroSQL	32.19	0.544	0.354
VAE	71.13	0.560	0.233
GAN	58.36	0.720	0.093

Appendix

Table 13: Results on AFHQ, an animal faces dataset, using ConvNet — with results averaged across latent dimensions.

Latent dimension	Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
2	NeuroSQL	7.267559	0.459967	0.267403
	VAE	15.259568	0.448991	0.235907
	GAN	39.100639	0.550003	0.091928
4	NeuroSQL	11.560558	0.460007	0.246023
	VAE	13.387914	0.432443	0.192707
	GAN	37.873501	0.356168	0.120893

Appendix

Table 14: Results on AFHQ, an animal faces dataset, using ResNet across latent dimensions. Lower is better for FID (proxy) and LPIPS; higher is better for SSIM.

Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
NeuroSQL	12.40	0.460	0.247
VAE	12.27	0.394	0.202
GAN	29.35	0.435	0.104

Appendix

Table 15: Results on AFHQ, an animal faces dataset, using ResNet — with results averaged across latent dimensions.

Latent dimension	Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
2	NeuroSQL	17.626825	0.595854	0.272737
	VAE	10.595321	0.620105	0.279837
	GAN	23.709433	0.405830	0.116950
3	NeuroSQL	10.193194	0.617973	0.278050
	VAE	29.156761	0.578171	0.173115
	GAN	17.098032	0.607903	0.074634

Appendix

Table 16: Results on AFHQ, an animal faces dataset, using U-Net across latent dimensions. Lower is better for FID (proxy) and LPIPS; higher is better for SSIM.

Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
NeuroSQL	12.486547	0.618680	0.268553
VAE	34.046180	0.578176	0.134123
GAN	50.295575	0.609927	0.047645

Appendix

Table 17: Results on AFHQ, an animal faces dataset, using U-Net — with results averaged across latent dimensions

\{2,3,4,8,16,32,64\}

Latent dimension	Seed	Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
2	11	NeuroSQL	0.696835	0.039503	0.564344
		VAE	1.070473	0.058065	0.200167
		GAN	2.157639	0.054278	0.245780
3	11	NeuroSQL	0.527451	0.030257	0.668574
		VAE	1.443453	0.060009	0.157831
		GAN	1.849820	0.057282	0.220785

Appendix

Table 18: Ablation results on MNIST, a database of handwritten digits (all runs). Lower is better for FID (proxy) and LPIPS; higher is better for SSIM.

Method	FID (proxy) $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
NeuroSQL	0.612143	0.034880	0.616459
VAE	1.256963	0.059037	0.178999
GAN	2.003730	0.055780	0.233283

Appendix

Table 19: Results on MNIST, a database of handwritten digits — averaged over latent dimensions

\{2,3\}

(seed

=11

Generative Modeling

Generative modeling learns data distributions to create new samples, from images and text to molecules, simulations, and structured records.

See all 493 papers

Jun 30, 2026

GEAR: Guided End-to-End AutoRegression for Image Synthesis

Visual generative models are typically trained in two stages. A tokenizer is first trained for reconstruction and then frozen, after which a generator is trained on its discrete indices or continuous latents. This decoupling leaves the tokenizer unaware of what the generator finds easy to model. We present GEAR (Guided End-to-end AutoRegression), which trains a vector-quantized (VQ) tokenizer and an autoregressive (AR) generator jointly and end-to-end, guided by representation alignment. The key obstacle is that the VQ index fed to the AR model is non-differentiable, so gradients cannot reach the tokenizer, and a straight-through estimator collapses. GEAR resolves this with a dual read-out of the codebook assignment. A hard, one-hot branch trains the AR with next-token prediction, while a differentiable soft branch carries a representation-alignment loss that flows back to guide only the tokenizer. The AR model thereby steers its tokenizer toward an index distribution it can predict more easily. This shifts the alignment burden from the tokenizer to the AR: the tokenizer's own features become less DINOv2-like while the AR's become more so, the opposite of diffusion-side recipes that make the latent itself semantic. GEAR speeds up ImageNet gFID convergence by up to 10x relative to the strong LlamaGen-REPA baseline, learns markedly better patch-level and spatially-coherent features, and generalizes across quantizers (VQVAE, LFQ, IBQ) and to text-to-image generation.

Bin Lin, Zheyuan Liu, Chenguo Lin +8

Peking University · Tencent Hunyuan

Vector Quantization Autoregressive Image Generation Representation Alignment

Ask PDF

May 13, 2026

AcquisitionSynthesis: Targeted Data Generation using Acquisition Functions

Data quality remains a critical bottleneck in developing capable, competitive models. Researchers have explored many ways to generate top quality samples. Some works rely on rejection sampling: generating lots of synthetic samples and filtering out low-quality samples. Other works rely on larger or closed-source models to extract model weaknesses, necessary skills, or a curriculum off of which to base data generation. These works have one common limitation: there is no quantitative approach to measure the impact of the generated samples on the downstream learner. Active learning literature provides exactly this, in the form of acquisition functions. Acquisition functions measure the informativeness and/or influence of data, providing interpretable, model-centric signals. Inspired by this, we propose AcquisitionSynthesis: using acquisition functions as reward models to train language models to generate higher-quality synthetic data. We conduct experiments on classic verifiable tasks of math, medical question-answering, and coding. Our experimental results indicate that (1) student models trained with AcquisitionSynthesis data achieve good performance on in-distribution tasks (2-7% gain) and is more robust to catastrophic forgetting, and (2) AcquisitionSynthesis models can generate data for other models and for low-to-high resource training paradigms. By leveraging acquisition rewards, we seek to demonstrate a principled path toward model-aware self-improvement that surpasses static datasets.

Ishika Agarwal, Sofia Stoica, Emre Can Acikgoz +4

University of Illinois Urbana-Champaign1 · Amazon2

Synthetic Data Generation Synthetic Data Generation for Language Models Language Model Self-Improvement Large Language Model-Based Data Augmentation

Ask PDF

Oct 14, 2026

Discriminative Score Function: Turning Pretrained Models into Functional Generative Priors

Discriminative models are trained with task-specific objectives such as classification, detection, or representation learning, yet their practical utility depends on generalizing beyond the training task to unseen samples from the same domain. This suggests that a discriminative model may retain an implicit trace of the data manifold beyond its task labels. We ask whether this trace can be exposed as a generative signal without task-specific guidance. We introduce the Discriminative Score Function (DSF), which extracts an image-space generative field from the loss-gradient geometry of a pretrained discriminative model. DSF provides a training-free update rule that moves images toward model-consistent visual regions without retraining or task-specific targets. We apply DSF to diverse pretrained discriminative models, including ResNet classifiers, DETR detectors, and DINOv2 ViT encoders. DSF generates unconditional images from noise and supports conditional guidance, editing, inpainting, and explanation through the same field. DSF-generated samples also provide effective zero-shot quantization calibration when real images or labels are unavailable, preserving feature geometry better than data-free calibration baselines. These results show that discriminative training leaves a generative functional signal for synthesis and calibration. Project page: https://junhoo.me/dsf.

Junhoo Lee, Hyeonjin Kim, Sangbum Han +1

Seoul National University

Generative Modeling Image Generation Score-Based Generative Modeling

Ask PDF

Abstract

Generative Modeling

Explore similar work

GEAR: Guided End-to-End AutoRegression for Image Synthesis

AcquisitionSynthesis: Targeted Data Generation using Acquisition Functions

Discriminative Score Function: Turning Pretrained Models into Functional Generative Priors