Generative Models

Follow this topic Ask about this topic

Momentum

94 papers in the last four weeks, up 119% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 813

Oct 10, 2025cs.CL

LLP: LLM-Based Product Pricing in E-commerce

Unlike Business-to-Consumer e-commerce platforms (e.g., Amazon), inexperienced individual sellers on Consumer-to-Consumer platforms (e.g., eBay) often face significant challenges in setting prices for their second-hand products efficiently. Therefore, numerous studies have been proposed for automating price prediction. However, most of them are based on static regression models, which suffer from poor generalization performance and fail to capture market dynamics (e.g., the price of a used iPhone decreases over time). Inspired by recent breakthroughs in Large Language Models (LLMs), we introduce LLP, the first LLM-based generative framework for second-hand product pricing. LLP first retrieves similar products to better align with the dynamic market change. Afterwards, it leverages the LLMs' nuanced understanding of key pricing information in free-form text to generate accurate price suggestions. To strengthen the LLMs' domain reasoning over retrieved products, we apply a two-stage optimization, supervised fine-tuning (SFT) followed by group relative policy optimization (GRPO), on a dataset built via bidirectional reasoning. Moreover, LLP employs a confidence-based filtering mechanism to reject unreliable price suggestions. Extensive experiments demonstrate that LLP substantially surpasses existing methods while generalizing well to unseen categories. We have successfully deployed LLP on Xianyu\footnote{Xianyu is China's largest second-hand e-commerce platform.}, significantly outperforming the previous pricing method. Under the same 30% product coverage, it raises the static adoption rate (SAR) from 40% to 72%, and maintains a strong SAR of 47% even at 90% recall.

Hairu Wang, Sheng You, Qiheng Zhang +5

Xianyu of Alibaba, Hangzhou, China · University of Science and Technology of China, Suzhou, China

Prices Generative Models

Oct 10, 2025cs.RO

Flow-Opt: Scalable Centralized Multi-Robot Trajectory Optimization with Flow Matching and Differentiable Optimization

Centralized trajectory optimization in the joint space of multiple robots allows access to a larger feasible space that can result in smoother trajectories, especially while planning in tight spaces. Unfortunately, it is often computationally intractable beyond a very small swarm size. In this paper, we propose Flow-Opt, a learning-based approach towards improving the computational tractability of centralized multi-robot trajectory optimization. Specifically, we reduce the problem to first learning a generative model to sample different candidate trajectories and then using a learned Safety-Filter(SF) to ensure fast inference-time constraint satisfaction. We propose a flow-matching model with a diffusion transformer (DiT) augmented with permutation invariant robot position and map encoders as the generative model. We develop a custom solver for our SF and equip it with a neural network that predicts context-specific initialization. The initialization network is trained in a self-supervised manner, taking advantage of the differentiability of the SF solver. We advance the state-of-the-art in the following respects. First, we show that we can generate trajectories of tens of robots in cluttered environments in a few tens of milliseconds. This is several times faster than existing centralized optimization approaches. Moreover, our approach also generates smoother trajectories orders of magnitude faster than competing baselines based on diffusion models. Second, each component of our approach can be batched, allowing us to solve a few tens of problem instances in a fraction of a second. We believe this is a first such result; no existing approach provides such capabilities. Finally, our approach can generate a diverse set of trajectories between a given set of start and goal locations, which can capture different collision-avoidance behaviors.

Simon Idoko, Prajyot Jadhav, Arun Kumar Singh

All authors are with the University of Tartu, Estonia.

Multi-Robot Motion Planning Robust Trajectory Collision Avoidance Differentiable Optimization Generative Models

Oct 3, 2025cs.CV

What Drives Compositional Generalization in Visual Generative Models? The Importance of Continuous Training Objectives

Compositional generalization, the ability to generate novel combinations of known concepts, is a key ingredient for visual generative models. Yet, not all mechanisms that enable or inhibit it are fully understood. In this work, we conduct a systematic study of which design choices critically determine compositional generalization in image and video generation. By isolating independent design axes, we identify two key factors strongly associated with compositional success: (i) whether the training objective operates on a discrete or continuous distribution, and (ii) the completeness of conditioning information about constituent factors during training. We also show that relaxing the discrete loss with an auxiliary continuous latent objective can partially recover compositional performance in discrete models like MaskGIT. Our findings, corroborated by diverse compositional tasks and preliminary evidence in world models and LLMs, motivate a shift toward continuous objectives for compositional generalization.

Karim Farid, Rajat Sahay, Yumna Ali Alnaggar +4

University of Freiburg · Bosch Center for Artificial Intelligence · Inria, École Normale Supérieure, CNRS, PSL Research University

Compositional Generalization Generative Models Loss Function Importance

Sep 28, 2025cs.CV

HunyuanImage 3.0 Technical Report

We present HunyuanImage 3.0, a native multimodal model that unifies multimodal understanding and generation within an autoregressive framework, with its image generation module publicly available. The achievement of HunyuanImage 3.0 relies on several key components, including meticulous data curation, advanced architecture design, a native Chain-of-Thoughts schema, progressive model pre-training, aggressive model post-training, and an efficient infrastructure that enables large-scale training and inference. With these advancements, we successfully trained a Mixture-of-Experts (MoE) model comprising over 80 billion parameters in total, with 13 billion parameters activated per token during inference, making it the largest and most powerful open-source image generative model to date. We conducted extensive experiments and the results of automatic and human evaluation of text-image alignment and visual quality demonstrate that HunyuanImage 3.0 rivals previous state-of-the-art models. By releasing the code and weights of HunyuanImage 3.0, we aim to enable the community to explore new ideas with a state-of-the-art foundation model, fostering a dynamic and vibrant multimodal ecosystem. All open source assets are publicly available at https://github.com/Tencent-Hunyuan/HunyuanImage-3.0

Tencent Hunyuan Foundation Model Team

Tencent Hunyuan Foundation Model Team

Generative Models Technical Report Autoregressive Model

Aug 9, 2025cs.AI

A memory-based active inference model of DishBrain-like adaptive behaviour

Recent and rapid advances in artificial intelligence (AI) make it increasingly important to understand the foundations of adaptive behaviour in autonomous agents, especially for building safe and efficient systems. While artificial neural networks have dominated the development of AI, recent work has begun to explore living biological neuronal networks as an alternative substrate for computation. These systems promise remarkable data and sample efficiency and rich dynamics, and may also inspire explainable and biologically plausible models. Here, we develop an experiment-informed active inference framework to model decision-making in closed-loop agents that mirror experimental setups using biological neurons. Using a generative model whose dimensions are matched to an experiment protocol, we systematically compare three decision-making schemes within this common generative model. Under matched episode counts (i.e. total data available for learning) to the in-vitro experiment, our simulations show that agents with short memory horizons reach a level of performance close to that of mouse and human cortical cultures (DishBrain platform), whereas longer memory horizons depart from it substantially. Increasing the planning horizon, by contrast, confers no comparable benefit. Because all model parameters are explicit, we can also track the quantities in our generative model that accompany this improvement, such as the risk term and the entropy of the transition and state-action mappings. Together, these results illustrate how active inference offers a formal language for comparing decision-making schemes in similar closed-loop control environments.

Aswin Paul, Moein Khajehnejad, Forough Habibollahi +2

Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Clayton 3800, Australia · VERSES, Los Angeles, California, USA · Cortical Labs Pty Ltd, Melbourne 3056, Australia +1

Brain-Inspired Synergistic Framework Inference-Time Steering Neurons Autonomous Agents Agent Behavior Modeling Generative Models

Aug 5, 2025cs.LG

Physics-Constrained Fine-Tuning of Flow-Matching Models for Generation and Inverse Problems

We present a framework for fine-tuning flow-matching generative models to enforce physical constraints and solve inverse problems in scientific systems. Starting from a model trained on low-fidelity or observational data, we apply a differentiable post-training procedure that minimizes weak-form residuals of governing partial differential equations (PDEs), promoting physical consistency and adherence to boundary conditions without distorting the underlying learned distribution. To infer unknown physical inputs, such as source terms, material parameters, or boundary data, we augment the generative process with a learnable latent parameter predictor and propose a joint optimization strategy. The resulting model produces physically valid field solutions alongside plausible estimates of hidden parameters, effectively addressing ill-posed inverse problems in a data-driven yet physicsaware manner. We validate our method on canonical PDE benchmarks, demonstrating improved satisfaction of PDE constraints and accurate recovery of latent coefficients. Our approach bridges generative modelling and scientific inference, opening new avenues for simulation-augmented discovery and data-efficient modelling of physical systems.

Jan Tauberschmidt, Sophie Fellenz, Sebastian J. Vollmer +1

DSA, German Research Center for Artificial Intelligence (DFKI) · Department of Computer Science, University of Kaiserslautern–Landau (RPTU) · Department of Mathematics, Imperial College London

Bayesian Inverse Problems Inverse Problem Generative Models Partial Differential Equations Model Fine-Tuning Scientific Discovery

Aug 3, 2025cs.CV

Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learning

3D affordance grounding aims to understand how diverse objects can be manipulated, making it a cornerstone of embodied interaction. However, prior works struggle to generalize to out-of-distribution, open-world scenarios, leaving a critical gap between limited dataset performance and real-world application needs. Inspired by the saying: \textit{\textbf{``What I can not create, I do not understand''}}, we find generative models can generate semantically valid HOI images, which indicates inherent encoding of affordance concepts. Building on this insight, we propose DAG, the first innovative diffusion-based 3D affordance grounding framework that extracts general affordance knowledge from text-to-image diffusion models for 3D affordance prediction. Specifically, we extract the affordance priors from a diffusion model to encode HOI priors, and design an affordance block with a multi-source affordance decoder for dense 3D affordance prediction. Extensive experiments show that DAG consistently outperforms state-of-the-art methods and exhibits strong open-world generalization, even in the challenging one-shot setting. The code of our method is released on \textcolor{blue}{\textit{https://github.com/hq-King/DAG}}.

Hanqing Wang, Zhenhao Zhang, Kaiyang Ji +12

The Hong Kong University of Science and Technology, Guangzhou, China · ShanghaiTech University, Shanghai, China · Zhejiang University, Hangzhou, China +3

Open-World Generative Models Diffusion Models Prior Knowledge Integration Unimodal Encoders

Jul 31, 2025cs.LG

L-GTA: Latent Generative Modeling for Time Series Augmentation

Data augmentation is becoming increasingly important across various areas of time series analysis, including forecasting, classification, and anomaly detection. We introduce the Latent Generative Temporal Augmentation (L-GTA) model, a generative approach based on a Variational Autoencoder with a Bi-LSTM backbone and temporal self-attention. The model learns a latent representation for each timestep and applies controlled perturbations such as jittering, magnitude warping, or drift. We define an equivariance objective to further encourage consistency between latent space and data space transformations. As a result, the augmented samples show predictable and interpretable transformation signatures. We evaluate L-GTA on several real-world datasets against SOTA generative methods, including TimeGAN, TimeVAE, and Diffusion-TS, as well as direct transformation approaches. Across experiments on downstream forecasting, distribution fidelity, and controllability of transformation intensity, L-GTA consistently outperforms competing approaches. In downstream forecasting, it reduces prediction error by up to 26% compared to the strongest generative method and 27% relative to using the original data without augmentation.

Luis Roque, Vitor Cerqueira, Carlos Soares +1

Faculdade de Engenharia da Universidade do Porto, Porto, Portugal · University of Coimbra, Coimbra, Portugal · Fraunhofer Portugal AICOS, Portugal +1

Time-Series Generation Augmentation Time Series Generative Models Cnn-Lstm

Jul 12, 2025cs.LG

Capturing Unseen Spatial Heat Extremes Through Dependence-Aware Generative Modeling

Observed records of climate extremes provide an incomplete view of plausible hazards, missing "unseen" events beyond historical experience. Ignoring spatial dependence further underestimates hazards striking multiple locations simultaneously. We introduce DeepX-GAN (Dependence-Enhanced Embedding for Physical eXtremes-Generative Adversarial Network), a deep generative model that explicitly captures the spatial structure of rare extremes. Its zero-shot generalizability enables the simulation of statistically plausible extremes beyond the observed record, evaluated against long climate model large-ensemble simulations. We define two unseen types: direct-hit extremes that affect the target, and near-miss extremes that narrowly miss. These unrealized events reveal hidden risks and can either prompt proactive adaptation or reinforce a false sense of resilience. Applying DeepX-GAN to the Middle East and North Africa shows that the probability of unseen heat extremes is disproportionately distributed toward countries with high vulnerability and low socioeconomic readiness. Using a representative climate simulation, we demonstrate how future warming could expand and shift these hazards, creating persistent hotspots in Northwest Africa and the Arabian Peninsula and new hotspots in Central Africa, necessitating spatially adaptive resilience planning.

Xinyue Liu, Xiao Peng, Shuyue Yan +5

Department of Civil and Environmental Engineering, National University of Singapore, Singapore. · School of Environmental Science and Engineering, Southern University of Science and Technology, Shenzhen, Guangdong, China. · Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, Zhejiang, China. +1

Global Climate Models Extremes Temporal Dependence Generative Models Generative Adversarial Network Resilience

Jul 1, 2025physics.flu-dyn

Guided Unconditional and Conditional Generative Models for Super-Resolution and Inference of Quasi-Geostrophic Turbulence

Typically, numerical simulations of Earth systems are coarse, and Earth observations are sparse and gappy. We apply four generative diffusion modeling approaches to super-resolution and inference of forced two-dimensional quasi-geostrophic turbulence on the beta-plane from coarse, sparse, and gappy observations. Two guided approaches minimally adapt a pre-trained unconditional model: SDEdit modifies the initial condition, and Diffusion Posterior Sampling (DPS) modifies the reverse diffusion process score. Two conditional approaches, a vanilla variant and classifier-free guidance, require training with paired high-resolution and observation data. We consider multiple test cases spanning: two regimes, eddy and anisotropic-jet turbulence; two Reynolds numbers, 10^3 and 10^4; and two observation types, 4x coarse-resolution fields and coarse, sparse and gappy observations. Our comprehensive skill metrics include norms of the reconstructed vorticity fields, turbulence statistical quantities, and quantifications of the super-resolved probabilistic ensembles and their errors. We also study the sensitivity to tuning parameters such as guidance strength. Results show that the generated super-resolution fields of SDEdit are unphysical, while those of DPS are reasonable but with smoothed fine-scale features; however, neither of these lower-cost models propagates observational information effectively to unobserved regions. The two conditional models require re-training, but reconstruct missing fine-scale features, are cycle-consistent with observations, and predict correct turbulence statistics, including the tails. Further, their mean errors are highly correlated with and predictable from their ensemble standard deviations. Results highlight the tradeoffs between ease of implementation, fidelity (sharpness), and cycle-consistency of the diffusion models, and offer practical guidance for deployment.

Anantha Narayanan Suresh Babu, Akhil Sadam, Pierre F. J. Lermusiaux

Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, USA · Center for Computational Science and Engineering, Massachusetts Institute of Technology, Cambridge, USA

Turbulent Flows Generative Super-Resolution Diffusion Sampling Generative Models Vortex Earth Observation+1

Jun 30, 2025q-bio.NC

Neural Langevin Machine: a local asymmetric learning rule can be creative

Fixed points of recurrent neural networks can be leveraged to store and generate information. These fixed points are captured by the Boltzmann-Gibbs measure, which leads to neural Langevin dynamics that relax to those fixed points for generative learning of a real dataset. We call this type of generative model a neural Langevin machine, which derives an asymmetric and firing-rate-speed-adjusted learning rule requiring only local neural signals, thereby bearing biological relevance in terms of local predictive learning. An out-of-equilibrium regime of the generative process is revealed, together with a memorization-to-generalization transition with increasing training data size. The neuro-inspired machine can also realize a continuous exploration of the phase space for different kinds of generative images and can denoise a corrupted image as well.

Zhendong Yu, Weizhong Huang, Haiping Huang

PMI Lab, School of Physics, Sun Yat-sen University, Guangzhou 510275, People’s Republic of China · Guangdong Provincial Key Laboratory of Magnetoelectric Physics and Devices, Sun Yat-sen University, Guangzhou 510275, People’s Republic of China

Langevin Dynamics Generative Models Boltzmann Distributions Information Asymmetry

Jun 23, 2025cs.LG

Local Learning Rules for Out-of-Equilibrium Physical Generative Models

We show that the out-of-equilibrium driving protocol of score-based generative models (SGMs) can be learned via local learning rules. The gradient with respect to the parameters of the driving protocol is computed directly from force measurements or from observed system dynamics. As a demonstration, we implement an SGM in a network of driven, nonlinear, overdamped oscillators coupled to a thermal bath. We first apply it to the problem of sampling from a mixture of two Gaussians in 2D. Finally, we train an oscillator network on the MNIST dataset to generate images of handwritten digits 0 and 1.

Cyrill Bösch, Geoffrey Roeder, Marc Serra-Garcia +1

Department of Computer Science, Princeton University, Princeton, NJ 08540, USA · AMOLF, Science Park 104, 1098 XG Amsterdam, The Netherlands

Statistical Physics Generative Models Gradient-Based Optimization Oscillatory Dynamics

Jun 9, 2025cs.GR

Squeeze3D: Extreme Neural Compression with Latent Space Bridging

We propose Squeeze3D, a novel framework that leverages implicit prior knowledge learnt by existing pre-trained encoders and decoders to compress 3D data at extremely high compression ratios. Our approach bridges the latent spaces between a pre-trained encoder and a pretrained decoder model through trainable mapping networks. Any 3D asset represented as a mesh, point cloud, or radiance field is first encoded by the pre-trained encoder and then transformed (i.e. compressed) into a highly compact latent code by a mapping network. This latent code can effectively be used as an extremely compressed representation of the mesh, point cloud, or radiance field. A mapping network transforms the compressed latent code into the latent space of a powerful generative model; the decoder of this generative model then recreates the original 3D asset (i.e. decompression). Squeeze3D is trained entirely on generated synthetic data and does not require any 3D datasets. The Squeeze3D architecture can be flexibly used with existing pre-trained 3D encoders and existing generative models. It can flexibly support different formats, including meshes, point clouds, and radiance fields. Our experiments demonstrate that Squeeze3D achieves compression ratios of up to 2187

\times

for textured meshes, 58.5

\times

for point clouds, and more than 650

\times

for radiance fields while maintaining visual quality comparable to many existing methods. Squeeze3D only incurs a small compression and decompression latency since it does not involve training object-specific networks to compress an object.

Rishit Dagli, Yushi Guan, Sankeerth Durvasula +2

University of Toronto

Large-Scale 3D Editing Dataset 3D Generation Data Compression Methods Point Clouds Latent Space Generative Models

Jun 3, 2025cs.CV

ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing

Scene synthesis and editing has emerged as a promising direction in computer graphics. Current trained approaches for 3D indoor scene generation either oversimplify object semantics through one-hot class encodings (e.g., 'chair' or 'table'), require masked diffusion for editing, ignore room boundaries, or rely on floor plan renderings that fail to capture complex layouts. LLM-based methods enable richer semantics via natural language, but lack editing functionality, are limited to rectangular layouts, or rely on weak spatial reasoning from implicit world models. We introduce ReSpace, a generative framework for autoregressive text-driven 3D indoor scene synthesis and editing. Our approach features a compact structured scene representation with explicit room boundaries that enables asset-agnostic deployment and frames scene manipulation as a next-token prediction task, supporting object addition, removal, and swapping via natural language. We employ supervised fine-tuning with a preference alignment stage to train a specialized language model for object addition that accounts for user instructions, spatial geometry, object semantics, and scene-level composition. We further introduce a voxelization-based evaluation metric capturing fine-grained geometric violations beyond 3D bounding boxes. Experiments surpass state-of-the-art on object addition and achieve superior human-perceived quality on the application of full scene synthesis, despite not being trained on it.

Martin JJ. Bucher, Iro Armeni

Stanford University

3D Scene Generation 3D Scene Understanding Text-Driven Generative Models Boundaries

May 29, 2025math.NA

A Jump-Diffusion Framework for Irregular Time Series Generation

We propose a framework for generative modeling of continuous-time processes from irregularly and asynchronously recorded data. It is based on the matching of generators and accommodates discontinuous trajectories. Analytical formulas for diffusion and jump bridges yield a family of reference generators that a neural network is trained to match. The key ingredient is that, for our constructed jump bridge, a parametrization of the jump kernel densities by scaled Gaussians admits closed-form expressions for the Kullback-Leibler divergence, allowing simulation-free training.

O. Pfohl, J. Chemseddine, P. Hagemann +3

Department of Mathematics, Humboldt-Universität zu Berlin, 12489, Berlin, Germany · Department of Mathematics, Technische Universität Berlin, 10623, Berlin, Germany · Bundesanstalt für Materialforschung und -prüfung, 12205, Berlin, Germany +1

Time-Series Generation Stochastic Processes Generative Models Diffusion Dynamics Kullback-Leibler Divergence

May 28, 2025cs.LG

Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation

Modeling physical systems in a generative manner offers several advantages, including the ability to handle partial observations, generate diverse solutions, and address both forward and inverse problems. Recently, diffusion models have gained increasing attention in the modeling of physical systems, particularly those governed by partial differential equations (PDEs). However, diffusion models only access noisy data

\boldsymbol{x}_t

at intermediate steps, making it infeasible to directly enforce constraints on the clean sample

\boldsymbol{x}_0

at each noisy level. As a workaround, constraints are typically applied to the expectation of clean samples

\mathbb{E}[\boldsymbol{x}_0|\boldsymbol{x}_t]

, which is estimated using the learned score network. However, imposing PDE constraints on the expectation does not strictly represent the one on the true clean data, known as Jensen's Gap. This gap creates a trade-off: enforcing PDE constraints may come at the cost of reduced accuracy in generative modeling. To address this, we propose a simple yet effective post-hoc distillation approach, where PDE constraints are not injected directly into the diffusion process, but instead enforced during a post-hoc distillation stage. We term our method as Physics-Informed Distillation of Diffusion Models (PIDDM). This distillation not only facilitates single-step generation with improved PDE satisfaction, but also support both forward and inverse problem solving and reconstruction from randomly partial observation. Extensive experiments across various PDE benchmarks demonstrate that PIDDM significantly improves PDE satisfaction over several recent and competitive baselines, such as PIDM, DiffusionPDE, and ECI-sampling, with less computation overhead. Our approach can shed light on more efficient and effective strategies for incorporating physical constraints into diffusion models.

Yi Zhang, Peng Wang, Difan Zou

Institute of Data Science · Department of Computer and Information Science · Institute of Data Science & School of Computing and Data Science

Partial Differential Equations Diffusion Models Generative Models Few-Step Generation

May 17, 2025eess.IV

Bridging the Inter-Domain Gap through Low-Level Features for Cross-Modal Medical Image Segmentation

This paper addresses cross-modal medical image segmentation, focusing on MRI-CT transfer in a source-only domain generalization setting. During training, only source-modality samples are available, while unlabeled target-modality images are used for testing. We propose LowBridge, which builds on the observation that cross-modal images share similar low-level features (e.g. edges) as they depict the same types of anatomical structures. Specifically, we first train a generative model to recover the source images from their edge features, followed by training a segmentation model on the generated source images, separately. At test time, edge features from the target images are input to the pretrained generative model to generate source-style target domain images, which are then segmented using the pretrained segmentation network. Experiments on various public datasets demonstrate that LowBridge achieves state-of-the-art performance, outperforming ten existing approaches. Ablation studies further show that LowBridge is compatible with different types of generative and segmentation models, suggesting its generalizability and potential to benefit from future advances in these models. The code will be available at https://github.com/JoshuaLPF/LowBridge.

Pengfei Lyu, Pak-Hei Yeung, Jing Xia +5

Northeastern University, Shenyang, China · Nanyang Technological University, Singapore

Semi-Supervised Medical Image Segmentation Multimodal Domain Generalization Generative Models Object-Level Control

May 16, 2025cs.CV

Conditioning Residuals for Diffusion Models via Representation Feedback

Diffusion models now serve as a common foundation for multimedia generation, and useful intermediate representations emerge during their generative training. Standard architectures, however, propagate these representations through the main feature stream, without explicitly reintroducing their encoded semantics to later denoising layers. Meanwhile, such backbones already provide a conditioning pathway for global modulation by predefined inputs. This work examines whether this native pathway can also route internally inferred semantics as evolving, sample-dependent cues. We propose Conditioning Residuals, a lightweight feedback mechanism that converts aggregated features into residuals added to condition embeddings. By feeding back compact feature summaries, it provides adaptive generative guidance and encourages a tighter semantic bottleneck, without external encoders, auxiliary objectives, or sampling-time changes. It supports feedback at one or multiple depths in UNet and DiT backbones, with negligible overhead. Across diffusion formulations, backbone configurations, and datasets, experiments show consistent gains in generative performance, along with stronger representations in downstream linear probing and segmentation. Mechanistic analyses reveal improved generative training dynamics and reshaped feature structure, suggesting a grounded, generalizable way to enhance diffusion backbones from within.

Weilai Xiang, Hongyu Yang, Di Huang +1

State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Beijing 100191, China · Institute of Artificial Intelligence, Beihang University, Beijing 100191, China · School of Computer Science and Engineering, Beihang University, Beijing 100191, China

Diffusion Models Generative Models Feedback Contextual Cues

May 7, 2025cs.LG

Riemannian Denoising Diffusion Probabilistic Models

We propose Riemannian Denoising Diffusion Probabilistic Models (RDDPMs) for learning distributions on submanifolds of Euclidean space that are level sets of functions, including most of the manifolds relevant to applications. Existing methods for generative modeling on manifolds rely on substantial geometric information such as geodesic curves or eigenfunctions of the Laplace-Beltrami operator and, as a result, they are limited to manifolds where such information is available. In contrast, our method, built on a projection scheme, can be applied to more general manifolds, as it only requires being able to evaluate the value and the first order derivatives of the function that defines the submanifold. We provide a theoretical analysis of our method in the continuous-time limit, which elucidates the connection between our RDDPMs and score-based generative models on manifolds. The capability of our method is demonstrated on datasets from previous studies and on new datasets sampled from two high-dimensional manifolds, i.e.

\mathrm{SO}(10)

and the configuration space of molecular system alanine dipeptide with fixed dihedral angle.

Zichen Liu, Wei Zhang, Christof Schütte +1

Center for Data Science, Peking University, Beijing 100871, P.R. China · Zuse Institute Berlin, Takustrasse 7, Berlin 14195, Germany · Institute of Mathematics, Freie Universität Berlin and Zuse Institute Berlin, Takustrasse 7, Berlin 14195, Germany +1

Data Manifold Score-Based Diffusion Model Denoising Diffusion Probabilistic Model Generative Models Graph Laplacians

May 1, 2025cs.CV

Multimodal Language Models as Text-to-Image Model Evaluators

The steady improvements of text-to-image (T2I) generative models lead to slow deprecation of automatic evaluation benchmarks that rely on static datasets, motivating researchers to seek alternative ways to evaluate T2I progress. We present Multimodal Text-to-Image Eval (MT2IE), an evaluation framework in which a single multimodal large language model (MLLM) acts as an evaluator agent, iteratively generating the evaluation prompts and scoring the resulting images. We show that MT2IE's image-text consistency scores have higher correlation with human judgment than metrics previously introduced in the literature. MT2IE generates prompts that are efficient at probing T2I model performance: closely recovering the official T2I model rankings of three structurally distinct benchmarks from just 20 generated evaluation prompts, 28-105x fewer than the benchmarks' own prompt sets. When compared to existing evaluation metrics such as CLIPScore, VIEScore, and VQAScore, MT2IE's T2I model rankings are more faithful and far more consistent across multiple evaluation seeds when using the same number of prompts. MT2IE can also adapt evaluation to the model being tested: rewriting each prompt based on the model's own measured performance to produce a bespoke per-model benchmark that still recovers the official rankings and keeps the evaluated model in an informative scoring range. We hope that these results will encourage the development of dynamic and interactive evaluation frameworks, and mitigate the deprecation of automatic evaluation benchmarks.

Jiahui Chen, Candace Ross, Reyhane Askari-Hemmat +5

1FAIR at Meta · University of Texas at Austin · ∗Work done at Meta +3

Modern Text-To-Image Models Text-To-Image Evaluation Benchmarks Multimodal Large Language Models Generative Models Similarity and Metrics

Apr 3, 2025cs.CV

Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systems

Recent advances in image and video generation raise hopes that these models possess world modeling capabilities-the ability to generate realistic, physically plausible videos. This could revolutionize applications in robotics, autonomous driving, and scientific simulation. However, before treating these models as world models, we must ask: Do they adhere to physical laws? Current evaluation methods rely on subjective judgments or trajectory matching, limiting their usage for physical reasoning estimation, where many generations could be physically plausible. Thus, we introduce Morpheus, one of the first physics-informed evaluation frameworks for measuring the ability of video generation models to comprehend Newtonian dynamics. Morpheus features 130 real-world videos capturing physical phenomena, guided by conservation laws. Using those as conditioning for video generation, we assess physical plausibility leveraging interpretable metrics evaluated with respect to infallible conservation laws known per physical setting, leveraging advances in physics-informed neural networks and vision-language foundation models. Importantly, Morpheus targets controlled Newtonian rigid-body settings to enable quantitative checks. Our findings reveal that even with advanced prompting and video conditioning, contemporary models struggle to encode physical principles despite generating aesthetically pleasing videos.

Antonios Tragoudaras, Chenyu Zhang, Daniil Cherniavskii +7

University of Amsterdam, Amsterdam, The Netherlands · University of Trento, Trento, Italy.

Generative Video Models Physical Plausibility Video Generation Generative Models Autonomous Driving

Apr 3, 2025cs.CV

Generative Classifier for Domain Generalization

Domain generalization (DG) aims to improve the generalizability of computer vision models toward distribution shifts. The mainstream DG methods predominantly focus on learning domain invariance across domains, however, such methods overlook the untapped potential inherent in domain-specific information. While the prevailing practice of discriminative linear classifier has been tailored to domain-invariant features, it struggles when confronted with diverse domain-specific information, \emph{e.g.,} intra-class shifts, that exhibits multi-modality. To address these issues, we explore the theoretical implications of relying on domain-invariant features, revealing the crucial role of domain-specific information in mitigating the target risk for DG. Drawing from these insights, we propose Generative Classifier-driven Domain Generalization (GCDG), introducing a generative paradigm for the DG classifier based on Gaussian Mixture Models (GMMs) for each class across domains. GCDG consists of three key modules: Heterogeneity Learning Classifier~(HLC), Spurious Correlation Blocking~(SCB), and Diverse Component Balancing~(DCB). Concretely, HLC attempts to model the feature distributions and thereby capture valuable domain-specific information via GMMs. SCB identifies the neural units containing spurious correlations and perturbs them, mitigating the risk of HLC learning irrelevant spurious patterns. Meanwhile, DCB ensures a balanced contribution of components within HLC, preventing the underestimation or neglect of critical components. In this way, GCDG excels in capturing the nuances of domain-specific information characterized by diverse distributions. Extensive experiments show GCDG's comparable performance on five DG benchmarks and one face anti-spoofing dataset, seamlessly integrating into existing DG methods with consistent improvements.

Shaocong Long, Qianyu Zhou, Dexuan Zhang +1

Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China · College of Computer Science and Technology and the Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, Jilin University, Changchun, China

Multimodal Domain Generalization Domain Invariance Generative Models Spurious Correlations

Mar 11, 2025cs.LG

A Generative-AI Modeling Framework for Explainable Decision Support in Complex Geosteering Scenarios

The real-time process of directional changes while drilling, known as geosteering, is crucial for hydrocarbon extraction and emerging directional drilling applications such as geothermal energy, civil infrastructure, and CO2 storage. The geo-energy industry seeks an automatic geosteering workflow that continually updates subsurface uncertainties and captures the latest geological understanding, informed by real-time observations. We propose a real-time, AI-driven geosteering workflow that integrates Generative Adversarial Networks (GANs) for geological parameterization, ensemble methods for model updating, and global discrete dynamic programming (DDP) optimization for complex decision-making during directional drilling operations. Our framework relies on offline training of a GAN model to reproduce relevant geology realizations and a Forward Neural Network (FNN) to model the response of Logging-While-Drilling (LWD) tools for a given geomodel. This paper introduces a first-of-its-kind workflow that progressively reduces GAN-geomodel uncertainty around and ahead of the drilling bit and adjusts the well plan accordingly. The workflow automatically integrates real-time around-bit LWD, which, through learned geological correlations, reduces uncertainty in predicted geology ahead of drilling. A DDP-based decision support system leverages probabilistic look-ahead predictions to suggest better steering strategies. We test the workflow prototype on a small yet challenging low-net-to-gross drilling scenario with several possible targets. The results show that the workflow produces meaningful steering recommendations and, through its probabilistic updates, automatically maps formation boundaries along the drilled well.

Sergey Alyaev, Kristian Fossum, Hibat Errahmen Djecta +2

NORCE Norwegian Research Centre AS, Bergen, Norway. · University of Stavanger, Stavanger, Norway. · Heriot-Watt University, Edinburgh, United Kingdom.

Drilling Data Geological Semantic Understanding Map Uncertainty Generative Models Software Engineering Decision Support

Feb 25, 2025stat.ML

Recursive Learning Without Collapse: A Weighting-Based Stabilization Framework

Recent studies identified an intriguing phenomenon in recursive generative model training known as model collapse, where models trained on data generated by previous models exhibit severe performance degradation. Addressing this issue and developing more effective training strategies have become central challenges in generative model research. In this paper, we investigate this phenomenon within a novel framework, where generative models are iteratively trained on a combination of newly collected real data and synthetic data from the previous training step. To develop an optimal training strategy for integrating real and synthetic data, we evaluate the performance of a weighted training scheme in various scenarios, including Gaussian distribution estimation, generalized linear models, and nonparametric estimation. We theoretically characterize the impact of the mixing proportion and weighting scheme of synthetic data on the final model's performance. Our key finding is that, across different settings, the optimal weighting scheme under different proportions of synthetic data asymptotically follows a unified expression, revealing a fundamental trade-off between leveraging synthetic data and model performance. In some cases, the optimal weight assigned to real data corresponds to the reciprocal of the golden ratio. Finally, we validate our theoretical results on extensive simulated datasets and a real tabular dataset.

Hengzhi He, Shirong Xu, Guang Cheng

Department of Statistics and Data Science, University of California, Los Angeles · Wang Yanan Institute for Studies in Economics, Xiamen University

Synthetic Data Generative Models Collapse Stabilization

Feb 20, 2025stat.ML

Bringing Generative Learning to Representation Learning: Self-Supervised Transfer Learning as Distribution Matching

Most self-supervised learning objectives defend against collapse but leave the target representation law unspecified. We formulate representation learning as Distribution Matching (DM), learning an augmentation-invariant encoder whose induced law matches an explicit geometric reference. The reference law specifies what the learned representation distribution should look like, whereas a separately chosen discrepancy determines how deviations from this target are measured; here we use Mallows distance. The DM framework reveals a directional inverse: generative learning maps a tractable reference to data, whereas representation learning maps data to a designed reference law. We connect the population objective to class-centre separation and classification error and prove a non-asymptotic neural-sieve guarantee. Simulations and image benchmarks show manifold rectification, fine-grained structure and transfer across label spaces.

Yuling Jiao, Wensen Ma, Defeng Sun +2

School of Artificial Intelligence, Wuhan University, 430072, Wuhan, China · School of Mathematics and Statistics, Wuhan University, Wuhan, 430072, China · Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong SAR, China +2

Self-Supervised Representations Representation Learning Transfer Learning Generative Models Distribution Matching

Jan 22, 2025stat.ML

Low-dimensional adaptation of diffusion models: Convergence in total variation

This paper investigates how diffusion generative models leverage (unknown) low-dimensional structure to accelerate sampling. Focusing on two mainstream samplers -- the denoising diffusion implicit model (DDIM) and the denoising diffusion probabilistic model (DDPM), we prove that their iteration complexities under exact score functions are at most the order of

k/\varepsilon

(up to log factor), where

\varepsilon

is the precision in total variation distance and

k

is some intrinsic dimension of the target distribution. We further extend these convergence guarantees to the setting in which the score functions are learned from data rather than known exactly, showing that the convergence performance degrades gracefully under suitable score estimation assumptions. We then show that these assumptions are attainable via kernel-based score estimators with finite-sample guarantees that also adapt to the low-dimensional structure. Our results apply to a broad family of target distributions without requiring smoothness or log-concavity. Our findings provide the first rigorous evidence for the adaptivity of the DDIM-type samplers to unknown low-dimensional structure, and improve over the state-of-the-art DDPM theory regarding total variation convergence.

Jiadong Liang, Zhihan Huang, Yuxin Chen

Department of Statistics and Data Science, the Wharton School, University of Pennsylvania

Score-Based Diffusion Model Diffusion Models Denoising Diffusion Probabilistic Model Low-Dimensional Structure Generative Models Convergence+1

Jan 3, 2025cs.LG

Active Learning Enables Generation of Molecules that Advance the Known Pareto Front

Although generative models hold promise for discovering molecules with optimized desired properties, they often fail to suggest synthesizable molecules that improve upon the properties of the structures represented in the training distribution. We find that this limitation arises not only from the molecule generation process itself, but also from the poor generalization capabilities of molecular property predictors. We address this challenge by creating a closed-loop molecule generation pipeline with iterative retraining on new quantum chemical simulation data. Compared against static, single-pass generative modeling approaches, only our closed-loop iterative workflow generates molecules with properties extending beyond the training distribution (up to 0.44 standard deviations beyond the original range) and achieves a 79% improvement in out-of-distribution molecule classification accuracy. Furthermore, by conditioning molecular generation on thermodynamic stability data obtained during the iterative loop, the proportion of stable and hence potentially synthesizable molecules generated is 3.5x higher than the next-best model.

Evan R. Antoniuk, Peggy Li, Nathan Keilbart +3

Molecular Generation Molecular Property Prediction Molecules Generative Models Active Learning Pareto Frontier

Dec 13, 2024cs.CL

Modeling Story Expectations: A Generative Framework using LLMs

Consumers' engagement with stories is shaped by their expectations about what will happen next, yet modeling these forward-looking beliefs over unstructured narrative content has remained challenging. We develop a framework that uses large language models to approximate consumers' story expectations. Our method generates multiple imagined story continuations from a pre-trained LLM and extracts interpretable, theory-motivated features from these continuations, such as emotion and narrative path features. We propose two complementary validation procedures suited to different data availability: a survey-based approach that compares LLM-derived expectations to human-reported beliefs, and a rational-expectations approach that compares them to actual story outcomes. Applying the framework to both survey data collected in a controlled lab setting and observational data from an online reading platform, we find that LLM-derived expectations correlate with human-reported beliefs as well as actual story continuations along all features studied. In both settings, forward-looking expectations are associated with reader engagement above and beyond features of the content already consumed. Our framework provides a scalable method for modeling consumer beliefs about narrative content, with implications for content creation, platform strategy, and the study of narrative media.

Hortense Fong, George Gui, Bo Yang

Columbia Business School

Narrative Generation Narratives Generative Models

Oct 3, 2024stat.ML

Local Flow Matching Generative Models

Flow Matching (FM) is a simulation-free method for learning a continuous, invertible flow that interpolates between two distributions, and in particular generates data from noise. Inspired by the variational nature of the diffusion process as a gradient flow, we introduce a stepwise FM model, Local Flow Matching (LFM), which sequentially learns a sequence of FM submodels, each matching a diffusion process up to the time-step size in the data-to-noise direction. In each step, the two distributions to be interpolated by the sub-flow model are closer than those in the full-flow matching model, which interpolates data to noise distributions, enabling smaller models with more efficient training. This variational perspective also allows us to prove a theoretical generation guarantee for the proposed flow model in terms of the

χ^2

-divergence between the generated and true data distributions, leveraging the contraction property of the diffusion process. In practice, the stepwise structure of LFM is naturally amenable to model distillation, and various distillation techniques can be applied to accelerate generation. We empirically demonstrate that LFM achieves competitive generative performance compared to FM on unconditional generation of tabular and image datasets, and on conditional generation of robotic manipulation policies.

Chen Xu, Xiuyuan Cheng, Yao Xie

H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology. · Department of Mathematics, Duke University

Generative Flow Networks Conditional Flow Matching Generative Models Invertible Flows Gradient

Apr 26, 2024cs.CV

MaPa: Text-driven Photorealistic Material Painting for 3D Shapes

This paper aims to generate materials for 3D meshes from text descriptions. Unlike existing methods that synthesize texture maps, we propose to generate segment-wise procedural material graphs as the appearance representation, which supports high-quality rendering and provides substantial flexibility in editing. Instead of relying on extensive paired data, i.e., 3D meshes with material graphs and corresponding text descriptions, to train a material graph generative model, we propose to leverage the pre-trained 2D diffusion model as a bridge to connect the text and material graphs. Specifically, our approach decomposes a shape into a set of segments and designs a segment-controlled diffusion model to synthesize 2D images that are aligned with mesh parts. Based on generated images, we initialize parameters of material graphs and fine-tune them through the differentiable rendering module to produce materials in accordance with the textual description. Extensive experiments demonstrate the superior performance of our framework in photorealism, resolution, and editability over existing methods. Project page: https://zju3dv.github.io/MaPa

Shangzhan Zhang, Sida Peng, Tao Xu +7

Zhejiang University · Ant Group · Shenzhen University

Differentiable Rendering Materials Generative Models

Feb 6, 2024stat.ML

PQMass: Probabilistic Assessment of the Quality of Generative Models using Probability Mass Estimation

We propose a likelihood-free method for comparing two distributions given samples from each, with the goal of assessing the quality of generative models. The proposed approach, PQMass, provides a statistically rigorous method for assessing the performance of a single generative model or the comparison of multiple competing models. PQMass divides the sample space into non-overlapping regions and applies chi-squared tests to the number of data samples that fall within each region, giving a p-value that measures the probability that the bin counts derived from two sets of samples are drawn from the same multinomial distribution. PQMass does not depend on assumptions regarding the density of the true distribution, nor does it rely on training or fitting any auxiliary models. We evaluate PQMass on data of various modalities and dimensions, demonstrating its effectiveness in assessing the quality, novelty, and diversity of generated samples. We further show that PQMass scales well to moderately high-dimensional data and thus obviates the need for feature extraction in practical applications.

Pablo Lemos, Sammy Sharief, Esmeralda S. Whitammer +4

1Mila – Québec Artificial Intelligence Institute · 2Ciela Institute · 3Université de Montréal +3

Generative Models Distributional Fidelity Data Quality Probability Rejection Sampling Comparison

Feb 1, 2024cs.LG

Building Expressive and Tractable Probabilistic Generative Models: A Review

We present a comprehensive survey of the advancements and techniques in the field of tractable probabilistic generative modeling, primarily focusing on Probabilistic Circuits (PCs). We provide a unified perspective on the inherent trade-offs between expressivity and tractability, highlighting the design principles and algorithmic extensions that have enabled building expressive and efficient PCs, and provide a taxonomy of the field. We also discuss recent efforts to build deep and hybrid PCs by fusing notions from deep neural models, and outline the challenges and open questions that can guide future research in this evolving field.

Sahil Sidheekh, Sriraam Natarajan

The University of Texas at Dallas

Probabilistic Model Generative Models Expressivity Tractability Circuits Predictive Coding+3

Oct 18, 2023cs.LG

Consistent Distributed Ranking of Generative Models via Kernel Distances

Ranking generative models based on the fidelity and diversity of their outputs is required to identify the best generator in a group of candidate generative AI models. To rank a group of models in a conventional centralized setting, a standard score is commonly evaluated for each involved model. The selection and design of reference-based evaluation scores have been extensively studied in centralized settings, where the reference samples are drawn from a single probability distribution. However, in practical scenarios including distributed learning contexts, reference samples are distributed across multiple clients, each potentially with a heterogeneous data distribution. In this work, we investigate the ranking of generative models in such distributed settings with heterogeneous data distributions across clients. We focus on the widely used family of kernel distance (KD) evaluation metrics. We prove that, for every kernel function, ranking models by the averaged KD scores of individual clients yields the same ordering as a centralized KD evaluation using the combined reference data from all the clients. We further extend our analysis to other popular metrics, including the Fréchet Distance (FD), for which the individual client scores could be insufficient for accurate model ranking. We present the numerical results of several experiments on standard image datasets and generative models to validate our theoretical findings regarding distributed ranking using various evaluation scores.

Zixiao Wang, Farzan Farnia, Zhenghao Lin +2

Model Evaluation Generative Models Score-Based Diffusion Model Ranking Kernel Method