Autoencoders

Follow this topic Ask about this topic

Autoencoders learn a compact or structured latent representation by encoding data and training a decoder to reconstruct it, supporting dimensionality reduction, compression, and feature learning without labels. Common approaches include convolutional, recurrent, and transformer autoencoders; variational and sparse variants; and models with geometric, physical, or domain-specific constraints. They are used on images, audio and video, scientific simulations, medical data, and industrial time series, with evaluation ranging from reconstruction and compression to anomaly detection and downstream prediction. A central tension is that latents optimized for faithful reconstruction are not always useful for generation, interpretation, or stable long-horizon prediction, so current work increasingly evaluates and shapes representations for their intended downstream use.

Latest papers

155 papers23 papers in the last four weeks, up 64% on the four weeks before. 0.1% of all new papers.

Jul 6, 2026

Orthogonal Dendritic Intrinsic Networks: An Architecture for Significance-Ordered, Orthogonal Latent Spaces

Principal Component Analysis or PCA-like properties (orthogonality, variance ranking) are seldom realized in deep autoencoder architectures. In this work, we present ODIN (Orthogonal Dendritic Intrinsic Network), a novel autoencoder architecture that recovers PCA-like latent structure in a fully non-linear regime. By incorporating a set of geometric constraints directly into the training objective, ODIN encourages latent dimensions to be mutually orthogonal and ordered by explained variance, mirroring the interpretable decomposition of PCA while retaining the expressive power of deep networks. We provide theoretical grounding for these constraints and demonstrate their compatibility with standard encoder-decoder frameworks. We also establish empirical results for both synthetic and real world datasets, establishing a principled path toward interpretable, structured feature learning and dimensionality reduction.

Jeanie Schreiber, Tyrus Berry, Zeeshan Ahmed

Department of Mathematical Sciences George Mason University Fairfax, VA 22030, USA · Sensor Science Division, Physical Measurement Laboratory NIST Gaithersburg, MD 20899, USA

Representation Learning Principal Component Analysis Autoencoders Unsupervised Representation Learning Orthogonal Representation Learning Dimensionality Reduction+1

Jul 6, 2026

Biologically Informed Deep Neural Networks for Multi-Omic Integration, Pathway Activity Inference and Risk Stratification in Cancer

Integrating complex, multi-omics data presents significant challenges. Existing approaches often face a trade-off between model interpretability and representational capacity, with most either relying on post-hoc interpretation or use linear models that may overlook complex interactions. We report Pathway Activity Autoencoders for the multi-omics setting, which embed prior knowledge via pathway-informed architectural constraints, fostering interpretability, while preserving representational power. Our multi-omic framework is applied in the context of breast cancer and is evaluated in survival prediction and subtype classification with results indicating a positive effect of integration. We conduct analysis of individual omics layer impact on end-task performance, revealing that gene, protein, and microRNA expression layers provide the strongest contribution. Repeatability studies indicate that, while dropout improves model robustness and consistency, excessive regularisation can reduce predictive performance. Finally, visualizations of the learned feature space illustrate the framework's intrinsic transparency and clinical relevance. The results underscore the value of multi-omic integration and delineate the impact of individual omics layers, establishing practical guidelines for integration within our framework. Overall, our pathway activity autoencoder frameworks yield superior latent representations that are biologically meaningful and are directly translatable into clinically relevant insights.

Pedro Henrique da Costa Avelar, Le Ou-Yang, Min Wu +1

Department of Informatics, King’s College London, London WC2B 4BG, United Kingdom · Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore · Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester +1

Cancer Survival Prediction Autoencoders Oncology Cancer Subtype Classification Multi-Omics Data Integration

Jun 26, 2026

Autoencoder Architectures for Athlete Performance Scoring from Wearable Telemetry

Wearable devices produce large, high dimensional training logs for everyday runners, and interpretation rather than data collection is now the limiting step. This paper evaluates five dimensionality reduction models, three autoencoder variants, PCA, and a Variational Autoencoder, on their ability to compress nine sensor runner profiles into a single scalar performance indicator, the latent score. Because the setting is fully unsupervised, model quality is assessed along two complementary axes: reconstruction error (Mean Squared Error) and latent score interpretability, measured via Spearman and Kendall rank correlations, Mutual Information, and Permutation Importance. These are combined into a composite selection criterion that prevents selecting models on reconstruction accuracy alone. Feature rankings from the four metrics are aggregated via a modified Borda count, and their stability is confirmed by bootstrap validation. A two feature linear baseline is included to anchor the comparison. Deep autoencoder achieved the lowest reconstruction error and the highest composite score. Once the PCA hidden layers were widened, the deeper variants became closely competitive with Deep AE on the composite criterion, indicating that the limiting factor was hidden layer capacity rather than the one dimensional bottleneck. Running pace, aerobic decoupling, and average heart rate emerged as the dominant latent score drivers across all models and resampling runs, consistent with established physiology.

Mateusz Kubita, Jan Zubalewicz, Krzysztof Siwek

Warsaw University of Technology Warsaw, Poland

Variational Autoencoders Autoencoders Dimensionality Reduction

Jun 25, 2026

Elastic Time: Dynamic Frame Rate Bottlenecks for Neural Audio Coding

Neural audio autoencoders have become a core component of compression, feature extraction, and generation. However, while existing systems support variable bitrate, the vast majority of models still operate at a fixed latent frame-rate, allocating equal temporal budget to regions with very different information density, which can result in unnecessarily long sequences. We introduce Elastic Time, a dynamic frame-rate bottleneck that converts fixed-frame-rate autoencoders to dynamic ones. Our method learns a lightweight latent predictor used to decide which frames can be skipped and later reconstructed, enabling efficient greedy boundary selection at inference. Experiments show our method enables deployment-time rate control while improving efficiency-quality tradeoffs relative to baselines. Overall, we provide a flexible mechanism for adjusting temporal resolution in audio autoencoders, potentially facilitating more efficient downstream modeling for generation and long-context tasks.

Dimitrios Bralios, Paris Smaragdis, Minje Kim

University of Illinois Urbana-Champaign, Urbana, IL, USA · Massachusetts Institute of Technology, Cambridge, MA, USA

Neural Audio Codecs Autoencoders Neural Network Compression

Jun 24, 2026

Variational Autoencoder Layer

Variational Autoencoders (VAEs) belong to a family of autoencoders with probabilistic properties, making them well suited for generating data by producing a smooth and continuous latent space. Despite being introduced over a decade ago, the method continues to be widely adopted in both research and industry for diverse applications. While VAEs are typically used as standalone models, this paper introduces a novel approach to integrate them as a neural network layer. Furthermore, a new training strategy is proposed for models incorporating these layers, and their performance is thoroughly analyzed.

Gananath R

Variational Autoencoders Autoencoders

Jun 24, 2026

Lightweight PCGAE-Net: Parallel CrossGate Attention and Bottleneck AutoEncoder for Efficient 5G Channel Prediction

Accurate channel state information (CSI) prediction is essential for proactive beamforming and resource management in 5G massive MIMO systems, yet the deployment of high-accuracy transformer-based predictors on base-station hardware remains challenging because the most capable models carry upwards of 30,M parameters. This paper introduces Lightweight PCGAE-Net, which addresses the efficiency problem not by post-hoc compression but by correcting two architectural flaws in the current state of the art. The first is a sequential attention ordering bias: in CS3T-UNet, group-wise temporal attention (GTA) always operates on features that have already been transformed by cross-shaped spatial attention (CSA), distorting what temporal information GTA can capture. We remove this dependency by routing both attention modules to the same layer-normalized input and combining their independent outputs through a learned per-channel sigmoid CrossGate. The second flaw is an uncompressed bottleneck: applying full self-attention at the deepest encoder stage, where channel depth reaches

4C

, is quadratically expensive and carries redundant features. A Bottleneck AutoEncoder (BAE) with

1\times1

convolutions halves this depth and uses an auxiliary reconstruction loss to prevent information collapse. Wrapping these components inside a shallower encoder-decoder with frequency-domain dimensionality reduction (

N_f\!=\!32

,

C\!=\!48

) produces a model with just 8.54,M parameters -- 58% fewer than the CS3T-UNet baseline -- that outperforms it by up to 3.26,dB at 5,km/h and 6.0,dB at 9,km/h in single-step prediction on QuaDriGa dataset.

Uma Kishore Godavarti, K. Giridhar, Vanani Prince Dharmendrabhai +2

Network Modem Team, Samsung R&D India Bangalore · Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai, India

Autoencoders Channel State Prediction Attention Mechanisms

Jun 19, 2026

Memory-Augmented LSTM Autoencoder for Unsupervised Activity Recognition with IMU Sensor Fusion

HAR using Inertial Measurement Unit (IMU) sensors is vital for healthcare monitoring and rehabilitation. Despite deep learning advancements, major challenges remain: reliance on labeled data, multi-sensor fusion complexity, and the limited ability of unsupervised methods to capture spatiotemporal dependencies. These issues are pronounced in real-world scenarios with noisy data, overlapping activities, and missing labels. We propose a fully unsupervised spatiotemporal feature fusion framework using a memory-augmented autoencoder. It enhances activity representations via short temporal windows of multi-sensor IMU data, enabling real-time applications. Our framework extracts hierarchical static features via a Stacked Autoencoder, fusing them within and across sensors. A sequence-to-sequence LSTM Autoencoder then temporally refines these features, incorporating historical motion patterns without labels. We analyze key hyperparameters to identify configurations that maximize feature separability under short-window constraints. Evaluated on DaLiAc and PAMAP2 using realistic inter-class window segmentation, our method achieves 96.6% and 98.4% accuracy, respectively, surpassing supervised baselines and unsupervised approaches. Our method improves feature separability by up to 9% despite shorter temporal windows. While our realistic inter-class segmentation reduces accuracy by ~7%, it was intentionally adopted to better reflect real-world activity transitions and practical relevance.

Saeid Arabzadeh, Farshad Almasganj, Mohammad Mahdi Ahmadi

Faculty of Biomedical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran

Human Activity Recognition Autoencoders Unsupervised Representation Learning Multimodal Sensor Fusion

Jun 18, 2026

Neural Events: Discrete Asynchronous Autoencoders for Event-Based Vision

Event cameras capture dynamic scenes with exceptional temporal fidelity by representing them as a continuous stream of microsecond resolution events. Each individual event, however, only carries minimal semantic value, merely signaling a localized brightness change. To derive meaningful signals, downstream algorithms need to quickly integrate cues from a potentially massive torrent of low-information events. Current architectures, however, are easily overwhelmed, struggling to balance capturing fine-grained temporal dynamics and maintaining a manageable data throughput. This paper proposes a framework to re-tokenize event streams into a small set of highly informative neural events, each representing a local spatio-temporal context window with a discrete learnable code. Every time this code flips, a neural event is triggered, yielding a highly compressed data stream. We demonstrate that, across object detection and classification, networks trained on neural events are on par or surpass the performance of state-of-the-art approaches while reducing the event rate by a factor of 2.0.

Roberto Pellerito, Daniel Gehrig, Shintaro Shiba +1

Robotics and Perception Group, University of Zurich · University of Pennsylvania · The University of Tokyo +1

Event-Based Vision Representation Learning Autoencoders Event-Based Object Detection

Jun 16, 2026

Effects of sparsity and superposition on loss in simple autoencoders

One of the major difficulties in the mechanistic interpretability of neural networks is the occurrence of polysemanticity, which suggests that each neuron is typically responsible for multiple different tasks, impeding a clean interpretation of their function. The seminal paper of Elhage et al. (2022) argues that this occurs due to superposition, a phenomenon where the neural network represents distinct features as non-orthogonal directions in a lower-dimensional space, a strategy that allows much greater compression of the data without sacrificing fidelity due to the feature sparsity of input vectors. Elhage et al. (2022) empirically validates these hypotheses in a rather natural and simple autoencoder with sparse inputs. The contribution of the present work is to analyze the mathematical basis for the occurrence and optimality of superposition, while rigorously corroborating some of their findings. In particular, we provide upper and lower bounds for the L2 reconstruction loss, tight in the very sparse regime, for power activation functions. A short list of interesting open problems are also included at the end.

Mriganka Basu Roy Chowdhury, Eric McLaughlin Weiner

Department of Statistics, UC Berkeley · Department of Materials Science, UC Berkeley

Feature Superposition Activation Sparsity Autoencoders Representation Disentanglement Neural Network Interpretability Neural Network Expressivity

Jun 15, 2026

Sensor-Conditioned Representation Learning via Scene-Relevant Observation Quotients

Learned representations in intelligent sensing systems are often evaluated by reconstruction fidelity or downstream prediction accuracy, but these criteria do not specify which latent distinctions are justified by the sensing process. In sensor-conditioned environments, nuisance factors can change measurements without changing the scene, while distinct scenes may be indistinguishable under limited sensing capability. This paper formulates sensor-conditioned representation correctness as preserving sensing-supported scene distinctions while suppressing nuisance-induced and sensor-unsupported variation. We introduce the scene-relevant observation quotient, a representation target induced by sensing-supported distinguishability after nuisance canonicalization, and develop Observation-Quotient Tucker-Structured Autoencoding (OQ-TSAE), a scene-nuisance factorized framework with diagnostics for false distinction, false merge, nuisance sensitivity, and latent ordering consistency. Experiments on a controlled benchmark show that quotient-consistent supervision improves representation-correctness diagnostics over reconstruction-oriented, metric-learning, and contrastive-learning baselines. Sensitivity, perturbation, and ablation studies show the importance of quotient-aligned supervision, reliable quotient relations, and quotient geometry. Complementary real-radar experiments show that a reconstruction-only OQ-TSAE variant retains competitive downstream utility, robustness under observation degradation, and low seed-to-seed variability. These results suggest that sensor-conditioned representations should be evaluated not only by predictive utility, but also by whether their latent geometry preserves sensing-justified scene distinctions.

Yan Jiao, Pin-Han Ho, Limei Peng

Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, China · Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Canada · School of Computer Science and Engineering, Kyungpook National University, Daegu, South Korea

Representation Learning Autoencoders Invariant Representation Learning

Jun 14, 2026

Distilling Drifting Transformers with Representation Autoencoders

Representation Autoencoders (RAEs) have improved diffusion and flow models by semantically richer latent space owing to the strongly label-wise clustered DINO features in the pretrained encoders. Yet in the distillation stage, the severe anisotropy and large curvatures caused by the rich semantic representations would hinder the convergence and performance, making the trajectory-based distillation unstable. In this work, we argue that the RAE latent space is compatible with distillation via the newly proposed Drifting Models. We first quantitatively study the curvatures and isotropy statistics across different autoencoders, and theoretically reveal that Drifting Model itself is highly likely to fail on extremely scattered spaces like reconstruction-based VAEs. These motivate us to apply the drifting paradigm directly to representation autoencoders. Our proposed method, Drift-RAE, distills pretrained flow models in RAE latent spaces using Drifting, together with insightful modifications that improve training stability by thereotically aligning drifting fields with other frameworks. Regarding the experimental evidences, we achieve 1.77 FID on ImageNet 256 dataset using only 10k distillation steps, surpassing state-of-the-art RAE distillation methods and appearing comparative with the original Drifting Model without requiring an auxiliary MAE feature extractor. The code will be made publicly available.

Jiawei Zhang, Mengfei Xia, Gen Li +1

Tsinghua University · Ant Group · CUHK

Diffusion Model Distillation Autoencoders Drifting Models

Jun 9, 2026

IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder

Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spaces for image generation. However, their reconstruction quality often remains suboptimal, largely because deep VFM representations do not preserve sufficient fine-grained visual detail. This limitation becomes even more severe after discretization, where missing low-level information is difficult to recover. In fact, we observe that shallow VFM features retain considerably richer local appearance and structural detail, which complements the high-level semantics carried by deep features used in existing RAEs. Motivated by this complementary property, we propose Ideal, an In-depth Alignment framework for discrete representation autoencoding. By jointly aligning quantized tokens with both shallow and deep VFM features, Ideal enables the resulting discrete visual tokens to preserve both visual fidelity and rich semantics. Extensive experiments demonstrate that Ideal yields superior reconstruction performance, achieving 0.61 rFID on ImageNet and outperforming the previous best method by 0.28. When used for autoregressive image generation, Ideal further produces a gFID of 1.89, establishing a new state of the art for autoregressive image generation.

Yitong Chen, Zijie Diao, Junke Wang +5

Institute of Trustworthy Embodied AI, Fudan University · Shanghai Innovation Institute · University of Maryland, College Park

Visual Representation Learning Image Tokenization Autoencoders Image Reconstruction Autoregressive Image Generation Visual Token Compression

Jun 6, 2026

TextEconomizer: Enhancing Lossy Text Compression with Denoising Transformers and Entropy Coding

Lossy text compression reduces data size while preserving core meaning, making it well-suited for summarization, automated analysis, and digital archives. Despite the dominance of transformer-based models in language modeling, integrating context vectors and entropy coding into Sequence-to-Sequence (Seq2Seq) generation remains underexplored. A key challenge lies in identifying the most informative context vectors from encoder output and incorporating entropy coding to enhance storage efficiency while maintaining high-quality outputs, even under noisy text. We introduce TextEconomizer, an encoder-decoder framework paired with a transformer neural network that reduces variable-sized inputs by 50% to 80% without prior knowledge of dataset dimensions. Our model achieves competitive compression ratios via entropy coding while delivering near-perfect text quality, assessed by BLEU, ROUGE, METEOR, and semantic similarity scores. TextEconomizer operates with approximately 153x fewer parameters than comparable models, achieving a 5.39x compression ratio without sacrificing semantic quality. We also evaluate an LSTM-based autoencoder achieving a state-of-the-art 67x compression ratio with 196x fewer parameters, and LLaMAFormer, a modified transformer with 263x fewer parameters than ICAE while maintaining competitive text quality. TextEconomizer significantly surpasses existing transformer-based models in balancing memory efficiency and high-fidelity outputs, marking a breakthrough in lossy compression with optimal space utilization.

Mahbub E Sobhani, Anika Tasnim Rodela, Chowdhury Mofizur Rahman +2

United International University, Department of Computer Science and Engineering Dhaka, Bangladesh · BRAC University, Department of Computer Science and Engineering Dhaka, Bangladesh · Southeast University, Department of Computer Science and Engineering Dhaka, Bangladesh

Transformer Autoencoders Entropy Coding

Jun 5, 2026

Beyond Linear and Overcomplete Regimes: A Mean-Field Analysis of Bottleneck Autoencoders

Autoencoders (AEs) learn low-dimensional representations by mapping data into a latent space while minimizing reconstruction error. Despite their empirical success, theoretical understanding remains limited and largely restricted to linear models or settings without a bottleneck. In this work, we study nonlinear AEs with a fixed finite-dimensional bottleneck in the mean-field (MF) regime. We derive explicit MF learning dynamics for both encoder and decoder, providing a tractable characterization of training in the nonlinear setting. We show that, over finite time horizons, the empirical risk of finite-width networks trained with stochastic gradient descent closely tracks the MF risk trajectory with high probability. At optimality, we further establish that the finite-width risk converges to the MF optimum, demonstrating that finite networks are sufficiently expressive to approximate the infinite-width solution.

Santanu Das, Ramyak Bilas, Pascal Esser +1

STCS department Tata Institute Of Fundamental Research Mumbai, India-400005 · Department of Mathematics Indiana University Bloomington Indiana, USA-47405 · Department of Mathematics Ludwig-Maximilians-Universität München Germany, München-80799 +1

Autoencoders Mean-Field Theory Neural Network Approximation Theory Neural Network Training Dynamics

Jun 4, 2026

F3-Tokenizer: Taming Audio Autoencoder Latents for Understanding and Generation

Continuous audio autoencoders reconstruct waveforms well but often produce latents with weak structure for understanding, while self-supervised audio encoders capture semantics but are not directly decodable. This mismatch complicates a single audio tokenizer that must support both understanding and generation. We adapt continuous autoencoder latents to this setting with two components: a noise-regularized autoencoder bottleneck and a latent-side representation encoder. The bottleneck uses channel normalization and stochastic perturbation instead of KL-based variational training, yielding scale-controlled continuous latents for reconstruction and autoregressive generation. The representation encoder is trained on frozen autoencoder latents with RQ-MTP and frozen-LLM supervision. The resulting tokenizer provides high-dimensional representations for understanding while preserving normalized continuous latents as generation targets

Dinghao Zhou, Xingchen Song, Di Wu +3

2WeNet Open Source Community · ∗Equal contribution · 1Nanjing University, China

Audio Representation Learning Autoencoders Audio Understanding Audio Tokenization Tokenizer Adaptation Audio Generation

Jun 3, 2026

A prism hierarchy of learning regimes in large linear autoencoders

Theoretical studies of machine learning models commonly consider different limiting regimes in which the learning dynamics of gradient descent becomes theoretically tractable. It is, however, desirable to have a systematically obtained picture of qualitatively different extreme learning regimes for a particular type of models. In this paper we propose such a picture for large weight-tied linear autoencoders characterized by input and latent dimensions, initialization magnitude, and training set size. This model is nonlinear in the weights and its gradient flow does not have a general theoretical solution. We show that at the level of the formal loss-expansion hierarchy, its extreme regimes are naturally associated with faces of a triangular prism. In particular, there are five basic extreme regimes associated with the 2-faces of the prism: (1) large-data, (2) small-data, (3) mean-field, (4) narrow-latent, and (5) free. For regimes (1,2,3,4), we derive and rigorously characterize the limiting train and population dynamics under gradient flow, rigorously prove convergence to these limits, and get good agreement with experimental results. The resulting limits reveal spectral critical slowing, initialization-driven trapping away from the PCA optimum, and regimes in which population loss converges exponentially while training loss relaxes only algebraically.

Eugene Golikov, Yaroslav Gusev, Dmitry Yarotsky

Applied AI Institute Moscow, Russia · Steklov Mathematical Institute of Russsian Academy of Sciences Moscow, Russia

Autoencoders Neural Network Training Dynamics Statistical Learning Theory Gradient Descent Dynamics

Jun 3, 2026

BBOmix: A Tabular Benchmark for Hyperparameter Optimization of Unsupervised Biological Representation Learning

The rapid advancement of high-throughput sequencing has led to large, high-dimensional omics datasets. Deep unsupervised learning architectures, particularly Autoencoders (AEs), are increasingly used for dimensionality reduction and representation learning in this domain. However, AEs are highly sensitive to architectural choices and hyperparameters, and unsupervised optimization typically relies on reconstruction loss, which may be a poor proxy for downstream utility. Exhaustive hyperparameter optimization (HPO) is computationally expensive, leading researchers to frequently rely on suboptimal default configurations. To democratize access to large-scale unsupervised HPO research, we introduce

\textbf{BBOmix}

, the first open-source tabular benchmark for unsupervised representation learning on real-world biological data. Our benchmark includes 105,000 evaluations across four AE architectures and seven multi-omics modalities from the TCGA and SCHC datasets. We quantify the correlation between reconstruction loss and downstream task performance and provide an extensive evaluation of state-of-the-art single-fidelity, multi-fidelity, and transfer learning HPO methods, establishing a rigorous baseline for future research in unsupervised biological representation learning.

Luca Thale-Bombien, Jan Ewald, Ralf König +1

Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig, Leipzig University · ELLIS Institute Tübingen

Representation Learning Autoencoders Unsupervised Representation Learning Hyperparameter Optimization

Jun 3, 2026

Learning symplectic model reduction based on a approximation theorem of symplectic embeddings

High-dimensional Hamiltonian systems play a central role in many scientific and engineering disciplines, with dynamics evolving on symplectic manifolds. Although deep learning provides powerful tools for constructing low-dimensional surrogates from data, the intrinsic symplectic structure is easily destroyed during model reduction. As a result, a standard autoencoder may produce latent coordinates that do not support a Hamiltonian flow, leading to unstable long-time prediction. In this paper, we first establish a universal approximation theorem for symplectic embeddings. Based on this theory, we propose symplecticity-preserving autoencoders (SpAE), in which the decoder is parameterized as a symplectic embedding and the encoder is constructed as the corresponding symplectic projection. This architecture is expressive enough to approximate nonlinear symplectic embeddings and the associated symplectic projections, preserves the symplectic structure exactly by construction, and can be trained by standard unconstrained optimization, thereby improving both reconstruction and prediction accuracy. Extensive experiments on high-dimensional lattice and particle systems demonstrate the effectiveness of the proposed method.

Liyi Feng, Yifa Tang, Yulin Xie +2

School of Mathematics and Statistics, Beijing Jiaotong University, Beijing 100044, China · State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China · Department of Mathematics, National University of Singapore, 119076, Singapore

Dynamical Systems Autoencoders Reduced-Order Modeling Hamiltonian Dynamics

Jun 3, 2026

ReSGA: A Large Tail Risk Model for Learning Value-at-Risk and Expected Shortfall

Learning Value-at-Risk (VaR) and Expected Shortfall (ES) is important for managing financial risks effectively. Existing approaches with limited parameters are vulnerable to model misspecification in the era of big data. To address this limitation, we propose a large tail risk model, the retrieval-enhanced self-grouping autoencoder (ReSGA), which is designed with millions of parameters to exploit the rich cross-sectional dependence and long-term temporal dynamics of assets using their characteristics. Applied to monthly US equity returns from 1926 to 2023 with 153 firm characteristics, ReSGA outperforms twelve econometric and machine learning competitors in terms of out-of-sample loss and statistical backtesting. In addition, its forecast advantages can translate into significant economic gains from long-short decile portfolios that are constructed by a new size-enhanced left-side momentum strategy. To clarify the role of complexity, we further conduct a systematic scaling analysis and demonstrate that improvements in joint VaR-ES forecasting are primarily driven by data complexity rather than model complexity. Finally, our analyses of group-importance and transfer-learning exhibit the interpretability and cross-market generalizability of ReSGA.

Yichi Zhang, Ke Zhu, Zhoufan Zhu

Department of Statistics and Actuarial Science, The University of Hong Kong, Hong Kong · Wang Yanan Institute for Studies in Economics, Xiamen University, China · Department of Finance, School of Economics, Xiamen University, China

Quantitative Finance Autoencoders Probabilistic Forecasting Financial Time Series

Jun 2, 2026

SEAOTTER: Sensor Embedded Autoencoding with One-Time Transcode for Efficient Reconstruction

In robotics systems, vast amounts of visual data are easily captured at high resolution using low-cost, low-power hardware. Yet, limited bandwidth and on-device compute resources prevent full utilization when transmitted via conventional codecs like JPEG/MPEG. Newer codecs, like AV1/AVIF, improve the rate-distortion trade-off, but demand far more resources for encoding, impractical without custom ASICs. Recent asymmetric autoencoders deliver high quality under extreme power and bandwidth constraints, but add prohibitive decoding cost and use bespoke formats that ignore decades of infrastructure built around standards like JPEG. To address these limitations, we introduce a compression framework for cloud robotics based on a Sensor Embedded Autoencoder paired with a One-Time Transcode for Efficient Reconstruction (SEAOTTER). Because the sensor, cloud, and consumer stages face very different power and bandwidth budgets, SEAOTTER combines the compactness of a learned latent with the broad usability of a standard JPEG file. Since naive transcoding degrades performance, we propose a learnable JPEG color and quantization transform that enables increased accuracy for global, dense, and vision-language-based perception. Using SEAOTTER, we train both general-purpose and task-aware transcoding pipelines for a pre-trained, frozen encoder. At a compression ratio of 200:1 and compared to AVIF, we observe 7 times faster encoding, 3.5 times faster decoding, and +8% ImageNet top-1 accuracy, while retaining compatibility with JPEG infrastructure. Our code is available at https://github.com/UT-SysML/seaotter .

Dan Jacobellis, Neeraja J. Yadwadkar

Department of Electrical and Computer Engineering The University of Texas at Austin Austin, TX 78712, USA

Learned Image Compression Autoencoders

May 29, 2026

Learning Permutation-invariant Macroscopic Dynamics

Accurately modeling the macroscopic dynamics of high-dimensional microscopic systems is of broad interest across the sciences. Many data-driven approaches learn a low-dimensional latent state through an autoencoder trained for pointwise input reconstruction. These methods typically assume a fixed ordering of microscopic degrees of freedom in the input. However, in many settings, such as particle systems, the microscopic state is inherently unordered. This motivates an autoencoder framework that learns permutation-invariant latent representations. To this end, we adopt a permutation-invariant encoder and design the decoder to reconstruct the mass distribution centered at the observed points rather than per-sample reconstruction. We then jointly learn the macroscopic dynamics of the observables together with the latent states. We demonstrate the effectiveness and robustness of the proposed method across a range of microscopic settings, including learning the energy dynamics in interacting particle systems, predicting mixing dynamics in Lennard-Jones fluids, and modeling the stretching dynamics from video data of polymers moving in an elongational force field.

Zhichao Han, Mengyi Chen, Qianxiao Li

Institute for Functional Intelligent Materials, National University of Singapore · Department of Mathematics, National University of Singapore

Dynamical Systems Latent Dynamics Modeling Autoencoders Permutation-Invariant Learning Invariant Representation Learning

May 28, 2026

A Fully Convolutional Approach to Denoising Structural Dynamics Data from X-Ray Photon Correlation Spectroscopy

We present a fully convolutional denoising autoencoder (FC-DAE) for denoising two-time intensity-intensity correlation functions (

C_2

) in X-ray photon correlation spectroscopy (XPCS). Unlike conventional denoising autoencoders that are typically restricted to fixed input sizes, the FC-DAE accepts inputs of arbitrary dimensions while preserving correlation structures across diverse dynamical regimes. The model is trained using experimentally derived

C_2

data collected at NSLS-II beamlines, with data augmentation applied to expand the diversity of the dataset and reduce overfitting. The FC-DAE successfully recovers intricate dynamical features in low signal-to-noise conditions while maintaining structural fidelity. To assess reconstruction reliability, we employ quantitative metrics to evaluate structural fidelity and identify potential model-induced bias. Our results demonstrate that the FC-DAE provides robust denoising performance with high computational efficiency, enabling recovery of XPCS dynamics under photon-limited and low-dose measurement conditions.

Nisar Nellikunnummel, Andi Barbour, Lutz Wiegart +2

Brookhaven National Laboratory, Upton, NY 11973, USA · Currently at: Amazon · Currently at: GE Aerospace Research

Denoising Autoencoders Autoencoders Convolutional Neural Networks

May 28, 2026

Open World Autoencoding Drift Detection with Novel Class Recognition in Tabular Non-stationary Data Streams

Data stream processing has become a landmark in modern machine learning applications, with concept drifts and novel class appearances posing the primary challenges faced by sophisticated recognition methods. This work proposes an unsupervised concept drift detection method that identifies shifts in known class distributions based on the reconstruction errors of an autoencoder, while also enabling the recognition of novel class samples through density estimation of a proxy representation of samples. Using mirrored autoencoders allows for independent incremental adaptation to changing problem distributions for the two considered tasks, resulting in continuous adjustment to evolving concepts and reliable recognition of unknown samples. Conducted experiments used a diverse set of synthetic tabular data streams, where both concept drifts and the emergence of novelties were observed. The results show that the proposed approach is competitive with current state-of-the-art unsupervised drift detectors and novelty classifiers.

Joanna Komorniczak

Department of Systems and Computer Networks, Wrocław University of Science and Technology

Concept Drift Autoencoders Open-Set Recognition Tabular ML OOD Detection

May 27, 2026

A self-supervised learning approach to deep filter banks for texture recognition

An important challenge in texture recognition is the limited amount of data for training frequently found in real-world applications. In computer vision in general, a successful strategy to mitigate this issue is the use of a pretraining stage where the neural network learns to identify relations between parts of the data in a self-supervised manner. A well-established framework in this direction is masked autoencoder. Nevertheless, these models usually rely on computationally intensive architectures, such as vision transformers. In the particular case of texture images, most of the relevant information is compacted within a delimited area around each pixel, which suggests that capturing long-range dependence via the attention mechanism may be unnecessary. Based on that assumption, here we propose a framework where the pretraining model is a convolutional autoencoder. To leverage the rich information conveyed by texture patterns, we employ deep filters coupled with Fisher vector pooling. In this way, we improve the performance of texture recognition without adding significant computational burden. Our approach is compared with several state-of-the-art methods in different texture databases, confirming its potential both in terms of classification accuracy and computational complexity.

Joao B. Florindo, Lucas O. Lyra, Antonio E. Fabris

Institute of Mathematics, Statistics and Scientific Computing of the University of Campinas, Rua Sergio Buarque de Holanda, 651, Campinas, 13083-859, Sao Paulo, Brazil · Institute of Mathematics and Statistics of the University of Sao Paulo, Rua do Matao, 1010, Sao Paulo, 05508-090, Sao Paulo, Brazil

Autoencoders Convolutional Autoencoder Self-Supervised Visual Representation Learning Image Classification

May 27, 2026

Residualized Temporal Sparse Autoencoders for Interpreting Diffusion Models

Text-to-image diffusion models generate images by iterative denoising, so their internal layers produce trajectories of activations rather than single static representations. Sparse autoencoders (SAEs) have recently been used to decompose diffusion activations into interpretable features, but most approaches analyze individual timesteps or condition on time rather than learning from full trajectories. Training one SAE on whole trajectories would make each feature a single trajectory across timesteps, but adjacent activations are largely linearly predictable from one another, so such an SAE spends its latents on content carried forward from step to step. We introduce residualized temporal SAEs (ReSAE), which fit linear predictors between neighboring timesteps and represent each trajectory by its initial activation and the residuals these dynamics leave unexplained. Training an SAE on this representation is equivalent to training it on raw trajectories under a metric induced by the linear dynamics, and it encourages latents to capture structure beyond what is linearly predictable. Each latent's decoder direction maps back to activation space as a feature trajectory over denoising time. Across Stable Diffusion~1.5 and a Diffusion Transformer, ReSAE features span the whole trajectory while pinpointing when changes enter it, making ReSAE a natural tool for studying how a diffusion model generates an image over time.

Calvin Yeung, Prathyush Poduval, Ali Zakeri +2

Department of Computer Science University of California, Irvine Irvine, CA 92697

Autoencoders Sparse Autoencoders T2I Diffusion Models Diffusion Models

May 23, 2026

High-Dimensional Latents Should Be Diagnosed Through Phase Structure

We study autoencoder and variational-autoencoder latent spaces through the lens of spin-glass theory. The paper has two components. First, we formalize a latent-space spin-glass dictionary: for a fixed decoder, the reconstruction term together with a hyperspherical coordinates prior induces a Hamiltonian on the latent sphere, where latent coordinates play the role of continuous spins and the prior acts as an external magnetic field. This allows us to import operational spin-glass diagnostics -- overlap distributions, susceptibility, and block-spin coarse-graining -- to detect ordered, disordered, and edge-of-stability phases in trained latent representations. Second, we show that deliberately driving the latent system toward the edge-of-stability of the topological trivialization regime has concrete downstream consequences. In generation, hyperspherical compression improves the reconstruction-generation trade-off on CIFAR-10 and CelebA64, yielding lower self-FID while preserving or improving reconstruction. In anomaly detection, the same semi-ordered latent geometry improves both fully unsupervised and conditional OOD detection, including real-world Mars Rover and Galaxy Zoo datasets, as well as CIFAR-10/100 and Imagenette-based OOD benchmarks. We therefore advocate a phase-aware evaluation paradigm for AEs/VAEs, in which spin-glass observables complement standard ML metrics and expose the latent regimes that underlie downstream success or failure in many cases.

Alejandro Ascarate, Leo Lebrat, Rodrigo Santa Cruz +2

School of Electrical Engineering and Robotics, Faculty of Engineering, Queensland University of Technology, Brisbane, Queensland, Australia

Variational Autoencoders Autoencoders Phase Transitions

May 22, 2026

MEDAL: Manifold Embedding Distillation via Autoencoder Learning

Low-dimensional embeddings are widely used as visual summaries of high-dimensional data and to enable downstream scientific discoveries. Yet, popular nonlinear dimension reduction methods, such as t-SNE and UMAP, are often selected based on visual appeal alone and without rigorous quantitative validation. A major reason is that manifold embeddings typically do not provide an out-of-sample map nor an inverse back to the original feature space; this makes held-out validation, the gold standard in supervised learning, all but impossible. To address these challenges, we develop a novel framework, MEDAL (Manifold Embedding Distillation via Autoencoder Learning), which distills a fitted manifold embedding into a reusable encoder--decoder model. MEDAL trains a constrained autoencoder whose bottleneck exactly matches any teacher embedding while the decoder reconstructs the original input; this yields an explicit map for new samples, an approximate inverse, and a pointwise reconstruction-based measure of distortion in the manifold space. This converts static manifold embeddings into models that can be evaluated on held-out data, enabling quantitative validation including comparing different dimension reduction methods as well as hyperparameter tuning. Across multiple benchmark and scientific case studies, we show that MEDAL enables held-out validation to determine optimal manifold embeddings and hyperparameters, reveals biologically coherent regions that are difficult to preserve in two dimensional embeddings, and detects distribution shift when new samples are mapped into a fixed reference manifold. MEDAL provides a general validation wrapper to any existing dimension reduction technique that will improve the rigor and reliability of dimension reduction in scientific workflows.

Irene Chang, Tarek M. Zikry, Genevera I. Allen

Department of Statistics, Columbia University, New York, NY 10027 · School of Data and Information Sciences, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 · Irving Institute for Cancer Dynamics, Columbia University, New York, NY 10027 +1

Autoencoders Manifold Learning Dimensionality Reduction

May 21, 2026

DecQ: Detail-Condensing Queries for Enhanced Reconstruction and Generation in Representation Autoencoders

Representation Autoencoders (RAEs) leverage frozen vision foundation models (VFMs) as tokenizer encoders, providing robust high-level representations that facilitate fast convergence and high-quality generation in latent diffusion models. However, freezing the VFM inherently constrains its spatial reconstruction capacity, limiting fine-grained generation and image editing; in contrast, incorporating reconstruction-oriented signals via fine-tuning disrupts the pretrained semantic space and degrades generative fidelity. To address this trade-off, we propose DecQ, a simple yet effective framework for RAEs. Specifically, DecQ introduces lightweight detail-condensing queries that extract fine-grained information from intermediate VFM features through condenser modules. These queries are incorporated into the decoder to support reconstruction and are jointly generated with patch tokens during generative modeling. By aggregating information from both shallow and deep layers, DecQ effectively mitigates the reconstruction--generation trade-off, improving both reconstruction quality and generative performance. Our experiments demonstrate that: (1) with only 8 additional queries and 3.9% extra computation, DecQ improves reconstruction over the frozen DINOv2-based RAE, increasing PSNR from 19.13 dB to 22.76 dB; and (2) for generative modeling, DecQ achieves 3.3

\times

faster convergence than RAE, attaining an FID of 1.41 without guidance and 1.05 with guidance.

Tianhang Wang, Yitong Chen, Wei Song +3

Zhejiang University · Shanghai Innovation Institute · Fudan University +1

Latent Diffusion Models Autoencoders Image Reconstruction Diffusion-Based Image Generation

May 18, 2026

SAME: A Semantically-Aligned Music Autoencoder

Latent representations are at the heart of the majority of modern generative models. In the audio domain they are typically produced by a neural-audio-codec autoencoder. In this work we introduce SAME (Semantically-Aligned Music autoEncoder), an autoencoder for stereo music and general audio that reaches a 4096

\times

temporal compression ratio while maintaining reconstruction quality and downstream generative performance. We achieve this by combining a tranformer-based backbone with set of semantic regularisation approaches, phase-aware reconstruction losses and improved discriminator designs. The architecture delivers substantial computational cost benefits, through both its high compression ratio and its reliance on well-optimised transformer primitives. Two variants (a large SAME-L and a CPU-deployable SAME-S) are released in open-weights form.

Julian D. Parker, Zach Evans, CJ Carr +4

Stability AI

Audio Representation Learning Neural Audio Codecs Autoencoders

May 18, 2026

Improved Baselines with Representation Autoencoders

Representation Autoencoders (RAE) replace traditional VAE with pretrained vision encoders. In this paper, we systematically investigate several design choices and find three insights which simplify and improve RAE. First, we study a generalized formulation where the representation is defined as sum of the last k encoder layers rather than solely the final layer. This simple change greatly improves reconstruction without encoder finetuning or specialized data (e.g., text, faces). Second, we study the prevalent assumption that RAE (using pretrained representation as encoder) replaces representation alignment (REPA), which distills the same representation to intermediate layers instead. Through large-scale empirical analysis, we uncover a surprising finding: RAE and REPA exhibit complementary working mechanisms, allowing the same representation to be used as both encoder and target for intermediate diffusion layers. Finally, the original RAE struggles with classifier-free guidance (CFG) and requires training a second, weaker diffusion model for AutoGuidance (AG). We show that REPA itself can be viewed as x-prediction in RAE latent space. By simply re-parameterizing the output of the DiT model, it can provide guidance for "free". Overall, RAEv2 leads to more than 10x faster convergence over the original RAE, achieving a state-of-the-art gFID of 1.06 in just 80 epochs on ImageNet-256. On FDr6, RAEv2 achieves a state-of-the-art 2.17 at just 80 epochs compared to the previous best 3.26 (800 epochs) without any post-training. This motivates EPFID@k (epochs to reach unguided gFID < k) as a measure of training efficiency. RAEv2 attains an EPFID@2 of 35 epochs, versus 177 for the original RAE. We also validate our approach across diverse settings for text-to-image generation and navigation world models, showing consistent improvements. The code is available at https://raev2.github.io.

Jaskirat Singh, Boyang Zheng, Zongze Wu +3

Adobe Research · ANU · New York University

Diffusion Model Guidance Diffusion Transformer Autoencoders Diffusion-Based Image Generation Diffusion Models Diffusion Autoencoder