Representation Learning

Ask about this topic

Overview

Representation learning studies how to turn raw data—such as images, text, graphs, time series, and sensor signals—into features that make important patterns easier to use. Common approaches include contrastive and masked prediction, autoencoders, predictive latent models, sparse coding, and methods that impose structure such as geometry, hierarchy, or disentangled subspaces. Good representations can reduce label needs and support transfer across tasks, domains, and modalities, with evaluations spanning linear probes, retrieval, classification, and downstream control. A central tension is that compression and invariance can discard information a later task needs, so no single objective or representation geometry works best across settings.

Latest papers

CardsList

799 papers130 papers in the last four weeks, up 97% on the four weeks before. 0.8% of all new papers.

Mar 3, 2026

LLM-MLFFN: Multi-Level Autonomous Driving Behavior Feature Fusion via Large Language Model

Accurate classification of autonomous vehicle (AV) driving behaviors is critical for safety validation, performance diagnosis, and traffic integration analysis. However, existing approaches primarily rely on numerical time-series modeling and often lack semantic abstraction, limiting interpretability and robustness in complex traffic environments. This paper presents LLM-MLFFN, a novel large language model (LLM)-enhanced multi-level feature fusion network designed to address the complexities of multi-dimensional driving data. The proposed LLM-MLFFN framework integrates priors from largescale pre-trained models and employs a multi-level approach to enhance classification accuracy. LLM-MLFFN comprises three core components: (1) a multi-level feature extraction module that extracts statistical, behavioral, and dynamic features to capture the quantitative aspects of driving behaviors; (2) a semantic description module that leverages LLMs to transform raw data into high-level semantic features; and (3) a dual-channel multi-level feature fusion network that combines numerical and semantic features using weighted attention mechanisms to improve robustness and prediction accuracy. Evaluation on the Waymo open trajectory dataset demonstrates the superior performance of the proposed LLM-MLFFN, achieving a classification accuracy of over 94%, surpassing existing machine learning models. Ablation studies further validate the critical contributions of multi-level fusion, feature extraction strategies, and LLM-derived semantic reasoning. These results suggest that integrating structured feature modeling with language-driven semantic abstraction provides a principled and interpretable pathway for robust autonomous driving behavior classification.

Xiangyu Li, Tianyi Wang, Xi Cheng +5

Department of Civil, Architectural, and Environmental Engineering, The University of Texas at Austin, Austin, TX 78712, USA. · Systems Engineering Program, Cornell University, Ithaca, NY 14850, USA. · Department of Electrical and Computer Engineering, University of Michigan, Ann Arbor, MI 48109, USA. +2

Autonomous DrivingMultimodal ClassificationRepresentation Learning
AskPDF
Mar 2, 2026

From Pixels to Patches: Pooling Strategies for Earth Embeddings

Geospatial foundation models increasingly expose pixel-level embedding products that can be downloaded and reused without access to the underlying encoder. In this setting, downstream tasks with patch- or region-level labels require a post-hoc aggregation step that maps dense pixel embeddings to a single representation. The default choice, mean pooling, discards within-patch variability and can underperform under spatial distribution shift. To study this setting, we introduce EuroSAT-Embed: 81,000 embedding GeoTIFFs derived from three foundation models: AlphaEarth, OlmoEarth, and Tessera. Using these fixed embedding products, we benchmark 11 training-free pooling methods and 2 train-set-fitted baselines under both random and geographically disjoint test splits. Richer pooling schemes reduce the geographic generalization gap by over 50% relative to mean pooling and improve accuracy by up to 6% on spatial splits. We recommend a three-tier strategy: (1) mean as a baseline, (2) stats pooling (min/max/mean/std) as the default at 4x the embedding dimension, and (3) covariance pooling for peak accuracy. Across all three embedding products, simple distributional statistics improve spatial-split performance over mean pooling.

Isaac Corley, Caleb Robinson, Inbal Becker-Reshef +1

Wherobots · Microsoft AI for Good Research Lab

Spatial GeneralizationGeospatial Foundation ModelsRepresentation LearningRemote Sensing Benchmarks
AskPDF
Mar 2, 2026

Never Saddle for Reparameterized Steepest Descent as Mirror Flow

How does the choice of optimization algorithm shape a model's ability to learn features? To address this question for steepest descent methods --including sign descent, which is closely related to Adam --we introduce steepest mirror flows as a unifying theoretical framework. This framework reveals how optimization geometry governs learning dynamics, implicit bias, and sparsity and it provides two explanations for why Adam and AdamW often outperform SGD in fine-tuning. Focusing on diagonal linear networks and deep diagonal linear reparameterizations (a simplified proxy for attention), we show that steeper descent facilitates both saddle-point escape and feature learning. In contrast, gradient descent requires unrealistically large learning rates to escape saddles, an uncommon regime in fine-tuning. Empirically, we confirm that saddle-point escape is a central challenge in fine-tuning. Furthermore, we demonstrate that decoupled weight decay, as in AdamW, stabilizes feature learning by enforcing novel balance equations. Together, these results highlight two mechanisms how steepest descent can aid modern optimization.

Tom Jacobs, Chao Zhou, Rebekka Burkholz

CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

Neural Network Training DynamicsGradient Descent DynamicsAdaptive Gradient MethodsGradient DescentMirror DescentRepresentation Learning
AskPDF
Mar 2, 2026

Spectral Overfitting in Noisy Linear Probing of Pretrained Representations

Frozen pretrained features are often treated as a safe interface for downstream learning: only a small linear readout is trained, while the backbone is fixed. We show that this readout can still overfit noisy labels in a structured way. A label-blind PCA rank sweep reveals a sharp spectral pattern: under label noise, exposing all pretrained directions can hurt clean accuracy, and intermediate ranks often recover much of the lost performance. Rank-matched random projections help less, and measured between-class signal is strongly concentrated in leading PCs. The pattern appears across three ImageNet-pretrained backbones on CIFAR-10, with gains up to 36.0±0.836.0\pm0.8 points over the default full-rank probe at 40% noise. Tuned full-rank probes outperform validation-selected PCA probes, so we present the sweep as a diagnostic of spectral overfitting rather than a competitive noisy-label method.

Zice Wang, Zhenyu Zhang

School of Future Technology, Northeastern University, Shenyang, China.

Representation LearningLinear ProbingLearning with Noisy Labels
AskPDF
Mar 2, 2026

Discrete World Models via Regularization

World models aim to capture the states and dynamics of an environment in a compact latent space. Moreover, using Boolean state representations is particularly useful for search heuristics and symbolic reasoning and planning. Existing approaches keep latents informative via decoder-based reconstruction, or instead via contrastive or reward signals. In this work, we introduce Discrete World Models via Regularization (DWMR): a reconstruction-free and contrastive-free method for unsupervised Boolean world-model learning. In particular, we introduce a novel world-modeling loss that couples latent prediction with specialized regularizers. Such regularizers maximize the entropy and independence of the representation bits through variance, correlation, and coskewness penalties, while simultaneously enforcing a locality prior for sparse action changes. To enable effective optimization, we also introduce a novel training scheme improving robustness to discrete roll-outs. Experiments on two benchmarks with underlying combinatorial structure show that DWMR learns more accurate representations and transitions than reconstruction-based alternatives. Finally, DWMR can also be paired with an auxiliary reconstruction decoder, and this combination yields additional gains.

Davide Bizzaro, Luciano Serafini

University of Padua, Padua, Italy · Fonadazione Bruno Kessler, Trento, Italy

World Model LearningWorld ModelsAction-Conditioned World ModelsRepresentation LearningUnsupervised Representation Learning
AskPDF
Mar 2, 2026

Asking the Right Questions: Ontology-Grounded Interpretable Embeddings for Biomedical Text

While dense biomedical embeddings achieve strong performance, their opaque dimensions limit transparency in biomedical NLP. Recent question-based interpretable embeddings represent text through binary answers to natural-language questions, but existing approaches rely primarily on corpus-driven signals, often capturing topical or stylistic differences rather than fine-grained biomedical distinctions. We propose QIME, an ontology-grounded framework for interpretable biomedical text embeddings in which each dimension corresponds to an explicit biomedical-domain yes/no question. QIME leverages a biomedical ontology to guide contrastive question generation from semantic clusters, producing atomic, domain-grounded questions. It constructs embeddings via similarity-based semantic activation with MMR-based diversity-aware dimension selection, yielding sparse representations efficiently. Experiments on biomedical clustering, STS and retrieval benchmarks show that QIME consistently outperforms prior interpretable embedding methods and substantially narrows the gap to strong black-box biomedical encoders. It also imposes substantially lower cognitive burden than existing methods, providing concise and domain-specific interpretations. The code is available at https://github.com/L1nzh/QIME.

Yixuan Tang, Zhenghong Lin, Yandong Sun +3

School of Computing, National University of Singapore

Text EmbeddingsRepresentation LearningText Embedding Models
AskPDF
Feb 28, 2026

ResGene-T: A Tensor-Based Residual Network Approach for Genomic Prediction

In this work, we propose a new deep learning model for Genomic Prediction (GP), which involves correlating genotypic data with phenotypic. The genotypes are typically fed as a sequence of characters to the 1D-Convolution Neural Network layer of the underlying deep learning model. Inspired by earlier work that represented genotype as a 2D-image for genotype-phenotype classification, we extend this idea to GP, which is a regression task. We use a ResNet-18 as the underlying architecture, and term this model as ResGene-2D. Although the 2D-image representation captures biological interactions well, it requires all the layers of the model to do so. This limits training efficiency. Thus, as seen in the earlier work that proposed a 2D-image representation, our ResGene-2D performs almost the same as other models (3% improvement). To overcome this, we propose a novel idea of converting the 2D-image into a 3D/ tensor and feed this to the ResNet-18 architecture, and term this model as ResGene-T. We evaluate our proposed models on three crop species having ten phenotypic traits and compare it with seven most popular models (two statistical, two machine learning, and three deep learning). ResGene-T performs the best among all these seven methods (gains from 14.51% to 41.51%).

Kuldeep Pathak, Kapil Ahuja, Eric de Sturler

Department of Computer Science and Engineering, Indian Institute of Technology Indore, India 453552 · Department of Mathematics, Virginia Tech, Blacksburg, USA 24061

GenomicsConvolutional Neural NetworksRepresentation Learning
AskPDF
Feb 28, 2026

Social-JEPA: Emergent Geometric Isomorphism

World models compress rich sensory streams into compact latent codes that anticipate future observations. We let separate agents acquire such models from distinct viewpoints of the same environment without any parameter sharing or coordination. After training, their internal representations exhibit a striking emergent property: the two latent spaces are related by an approximate linear isometry, enabling transparent translation between them. This geometric consensus survives large viewpoint shifts and scant overlap in raw pixels. Leveraging the learned alignment, a classifier trained on one agent can be ported to the other with no additional gradient steps, while distillation-like migration accelerates later learning and markedly reduces total compute. The findings reveal that predictive learning objectives impose strong regularities on representation geometry, suggesting a lightweight path to interoperability among decentralized vision systems. The code is available at https://anonymous.4open.science/r/Social-JEPA-5C57.

Youjin Wang

Renmin University of China

Representation LearningVisual Representation LearningLatent Space AlignmentGeometric Representation LearningWorld Model Learning
AskPDF
Feb 27, 2026

InfoNCE Induces Gaussian Distribution

Contrastive learning has become a cornerstone of modern representation learning, allowing training with massive unlabeled data for both task-specific and general (foundation) models. A prototypical loss in contrastive training is InfoNCE and its variants. In this work, we show that the InfoNCE objective induces Gaussian structure in representations that emerge from contrastive training. We establish this result in two complementary regimes. First, we show that under certain alignment and concentration assumptions, projections of the high-dimensional representation asymptotically approach a multivariate Gaussian distribution. Next, under less strict assumptions, we show that adding a small asymptotically vanishing regularization term that promotes low feature norm and high feature entropy leads to similar asymptotic results. We support our analysis with experiments on synthetic and CIFAR-10 datasets across multiple encoder architectures and sizes, demonstrating consistent Gaussian behavior. This perspective provides a principled explanation for commonly observed Gaussianity in contrastive representations. The resulting Gaussian model enables principled analytical treatment of learned representations and is expected to support a wide range of applications in contrastive learning.

Roy Betser, Eyal Gofer, Meir Yossef Levi +1

Technion - Israel Institute of Technology

Contrastive LearningRepresentation Learning
AskPDF
Feb 26, 2026

SegReg: Latent Space Regularization for Improved Medical Image Segmentation

Medical image segmentation models are typically optimised with voxel-wise losses that constrain predictions only in the output space. This leaves latent feature representations largely unconstrained, potentially limiting generalisation. We propose {SegReg}, a latent-space regularisation framework that operates on feature maps of U-Net models to encourage structured embeddings while remaining fully compatible with standard segmentation losses. Integrated with the nnU-Net framework, we evaluate SegReg on prostate, cardiac, and hippocampus segmentation and demonstrate consistent improvements in domain generalisation. Furthermore, we show that explicit latent regularisation improves continual learning by reducing task drift and enhancing forward transfer across sequential tasks without adding memory or any extra parameters. These results highlight latent-space regularisation as a practical approach for building more generalisable and continual-learning-ready models.

Puru Vaish, Amin Ranem, Felix Meister +3

Department of Applied Mathematics, Technical Medical Centre, University of Twente, The Netherlands · Digital Technology and Innovation, Siemens Healthineers, Erlangen, Germany

Latent-Space RegularizationMedical Image SegmentationDomain GeneralizationRepresentation LearningMedical Image Representation LearningContinual Learning
AskPDF
Feb 26, 2026

Beyond NNGP: Large Deviations and Feature Learning in Bayesian Neural Networks

We study wide Bayesian neural networks focusing on the rare but statistically dominant fluctuations that govern posterior concentration, beyond Gaussian-process limits. Large-deviation theory provides explicit variational objectives-rate functions-on predictors, providing an emerging notion of complexity and feature learning directly at the functional level. We show that the posterior output rate function is obtained by a joint optimization over predictors and internal kernels, in contrast with fixed-kernel (NNGP) theory. Numerical experiments demonstrate that the resulting predictions accurately describe finite-width behavior for moderately sized networks, capturing non-Gaussian tails, posterior deformation, and data-dependent kernel selection effects.

Katerina Papagiannouli, Dario Trevisan, Giuseppe Pio Zitto

Dipartimento di Matematica, Università di Pisa

Large Deviation PrinciplesBayesian Neural NetworksBayesian InferenceRepresentation Learning
AskPDF
Feb 26, 2026

Robust Human Trajectory Prediction via Self-Supervised Skeleton Representation Learning

Human trajectory prediction plays a crucial role in applications such as autonomous navigation and video surveillance. While recent works have explored the integration of human skeleton sequences to complement trajectory information, skeleton data in real-world environments often suffer from missing joints caused by occlusions. These disturbances significantly degrade prediction accuracy, indicating the need for more robust skeleton representations. We propose a robust trajectory prediction method that incorporates a self-supervised skeleton representation model pretrained with masked autoencoding. Experimental results in occlusion-prone scenarios show that our method improves robustness to missing skeletal data without sacrificing prediction accuracy, and consistently outperforms baseline models in clean-to-moderate missingness regimes.

Taishu Arashima, Hiroshi Kera, Kazuhiko Kawamoto

Chiba University

Human Trajectory PredictionTrajectory PredictionLearning with Missing DataRepresentation Learning
AskPDF
Feb 25, 2026

DexRepNet++: Learning Dexterous Robotic Manipulation with Geometric and Spatial Hand-Object Representations

Robotic dexterous manipulation is a challenging problem due to high degrees of freedom (DoFs) and complex contacts of multi-fingered robotic hands. Many existing deep reinforcement learning (DRL) based methods aim at improving sample efficiency in high-dimensional output action spaces. However, existing works often overlook the role of representations in achieving generalization of a manipulation policy in the complex input space during the hand-object interaction. In this paper, we propose DexRep, a novel hand-object interaction representation to capture object surface features and spatial relations between hands and objects for dexterous manipulation skill learning. Based on DexRep, policies are learned for three dexterous manipulation tasks, i.e. grasping, in-hand reorientation, bimanual handover, and extensive experiments are conducted to verify the effectiveness. In simulation, for grasping, the policy learned with 40 objects achieves a success rate of 87.9% on more than 5000 unseen objects of diverse categories, significantly surpassing existing work trained with thousands of objects; for the in-hand reorientation and handover tasks, the policies also boost the success rates and other metrics of existing hand-object representations by 20% to 40%. The grasp policies with DexRep are deployed to the real world under multi-camera and single-camera setups and demonstrate a small sim-to-real gap.

Qingtao Liu, Zhengnan Sun, Yu Cui +5

the College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China · the College of Control Science and Engineering and the State Key Laboratory of Industrial Control Technology, Zhejiang University, China · the Department of Computer Science, National University of Singapore, 15 Computing Dr, 117418, Singapore

Dexterous ManipulationRepresentation LearningRobotic GraspingGeneralization in Robotic ManipulationRobotic RLBimanual Robotic Manipulation
AskPDF
Feb 24, 2026

CG-DMER: Hybrid Contrastive-Generative Framework for Disentangled Multimodal ECG Representation Learning

Accurate interpretation of electrocardiogram (ECG) signals is crucial for diagnosing cardiovascular diseases. Recent multimodal approaches that integrate ECGs with accompanying clinical reports show strong potential, but they still face two main concerns from a modality perspective: (1) intra-modality: existing models process ECGs in a lead-agnostic manner, overlooking spatial-temporal dependencies across leads, which restricts their effectiveness in modeling fine-grained diagnostic patterns; (2) inter-modality: existing methods directly align ECG signals with clinical reports, introducing modality-specific biases due to the free-text nature of the reports. In light of these two issues, we propose CG-DMER, a contrastive-generative framework for disentangled multimodal ECG representation learning, powered by two key designs: (1) Spatial-temporal masked modeling is designed to better capture fine-grained temporal dynamics and inter-lead spatial dependencies by applying masking across both spatial and temporal dimensions and reconstructing the missing information. (2) A representation disentanglement and alignment strategy is designed to mitigate unnecessary noise and modality-specific biases by introducing modality-specific and modality-shared encoders, ensuring a clearer separation between modality-invariant and modality-specific representations. Experiments on three public datasets demonstrate that CG-DMER achieves state-of-the-art performance across diverse downstream tasks.

Ziwei Niu, Hao Sun, Shujun Bian +4

Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · College of information science and engineering, Ritsumeikan university, ibaraki Osaka, Japan +1

Electrocardiogram Representation LearningMultimodal Disentangled Representation LearningRepresentation LearningElectrocardiogram InterpretationElectrocardiographyMultimodal Contrastive Learning
AskPDF
Feb 23, 2026

A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning

Pretraining and fine-tuning are central stages in modern machine learning systems. In practice, feature learning plays an important role across both stages: deep neural networks learn a broad range of useful features during pretraining and further refine those features during fine-tuning. However, an end-to-end theoretical understanding of how choices of initialization impact the ability to reuse and refine features during fine-tuning has remained elusive. Here we develop an analytical theory of the pretraining fine-tuning pipeline in diagonal linear networks, deriving exact expressions for the generalization error as a function of initialization parameters and task statistics. We find that different initialization choices place the network into four distinct fine-tuning regimes that are distinguished by their ability to support feature learning and reuse and therefore by the task statistics for which they are beneficial. In particular, a smaller initialization scale in earlier layers enables the network to both reuse and refine its features, leading to superior generalization on fine-tuning tasks that rely on a subset of pretraining features. We demonstrate empirically that the same initialization parameters impact generalization in ResNets trained on CIFAR-100 and SVHN as well as Transformers trained on modular arithmetic tasks. Overall, our results demonstrate an alytically how data and network initialization interact to shape fine-tuning generalization, highlighting an important role for the relative scale of initialization across different layers in enabling continued feature learning during fine-tuning.

Nicolas Anguita, Francesco Locatello, Andrew M. Saxe +4

Department of Engineering, University of Cambridge · Institute of Science and Technology, Austria (ISTA) · Gatsby Computational Neuroscience Unit and Sainsbury Wellcome Centre, UCL +1

Neural Network GeneralizationRepresentation LearningFine-TuningNeural Network InitializationStatistical Learning Theory
AskPDF
Feb 23, 2026

Cross-lingual Matryoshka Representation Learning across Speech and Text

Speakers of under-represented languages face both a language barrier, as most online knowledge is in a few dominant languages, and a modality barrier, since information is largely text-based while many languages are primarily oral. We address this for French-Wolof by training the first bilingual speech-text Matryoshka embedding model, enabling efficient retrieval of French text from Wolof speech queries without relying on a costly ASR-translation pipelines. We introduce large-scale data curation pipelines and new benchmarks, compare modeling strategies, and show that modality fusion within a frozen text Matryoshka model performs best. Although trained only for retrieval, the model generalizes well to other tasks, such as speech intent detection, indicating the learning of general semantic representations. Finally, we analyze cost-accuracy trade-offs across Matryoshka dimensions and ranks, showing that information is concentrated only in a few components, suggesting potential for efficiency improvements.

Yaya Sy, Dioula Doucouré, Christophe Cerisara +1

LORIA, CNRS · Soynade Research

Representation LearningAudio-Text RetrievalCross-Lingual Representation AlignmentMatryoshka Representation LearningLow-Resource Language Processing
AskPDF
Feb 23, 2026

PaReGTA: A Temporally Aware LLM-Based Patient Representation Framework for EHR Analytics

Temporal information in structured electronic health records (EHRs) is often lost in sparse one-hot or count-based representations, while sequence models can be costly and data-hungry. We propose PaReGTA, an LLM-based encoding framework that (i) converts longitudinal EHR events into visit-level templated text with explicit temporal cues, (ii) learns domain-adapted visit embeddings via lightweight contrastive fine-tuning of a sentence-embedding model, and (iii) aggregates visit embeddings into a fixed-dimensional patient representation using hybrid temporal pooling that captures both recency and globally informative visits. The resulting fixed-dimensional patient representations can be used with conventional downstream machine-learning models. To examine factor-level sensitivity, we use PaReGTA-RSS (Representation Shift Score), a prespecified factor-removal analysis that recomputes patient representations after removing clinically defined factor groups and quantifies the resulting change in the fitted logit of a fixed logistic-regression model. We evaluated PaReGTA in a cohort of 39,088 patients with migraine from the All of Us Research Program (AoU) on a retrospective patient-level classification task with an EHR-derived target. In an exploratory comparison on the fixed held-out test cohort of 7,818 patients, PaReGTA-Gap + LightGBM, used as a post hoc analytical reference, had the highest reported AUC, accuracy, and F1 values among the evaluated sparse, BERT-based EHR, and recurrent approaches in this cohort.

Kihyuk Yoon, Lingchao Mao, Catherine Chong +3

Georgia Institute of Technology · Meta · Mayo Clinic

Longitudinal EHR ModelingRepresentation LearningClinical Prediction
AskPDF
Feb 22, 2026

Stable Deep Reinforcement Learning via Isotropic Gaussian Representations

Deep reinforcement learning systems often suffer from unstable training dynamics due to non-stationarity, where learning objectives and data distributions evolve over time. We show that under non-stationary targets, isotropic Gaussian embeddings are provably advantageous. In particular, they induce stable tracking of time-varying targets for linear readouts, achieve maximal entropy under a fixed variance budget, and encourage a balanced use of all representational dimensions--all of which enable agents to be more adaptive and stable. Building on this insight, we propose the use of Sketched Isotropic Gaussian Regularization for shaping representations toward an isotropic Gaussian distribution during training. We demonstrate empirically, over a variety of domains, that this simple and computationally inexpensive method improves performance under non-stationarity while reducing representation collapse, neuron dormancy, and training instability.

Ali Saheb Pasand, Johan Obando-Ceron, Aaron Courville +2

Mila – Quebec AI Institute · McGill · Université de Montréal +1

RL StabilityRepresentation Learning for RLRepresentation LearningNon-Stationary RLRepresentation Collapse
AskPDF
Feb 22, 2026

LWM-Temporal: Sparse Spatio-Temporal Attention for Wireless Channel Representation Learning

LWM-Temporal is a new member of the Large Wireless Models (LWM) family that targets the spatiotemporal nature of wireless channels. Designed as a task-agnostic foundation model, LWM-Temporal learns universal channel embeddings that capture mobility-induced evolution and are reusable across various downstream tasks. To achieve this objective, LWM-Temporal operates in the angle-delay-time domain and introduces Sparse Spatio-Temporal Attention (SSTA), a propagation-aligned attention mechanism that restricts interactions to physically plausible neighborhoods, reducing attention complexity by an order of magnitude while preserving geometry-consistent dependencies. LWM-Temporal is pretrained in a self-supervised manner using a physics-informed masking curriculum that emulates realistic occlusions, pilot sparsity, and measurement impairments. Experimental results on channel prediction across multiple mobility regimes show consistent improvements over strong baselines, particularly under long horizons and limited fine-tuning data, highlighting the importance of geometry-aware architectures and geometry-consistent pretraining for learning transferable spatiotemporal wireless representations.

Sadjad Alikhani, Akshay Malhotra, Shahab Hamidi-Rad +1

Spatio-Temporal AttentionWireless CommunicationsWireless Foundation ModelsRepresentation LearningSparse AttentionSelf-Supervised Time Series Representation Learning
AskPDF
Feb 22, 2026

INTACT: Intent-Aware Representation Learning for Cryptographic Traffic Violation Detection

Security monitoring systems typically treat anomaly detection as identifying statistical deviations from observed data distributions. In cryptographic traffic analysis, however, violations are defined not by rarity but by explicit policy constraints, including key reuse prohibition, downgrade prevention, and bounded key lifetimes. This fundamental mismatch limits the interpretability and adaptability of conventional anomaly detection methods. We introduce INTACT (INTent-Aware Cryptographic Traffic), a policy-conditioned framework that reformulates violation detection as conditional constraint learning. Instead of learning a static decision boundary over behavioral features, INTACT models the probability of violation conditioned on both observed behavior and declared security intent. The architecture factorizes representation learning into behavioral and intent encoders whose fused embeddings produce a violation score, yielding a policy-parameterized family of decision boundaries. We evaluate the framework on a real-world network flow dataset and a 210,000-trace synthetic multi-intent cryptographic dataset. INTACT matches or exceeds strong unsupervised and supervised baselines, achieving near-perfect discrimination (AUROC up to 1.0000) in the real dataset and consistent superiority in detecting relational and composite violations in the synthetic setting. These results demonstrate that explicit intent conditioning improves discrimination, interpretability, and robustness in cryptographic monitoring.

Rahul D Ray

Department of Electronics & Electrical Engineering, BITS Pilani, Hyderabad Campus

Encrypted Network Traffic AnalysisAnomaly DetectionRepresentation Learning
AskPDF
Feb 19, 2026

Position: Evaluation of ECG Representations Must Be Fixed

This position paper argues that current benchmarking practice in 12-lead ECG representation learning must be fixed to ensure progress is reliable and aligned with clinically meaningful objectives. The field has largely converged on three public multi-label benchmarks (PTB-XL, CPSC2018, CSN) dominated by arrhythmia and waveform-morphology labels, even though the ECG is known to encode substantially broader clinical information. We argue that downstream evaluation should expand to include an assessment of structural heart disease and patient-level forecasting, in addition to other evolving ECG-related endpoints, as relevant clinical targets. Next, we outline evaluation best practices for multi-label, imbalanced settings, and show that when they are applied, the literature's current conclusion about which representations perform best is altered. Furthermore, we demonstrate the surprising result that a randomly initialized encoder with linear evaluation matches state-of-the-art pre-training on many tasks. This motivates the use of a random encoder as a reasonable baseline model. We substantiate our observations with an empirical evaluation of five representative ECG pre-training approaches across six evaluation settings: the three standard benchmarks, a structural disease dataset, hemodynamic inference, and patient forecasting.

Zachary Berger, Daniel Prakah-Asante, John Guttag +1

Massachusetts Institute of Technology, Cambridge, MA, USA · Massachusetts General Hospital, Boston, MA, USA

Electrocardiogram Representation LearningRepresentation LearningElectrocardiogram ClassificationElectrocardiogram InterpretationClinical PredictionMedical Diagnosis
AskPDF
Feb 19, 2026

Unified Latents (UL): How to train your latents

We present Unified Latents (UL), a framework for learning latent representations that are jointly regularized by a diffusion prior and decoded by a diffusion model. By linking the encoder's output noise to the prior's minimum noise level, we obtain a simple training objective that provides a tight upper bound on the latent bitrate. On ImageNet-512, our approach achieves competitive FID of 1.4, with high reconstruction quality (PSNR) while requiring fewer training FLOPs than models trained on Stable Diffusion latents. On Kinetics-600, we set a new state-of-the-art FVD of 1.3.

Jonathan Heek, Emiel Hoogeboom, Thomas Mensink +1

Google DeepMind Amsterdam

Diffusion AutoencoderLatent Diffusion ModelsRepresentation LearningDiffusion ModelsDiffusion-Based Image GenerationVideo Diffusion Models
AskPDF
Feb 19, 2026

Learning a Latent Pulse Shape Interface for Photoinjector Laser Systems

Controlling the longitudinal laser pulse shape in photoinjectors of Free-Electron Lasers is a powerful lever for optimizing electron beam quality, but systematic exploration of the vast design space is limited by the cost of brute-force pulse propagation simulations. We present a generative modeling framework based on Wasserstein Autoencoders to learn a differentiable latent interface between pulse shaping and downstream beam dynamics. Our empirical findings show that the learned latent space is continuous and interpretable while maintaining high-fidelity reconstructions. Pulse families such as higher-order Gaussians trace coherent trajectories, while standardizing the temporal pulse lengths shows a latent organization correlated with pulse energy. Analysis via principal components and Gaussian Mixture Models reveals a well behaved latent geometry, enabling smooth transitions between distinct pulse types via linear interpolation. The model generalizes from simulated data to real experimental pulse measurements, accurately reconstructing pulses and embedding them consistently into the learned manifold. Overall, the approach reduces reliance on expensive pulse-propagation simulations and facilitates downstream beam dynamics simulation and analysis.

Alexander Klemps, Denis Ilia, Pradeep Kr. Banerjee +3

Institute for Data Science Foundations, Hamburg University of Technology, Germany · Deutsches Elektron-Synchrotron (DESY), Hamburg, Germany

Representation LearningGenerative Modeling
AskPDF
Feb 19, 2026

IntentCUA: Learning Intent-level Representations for Skill Abstraction and Multi-Agent Planning in Computer-Use Agents

Computer-use agents operate over long horizons under noisy perception, multi-window contexts, evolving environment states. Existing approaches, from RL-based planners to trajectory retrieval, often drift from user intent and repeatedly solve routine subproblems, leading to error accumulation and inefficiency. We present IntentCUA, a multi-agent computer-use framework designed to stabilize long-horizon execution through intent-aligned plan memory. A Planner, Plan-Optimizer, and Critic coordinate over shared memory that abstracts raw interaction traces into multi-view intent representations and reusable skills. At runtime, intent prototypes retrieve subgroup-aligned skills and inject them into partial plans, reducing redundant re-planning and mitigating error propagation across desktop applications. In end-to-end evaluations, IntentCUA achieved a 74.83% task success rate with a Step Efficiency Ratio of 0.91, outperforming RL-based and trajectory-centric baselines. Ablations show that multi-view intent abstraction and shared plan memory jointly improve execution stability, with the cooperative multi-agent loop providing the largest gains on long-horizon tasks. These results highlight that system-level intent abstraction and memory-grounded coordination are key to reliable and efficient desktop automation in large, dynamic environments.

Seoyoung Lee, Seobin Yoon, Seongbeen Lee +4

Sookmyung Women’s University, Seoul, Republic of Korea

Computer-Use AgentsLong-Horizon Agent TasksMulti-Agent PlanningRepresentation LearningShared-Memory Multi-Agent Systems
AskPDF
Feb 18, 2026

Coverage-aware Semantic Representation Learning for Underrepresented Visual Concepts

Modern vision models increasingly rely on rich semantic representations that extend beyond class labels to include descriptive concepts, attributes, and contextual cues. However, semantic concepts are not uniformly represented across classes: a concept may be frequent globally yet remain underrepresented within a class, resulting in low class-concept coverage. We formalize this class-concept phenomenon as Semantic Coverage Imbalance (SCI) and study its relationship with learned semantic representations. We find that lower class-concept coverage is consistently associated with weaker semantic representations. To address this, we introduce Coverage-aware Semantic Representation Learning, which shares more semantic structure for low coverage or uncertain relations while preserving more pair-specific structure for high coverage relations. We instantiate this principle in SemCovNet with Coverage-Calibrated Semantic Sharing (CCSS), a coverage- and uncertainty-aware partial-pooling mechanism. Using a range of vision datasets spanning facial attributes and fine-grained recognition, as well as real-world medical datasets, we show that SCI is widespread, degrades representation quality, and can be mitigated through coverage-aware semantic representation learning.

Sakib Ahammed, Xia Cui, Wenqi Lu +4

Department of Computing and Mathematics, Manchester Metropolitan University The Dalton Building, Chester Street, M1 5GD Manchester

Representation LearningVisual Representation LearningSemantic Representation LearningFine-Grained Visual Perception
AskPDF
Feb 18, 2026

Knowledge-Embedded Latent Projection for Robust Representation Learning

Latent space models are widely used for analyzing high-dimensional discrete data matrices, such as patient-feature matrices in electronic health records (EHRs), by capturing complex dependence structures through low-dimensional embeddings. However, estimation becomes challenging in the imbalanced regime, where one matrix dimension is much larger than the other. In EHR applications, cohort sizes are often limited by disease prevalence or data availability, whereas the feature space remains extremely large due to the breadth of medical coding system. Motivated by the increasing availability of external semantic embeddings, such as pre-trained embeddings of clinical concepts in EHRs, we propose a knowledge-embedded latent projection model that leverages semantic side information to regularize representation learning. Specifically, we model column embeddings as smooth functions of semantic embeddings via a mapping in a reproducing kernel Hilbert space. We develop a computationally efficient two-step estimation procedure that combines semantically guided subspace construction via kernel principal component analysis with scalable projected gradient descent. We establish estimation error bounds that characterize the trade-off between statistical error and approximation error induced by the kernel projection. Furthermore, we provide local convergence guarantees for our non-convex optimization procedure. Extensive simulation studies and a real-world EHR application demonstrate the effectiveness of the proposed method.

Weijing Tang, Ming Yuan, Zongqi Xia +1

Department of Statistics and Data Science, Carnegie Mellon University · Department of Statistics, Columbia University · Department of Neurology, University of Pittsburgh +1

Representation LearningLatent Variable ModelsElectronic Health Records
AskPDF
Feb 18, 2026

On the Power of Source Screening for Learning Shared Feature Extractors

Learning with shared representation is widely recognized as an effective way to separate commonalities from heterogeneity across various heterogeneous sources. Most existing work includes all related data sources via simultaneously training a common feature extractor and source-specific heads. It is well understood that data sources with low relevance or poor quality may hinder representation learning. In this paper, we further dive into the question of which data sources should be learned jointly by focusing on the traditionally deemed ``good'' collection of sources, in which individual sources have similar relevance and qualities with respect to the true underlying common structure. Towards tractability, we focus on the linear setting where sources share a low-dimensional subspace. We find that source screening can play a central role in statistically optimal subspace estimation. We show that, for a broad class of problem instances, training on a carefully selected subset of sources suffices to achieve minimax optimality, even when a substantial portion of data is discarded. We formalize the notion of an informative subpopulation, develop algorithms and practical heuristics for identifying such subsets, and validate their effectiveness through both theoretical analysis and empirical evaluations on synthetic and real-world datasets.

Leo Muxing Wang, Connor Mclaughlin, Lili Su

Northeastern University

Shared Representation LearningRepresentation LearningData SelectionTraining Data SelectionMulti-Task LearningStatistical Learning Theory
AskPDF
Feb 17, 2026

A unified theory of feature learning in RNNs and DNNs

Recurrent and deep neural networks (RNNs/DNNs) are cornerstone architectures in machine learning. Remarkably, RNNs differ from DNNs only by weight sharing, as can be shown through unrolling in time. How does this structural similarity fit with the distinct functional properties these networks exhibit? To address this question, we here develop a unified mean-field theory for RNNs and DNNs in terms of representational kernels, describing fully trained networks in the feature learning (μμP) regime. This theory casts training as Bayesian inference over sequences and patterns, directly revealing the functional implications induced by the RNNs' weight sharing. In DNN-typical tasks, we identify a phase transition when the learning signal overcomes the noise due to randomness in the weights: below this threshold, RNNs and DNNs behave identically; above it, only RNNs develop correlated representations across timesteps. For sequential tasks, the RNNs' weight sharing furthermore induces an inductive bias that aids generalization by interpolating unsupervised time steps. Overall, our theory offers a way to connect architectural structure to functional biases.

Jan P. Bauer, Kirsten Fischer, Moritz Helias +1

Gatsby Computational Neuroscience Unit, UCL, London, UK · IAS-6, Forschungszentrum Juelich, Juelich, Germany

Recurrent Neural NetworksRepresentation LearningNeural Network Generalization
AskPDF
Feb 17, 2026

Logit Distance Bounds Representational Similarity

For a broad family of discriminative models that includes autoregressive language models, identifiability results imply that if two models induce the same conditional distributions, then their internal representations are equal up to an invertible linear transformation. We ask whether an analogous conclusion holds approximately when the distributions are close instead of equal. Building on the observation of Nielsen et al. (2025) that closeness in KL divergence need not imply high linear representational similarity, we study a distributional distance based on logit differences and show that closeness in this distance does yield linear similarity guarantees. Specifically, we define a representational dissimilarity measure based on the models' identifiability class and prove that it is bounded by the logit distance. We further show that, when model probabilities are bounded away from zero, KL divergence upper-bounds logit distance; yet the resulting bound fails to provide nontrivial control in practice. As a consequence, KL-based distillation can match a teacher's predictions while failing to preserve linear representational properties, such as linear-probe recoverability of human-interpretable concepts. In distillation experiments on synthetic and image datasets, logit-distance distillation yields students with higher linear representational similarity and better preservation of the teacher's linearly recoverable concepts.

Beatrix M. G. Nielsen, Emanuele Marconato, Luigi Gresele +2

IT University of Copenhagen, Denmark · University of Trento, Italy · University of Copenhagen, Denmark +2

Representational Similarity AnalysisRepresentation LearningRepresentation IdentifiabilityKnowledge DistillationRepresentation Alignment
AskPDF
Feb 16, 2026

Symmetry in language statistics shapes the geometry of model representations

The internal representations learned by language models consistently exhibit striking geometric structure: calendar months organize into a circle, historical years form a smooth one-dimensional manifold, and cities' latitudes and longitudes can be decoded using a linear probe. To explain this neural code, we first show that language statistics exhibit translation symmetry (for example, the frequency with which any two months co-occur in text depends only on the time interval between them). We prove that this symmetry governs these geometric structures in high-dimensional word embedding models, and we analytically derive the manifold geometry of word representations. These predictions empirically match large text embedding models and large language models. Moreover, the representational geometry persists at moderate embedding dimension even when the relevant statistics are perturbed (e.g., by removing all sentences in which two months co-occur). We prove that this robustness emerges naturally when the co-occurrence statistics are controlled by an underlying latent variable. Our results indicate that these representational manifolds originate in the statistical symmetries of natural language.

Dhruva Karkada, Daniel J. Korchinski, Andres Nava +2

1UC Berkeley · 2EPFL · 3Johns Hopkins +1

Representation LearningNeural Representation GeometryWord EmbeddingsRepresentation Geometry in Language ModelsLatent Space Geometry
AskPDF