Frozen Backbone

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, up 25% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 46

All topics

Universal interpolation for deep residual self-attention networks
Oct 1, 2026Sibylle Marcotte, Joan BrunaTransformer Architectures Approximation
Department of Computer Science New York University
Grounding Time-Series Foundation Models in Digital Twin Topology for Predictive Maintenance
Sep 30, 2026Sizhe Ma, Katherine A. Flanigan, Mario BergésTime Series Foundation Models Predictive Maintenance
Department of Civil & Environmental Engineering, Carnegie Mellon University 5000 Forbes Ave., Pittsburgh, PA USA 15213
OccluDex: Hierarchical 3D Visuo-Tactile Representation Learning for Egocentric Dexterous Manipulation under Self-Occlusion
Sep 30, 2026Ziheng Xu, Yueyuan Chen, Xinyuan He +6Contact-Rich Dexterous Manipulation Tactile
Shanghai Jiao Tong University · Tongji University · The Chinese University of Hong Kong, Shenzhen
Self-Distilled Pronunciation and Accent Control for Neural Text-to-Speech
Sep 15, 2026Shuhei KatoFlow-Matching Text-To-Speech Accents
Independent Researcher, Japan · KOWRO Inc., Japan
BarkNet-Lite: A Lightweight Texture and Colour Network with the BarkBD Benchmark for Bark-Based Tree Species Recognition in Bangladesh
Sep 7, 2026Aroshi Ali, Saad Ahmed, Md. Khalid SyfullahSpecies Identification Benchmark Datasets
Department of Computer Science and Engineering, Khulna University of Engineering & Technology, Khulna, 9203, Bangladesh · Department of Computer Science and Engineering, Bangladesh Army University of Science & Technology, Saidpur, Bangladesh
SPARK: Input-Conditioned Sparse Activation Modulation for Frozen DiT-based Super-Resolution
Sep 3, 2026Federico Putamorsi, Leonardo Zini, Marcella Cornia +1Diffusion Transformers Frozen Backbone
University of Modena and Reggio Emilia, Italy
ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits
Sep 1, 2026Sethuraman T, Savya Khosla, Onkar Kishor Susladkar +6Video-Language Models Image-Text Alignment
University of Illinois Urbana-Champaign · Adobe Research
Adapting Without Gradients: Affine Statistics Transport and What Its Certificate Can Tell You
Aug 31, 2026Salim Khazem, Ibrahim Mohamed SerouisTest-Time Adaptation Frozen Backbone
Talan Research Center · Paris, France · Toulouse, France
Rethinking Data Efficiency in Industrial Dense Prediction: Pretraining Coherence, Not Inductive Bias, Determines ViTs Low-Data Advantage
Aug 11, 2026Haoran Sui, Yaoyuan JiaVision Transformer Frozen Backbone
ZTE Corporation, Shenzhen, Guangdong Province, P.R. China · The University of Hong Kong, Hong Kong SAR, P.R. China
Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses
Aug 9, 2026Tailin ZhouEvolving Harness Large Language Model Agents
HKUST
SCoPE: Training-Free Audio-Visual Event Perception via Sparse Cross-Modal Prior Exchange
Aug 8, 2026Jaemo Jeong, Junho Yoon, Hyunju Kim +1Audio-Visual Reasoning Fine-Grained Video Understanding
KAIST
Align-RAG: Alignment Is All You Need for TSFM In-Context Learning
Aug 6, 2026Mohammad Asadi, Soheil Hor, Bardiya Akhbari +6Retrieval-Augmented Forecasting Time Series Foundation Models
1Stanford University 2Amazon
PRISM: Powerful Time Series to Image (TS2I) Representations for Multivariate Anomaly Detection
Aug 4, 2026Mateusz Smendowski, Kamil Faber, Piotr Nawrocki +2Time-Series Anomaly Detection Time Series
Faculty of Computer Science, AGH University of Krakow, al. A. Mickiewicza 30, 30-059 Krakow, Poland · Department of Computer Science, American University, 4400 Massachusetts Ave NW, Washington, DC 20016, USA
Breaking Diversity Collapse in Spiking Pseudo-Ensembles for Efficient OOD Detection in Remote Sensing
Aug 2, 2026Srinivas Anumasa, Rushi Shah, Qiran Zou +1Spiking Neural Networks Out-Of-Distribution Detection
National University of Singapore
Rescene: band-limited stochastic forcing turns a frozen neural weather operator into a climate emulator
Jul 30, 2026Minjong CheonEra5 Reanalysis Data Climate
Department of Computer Science and Engineering, Sejong University, Seoul, South Korea
Deep Learning Estimation of Sex, Age, Height, and Weight from CT-derived Digitally Reconstructed Radiographs
Jul 21, 2026Tomohiro Kikuchi, Kohei Yamamoto, Yukihiro Nomura +6Digitally Reconstructed Radiographs Frozen Backbone
Department of Radiology, Jichi Medical University, 3311-1 Yakushiji, Shimotsuke-shi, Tochigi, 329-0498, Japan. · Data Science Center, Jichi Medical University, 3311-1 Yakushiji, Shimotsuke-shi, Tochigi, 329-0498, Japan. · Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan. +3
Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator
Jul 20, 2026Mateusz Piechocki, Alessandro Capotondi, Marek KraftEdge Devices Parameter-Efficient Adaptation
Poznan University of Technology, Institute of Robotics and Machine Intelligence, Poznan, Poland · University of Modena and Reggio Emilia, The High-Performance Real-Time Laboratory, Modena, Italy
Lightweight Wrappers for Adapting Time Series Foundation Models to Regional Drought Forecasting
Jul 20, 2026Wentao Gao, Jiuyong Li, Lin Liu +4Time Series Foundation Models Zero-Shot Forecasting
Adelaide University · Adelaide, Australia · CSIRO Technology +1
Transferable Low-Rank Convolutional Bases for Onboarding Unseen Medical Imaging Modalities
Jul 18, 2026Ranat Das Prangon, Istiaque Ahmed, Shajid Hasan Naim +2Frozen Backbone Modalities
Dept. of Chemical Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh. · Graduate School of Informatics, Osaka Metropolitan University, Osaka, Japan. · Dept. of Mechanical Engineering, Chittagong University of Engineering and Technology (CUET), Chattogram, Bangladesh. +2
Is Domain Adaptation Always Helpful? A Frozen-Backbone Study of Cross-Domain Sentiment Transfer
Jul 7, 2026Phat Tran, Artin Lahni, Pranav Kulkarni +1Domain Adaptation Frozen Backbone
Oregon State University
Separating Representation from Reconstruction Enables Scalable Text Encoders
Jul 4, 2026Megi Dervishi, Mathurin Videau, Yann LeCunSentence Encoders Encoders
FAIR · 3New York University
When Does Resolution Help a Frozen Backbone? Global Attention at Resolution Predicts Scalable Adaptation for Camouflaged and Marine Animal Segmentation
Jul 2, 2026Tyler Rust, Chandra KambhamettuFrozen Backbone Self-Supervised Vision Transformers
University of Delaware · University of South Florida
JuZhou 1.0 Technical Report: The First Edge-Native Text-to-Image Foundation Model Trained Entirely on China-Developed AI Accelerators
Jun 25, 2026Ce Chen, Congrui Wang, Yonglin Li +20Text-To-Image Frozen Backbone
JuZhou Team, HSW Group
Data-Free Reservoir Features for Efficient Long-Horizon Cold-Start Continual Learning
Jun 25, 2026Augustinas Jučas, Yangchen PanClass-Incremental Learning Replay-Based Continual Learning
Department of Computer Science University of Oxford · Department of Engineering Sciences University of Oxford
Naturalness Predicts but Does Not Cause Transferability in Image Encodings of Real-World Streams
Jun 24, 2026Faruk Alpay, Baris BasaranFrozen Backbone
Department of Computer Engineering, Bahçeşehir University, Istanbul, Turkey
TrOCR for Medieval HTR: A Systematic Ablation Study with Cross-Dataset Validation
Jun 23, 2026Sachin Sharma, Michele Flammini, Federico SimonettaHandwritten Text Recognition Historical Manuscripts
GSSI – Gran Sasso Science Institute L’Aquila, Italy
Reliability-Aware Prototype Calibration for Frozen Pose-Flow Video Anomaly Detection
Jun 18, 2026Ning Dong, Yingna Su, Xin Dong +3Training-Free Video Anomaly Detection Area Under The Receiver Operating Characteristic Curve
School of Information Engineering Suqian University Suqian, China · School of Computer Science and Engineering University of Electronic Science and Technology of China Chengdu, China · Institute for Advanced Studies Universiti Malaya Kuala Lumpur, Malaysia
EAGG: Embodiment-Aligned Grasp Generation via Geometry-Aware Graph Conditioning
Jun 16, 2026Wanhao Niu, Qiyan Ke, Yuan Sun +5Robotic Grasping End-Effector Pose
Department of Computer Science and Technology, Tsinghua University, Beijing, China · Beijing Moce Future Technology Co., Ltd., Beijing, China
EPIG: Emotion-Based Prompting for Personalised Image Generation
Jun 11, 2026Emna Othmen, Mohamed Yassine Landolsi, Lotfi Ben RomdhaneImage Generation Contrastive Language-Image Pre-Training Model
MARS Research Lab LR17ES05, ISITCom, University of Sousse, Sousse, Tunisia.
EWAM: An Enhanced World Action Model for Closed-Loop Online Adaptation in Embodied Intelligence
Jun 10, 2026Xin Zhou, Cong MiaoEfficient World-Action Model Recent World-Action Models
Astronex Robotics · Nanjing University of Information Science and Technology
CFRNet: Cycle-Consistent Fixed-Point Training for Real-Time Blind Face Restoration on Consumer Embedded NPUs
Jun 5, 2026Fuchen Li, Xinyang Wang, Yahui Zhang +4Blind Face Restoration Image Restoration
University of Florida, Gainesville, FL, USA · University of Southampton, Southampton, UK · Chongqing University, Chongqing, China +2
MemoGen: Can Past Experience Improve Future Text-to-Image Generation?
Jun 2, 2026Wenshuo Chen, Kuimou Yu, Bowen Tian +10Text-To-Image Frozen Backbone
The Hong Kong University of Science and Technology (Guangzhou) · Scholarly, Guangzhou Ziyan Technology Co., Ltd. · LimX Dynamics Technology Co., Ltd. +4
PRISM: Progressive Reasoning through Iterative Slot Memory for Vision
May 29, 2026Ziyu Wang, Shuangpeng Han, Mengmi ZhangVisual Representations Frozen Backbone
Deep NeuroCognition Lab, Nanyang Technological University, Singapore
Rethinking Transfer Learning for Industrial Inspection: DINOv3 vs. ImageNet Pretraining Across RGB and X-ray Tasks
May 22, 2026Mehdi Gharbage, Céline Teulière, Pierre Bouges +1Recent Vision Foundation Models Dinov3
1Michelin Tyres Manufacturer · 2Universit´e Clermont Auvergne, CNRS, Institut Pascal, Clermont-Ferrand, France
Entropy-Guided Self-Supervised Learning for Medical Image Classification
May 21, 2026Joao Florindo, Viviane MouraMedical Image Classification Medical Imaging Datasets
Institute of Mathematics, Statistics and Scientific Computing Department of Applied Mathematics, University of Campinas Rua Sergio Buarque de Holanda, 651, Campinas, 13083-859, Sao Paulo, Brazil
MAPS: A Synthetic Dataset for Probing Vision Models in a Controlled 3D Scene Space
May 19, 2026Santiago Galella, Pamela Osuna-Vargas, Maren Wehrheim +3Vision Foundation Models Indoor Scene Generation
FIAS & Institute of Computer Science · York University · Mila & Department of Biology +1
FullFlow: Upgrading Text-to-Image Flow Matching Models for Bidirectional Vision--Language Generation
May 19, 2026Eric Tillmann Bill, Enis Simsar, Alessio Tonioni +1Text-To-Image Diffusion Models Recent Vision-Language Models
ETH Zurich · Google
AgriMind: An Ensemble Deep Learning Framework for Multi-Class Plant Disease Classification
May 15, 2026Salma Hoque Talukdar Koli, Fahima Haque Talukder JelyLeaf Disease Classification Tomato Diseases
RTM Al-Kabir Technical University, Sylhet-3100, Bangladesh · North East University Bangladesh, Sylhet, Bangladesh
Inline Critic Steers Image Editing
May 12, 2026Weitai Kang, Xiaohang Zhan, Yizhou Wang +4Image Editing Multi-Reference Image Generation
University of Illinois Chicago · Adobe
$δ$ -mem: Efficient Online Memory for Large Language Models
May 12, 2026Jingdi Lei, Di Zhang, Junxian Li +7Peak Memory Efficient Long-Context Inference
Nanyang Technological University · Mind Lab · Fudan University +3
Backbone-Equated Diffusion OOD via Sparse Internal Snapshots
May 10, 2026Yadang Alexis Rouzoumka, Jean Pinsolle, Eugénie Terreaux +3Out-Of-Distribution Detection Frozen Backbone
DEMR, ONERA & SONDRA · Université Paris-Saclay · SONDRA, CentraleSupélec +1
MC-RFM: Geometry-Aware Few-Shot Adaptation via Mixed-Curvature Riemannian Flow Matching
May 8, 2026Salim Khazem, Ibrahim Mohamed Serouis, Zakaria EzzahedFew-Shot Learning Frozen Backbone
Talan Research Center, Paris, France
HALO: Hybrid Adaptive Latent Reasoning for Language Models
May 3, 2026Micah ZhangFrozen Backbone Efficient Latent Reasoning
Lockheed Martin
Not All Forgetting Is Equal: Retention Dynamics in Fine-Tuned Image Classifiers
Apr 13, 2026Miit Daga, Swarna Priya RamuRetraining Retention
School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, 632014, Tamil Nadu, India
SK-Adapter: Skeleton-Based Structural Control for Native 3D Generation
Mar 14, 2026Anbang Wang, Yuzhuo Ao, Shangzhe Wu +1High-Fidelity 3D Generation 3D Generation
The Hong Kong University of Science and Technology · University of Cambridge
Spectral Overfitting in Noisy Linear Probing of Pretrained Representations
Mar 2, 2026Zice Wang, Zhenyu ZhangNoisy Labels Overfitting
School of Future Technology, Northeastern University, Shenyang, China.

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Universal interpolation for deep residual self-attention networks

Sibylle Marcotte, Joan Bruna

Department of Computer Science New York University

Transformer Architectures Approximation Frozen Backbone

Open paper AskPDF arXiv

Universal approximation is a necessary qualitative property of learning architectures to benefit from scaling laws. While it is generically verified on a variety of neural architectures and random feature models, it typically involves infinite width limits. In this work, we focus on deep self-attention models and consider instead the `dual' regime, where approximation power is enabled entirely by depth, and featuring strong parameter sharing across layers, motivated by recent models such as the Looped Transformers. More specifically, we ask whether one can find a predefined finite set of parameters, each defining an attention block, such that the resulting finite set of transformations can map any collection of

N

sequences of

n

tokens to any other collection of

N

sequences of

n

tokens. Crucially, these transformations are \emph{fixed independently of the input and output} collections: only the order in which the blocks are applied, their signs, and their durations depend on the particular interpolation task. Our main result establishes it for residual softmax attention using only two frozen single-head blocks with Gaussian-initialized projection matrices. The result holds at both continuous and finite depth. We also characterize the restrictions imposed by causal masking and establish corresponding universal interpolation guarantees.

Momentum

Related topics

Latest papers 46