Sparse Structural Specialization

Follow this topic Ask about this topic

Momentum

1 paper in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 14

All topics

Persistence Forcing: Exploiting Feature Specialization in Pixel-Space Diffusion
Sep 28, 2026Chong Wang, Zixuan Fu, Shiqi Huang +3Imagenet Sparse Structural Specialization
Nanyang Technological University · KTH Royal Institute of Technology · Hebei University of Technology
From Intents to Algorithms: Verified Algorithm Discovery for Transport Networks
Sep 23, 2026Behnam Ojaghi, Ricard Vilalta, Raul MuñozOptical Networks Transport
CTTC, Castelldefels, Barcelona, Spain
FUSE: Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching
Aug 7, 2026Suman Cha, Seongchan Lee, Dohyun Ko +1Conditional Flow Matching Mixture Models
Department of Statistics and Data Science, Yonsei University · Department of Mathematical Sciences, KAIST
Hidden Gauge Controls Feature Specialization in ReLU Networks
Aug 7, 2026Tongxi WangFeature Learning Rectified Linear Unit Networks
School of Future Technology Southeast University Nanjing, China
STEAM: A Spatio-TEmporal Alignment Mixture-of-Experts Model with Hierarchical Pre-training for EEG Decoding
Aug 3, 2026Zhu Chen, Dingkun Liu, Yuheng Chen +1Electroencephalography Foundation Models Electroencephalography
Ministry of Education Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China · Zhongguancun Academy, Beijing 100094, China
SpecAHD: Localize to Specialize for Automated Heuristic Design in Large-Scale Routing Problems
Jul 26, 2026Kezhao Lai, Yutao Lai, Hai-Lin LiuAutomated Heuristic Design Vehicle Routing Problem
School of Mathematics and Statistics, Guangdong University of Technology, China
MixCompress: Mixture of Experts for Variable Rate Learned Image Compression
Jul 15, 2026Calvin-Khang Ta, Praneet Singh, Tong Shao +1Learned Image Compression Ultra-Low Bitrates
Dolby Laboratories, USA
Ranking the Impact of Contextual Specialization in Neural Speech Enhancement
Jul 6, 2026Peter Leer, Svend Feldt, Zheng-Hua Tan +2Speech Enhancement Speaker
Eriksholm Research Centre, Snekkersten, Denmark · Aalborg University, Department of Electronic Systems, Aalborg, Denmark
Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter
Jun 2, 2026Zhengbao He, Ruiqi Ding, Zhehao Huang +3Low-Rank Adaptation Framework Large Language Model Compression
Institute of Image Processing and Pattern Recognition, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China · Shanghai Key Laboratory of Flexible Medical Robotics, Tongren Hospital, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
Learn from Weaknesses: Automated Domain Specialization for Small Computer-Use Agents
May 27, 2026Suji Kim, Kangsan Kim, Sung Ju HwangComputer-Use Agents Sparse Structural Specialization
KAIST · Samsung Electronics
From Latent Space to Training Data: Explainable Specialization in Minimal MLPs
May 25, 2026Enrique Alba, Ezequiel Lopez-RubioMultilayer Perceptrons Sparse Structural Specialization
aITIS Software, University of Malaga, Spain
Learning Dynamic Structural Specialization for Underwater Salient Object Detection
May 15, 2026Lin Hong, Chenhui Wang, Linan Deng +7Underwater Object Detection Underwater Acoustic Target Recognition
School of Automation, Southeast University, Nanjing, China.
Polysemantic Experts, Monosemantic Paths: Routing as Control in MoEs
Apr 20, 2026Charles Ye, Bo Yuan, Lee SharkeyMixture-Of-Experts Layer-Wise
Independent · Georgia Institute of Technology · Goodfire
Understanding and Enforcing Weight Disentanglement in Task Arithmetic
Apr 18, 2026Shangge Liu, Yuehan Yin, Lei Wang +5Orthogonality Regularization
State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia · Nanyang Technological University, Singapore

Sep 28, 2026, cs.CVJ/K move · Enter open · S save

Persistence Forcing: Exploiting Feature Specialization in Pixel-Space Diffusion

Chong Wang, Zixuan Fu, Shiqi Huang, Siyuan Yang+2

Nanyang Technological University · KTH Royal Institute of Technology · Hebei University of Technology

Imagenet Sparse Structural Specialization Persistence

Open paper AskPDF arXiv

Pixel-space diffusion Transformers (DiTs) directly operate on high-dimensional visual data, yet their hidden representations typically undergo uniform refinement across depth. Natural images, however, are inherently organized at different levels of granularity. Global structure can often be represented compactly, whereas local textures and fine details require richer representations. Motivated by this, we introduce heterogeneous refinement in pixel-space DiTs, assigning different feature groups distinct refinement budgets across depth. Consequently, an ordered feature specialization emerges: sparsely refined features predominantly encode global visual structure, whereas more frequently refined features increasingly specialize toward localized, high-frequency details. We refer to these two groups as persistent and active features, respectively. Building on this emergent specialization, we introduce Persistence Forcing (PerF), which explicitly exploits this persistent--active feature organization for pixel-space image generation. This enables persistent features to continuously condition actively refined features, allowing stable global information to guide the ongoing refinement of finer visual details. During generative sampling, this interaction further induces a meaningful guidance direction that promotes coherent global structure and naturally complements classifier-free guidance. On ImageNet

256\times256

, PerF-L achieves FID of

1.91

, approaching

1.86

of JiT-H with only half the parameters, while PerF-H further achieves FID of

1.63

and

1.76

on ImageNet

256\times256

and

512\times512

, respectively.

Momentum

Related topics

Latest papers 14