Effective Distillation

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 20

All topics

AHMAD: Adaptive Hybrid Multi-task Vision Learning with Assisted Distillation for Keypoint Detection
Sep 28, 2026Mohammad Mahdi, Nedyalko Prisadnikov, Yuqian Fu +3Multi--Task Learning Keypoint Detection
INSAIT, Sofia University “St. Kliment Ohridski”
On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics
Sep 28, 2026Julianna Piskorz, Antonin Berthon, Mihaela van der SchaarOff-Policy Learning On-Policy
University of Cambridge Cambridge, UK
AnyStep-WAM: Budget-Aligned Distillation and Adaptive Inference for World Action Models
Sep 27, 2026Rui Wang, Xiangyu Wang, Donglin Yang +4Recent World-Action Models Faster-Wam
Southern University of Science and Technology · Shenzhen Loop Area Institute · The University of Hong Kong
Reward Lightning: Fast Video Generation via Homologous Preference Distillation
Jul 4, 2026Jiaxiang Cheng, Bing Ma, Xuhua Ren +4Video Diffusion Models Long Video Generation
Tencent Hunyuan, China
A Decomposable Probe for Few-Step Diffusion Models: Prompt, Latent, and Score Selectivity across Backbone Families and Distillation Paradigms
Jul 3, 2026Patrick Mu HaojieEffective Distillation Iterative Denoising Methods
Independent Researcher.
Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation
Jul 1, 2026Shayan Talaei, Abhinav Chinta, Devvrit Khatri +3Large Language Model Bias Biases
1Stanford University · 3Foundation AI–Cisco Systems Inc. · University of Texas at Austin
SATB-VR: Training Few-Step Video Restoration Diffusion Model using SNR-Aware Trajectory Blending
Jun 27, 2026Haoran Bai, Xiaoxu Chen, Xiaoyu Liu +4Video Restoration Iterative Denoising Methods
1Alibaba Group · 2Harbin Institute of Technology · 3Xi’an Jiaotong University
RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting
May 29, 2026Yuduo Li, Xiaofeng Shi, Qian Kou +2Supervised Finetuning Dataset Distillation
Beijing Academy of Artificial Intelligence (BAAI) · Beijing Jiaotong University (BJTU)
When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection
May 26, 2026Zhengyu Hu, Zheyuan Xiao, Linxin Song +10Teacher Effective Distillation
University of Washington · University of Texas at Austin · University of Southern California +6
Fast Image Super-Resolution via Consistency Rectified Flow
May 12, 2026Jiaqi Xu, Wenbo Li, Haoze Sun +8Real-World Image Super-Resolution Generative Super-Resolution
The Chinese University of Hong Kong · Huawei Noah’s Ark Lab · HKUST (GZ) +1
Federated Language Models Under Bandwidth Budgets: Distillation Rates and Conformal Coverage
May 11, 2026Prasanjit Dubey, Xiaoming HuoFederated Learning Probe-Logit Distillation
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332, U.S.A.
Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation
May 8, 2026Amin Karimi Monsefi, Dominic Culver, Nikhil Bhendawade +3Conditional Flow Matching Few-Step Distillation
1The Ohio State University · 2Apple
Learned Neighbor Trust for Collaborative Deployment in Model-Agnostic Decentralized Learning
May 6, 2026Michael Lanier, Luise Ge, Sastry Kompella +1Log-Ratio-Based Gaussian Trust Weight Decentralized Learning
Department of Computer Science Washington University in St. Louis and Nexcepta
Deep Reprogramming Distillation for Medical Foundation Models
May 6, 2026Siyuan Du, Yuhang Zhou, Haolin Li +5Knowledge Distillation Wireless Foundation Models
School of Computer Science, Fudan University and with the Shanghai AI Laboratory, Shanghai, China · Shanghai AI Laboratory, Shanghai, China · Cooperative Medianet Innovation Center, Shanghai Jiao Tong University and with the Shanghai AI Laboratory, Shanghai, China +3
Hiding in Plain Sight: Detectability-Aware Antidistillation of Reasoning Models
Apr 25, 2026Max Hartman, Vidhata Jayaraman, Moulik Choraria +2Effective Distillation Reasoning Traces
University of Illinois Urbana-Champaign · Carnegie Mellon University · Stony Brook University
Distillation Traps and Guards: A Calibration Knob for LLM Distillability
Apr 21, 2026Weixiao Zhan, Yongcheng Jing, Leszek Rutkowski +1Knowledge Distillation Dataset Distillation
Generative AI Lab, College of Computing and Data Science Nanyang Technological University, Singapore 639798 · Systems Research Institute of the Polish Academy of Sciences, AGH University of Krakow, 30-059 Kraków, and the SAN University, 90-113, Łód´z, Poland
When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models
Mar 27, 2026Juan Gabriel Kostelec, Qinghai GuoEffective Distillation Symbolic Distillation
Huawei Switzerland · ACS Lab, Huawei Technologies
Effective Distillation to Hybrid xLSTM Architectures
Mar 16, 2026Lukas Hauzenberger, Niklas Schmidinger, Thomas Schmied +7Probe-Logit Distillation Effective Distillation
1NXAI · 2Johannes Kepler University Linz
TimeWarp: Evaluating Web Agents by Revisiting the Past
Mar 5, 2026Md Farhan Ishmam, Kenneth MarinoWeb Agents Agentic Benchmarks
University of Utah, USA
When Distillation Breaks Motion Control: Restoring Generative Trajectories for Fast Video Generators
Jun 24, 2025Jintao Rong, Xin Xie, Xinyi Yu +4Student-Generated Trajectories Effective Distillation
Zhejiang University of Technology · University of New South Wales (UNSW Sydney) · University of Auckland +1

Sep 28, 2026, cs.CVJ/K move · Enter open · S save

AHMAD: Adaptive Hybrid Multi-task Vision Learning with Assisted Distillation for Keypoint Detection

Mohammad Mahdi, Nedyalko Prisadnikov, Yuqian Fu, Carmelo Scribano+2

INSAIT, Sofia University “St. Kliment Ohridski”

Multi--Task Learning Keypoint Detection Object Detection Multiple Vision Tasks Decoder Layers Projection Heads+1

Open paper AskPDF arXiv

Generalist multitasking vision models aim to unify multiple vision tasks within a single framework, enabling more efficient and versatile learning. However, handling diverse vision tasks -- spanning dense and sparse predictions -- remains challenging due to their inherently varying output structures. In this paper, we propose AHMAD, a simple yet effective framework for generalist multitask learning that integrates different key vision tasks: semantic segmentation, instance segmentation, depth estimation, keypoint detection, and object detection. Our approach incorporates these five tasks into a unified structure: a shared encoder-decoder with several lightweight task-specific projectors. Under the multitask learning paradigm, we observed a complementary performance gain, achieving a state-of-the-art PQ of 53.1 and an mIoU of 66.5 for COCO-val panoptic and semantic segmentation, respectively. Additionally, for top-down keypoint detection, which typically incurs high computational overhead due to multiple forward passes, we introduce a knowledge distillation-based method that enables a single forward pass over the entire image, greatly improving efficiency. Ultimately, our model delivers a lightweight yet effective generalist multitask learning framework, demonstrating strong performance across five vision tasks.

Momentum

Related topics

Latest papers 20