Deep Supervision

Follow this topic Ask about this topic

Latest papers

8 papers0 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Best-of-Depth: Learning Exploratory Dynamics in Looped Transformers
Nov 3, 2026Shota Takashiro, Takeshi Kojima, Masanori Koyama +2Deep Supervision Recurrent Transformers
The University of Tokyo
Training Crossroads for Recurrent Vision Transformers: Recurrence, Neural ODEs, and Deep Supervision
Aug 5, 2026Grzegorz Gruszczynski, Pawel Olszowiec, Michal Byra +2Recurrent Transformers Neural ODEs
Samsung AI Center, Warsaw, Poland · Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models
Aug 3, 2026Jing Wu, Jianhua Wu, Jiayi Guan +5Vision-Language Models Visual Spatial Reasoning
Xiaomi EV, Beijing, China · College of Automotive and Energy Engineering, Tongji University, Shanghai, China · Independent Researcher
PRISM: Privileged Probabilistic Latent Supervision for End-to-End Autonomous Driving Motion Planning
Aug 2, 2026Volodymyr Havrylov, Faris Janjoš, Andreas Look +2End-to-End Autonomous Driving Autonomous Driving Planning
University of Tübingen · Bosch Center for Artificial Intelligence · Coburg University +1
U $^2$ Mamba: A Two-level Nested U-structure Mamba for Salient Object Detection
Jun 18, 2026Junhui Li, Jialu Li, Youshan ZhangMamba U-Net
School of Electronics and Information Engineering, University of Science and Technology Liaoning, China · School of Artificial Intelligence, Chuzhou University, Anhui, China · MS in Artificial Intelligence, Yeshiva University, New York, USA
Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models
Jun 12, 2026Rituraj Sharma, Tu VuRecurrent Neural Networks Recurrent Transformers
Virginia Tech
QDS-SNN: Energy-efficient Quantum Deeply-Supervised Spiking Neural Network Algorithm for Traffic Sign Recognition
Jun 3, 2026Zhiguo Qu, Keqi Li, Le Sun +4Energy-Efficient ML Autonomous Driving Perception
School of Computer Science, School of Software, Nanjing University of Information Science and Technology, Nanjing 210044, China · School of Information, Yunnan University of Finance and Economics, Kunming 650221, China · Qatar Center for Quantum Computing, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar +1
Decoding Matters: Efficient Mamba-Based Decoder with Distribution-Aware Deep Supervision for Medical Image Segmentation
Mar 13, 2026Fares Bougourzi, Fadi Dornaika, Abdenour HadidDeep Supervision Medical Image Segmentation
IEMN UMR CNRS 8520, Université Polytechnique Hauts-de-France, 59313 Valenciennes, France · University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain · Sorbonne Center for Artificial Intelligence, Sorbonne University Abu Dhabi, Abu Dhabi, UAE

Nov 3, 2026J/K move · Enter open · S save

Best-of-Depth: Learning Exploratory Dynamics in Looped Transformers

Shota Takashiro, Takeshi Kojima, Masanori Koyama, Kohei Hayashi+1

The University of Tokyo

Deep Supervision Recurrent Transformers

Open paper AskPDF arXiv

Looped Transformers extend inference-time computation through repeated latent-state updates, but final-depth training does not directly encourage useful intermediate alternatives. We introduce Best-of-Depth (BoD), which trains a recurrent trajectory as a candidate set by minimizing prediction loss across depths. Different valid targets can be matched at different depths without architectural changes or depth-specific labels. An exact-match analysis connects this objective to solution coverage. On N-Queens completion and graph coloring, BoD improves coverage over final-depth and intermediate-supervision objectives. N-Queens coverage reaches 91.03%, versus 45.75% for final-depth training. Further experiments show discovery beyond the training depth and recovery of solutions withheld from teacher pools on familiar inputs. Fine-tuning a pretrained looped language model with BoD yields higher pooled code-generation success than final-depth training when programs are generated separately at evaluated fixed depths. These results support training recurrent computation for solution discovery along its trajectory.