Classification

Follow this topic Ask about this topic

Momentum

45 papers in the last four weeks, up 137% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 254

All topics

Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces
Jun 5, 2026Debjyoti Saha RoyMulti-Label Classification CoT Distillation
Khoury College of Computer Sciences, Northeastern University
In-Context Multiple Instance Learning
Jun 4, 2026Alexander Möllers, Marvin Sextro, Julius Hense +2Synthetic Data Pretraining Few-Shot Learning
Berlin Institute for the Foundations of Learning and Data, Berlin, Germany · Machine Learning Group, Technische Universität Berlin, Berlin, Germany · Aignostics, Berlin, Germany +3
Long Live Fine-Tuning: Task-Specific Transformers Outperform Zero-Shot LLMs for Misinformation Response Classification on Reddit
Jun 2, 2026JooYoung Lee, Lin Tian, Angela Brillantes +2LLM Evaluation Zero-Shot Text Classification
University of Technology Sydney, Sydney, Australia.
MimeLens: Position-Agnostic Content-Type Detection for Binary Fragments
Jun 2, 2026Michael J. BommaritoDigital Forensics Classification
AutoTail-BSFGM: Class-Balance-Aware Fine-Tuning for Chinese Scholarly Text Classification
Jun 2, 2026Anling Xiang, Yuwen Yang, Yang ShenClass-Imbalanced Learning Long-Tailed Classification
Department of Intelligent Communication, School of Journalism and Communication, Minzu University of China, Beijing, China · ZeeLin (Beijing) Technology Co., Ltd., Beijing, China · School of Journalism and Communication, Tsinghua University, Beijing, China +1
ToolFG: Towards Well-Grounded Fine-Grained Image Classification
Jun 1, 2026Yu Xue, Haoxuan Qu, Zhuoling Li +4Tool Use in VLMs Multimodal Large Language Models
1Lancaster University · 2Peking University
Machine Learning for Coding Retail Product Names to Consumer-Price Categories: A Rule-plus-Bag-of-Words Pipeline with Reliability-Weighted Human-in-the-Loop Labeling
Jun 1, 2026Vladimir BeskorovainyiHuman-in-the-Loop Annotation Text Classification
Besk Tech · Moscow Institute of Physics and Technology (MIPT)
Linear Strategic Classification with Endogenous Improvements
May 31, 2026Siddharth Shrivastava, Mahvith Akshintala, B Vamsha Vardhan Reddy +3Strategic Classification Classification
Department of Artificial Intelligence IIT Hyderabad · IIIT Hyderabad · DepartementIITofHyderabadArtifical Intelligence
The role of class encoding in neural collapse
May 29, 2026Bastien Massion, Roy Makhlouf, Estelle MassartNeural Representation Geometry Multiclass Classification
ICTEAM, UCLouvain Louvain-la-Neuve, Belgium
FlagGAM: Rule-Basis Generalized Additive Models for Explainable Tabular Prediction
May 29, 2026Zijie Zhao, Roy E. WelschInterpretable ML Corruption Robustness
EECS Department, Massachusetts Institute of Technology, Cambridge, MA, USA · Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA
Improving Selective Classification with Pairwise Queries for Binary Classification
May 28, 2026Harsh Vardhan, Sunav Choudhary, Natwar Modani +1Pairwise Comparison Selective Prediction
CSE, UCSD, 9500 Gilman Drive, La Jolla, 92092, CA, USA. · Adobe Research, 345 Park Ave, San Jose, 95110, CA, USA. · Adobe Research, Marathahalli, Bengaluru, 560087, Karnataka, India. +1
Universal Multiclass Transductive Online Learning
May 28, 2026Steve Hanneke, Hongao WangMulticlass Classification Transductive Learning
Department of Computer Science, Purdue University, West Lafayette, IN 47907, USA
CalArena: A Large-Scale Post-Hoc Calibration Benchmark
May 28, 2026Eugène Berta, David Holzmüller, Francis Bach +1Post-Hoc Calibration Probability Calibration
Inria - Ecole Normale Supérieure PSL Research University · Inria
Code as a Weapon: A Consensus-Labeled Prompt Bank for Measuring Coding-Model Compliance with Malicious-Code Requests
May 27, 2026Richard J. Young, Gregory D. MoodyClassification LLM Safety Evaluation
University of Nevada Las Vegas · Department of Information Systems · Las Vegas, NV, USA
Where LLM Annotators Fail: Label-Free Learning on Graphs with LLMs
May 27, 2026Safal Thapaliya, Jiatan Huang, Chuxu ZhangLLM-Assisted Annotation Text-Attributed Graph Learning
University of Connecticut, USA
PAC Learning with Bandit Feedback: Sharp Sample Complexity in the Realizable Setting
May 25, 2026Steve Hanneke, Qinglin Meng, Shay Moran +1Multiclass Classification Classification
Department of Computer Science Purdue University West Lafayette, IN, USA · Faculties of Mathematics, Computer Science, and Data and Decision Sciences Technion – Israel Institute of Technology and Google Research Haifa, Israel
TypedCSIP: Typed Counterfactual Pretraining for Chinese Legislative Conflict Classification
May 25, 2026Yao LiuLegal NLP Classification
Chengdu University of Technology, Leshan, China
MedicalRec: Medical recommender system for image classification without retraining
May 23, 2026Roghayeh Taghavi, Aysa Hasanazde Bashkandi, Amir Ali Bengari +5Model Selection Healthcare
ChainzRule: Sample-Efficient, Robust Deep Learning Across Tabular, NLP, and Vision Tasks
May 23, 2026Rowan MartnishnPolynomial Neural Networks Neural Network Robustness
Sentivity AI
Improving Labeling Consistency with Detailed Constitutional Definitions and AI-Driven Evaluation
May 22, 2026Konstantin Berlin, Adam SwandaAnnotator Disagreement Language Model Safety Evaluation
Cisco AI Defense
A Fine-Tuned BERT Classifier for Personal-Letter Titles in Late-Ming and Early-Qing Collected Works
May 21, 2026Queenie LuoHistorical Document Analysis Computational Literary Studies
Harvard University
Smoothed Elicitation Complexity for Approximate $Γ$ -calibration of Discrete Classification Tasks
May 21, 2026Jessica Finocchiaro, Victor Ganson, Drona KhuranaModel Calibration Multiclass Classification
Computer Science, Boston College · Computer Science, University of Colorado Boulder
Ternary Decision Trees with Locally-Adaptive Uncertainty Zones
May 21, 2026William SmitsUncertainty Quantification Selective Prediction
Avathon, Austin, TX, USA
Is a Document Educational or Just Wikipedia-Style? -- Pitfalls of Classifier-Based Quality Filtering
May 21, 2026Mateusz Klimaszewski, Piotr AndruszkiewiczLanguage Model Pretraining Training Data Curation
Warsaw University of Technology · 2IDEAS Research Institute
Algebraic Machine Learning for Small-to-Medium Datasets Is Competitive against Strong Standard Baselines
May 21, 2026David Mendez, Fernando Martin-Maroto, Gonzalo G. de PolaviejaClassification Image Classification
Mathematics of Behavior and Intelligence Lab, Champalimaud Foundation, Lisbon
Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction
May 20, 2026Alexander SmirnovAI-Generated Text Detection Classification
University College London
Polynomial-Time Robust Multiclass Linear Classification under Gaussian Marginals
May 20, 2026Ilias Diakonikolas, Giannis Iakovidis, Mingchen MaMulticlass Classification Classification
University of Wisconsin-Madison
Holistic Reliability Propagation: Decoupling Annotation and Prediction for Robust Noisy-Label
May 20, 2026Jingyang Mao, Ningkang Peng, Yanhui GuClassification Learning with Noisy Labels
Nanjing Normal University Nanjing, China
Same Target, Different Basins: Hard vs. Soft Labels for Annotator Distributions
May 20, 2026Mirerfan Gheibi, Gashin GhazizadehAnnotator Disagreement Label Distribution Learning
Independent Researcher.
Efficient and Noise-Tolerant PAC Learning of Multiclass Linear Classifiers
May 18, 2026Rita Adhikari, Shiwei ZengMulticlass Classification Classification
Augusta University

Jun 5, 2026, cs.CLJ/K move · Enter open · S save

Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces

Debjyoti Saha Roy

Khoury College of Computer Sciences, Northeastern University

Multi-Label Classification CoT Distillation Mechanistic Interpretability Classification Knowledge Distillation

Open paper AskPDF arXiv

Reasoning-trained language models can perform, zero-shot, multi-label tasks that require selecting a small set of relevant labels from a universe of thousands to hundreds of thousands of candidates. We ask how they do it mechanistically, and whether the mechanism can be distilled. We make the question measurable by treating each decision as a token-level event scored by the model's own decision margin: the token that picks a coarse region of the label space, the tokens that pick a label within it, and the token where the output departs from a close alternative (a near-miss) named earlier in the reasoning. Attribution, exact mean-ablation, knock-in into another example's context, and a null calibration that discounts generic heads then give individual attention heads causal standing. On clinical coding of hospital discharge summaries (MIMIC-IV), with all 5,651 candidate diagnosis codes in context, a small, global, phase-structured set of heads is necessary and sufficient, by ablation and knock-in, on essentially every summary; distinct head families attend to the candidate region and back to the near-miss named earlier; and, for the mentions decided in the reasoning, the region can already be elicited several tokens before the code, from a disjoint mid-layer set that reads the input. We introduce MISTILL: unlike chain-of-thought distillation, which transfers only the teacher's reasoning text, it also supervises the student's pooled attention at exactly these decision events. Read on heads found after training, it nearly doubles the causal recovery of the contrastive decision in a cross-family student and adds a small, seed-stable gain in one that already carries most of it, with no detected task difference when both objectives train bf16 weights and a task cost with fp32 master weights.

Momentum

Related topics

Latest papers 254