Classification

Follow this topic Ask about this topic

Momentum

45 papers in the last four weeks, up 137% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 254

May 18, 2026cs.LG

Pocket Foundation Models: Distilling TFMs into CPU-Ready Gradient-Boosted Trees

A fraud scorer needs to answer in under 2 ms. The best tabular foundation models (TFMs) take 151-1,275 ms on GPU. We close this gap by distilling the TFM offline into an XGBoost or CatBoost student that runs natively on CPU. The central obstacle is specific to in-context learning (ICL) teachers: they leak labels when scoring their own training set, so the soft targets collapse to near-one-hot vectors with no inter-class structure left to distill. Stratified out-of-fold (OOF) teacher labeling prevents this. Across 153 classification datasets drawn from TALENT, OpenML-CC18, TabZilla, and TabArena, distilling TabICLv2 into XGBoost gives 0.882 macro-mean AUC (96.5% of teacher AUC) at 1.9 ms on CPU, a 38x to 860x speedup across teacher-student pairs with a statistically significant edge over a tuned CatBoost baseline (Wilcoxon p = 0.0008; 51% win rate). Four further findings: teacher rank transfers exactly to student rank; gains concentrate on low-dimensional data (< 21 features: +0.011 over CatBoost vs. >21 features: +0.001); multi-teacher averaging helps MLP students (+0.006, p = 0.003) but adds less than 0.001 for tree students; and on high-dimensional tasks where the teacher itself trails CatBoost, distillation makes things worse rather than better. The full pipeline is open-sourced as part of the TabTune library.

Aditya Tanna, Nassim Bouarour, Mohamed Bouadi +2

Lexsi Labs

Tabular Foundation Models Gradient-Boosted Decision Trees Cost-Aware Inference Classification Teacher-Student Learning Tabular ICL+1

May 18, 2026cs.LG

Graph Transductive Sharpening: Leveraging Unlabeled Predictions in Node Classification

In the transductive setting, where the full graph is observed but node labels are only partially available, progress in semi-supervised node classification has largely focused on architectural innovation. In this paper, we revisit an orthogonal axis: the training objective. We start from a simple observation: transductive models produce predictions for every node during training, including nodes without labels. These unlabeled-node predictions may contain useful training signal, but standard supervised objectives discard them because no ground-truth labels are available. Inspired by the decomposition of cross-entropy into a label-dependent alignment term and a label-independent entropy term, we propose prediction confidence as a natural way to extract this signal in the absence of labels. This motivates Transductive Sharpening (TS): a loss-level modification that minimizes prediction entropy on unlabeled nodes while counterbalancing this effect on labeled nodes. We evaluate Transductive Sharpening across a wide range of node-classification benchmarks and observe consistent performance improvements without requiring any changes to the backbone architecture. Code is available at https://github.com/transductive-sharpening/tunedGNN.

Brown Zaz, Mar Gonzàlez I Català, Ferran Hernandez Caralt +2

University of Cambridge

Graph Neural Networks Graph-Based Semi-Supervised Learning Transductive Learning Classification Semi-Supervised Learning Node Classification+1

May 18, 2026cs.CV

Evidence-Guided Unknown Rejection for High-Confidence Near-Known Unknowns

Open-set recognition systems face a neglected failure mode: high-confidence near-known unknowns, which lie outside the known label set but are close enough to known classes that a closed-set classifier accepts them with high confidence. We show that this failure is widespread across scalar-threshold methods, including recent post-hoc detectors, and that stronger encoders can amplify rather than remove the risk. We propose EGUR-A, which changes the decision from is this sample's score high enough?'' to does this predicted known class have sufficient evidence to accept this sample?'' EGUR-A combines class-conditional local acceptance evidence with global residual evidence, and selects their relative weight from known-sample statistics without unknown validation data. Across CUB, FGVC-Aircraft, and ImageNet-hard, EGUR-A substantially reduces high-confidence false known acceptance at matched known-rejection operating points. The result is not a stronger threshold; it is a different question: whether a known class is entitled to accept a sample.

Xi Chen, Yingjun Xiao, Gang Fang

School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou, China · School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou 510006, China · School of Artificial Intelligence, Guangzhou University, Guangzhou, China +3

Open-Set Recognition Classification OOD Detection

May 17, 2026cs.LG

The Neural Tangent Kernel for Classification

In wide neural networks, the Neural Tangent Kernel (NTK) remains approximately constant during training, providing a powerful theoretical tool for studying training dynamics, generalization, and connections to kernel methods. However, this theory is largely restricted to regression losses. It was previously thought that training on a classification loss, or more generally losses involving nonlinear output transformations, breaks this property, leading to divergent logits and a breakdown of the linearization. In this paper, we extend NTK theory to classification by identifying conditions under which wide neural networks remain in the lazy training regime. We show that parameter-space regularization ensures a constant NTK during training for cross-entropy loss, while in the absence of regularization the regime is recovered when targets are non-degenerate, i.e. when all classes have strictly positive probability. Under these conditions, training is well-approximated by the linearized model, yielding an explicit characterization of the solution in terms of the NTK. We further analyze the distribution of trained predictors induced by random initialization and relate this notion of model uncertainty to Bayesian methods.

Jonathan Plenk, Sergio Calvo-Ordonez, Alvaro Cartea +3

Mathematical Institute, University of Oxford · Oxford-Man Institute of Quantitative Finance, University of Oxford · OATML, University of Oxford +2

Classification Neural Network Training Dynamics Neural Tangent Kernel

May 15, 2026cs.CL

Few-Shot Large Language Models for Actionable Triage Categorization of Online Patient Inquiries

Online patient inquiries are often informal, incomplete, and written before professional assessment, yet they must still be routed to an appropriate level of clinical follow-up. We study this as a four-class actionable triage task -- self-care, schedule-visit, urgent-clinician-review, or emergency-referral, and ask whether prompted large language models (LLMs) can support such routing under low-resource labeling conditions. Using the public HealthCareMagic-100K corpus, we construct a 300-example human calibrated gold evaluation set, a 700-example auto-labeled silver training set, and a 40-example few-shot pool. We compare Term Frequency-Inverse Document Frequency (TF-IDF) and Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT) baselines train on silver labels against six prompted LLMs under 0-shot, 4-shot, and 12-shot conditions respectively. Accordingly, we evaluate with macro-

F_1

alongside safety-aware metrics, including emergency-recall, under-triage rate, and severe under-triage rate. The strongest LLM (Claude Haiku 4.5, 12-shot) reaches macro-

F_1

0.475, exceeding the best supervised baseline (BioBERT, 0.378) on point estimate, with overlapping confidence intervals. Few-shot prompting and two-model agreement help in label-dependent ways: self-care agreement is reliable, urgent-clinician-review is not. We conclude that LLMs can support triage prioritization and selective human review, but not autonomous deployment.

Liqi Zhou, Jiafu Li

Healthcare Clinical Triage Few-Shot Prompting Clinical Decision-Making Classification Clinical Language Model Evaluation+1

May 14, 2026cs.AI

A Deterministic Agentic Workflow for HS Tariff Classification: Multi-Dimensional Rule Reasoning with Interpretable Decisions

Harmonized System (HS) tariff classification is a high-stakes, expert-level task in which a free-form product description must be mapped to a specific six- or eight-digit code under the General Interpretive Rules (GIR), section notes, chapter notes, and Explanatory Notes. The difficulty lies not in knowledge volume but in multi-dimensional rule reasoning: a correct classification must satisfy competing priority rules along several axes simultaneously, including material, form, function, essential character, the part-versus-whole boundary, and specific listing versus residual headings. End-to-end prompting of large language models fails characteristically by resolving one axis while ignoring the priority constraints on the others. We present a deterministic agentic workflow in contrast to self-planning agents: the control flow is fixed, language model calls are confined to narrow stages, and reflection and verification are retained as local mechanisms. This design yields interpretability by construction--each decision is decomposed into stage-wise structured outputs with verbatim citation of the chapter or section notes that bear on it. The architecture combines offline knowledge-engineering of the Chinese HS tariff with an online six-stage pipeline. Evaluated on HSCodeComp at the six-digit level, the workflow reaches 75.0% top-1 and 91.5% top-3 at four digits, and 64.2% top-1 and 78.3% top-3 at six digits with Qwen3.6-plus; an open-weight Qwen3.6-27B-FP8 backbone in non-thinking mode achieves 84.2% four-digit and 77.4% six-digit top-1 agreement with the frontier model. A two-stage manual audit of 226 six-digit disagreements suggests that a non-trivial fraction of HSCodeComp ground-truth labels may deviate from HS general rules; full adjudication records are released in the appendix as preliminary findings for community review.

Yu Zhang, Dongjiang Zhuang, Qu Zhou +4

School of Information and Electronic Engineering, Shanghai Jiao Tong University, Shanghai, China · Department of Science and Technology (Shanghai), General Administration of Customs of the P.R.C. (GACC), Shanghai, China · Nanjing Jiyun Information Technology Co., Ltd., Nanjing, China +2

Agentic Workflows Classification LLM Reasoning

May 14, 2026cs.LG

Focused PU learning from imbalanced data

We propose a new method of learning from positive and unlabeled (PU) examples in highly imbalanced datasets. Many real-world problems, such as disease gene identification, targeted marketing, fraud detection, and recommender systems, are hard to address with machine learning methods, due to limited labeled data. Often, training data comprises positive and unlabeled instances, the latter typically being dominated by negative, but including also several positive instances. While PU learning is well-studied, few methods address imbalanced settings or hard-to-detect positive examples that resemble negative ones. Our approach uses a focused empirical risk estimator, incorporating both positive and unlabeled examples to train binary classifiers. Empirical evaluations demonstrate state-of-the-art performance on imbalanced datasets under two labeling mechanisms - selecting positives completely at random (SCAR) and selecting at random (SAR). Beyond these controlled experiments, we demonstrate the value of the proposed method in the real-world application of financial misstatement detection.

Elias Zavitsanos, Georgios Paliouras

Institute of Informatics and Telecommunications, NCSR “Demokritos”, Patriarhou Gregoriou E and 27 Neapoleos St., Agia Paraskevi, 15310, Attica, Greece.

Class-Imbalanced Learning Binary Classification Weakly Supervised Learning Classification Positive-Unlabeled Learning Empirical Risk Minimization

May 13, 2026cs.LG

Reducing cross-sample prediction churn in scientific machine learning

Scientific machine learning reports predictive performance. It does not report whether the same prediction would survive a different draw of training data. Across

9

chemistry benchmarks, two classifiers trained on independent bootstraps of the same training set agree on aggregate accuracy to within

1.3\text{--}4.2

percentage points but disagree on the class label of

8.0\text{--}21.8\%

of test molecules. We call this gap \emph{cross-sample prediction churn}. The standard parameter-side techniques (deep ensembles, MC dropout, stochastic weight averaging) do not reduce this gap; two data-side methods do. The first is

K

-bootstrap bagging, which cuts the rate

40\text{--}54\%

on every dataset at no accuracy cost (

K{\times}

-ERM compute). The second is \emph{twin-bootstrap}, our proposal: two networks trained jointly on independent bootstraps with a sym-KL consistency loss between their predictions, which at matched

2{\times}

-ERM compute reduces churn a further median

45\%

beyond bagging-

K{=}2

. Cross-sample prediction churn deserves a column alongside predictive performance in scientific-ML benchmark reports, because without it the parameter-side and data-side methods are indistinguishable on the metric they actually differ on.

Gordan Prastalo, Kevin Maik Jablonka

Helmholtz-Zentrum Berlin für Materialien und Energie GmbH, Hahn-Meitner-Platz 1, 14109 Berlin, Germany · HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena), Lessingstrasse 12–14, 077432026 Jena, Germany · Laboratory of Organic and Macromolecular Chemistry (IOMC), Friedrich Schiller University Jena, Humboldtstrasse 10, 07743 Jena, Germany +2

Ensemble Learning Classification Scientific ML Consistency Regularization

May 13, 2026cs.CV

Concepts Worth Having: Refining VLM-Guided Concept Bottleneck Models with Minimal Annotations

Concept-bottleneck models (CBMs) are neural classifiers that compute predictions from high-level concepts extracted from the input. CBMs ensure stakeholders can understand the concepts -- and the predictions they entail -- by learning these from concept-level annotations, which are however seldom available. Recent CBM architectures work around this issue by obtaining annotations from Vision-Language Models (VLMs). While greatly broadening applicability, doing so can yield lower quality concepts and therefore less interpretable models. We strike for a middle ground by introducing Vision-plus-Human-guided CBM (VH-CBM), a hybrid approach that exploits both VLMs and a small amount of dense annotations. VH-CBM employs a Gaussian Process in the VLM's embedding space, which captures useful global information about the target domain, to propagate the expert's supervision to any target data point. Our empirical evaluation shows how VH-CBM predicts more accurate concepts than VLM-guided CBMs even when annotating as little as 1% of the data, while sporting better concept calibration and supporting active learning.

Nicola Debole, Andrea Passerini, Stefano Teso +2

DISI, University of Trento, Italy · CIMeC, University of Trento, Italy

Concept Bottleneck Models Human-in-the-Loop Annotation Explainable Image Classification Classification

May 12, 2026cs.AI

Classifier Context Rot: Monitor Performance Degrades with Context Length

Monitoring coding agents for dangerous behavior using language models requires classifying transcripts that often exceed 500K tokens, but prior agent monitoring benchmarks rarely contain transcripts longer than 100K tokens. We show that when used as classifiers, current frontier models fail to notice dangerous actions more often in longer transcripts. In particular, on a dataset that requires identifying when a coding agent takes a subtly dangerous action, Opus 4.6, GPT 5.4, and Gemini 3.1 miss these actions

2\times

to

30\times

more often when they occur after 800K tokens of benign activity than when they occur on their own. We also show that these weaknesses can be partially mitigated with prompting techniques such as periodic reminders throughout the transcript and may be mitigated further with better post-training. Monitor evaluations that do not consider long-context degradation are likely overestimating monitor performance.

Sam Martin, Fabien Roger

Anthropic Fellows Program · Anthropic

Language Model Safety Evaluation Long-Context Language Model Evaluation AI Agent Monitoring Long-Context Modeling LLM Reliability Classification

May 12, 2026cs.AI

How Useful Is Cross-Domain Generalization for Training LLM Monitors?

Using prompted language models as classifiers enables classification in domains with limited training data, but misses some of the robustness and performance benefits that fine-tuning can bring. We study whether training on multiple classification tasks, each with its own prompt, improves performance on new domains with new classification prompts. We show that such training partially generalizes to adjacent domains, improving classification performance on tasks that are unseen during training. However, we identify specific edge cases where the fine-tuned models fail to follow prompts, such as when the classification prompt changes completely while the data domain remains the same as during training. We show that classification training can be mixed with general instruction following training, and that (when done well) such training keeps the benefits of classification training and mitigates its generalization failures. Surprisingly, we see that this no-thinking supervised classification training can generalize to with-thinking classification and summarization, suggesting that no-thinking classification training might be instrumentally useful in building other kinds of classifiers and monitoring systems.

Sam Martin, Fabien Roger

Anthropic Fellows Program · Anthropic

Supervised Fine-Tuning Cross-Domain Generalization Classification Instruction Tuning Language Model Robustness

May 12, 2026cs.CL

Choosing features for classifying multiword expressions

Multiword expressions (MWEs) are a heterogeneous set with a glaring need for classifications. Designing a satisfactory classification involves choosing features. In the case of MWEs, many features are a priori available. Not all features are equal in terms of how reliably MWEs can be assigned to classes. Accordingly, resulting classifications may be more or less fruitful for computational use. I outline an enhanced classification. In order to increase its suitability for many languages, I use previous works taking into account various languages.

Eric Laporte

Université Paris-Est, Laboratoire d’informatique Gaspard-Monge CNRS UMR 8049, F77454 Marne-la-Vallée, France

Text Classification Classification Feature Selection

May 12, 2026cs.LG

MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound

Streaming decision trees are natural candidates for open-world continual learning, as they perform local updates, enjoy bounded memory, and static decision boundaries. Despite these, they still fail in online class-incremental learning due to two coupled miscalibrations: (i) their split criterion grows unreliable as the class count K expands, and (ii) the absence of knowledge transfer at split time. Both failures share a common root: the range of Information Gain intrinsically scales with log2 K. Consequently, any Hoeffding-style confidence radius derived from it must inevitably grow with the class count, making a K-independent split criterion structurally impossible, taking away the potential benefits of applying streaming decision trees to continual learning. To fix this issue, we present MIST (McDiarmid Incremental Streaming Tree), which resolves both failures through three integrated components: (i) a tight, K-independent McDiarmid confidence radius for Gini splitting that acts as a structural regulariser; (ii) a Bayesian inheritance protocol that projects parent statistics to child nodes via truncated-Gaussian moments, with variance reduction guarantees strongest precisely when splitting is most conservative; and (iii) per-leaf KLL quantile sketches that support both continuous threshold evaluation and geometry-adaptive leaf prediction from a single data structure. On standard and stress-test tabular streams, MIST is competitive with global parametric methods on near-Gaussian benchmarks and uniquely robust on non-Gaussian geometry where SOTA benchmarks collapse.

Phu-Hoa Pham, Chi-Nguyen Tran, Nguyen Lam Phu Quy +3

Faculty of Information and Technology University of Science Vietnam National University Ho Chi Minh City, Vietnam · Department of Computer Science University of Warwick Coventry, United Kingdom

Class-Incremental Learning Continual Learning Decision Tree Learning Classification Online Continual Learning Incremental Learning+1

May 11, 2026cs.LG

DANCE: Detect and Classify Events in EEG

Event identification in continuous neural recordings is a critical task in neuroscience. Decoding in EEG is dominated by classifying windows aligned to known event onsets. However, while available in controlled experiments, such onsets are absent in continuous real-world monitoring. Here, we introduce DANCE, a deep learning pipeline that frames neural decoding as a set-prediction problem and jointly detects and classifies events directly from raw, unaligned signals. Evaluated separately on ten datasets curated from the literature with a wide variety of event types (ranging from milliseconds to minutes in duration), our model outperforms existing methods on a broad range of cognitive, clinical and BCI tasks. This single architecture establishes a new state of the art in the competitive task of seizure monitoring and matches the accuracy of onset-informed models for BCI tasks. Overall, our method marks a step towards end-to-end asynchronous neural decoding models

Jarod Lévy, Hubert Banville, Jérémy Rapin +3

1Meta AI · 2Inria, Université Paris-Saclay, Palaiseau, France

Electroencephalography Brain-Computer Interfaces EEG Decoding Set Prediction Classification Structured Prediction

May 11, 2026cs.AI

Deep Arguing

Deep learning has become the dominant approach for creating high capacity, scalable models across diverse data modalities. However, because these models rely on a large number of learned parameters, tightly couple feature extraction with task objectives, and often lack explicit reasoning mechanisms, it is difficult for humans to understand how they arrive at their predictions. Understanding what representations emerge and why they arise from the training data remains an open challenge. We introduce Deep Arguing, a novel neurosymbolic approach that integrates deep learning with argumentation construction and reasoning for interpretable classification with different data modalities. In our approach deep neural networks construct an argumentation structure wherein data points support their assigned label and attack different ones. Using differentiable argumentation semantics for reasoning, the model is trained end-to-end to jointly learn feature representation and argumentative interactions. This results in argumentation structures providing faithful case-based explanations for predictions. Structure constraints over the argumentation graph guide learning, improving both interpretability and predictive performance. Experiments with tabular and imaging datasets show that Deep Arguing achieves performance competitive with standard baselines whilst offering interpretable argumentative reasoning.

Adam Gould, Francesca Toni

Department of Computing Imperial College London

Neural Network Interpretability Computational Argumentation Classification Neuro-Symbolic Reasoning

May 11, 2026cs.CL

Multi-domain Multi-modal Document Classification Benchmark with a Multi-level Taxonomy

Document classification forms the backbone of modern enterprise content management, yet existing benchmarks remain trapped in oversimplified paradigms -- single domain settings with flat label structures -- that bear little resemblance to the hierarchical, multi-modal, and cross-domain nature of real-world business documents. This gap not only misrepresents practical complexity but also stifles progress toward industrially viable document intelligence. To bridge this gap, we construct the first Multi-level, Multi-domain, Multi-modal document classification Benchmark (MMM-Bench). MMM-Bench includes (1) a deeply hierarchical taxonomy spanning five levels that capture the authentic organizational logic of business documentation; and (2) 5,990 real-world multi-modal documents meticulously curated from 12 commercial domains in Alibaba. Each document is manually annotated with a complete hierarchical path by domain experts. We establish comprehensive baselines on MMM-Bench, which consists of open-weight models and API-based models. Through systematic experiments, we identify four fundamental challenges within MMM-Bench and propose corresponding insights. To provide a solid foundation for advancing research in multi-level, multi-domain document classification, we release all of the data and the evaluation toolkit of MMM-Bench at https://github.com/MMMDC-Bench/MMMDC-Bench.

Denghao Ma, Qing Liu, Zulong Chen +5

Beijing Information Science and Technology University, China · Alibaba Group, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), China +2

Hierarchical Classification Text Classification Multimodal Classification Classification

May 10, 2026cs.AI

Empowering VLMs for Few-Shot Multimodal Time Series Classification via Tailored Agentic Reasoning

In this paper, we propose the first VL

\underline{\textbf{M}}

\underline{\textbf{a}}

gentic

\underline{\textbf{r}}

easoning framework for few-

\underline{\textbf{s}}

hot multimodal

\underline{\textbf{T}}

ime

\underline{\textbf{S}}

eries

\underline{\textbf{C}}

lassification (

\textbf{MarsTSC}

), which introduces a self-evolving knowledge bank as a dynamic context iteratively refined via reflective agentic reasoning. The framework comprises three collaborative roles: i) Generator conducts reliable classification via reasoning; ii) Reflector diagnoses the root causes of reasoning errors to yield discriminative insights targeting the temporal features overlooked by Generator; iii) Modifier applies verified updates to the knowledge bank to prevent context collapse. We further introduce a test-time update strategy to enable cautious, continuous knowledge bank refinement to mitigate few-shot bias and distribution shift. Extensive experiments across 12 mainstream time series benchmarks demonstrate that

\textbf{MarsTSC}

delivers substantial and consistent performance gains across 6 VLM backbones, outperforming both classical and foundation model-based time series baselines under few-shot conditions, while producing interpretable rationales that ground each classification decision in human-readable feature evidence.

Lin Li, Jiawei Huang, Qihao Quan +7

Sun Yat-sen University Zhuhai, Guangdong, China · Xiaomi Corporation Beijing, China · University of Science and Technology of China Hefei, Anhui, China +1

Multivariate Time Series Classification Time Series Classification Few-Shot Learning Multimodal Reasoning VLM Reasoning Agentic Reasoning+1

May 9, 2026cs.LG

Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation

Class-level machine unlearning aims to remove the influence of specified classes while preserving model utility on retained classes. Existing methods are commonly evaluated by retain-set accuracy, forget-set accuracy, and unlearning time, but these metrics provide limited insight into how forgetting is achieved internally. In this paper, we reveal a bias-dominated shortcut in class-level unlearning: the prediction of forgotten classes can be suppressed by decreasing the corresponding bias terms in the final classification head. We first analyze the gradient dynamics of classification-head biases under softmax cross-entropy training, explaining why retain-set-only optimization tends to reduce the biases of absent classes. Based on this observation, we introduce BiasShift as a diagnostic baseline, showing that simple bias manipulation can satisfy conventional unlearning metrics while leaving abnormal bias patterns that reveal forgotten labels. To mitigate excessive forgotten-class bias suppression, we propose two bias-aware mechanisms, namely Two-Stage Bias Gradient Reversal Mechanism (TS-BGRM) and Lower-Bound Hinge Regularization (LB-HR). We further introduce three bias-oriented metrics, including Bias Stability Coefficient (BSC), Median Bias Gap (MBG), and Minimal Bias Score (MBS), to quantify bias dependence and potential leakage. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that the proposed methods maintain competitive unlearning performance while producing more stable bias distributions. We have released our code at {https://github.com/zwd2024/Beyond-the-Shadow-of-Bias-From-Classification-Head-Bias-to-Parameter-Redistribution}.

Weidong Zheng, Kongyang Chen, Yuanwei Guo +1

School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou 510335, China · School of Artificial Intelligence, Guangzhou University, Guangzhou 510006, China · Pazhou Lab, Guangzhou 510006, China +1

Classification Machine Unlearning Machine Unlearning Evaluation

May 8, 2026cs.LG

Black-box model classification under the discriminative factorization

Access to modern generative systems is often restricted to querying an API (the ``black-box" setting) and many properties of the system are unknown to the user at inference time. While recent work has shown that low-dimensional representations of models based on the relationship between their embedded responses to a set of queries are useful for inferring model-level properties, the quality of these representations is highly sensitive to the query set. We introduce the \emph{discriminative factorization} to distinguish between high- and low-quality query sets in the context of black-box model-level classification. Under this framework, the probability of chance-level classification decays exponentially in the query budget. On three auditing tasks, estimated factorization parameters predict the empirical performance decay rate. We conclude by showing that query sets selected using the estimated discriminative field reproduce the empirical ordering of oracle query sets.

Hayden Helm, Merrick Ohata, Carey Priebe

Helivan · JHU

Classification Black-Box Model Auditing

May 8, 2026cs.LG

Flexible Routing via Uncertainty Decomposition

A key strategy for balancing performance and cost in modern machine learning systems is to dynamically route queries to either a low-cost model or a more expensive oracle (such as a large pretrained model or human expert), an approach known as model routing. In this work we present a new uncertainty-aware router that (1) avoids unnecessary oracle calls on inherently ambiguous queries, and (2) adapts dynamically to different loss functions and cost parameters through simple hyperparameter changes, without retraining. Our method, applicable to any classification setting where multiple independent annotations per input are available, is based on decomposing total uncertainty into irreducible and reducible components using higher-order predictors [Ahdritz et al., 2025]. This enables a unified approach to both routing and abstention: predict with the weak model when uncertainty is low, route to the oracle when reducible uncertainty is high, and abstain when irreducible uncertainty is high. Our router comes with strong theoretical guarantees bounding regret relative to optimal task-specific routers. We conduct experiments on both synthetic and real-world datasets that demonstrate the benefits of our approach in suitable regimes -- in particular, whenever reducible and irreducible uncertainty are not too correlated.

Charlotte Peale, Siddartha Devic, Parikshit Gopalan +2

1Stanford University · University of Southern California · 3Apple

Uncertainty Quantification Adaptive Model Routing Selective Prediction Cost-Aware Inference Classification Uncertainty Decomposition

May 8, 2026cs.LG

In-Context Fixation: When Demonstrated Labels Override Semantics in Few-Shot Classification

While random demonstration labels barely hurt in-context learning (Min et al., 2022), we show that homogeneous labels--even semantically valid ones--collapse accuracy to <=12% across six models (Pythia, Llama, Qwen; 0.8B--8B) and four tasks. The trigger is label-slot content: the model treats tokens occupying the label position as an exhaustive answer vocabulary, with homogeneity as the maximally collapsed case. A novel set-level fixation finding confirms this: when demonstrations carry varied nonsense tokens from {foo,bar,vex,nit,orb}, the model places 42--67% of probability on the demonstrated set while P(dog) remains below 0.2%. This is inconsistent with latent-concept Bayesian accounts (Xie et al., 2022) and reveals that ICL output is constrained vocabulary retrieval--the model binds its output to the demonstrated token inventory regardless of semantic plausibility. The effect generalizes to 4-way classification (0% accuracy across three models, 1B--8B) and multi-token verbalizers ("very positive"), where we decompose fixation into format-level (template adoption) and content-level (polarity override) components that are experimentally dissociable. Mechanistically, per-item paired activation patching on Pythia-1B recovers 98.4% of the gap (95% CI [84%, 112%]), localizing fixation to a layer-7-centered circuit (rank 2/560, 99.8th percentile; 4-fold CV mean 103%). Cross-architecture logit lens on Llama-3.2-1B replicates the encode-then-override trajectory with causal confirmation (top-5 layers: 89% recovery).

Ming Liu

Amazon

LLM Interpretability Few-Shot Learning In-Context Learning Classification

May 8, 2026cs.LG

Risk-Consistent Multiclass Learning from Random Label-Subset Membership Queries

Obtaining accurate class labels is often costly or unreliable, and may also be limited by privacy or other practical conditions. Compared with asking an annotator to provide the exact class, it is often easier to ask whether the true label belongs to a certain label subset. This query-response form defines a distinct weak-supervision mechanism: weak supervision information is generated through feedback on a label subset. Although weakly supervised learning has studied many learning frameworks, most existing work starts from established weak label objects. A systematic characterization is still lacking for weakly supervised learning generated directly by such query response observations. This paper proposes a multiclass learn ing framework under random label-subset queries. We model the data-generating distribution of query-response observations and derive an unbiased estimator of the target risk under the empirical risk minimization (ERM) framework. To address negative empirical risk and the associated overfitting problem, we introduce corrected risk estimators based on non-negative and absolute-value corrections. Theoretical analysis establishes a conditional generalization and excess-risk bound for the unbiased estimator, and a bias-and-consistency result for the corrected risk estimator. Experiments under the matched random-query mechanism demonstrate the feasibility of direct query-response learning and the stabilization effect of risk correction.

Jiaxu Su, Junpeng Li, Changchun Hua +1

Engineering Research Center of the Ministry of Education for Intelligent Control System and Intelligent Equipment, Yanshan University, Qinhuangdao, China

Multiclass Classification Weakly Supervised Learning Classification Statistical Learning Theory Empirical Risk Minimization

May 8, 2026cs.LG

When Symbol Names Should Not Matter: A Logistic Theory of Fresh-Symbol Classification

Template tasks have emerged as a clean testbed for asking whether transformers reason with abstract symbols rather than concrete token names. We study the fixed-label classification version of this problem, where train and test examples share latent templates but may use disjoint vocabularies. Unlike next-token prediction, the model need not emit unseen symbols; it must learn a decision rule invariant to symbol renaming. We analyze regularized kernel logistic classification in the transformer-kernel regime. Our main result decomposes the learned predictor into an ideal template-level classifier and a finite-sample perturbation caused by accidental token overlaps in the training data. We encode these overlaps by a colored collision graph and prove high-probability margin-transfer guarantees for fresh-symbol classification. This perspective extends template-based analyses to logistic classification and refines scalar diversity conditions: vocabulary size controls the average rate of collisions, but collision geometry controls whether the ideal classification margin is preserved. More broadly, the same perturbation framework applies to abstraction-augmented inputs, yielding a general margin-versus-collision criterion for identifying when prompting strategies improve fresh-symbol generalization. Synthetic template experiments illustrate the predicted roles of regularization, sample size, and transformer-kernel structure.

Wenjie Guan, Jelena Bradic

Department of Statistics and Data Science, Cornell University

Transformer Classification Kernel Methods Invariant Representation Learning Compositional Generalization

May 7, 2026cs.CV

Empirical Evidence for Simply Connected Decision Regions in Image Classifiers

The topology of a classifier's decision regions determines how inputs with the same predicted label can be connected and deformed without changing that prediction. Prior empirical work constructed paths between same-label images within a single region, but did not examine whether loops bound surfaces within that region. We investigate this question using adaptive quadrilateral meshes with targeted repair of off-label interior vertices, while holding the same-label boundary loop fixed. A finite-resolution acceptance criterion distinguishes completed constructions from those left unresolved at the refinement ceiling. Across the pretrained classifiers studied, every tested loop admits an accepted filling. Construction effort varies by orders of magnitude within classes and is greater for mean-score-adjusted randomly initialised classifiers than for trained classifiers. An analytic control with a known hole leaves winding loops unresolved at the tested hole radii at or above the resolution threshold. These results provide empirical evidence consistent with simply connected decision regions at the tested resolution.

Arjhun Swaminathan, Mete Akgün

Medical Data Privacy and Privacy-preserving Machine Learning (MDPPML), University of Tübingen. · Institute for Bioinformatics and Medical Informatics (IBMI), University of Tübingen.

Classification Image Classification

May 7, 2026cs.LG

When Labels Have Structure: Improving Image Classification with Hierarchy-Aware Cross-Entropy

Standard cross-entropy is the default classification loss across virtually all of machine learning, yet it treats all misclassifications equally, ignoring the semantic distances that a class hierarchy encodes. We propose Hierarchy-Aware Cross-Entropy (HACE), a drop-in replacement for standard cross-entropy that incorporates a known class hierarchy directly into the loss. HACE combines two components: prediction aggregation, which propagates the model's probability mass upward through the class hierarchy to ensure that parent nodes accumulate the confidence of their children; and ancestral label smoothing, which distributes the ground-truth signal along the path from the true class to the root. We evaluate HACE on CIFAR-100, FGVC Aircraft, and NABirds in two regimes: end-to-end training across six architectures spanning convolutional and attention-based designs, and linear probing on frozen DINOv2-Large features. In end-to-end training, HACE improves accuracy over standard cross-entropy in 15 out of 18 architecture--dataset pairs, with a mean gain of 4.66%. In linear probing on frozen DINOv2-Large features, HACE outperforms all competing methods on all three datasets, with a mean improvement of 2.18% over the next best baseline.

April Chan, Davide D'Ascenzo, Sebastiano Cultrera di Montesano

Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Computer Science, University of Milan, Milan, Italy · Department of Control and Computer Engineering, Politecnico di Torino, Torino, Italy +1

Hierarchical Classification Fine-Grained Image Classification Classification Image Classification

May 5, 2026cs.LG

A Closed-Form Adaptive-Landmark Kernel for Certified Point-Cloud and Graph Classification

We introduce PALACE (Persistence Adaptive-Landmark Analytic Classification Engine), the data-adaptive companion to PLACE, paying a small cross-validation tier on three knobs (budget, radii, bandwidth;

\leq 5

choices each). A cover-theoretic core (Lebesgue-number criterion on the landmark cover) yields four closed-form guarantees. (i) A structural lower distortion bound

λ(τ;ν)

on

\mathcal{D}_n

under cross-diagram non-interference, with a

(D/L)^2

budget reduction over the uniform grid when diagrams concentrate. (ii) Equal weights

w_k = K^{-1/2}

maximizing

λ

, and farthest-point-sampling positions

2

-approximating the optimal

k

-center covering radius; both derived from training labels alone, no gradient training. (iii) A kernel-RKHS classification rate

O((k-1)\sqrt{K}/(γ\sqrt{m_{\min}}))

with binary necessity threshold

m = Ω(\sqrt K/γ)

from a matching Le Cam lower bound, and a closed-form filtration-selection rule. The kernel-Mahalanobis margin

\hatρ_{\mathrm{Mah}}

is the strongest closed-form ranker across the chemical-graph pool (mean Spearman

ρ\approx +0.60

); the isotropic surrogate

\hatγ/\sqrt{K}

admits a selection-consistency rate, and

\widehatλ

from (i) provides an independent data-level signal (positive on COX2 and PTC). (iv) A per-prediction certificate, in non-asymptotic Pinelis and asymptotic Gaussian forms, with no calibration split. Empirically, PALACE is the strongest closed-form diagram-based method on Orbit5k (

91.3 \pm 1.0\%

, matching Persformer), leads every diagram-based competitor on COX2 and MUTAG, and is competitive on DHFR (within 1 pp of ECP). At

8\times

domain inflation, adaptive placement maintains

94\%

while the uniform grid collapses to chance (

25\%

on 4-class data).

Sushovan Majhi, Atish Mitra, Žiga Virk +1

Data Science, George Washington University, USA · Department of Mathematical Sciences, Montana Technological University, USA · Faculty of Computer and Information Science, University of Ljubljana, Slovenia +1

Point Cloud Classification Classification Kernel Methods Graph Classification Persistent Homology Statistical Learning Theory

May 5, 2026stat.ML

The Manokhin Probability Matrix: A Diagnostic Framework for Classifier Probability Quality

The Brier score conflates two distinct properties of probabilistic predictions: reliability (calibration error) and resolution (discriminatory power). We introduce the Manokhin Probability Matrix, a BCG-style two-dimensional diagnostic framework that separates them. Classifiers are placed on a 2x2 grid by Spiegelhalter Z-statistic and AUC-ROC expected rank, then assigned to one of four archetypes: Eagle (good on both axes), Bull (strong discrimination, poor calibration), Sloth (well-calibrated, weak discriminator), and Mole (poor on both). Each archetype carries a distinct prescription. We populate the matrix from a large-scale empirical study spanning 21 classifiers, 5 post-hoc calibrators, and 30 real-world binary classification tasks from the TabArena-v0.1 suite. The assignment is unambiguous. CatBoost, TabICL, EBM, TabPFN, GBC, and Random Forest are Eagles. XGBoost, LightGBM, and HGB are Bulls; Venn-Abers calibration cuts log-loss by 6.5 to 12.6% on Bulls but degrades Eagles by 2.1%. SVM, LR, LDA, and the empirical base-rate predictor are Sloths. MLP, KNN, Naive Bayes, and ExtraTrees are Moles. A theoretical asymmetry follows: no order-preserving post-hoc calibrator can add discriminatory power (Proposition 1), so calibration is the fixable part and discrimination is the hard part. The practical rule is direct: do not optimise aggregate Brier score without first decomposing it; optimise discrimination first, then fix calibration post-hoc. Code and raw experimental data are available at https://github.com/valeman/classifier_calibration.

Valery Manokhin

Independent Researcher

Post-Hoc Calibration Probability Calibration Model Calibration Binary Classification Classification

May 5, 2026stat.ML

Imbalanced Classification under Capacity Constraints

Detecting observations from a minority class under severe class imbalance is a central challenge in applications such as fraud detection, medical screening, and industrial quality control. In these settings, each positive prediction triggers a costly follow-up action, an MRI scan, a transaction audit, whose execution is subject to real operational constraints. This paper proposes a formal classification framework under capacity constraints: given a user-defined bound limit

b

on the proportion of observations that can be labeled as belonging to the minority class, the goal is to find the classifier that maximizes sensitivity on that class. We characterize the optimal classifier under this constraint and establish its equivalence with the classical Bayes classifier under a reweighting of the prior probabilities. We also introduce a capacity-adjusted performance metric

M

that accounts for the effective detection rate when the capacity constraint is binding. The framework is implemented on top of standard learning methods, k-NN, SVM, random forests, and neural networks, and statistical consistency is established for each. We further show that these methods reduce to post-hoc thresholding when no hyperparameters are oriented toward the capacity-constrained objective, and introduce a capacity-aware support vector machine that exploits the constraint during training and achieves the strongest empirical performance. Experiments on the Taiwanese credit card default dataset confirm that capacity-constrained classifiers substantially outperform both classical approaches and SMOTE under high imbalance regimes. The framework extends naturally to multiclass settings and online environments.

Daniel Fraiman, Ricardo Fraiman

Departamento de Matem´atica y Ciencias, Universidad de San Andr´es, Buenos Aires, Argentina · CONICET, Argentina. · PEDECIBA, Matem´atica, Uruguay.

Class-Imbalanced Learning Binary Classification Classification

May 4, 2026cs.LG

Instance-Level Costs for Nuanced Classifier Evaluation

Standard classification treats all errors equally, but in applications such as content moderation and medical screening, mistakes on clear-cut cases are more costly than errors on ambiguous ones. From a contextual bandit framework, we propose normalized excess cost (NEC), a metric that weighs classification errors by per-example costs and reduces to standard error rate when costs are uniform. Costs can derive from annotator vote margins, distance from decision thresholds, or confidence ratings. Across text, image, and tabular benchmarks, we find that NEC is often substantially lower than error rate: models with 5% error rate can achieve 1.8% NEC, revealing that most mistakes concentrate on ambiguous, low-cost examples. We also find that incorporating costs into training via loss weighting, sampling strategies, or regression yields inconsistent benefits. Our framework provides a practical methodology for deriving and evaluating instance-level misclassification costs, even if cost-sensitive training offers limited benefit.

Kabir Kang, Stephen Mussmann

School of Computer Science, Georgia Institute of Technology, Atlanta, GA, USA.

Cost-Sensitive Learning Classification

May 4, 2026cs.LG

A Closed-Form Persistence-Landmark Pipeline for Certified Point-Cloud and Graph Classification

We introduce PLACE (Persistence-Landmark Analytic Classification Engine), a closed-form pipeline for classifying point clouds and graphs through their persistent-homology signatures. Three quantitative guarantees -- a margin-based excess-risk rate, a closed-form descriptor-selection rule, and a per-prediction certificate -- are derived from training labels alone, with no learned weights or held-out calibration. The embedding sums Mitra-Virk single-point coordinate functions over a sparse landmark grid; the closed-form weight rule

w_k^2 \propto (d_{k+1}^2 - d_k^2)/R_k^2

maximizes the distortion slope in Mitra-Virk's affine certificate under

ν

-coherence. (i) An

O(kR/(Δ\sqrt{m_{\min}}))

margin bound, driven by class-mean separation

Δ

and embedding radius

R

, matched in the sample-starved regime

m \lesssim R/Δ

by a Le Cam minimax lower bound. (ii) The Mahalanobis margin under Ledoit-Wolf-shrunk covariance is the strongest closed-form ranker on a 64-descriptor chemical-graph pool (mean Spearman

ρ= +0.56

across 11 benchmarks, positive on 10 of 11); the isotropic surrogate

Δ/\sqrt{\ell}

admits a closed-form selection-consistency rate on the homogeneous protein/social pools. (iii) A training-time-decided certificate, with no per-prediction overhead, in three concrete radii (Pinelis, Gaussian plug-in, and variance-aware Pinelis-Bernstein). Empirically, PLACE is the strongest diagram-based method on Orbit5k and matches the strongest topology-based baseline within statistical noise on MUTAG and COX2; remaining gaps fall into two diagnosable regimes (descriptor blindness on NCI1/NCI109; pool-coverage limits elsewhere). The Pinelis-Bernstein radius fires on 8 of the 12 benchmarks; on MUTAG the empirical and population nearest-centroid rules agree on every one of 940 held-out test predictions, validating the certificate's mechanism.

Sushovan Majhi, Atish Mitra, Žiga Virk +1

Data Science, George Washington University, USA · Department of Mathematical Sciences, Montana Technological University, USA · Faculty of Computer and Information Science, University of Ljubljana, Institute IMFM, Slovenia +1

Point Cloud Classification Classification Graph Classification Persistent Homology Statistical Learning Theory