Self-Training

Follow this topic Ask about this topic

Self-training studies how models can learn from unlabeled examples or their own outputs, most commonly by generating pseudo-labels and training on selected predictions. Teacher–student and Mean Teacher methods, confidence filtering, consistency losses, and class balancing are common approaches, especially in semi-supervised learning and domain adaptation. Newer methods also use model-generated reasoning, rewards, or task-specific verifiers to improve vision, language, audio, and 3D systems when human annotations are scarce. A central challenge is that incorrect pseudo-labels and unreliable self-checks can reinforce errors, so reliable filtering and independent verification remain important.

Latest papers

43 papers6 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Statistical Analysis of Synthetic Data with Verification: A Linear Regression Framework
Oct 29, 2026Heejune Sheen, Harrison H. ZhouSynthetic Data Selection Self-Training
Department of Statistics and Data Science Yale University New Haven, CT 06511 USA
AC-JEPA: Anchored Cross-Modal Joint Embedding Predictive Architecture for Visual-to-Symbolic Translation
Oct 27, 2026Chaofei Hong, Ziyang Chen, Baochen Wang +2Joint-Embedding Predictive Architecture Cross-Modal Representation Learning
Zhejiang Lab
FedBST: Bilateral Self-Training for Unsupervised Federated Classification
Oct 22, 2026Aijing Yu, Kuangpu Guo, Jian Liang +1Federated Learning Self-Training
School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences, Beijing, China +2
Distillation, Self-Training, and the Fragility of Learned Chemical Reasoning
Oct 16, 2026Anagha Aneesh, Sebastian Ibarraran, Frank Hu +2Chemical Reasoning in LLMs Chemical Reaction Prediction
Stanford University
Decoupled Optimization for Teacher-Student Semi-Supervised Learning via a Pioneer Student
Oct 7, 2026Haorong Han, Jidong Yuan, Chixuan Wei +1Semi-Supervised Learning Teacher-Student Learning
School of Computer Science and Technology Beijing Jiaotong University · School of Data Science and Intelligent Media Communication University of China
RBMatch: Dual-Level Class Rebalancing for Semi-Supervised Building Footprint Extraction
Oct 6, 2026Akil Ahmad Taki, Shaikh Anowarul FattahClass-Imbalanced Learning Remote Sensing Image Segmentation
Department of EEE, Bangladesh University of Engineering and Technology(BUET) · Department of EEE, University of Asia Pacific
Robust Surgical Robotic Instrument Tracking via Sequential Multi-Cue Fusion and Sim-to-Real Self-Training
Oct 4, 2026Hanyang Hu, Zekai Liang, Florian Richter +1Self-Training Sim-to-Real Transfer
Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093 USA.
Improving Generative Model Self-Training with Geometrically Modified Outputs
Sep 28, 2026Patrick Batsell, Thomas Walker, Richard BaraniukSelf-Training
Department of Electrical and Computer Engineering Rice University Houston, TX 77005, USA
Re:Cognize -- A Framework For Open-Set Comic Character Re-Identification
Sep 27, 2026Aaditya Baranwal, Madhav Kataria, Yogesh S Rawat +1Self-Training Open-Set Recognition
Institute of Artificial Intelligence, University of Central Florida, Orlando, FL, USA
OREO: Fidelity Alignment in 3D Generation via On-the-fly Rendering-Editing Optimization
Sep 24, 2026Zhiyuan Ma, Wenbo Hu, Wang Zhao +33D Asset Generation Self-Training
The Hong Kong Polytechnic University · Tencent ARC Lab
MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training
Sep 14, 2026Tianyi Xu, Daniel Pimentel-Alarcón, Zuzana Buřivalová +1Self-Training Sound Event Detection
University of Wisconsin–Madison
An Ensemble-Based Self-Taught Learning Approach for Parking Space Classification Under Limited Data
Sep 3, 2026Lucas de Oliveira Cunha, Joelton Deonei Gotz, Paulo Lisboa de Almeida +1Ensemble Learning Autonomous Parking
Universidade Federal do Paraná, Department of Informatics, Curitiba, Paraná, Brazil · Pontifícia Universidade Católica do Paraná, Graduate Program in Informatics (PPGIa), Curitiba, Paraná, Brazil
Generalizable Brain Tumor Segmentation with Self-Training and Tumor-Aware Deformations
Sep 2, 2026Henrique Zan Grande, Jeovane Honorio Alves, Rayson Laroca +1Brain Tumor Segmentation BraTS
Pontifícia Universidade Católica do Paraná (PUCPR), Programa de Pós-Graduação em Informática (PPGIa), Curitiba, Paraná, Brazil · University of Luxembourg, SEDAN - SnT, Luxembourg, Luxembourg
Cheap Verifiers, Large Blind Spots: Measuring the Reliability Cost of Cost-Saving Cascades
Sep 1, 2026Dushyant Rajput, Nirdesh Chauhan, Siddharth KosarajuLLM Inference Efficiency LLM Answer Verification
AltSlate Labs LLP
Moving the Mean Toward the Known Good, Not Beyond It: What Inference-Time Interventions and Weight Consolidation Buy in Open-Ended Generation
Aug 28, 2026Roberto I. Ono FilhoInference-Time Optimization Self-Training
Independent researcher
Unsupervised Post-Training of Foundation Models: A Survey
Aug 25, 2026Yijie Xu, Qianyi Cai, Huizai Yao +9LLM Post-Training Self-Training
HKUST(GZ) · Xiaohongshu Inc. · WeChat, Tencent +3
Test-Time Adaptation for ECG Classification via SQI-Gated Self-Training and Beat-Rhythm Consistency
Aug 24, 2026Wenhan Jiang, Zhipeng Deng, Jiale Zhou +3Test-Time Adaptation Electrocardiogram Classification
Westlake University, Hangzhou, China · Hokkaido University, Sapporo, Japan · RIKEN, Japan
Large-Small Model Collaboration for Zero-Shot Surgical Phase Recognition
Aug 24, 2026Yiyi Zhang, Ying Zheng, Wenxin Fan +5Surgical Video Understanding Self-Training
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, China · Department of Surgery, The Chinese University of Hong Kong, Hong Kong, China +1
Governance Records as Supervision: Verifier-Selected Self-Training for Structured Workflow Repair
Aug 18, 2026Jesus SalasSelf-Training for Language Models Self-Training
Microsoft
Domain-Division based Progressive Learning for Source-Free Domain Adaptation
Jul 31, 2026Pan Liu, Jing Li, Meng Zhao +3Source-Free Domain Adaptation Unsupervised Domain Adaptation
School of Computer Science and Engineering, Tianjin University of Technology, Tianjin 300384, China
CVKD-UDA: Cross-View Knowledge Distillation for 3D Unsupervised Domain Adaptive Segmentation
Jul 11, 2026Zhimin Yuan, Ming Cheng, Shangshu Yu +43D Point Cloud Segmentation Unsupervised Domain Adaptation
School of Artificial Intelligence, Nanyang Normal University, Nanyang 473061, China · Fujian Key Laboratory of Sensing and Computing for Smart Cities, School of Informatics, Xiamen University, Xiamen 361005, China · School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China
STEAM: Stable Self-Training with Elastic Matching and Adaptive Purification
Jul 10, 2026Shaoxiang Wang, Kejia Zhang, Haiwei Pan +2Cross-View Geo-Localization Self-Training
Harbin Engineering University, School of Computer Science and Technology, Harbin, China · Northeast Forestry University, School of Computer and Artificial Intelligence, Harbin, China
BAT3R: Bootstrapping Articulated 3D Reconstruction from 2D Image Collections
Jul 4, 2026Jakub Zadrozny, Oisin Mac Aodha, Hakan BilenArticulated Object Reconstruction Synthetic Data Generation
University of Edinburgh
SemiScope: Disentangling Classifier Tuning and Joint Optimization in Semi-Supervised Security Classification
Jun 30, 2026Rui Shu, Tianpei Xia, Jingzhu HeSelf-Training Cybersecurity
North Carolina State University · ShanghaiTech University
Real-Time Source-Free Object Detection
Jun 30, 2026Sairam VCR, Varun Gopal, Poornima Jain +2Source-Free Domain Adaptation Self-Training
1IIT Hyderabad, India · 2Microsoft Research · 3MBZUAI
Domain Adaptive Object Detection via Dual-Stream Bilevel-Cycle Optimization
Jun 30, 2026Yannan Chen, Wei Wang, Wenqiang Wang +5Unsupervised Domain Adaptation Self-Training
School of Cyber Science and Technology, Sun Yat-sen University, Shenzhen Campus, Guangdong 518107, China · PC Laboratory, Shenzhen, Guangdong 518055, China · University of Chinese Academy of Sciences, Beijing 100049, China +1
When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs
Jun 26, 2026Xinyuan Song, Zekun Cai, Liang ZhaoSelf-Training Model Collapse
Emory University, Atlanta, GA, USA · The University of Tokyo, Tokyo, Japan · LocationMind, Tokyo, Japan
Data Selection Through Iterative Self-Filtering for Vision-Language Settings
Jun 22, 2026Andrei Liviu Nicolicioiu, Sarvjeet Singh Ghotra, Morgane M. Moss +1Vision-Language Models Training Data Curation
Mila, Université de Montréal
Spatially Selective Self-Training for Unsupervised Building Change Detection
Jun 9, 2026Wafaa I. M. Hussin, Zhi Lu, Anas M. I. Mohammed +3Self-Training Remote Sensing Change Detection
School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China · Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Civil Engineering, University of Khartoum, Khartoum, Sudan +2
Teacher-Free Self-Training Amplifies but Does Not Compound: A Pass@ $K$ Crossover on a Free-Verifier Domain
Jun 5, 2026Igor Lima StrozziSelf-Training for Language Models Self-Training
Applied Mathematics Department Federal University of Rio de Janeiro

Oct 29, 2026J/K move · Enter open · S save

Statistical Analysis of Synthetic Data with Verification: A Linear Regression Framework

Heejune Sheen, Harrison H. Zhou

Department of Statistics and Data Science Yale University New Haven, CT 06511 USA

Synthetic Data Selection Self-Training Statistical Learning Theory Synthetic Data Generation

Open paper AskPDF arXiv

Synthetic data are widely used for training modern AI systems. However, their statistical understanding remains limited. It is well recognized in statistics that repeated self-training on synthetic data can amplify errors inherited from pre-trained models, leading to model collapse. Verification plays a crucial role in selecting high-quality synthetic data to help prevent model collapse and improve model performance. We study verified self-training in a linear regression framework, where a pre-trained estimator generates synthetic data with injected generation noise and a verifier selects samples according to a verification threshold. Post-training on the accepted samples updates either all coordinates (full fine-tuning) or only an active subset (partial fine-tuning). Our analysis reveals three potentially counterintuitive effects. We establish that tighter verification does not always lead to lower prediction risk. We also find that a suitable level of injected generation noise can reduce prediction risk, while excessive noise can increase it. Moreover, full and partial fine-tuning can respond differently to the same verification rule, since pre-training error in the frozen coordinates affects both sample selection and post-training risk. We analyze these effects by deriving finite sample risk bounds for four separate regimes determined by the relative scales of the verification threshold, generation noise, and pre-training error. The bounds precisely characterize how verification affects prediction risk across these regimes. In particular, we quantify how the generation noise, the effective accepted sample size, and the covariance of the accepted covariates affect prediction risk. The analysis extends to iterative verified full fine-tuning, where sufficiently tight verification reduces the initial error contribution at a geometric rate.