Self-Training

Follow this topic Ask about this topic

Self-training studies how models can learn from unlabeled examples or their own outputs, most commonly by generating pseudo-labels and training on selected predictions. Teacher–student and Mean Teacher methods, confidence filtering, consistency losses, and class balancing are common approaches, especially in semi-supervised learning and domain adaptation. Newer methods also use model-generated reasoning, rewards, or task-specific verifiers to improve vision, language, audio, and 3D systems when human annotations are scarce. A central challenge is that incorrect pseudo-labels and unreliable self-checks can reinforce errors, so reliable filtering and independent verification remain important.

Latest papers

43 papers6 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loop
May 28, 2026Yang Zhang, Xiukun Wei, Xueru ZhangLLM Alignment Self-Training
Department of Computer Science and Engineering, The Ohio State University, Columbus, Ohio.
RISE: Reliable Improvement in Self-Evolving Vision-Language Models
May 20, 2026Chaoran Xu, Yingmao Miao, Pengfei Zhang +3Vision-Language Models VLM Adaptation
AMAP, Alibaba Group
EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant
May 11, 2026Zichen Wen, Boxue Yang, Junlong Ke +5Streaming Video Understanding Self-Training
EPIC Lab, Shanghai Jiao Tong University · Tsinghua University · The Hong Kong University of Science and Technology (Guangzhou) +1
RemoteZero: Geospatial Reasoning with Zero Human Annotations
May 6, 2026Liang Yao, Fan Liu, Shengxiang Xu +4Self-Training Remote Sensing
1Hohai University · 2Southeast University
Unpaired Image Deraining Using Reward-Guided Self-Reinforcement Strategy
May 1, 2026Yinghao Chen, Yeying Jin, Xiang Chen +3Unsupervised Image Restoration Self-Training
1National University of Defense Technology · National University of Singapore · 3Nanjing University of Science and Technology +2
SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments
Apr 15, 2026Dinging Li, Yingxiu Zhao, Xinrui Cheng +163D Spatial Reasoning Self-Training
Zhejiang University · StepFun
GIFT: Bootstrapping Image-to-CAD Program Synthesis via Geometric Feedback
Mar 28, 2026Giorgio Giannone, Anna Clare Doris, Amin Heyrani Nobari +3Synthetic Data Augmentation Self-Training
AI Innovation Team, Red Hat · DeCoDE Lab, MIT · Core AI, IBM +1
Semi-Supervised Learning with Balanced Deep Representation Distributions
Mar 22, 2026Changchun Li, Ximing Li, Bingjie Zhang +2Semi-Supervised Learning Self-Training
College of Computer Science and Technology, Jilin University, Changchun 130012, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, China · Department of Computer Science, University of Texas at Dallas, Richardson, TX, USA
Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision
Mar 18, 2026Nikhil Gosala, B. Ravi Kiran, Senthil Yogamani +1Visual Object Tracking 3D MOT
University of Freiburg, Germany · Qualcomm SARL France · Automated Driving, Qualcomm Technologies, Inc., USA
Unsupervised Adaptation from FDG to PSMA PET/CT for 3D Lesion Detection under Label Shift
Mar 14, 2026Xiaofeng Liu, Menghua Xia, Yanis Chemli +3Unsupervised Domain Adaptation Label Shift
Yale Biomedical Imaging Institute and Department of Radiology & Biomedical Imaging, Yale University, New Haven, CT 06520, USA
Dream2Learn: Structured Generative Dreaming for Continual Learning
Mar 2, 2026Salvatore Calcagno, Matteo Pennisi, Federica Proietto Salanitri +4Class-Incremental Learning Self-Training
PeRCeiVe Lab, University of Catania, Italy
Survival is the Only Reward: Sustainable Self-Training Through Environment-Mediated Selection
Jan 18, 2026Jennifer Dodgson, Alfath Daryl Alhajir, Michael Joedhitya +6Self-Training Self-Improving Agents
See, Think, Learn: A Self-Taught Multimodal Reasoner
Dec 2, 2025Sourabh Sharma, Sonam Gupta, SadbhawnaSelf-Training VLM Reasoning
Malaviya National Institute of Technology Jaipur · IBM Research

May 28, 2026J/K move · Enter open · S save

When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loop

Yang Zhang, Xiukun Wei, Xueru Zhang

Department of Computer Science and Engineering, The Ohio State University, Columbus, Ohio.

LLM Alignment Self-Training Model Collapse

Open paper AskPDF arXiv

Foundation models are increasingly trained on synthetic data generated by prior model iterations rather than exclusively on real data. This self-consuming training paradigm can lead to model collapse, divergence, or bias amplification. Recent work (Ferbach et al., 2024) shows that incorporating human curation into the loop can steer a self-consuming model toward human-aligned behavior, but these analyses focus on a single, isolated model that solely consumes its own outputs. In practice, however, models often interact and train on input-output pairs produced by other models. This paper studies self-consuming training in the multi-model regime. We first formalize a framework for interacting self-consuming models and characterize when the resulting dynamical system converges to a stable point. We then examine how human curation of one model affects its own alignment (self-influence) and how such effects propagate to other models (cross-influence). Unlike isolated settings where human curation always enhances model alignment, we show that cross-model interactions can dampen or even invert this effect, ultimately degrading long-term alignment.