Logit Adjustment

Follow this topic Ask about this topic

Latest papers

8 papers0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Safe Error Correction for Language Models: Frozen-Base Adjustment with Capability Preservation
Sep 14, 2026Gautam KishoreLogit Adjustment Parameter-Efficient Fine-Tuning
Eulogik
Logit-Origin Centering for Singleton Test-Time Adaptation
Aug 2, 2026Mayank Sharma, Rohit Kumar Mourya, Pratik MazumderTest-Time Adaptation Logit Adjustment
Indian Institute of Technology Jodhpur, Jodhpur, India
Generalised Balanced Softmax: A Finite-Data Perspective on Logit Adjustment for Long-Tailed Recognition
Jul 24, 2026Yi-Hang Zhu, Rajeev Raman, Shiqi Su +4Class-Imbalanced Learning Long-Tailed Classification
School of Computing and Mathematical Sciences, University of Leicester, Leicester, LE1 7RH, UK · Department of Physics and Astronomy, University of Leicester, Leicester, LE1 7RH, UK · Scientific Computing, Rutherford Appleton Laboratory, Science and Technology Facilities Council, Didcot, OX11 0QX, UK +2
AdaPCLA: Adaptive Prior-Calibrated Logit Adjustment for Long-Tailed Longitudinal EHR Generation
Jul 14, 2026Shuai Cui, Chen Wenxuan, Wenjie Du +3Logit Adjustment Long-Tail Learning
University of Science and Technology of China · Sun Yat-sen University
TCLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models
Jul 10, 2026Tianyou Jiang, Ziyu ZhouFew-Shot Domain Adaptation Logit Adjustment
University of Bern, Bern, Switzerland · Shanghai Jiao Tong University, Shanghai, China
Quantized Reasoning Models Think They Need to Think Longer, but They Do Not
May 29, 2026Sanae Lotfi, Polina Kirichenko, Steven Li +1Logit Adjustment Overthinking in Language Models
1FAIR at Meta · 2Meta AI
CUE: Concept-Aware Multi-Label Expansion to Mitigate Concept Confusion in Long-Tailed Learning
May 2, 2026Ruichi Zhang, Chikai Shang, Jiacheng Yang +4Long-Tailed Classification Logit Adjustment
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China · College of Computer Science and Software Engineering, Shenzhen University, China · Institute of High Performance Computing, A*STAR, Singapore
Neural Prior Estimation: Learning Class Priors from Latent Representations
Feb 19, 2026Masoud Yavari, Payman MoallemLogit Adjustment Class-Imbalanced Learning
Independent Researcher Isfahan, Iran · Department of Electrical Engineering Faculty of Engineering University of Isfahan, Iran

Sep 14, 2026J/K move · Enter open · S save

Safe Error Correction for Language Models: Frozen-Base Adjustment with Capability Preservation

Gautam Kishore

Eulogik

Logit Adjustment Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

We study a practical question: can a small correction module fix errors in a frozen language model's outputs without degrading its base capabilities? We propose CRN v2, a lightweight logit-level correction module (~34M trainable parameters, 0.73% of the 4.65B text module) that sits atop a fully frozen Gemma 4 E2B model. The base model is never updated; only the correction module learns, via supervised fine-tuning followed by reference-free DPO on 83,400 error-correction pairs. On a 60-question domain exam (CEHRI: Certified Human-Robot Intelligence, covering facts, arithmetic, and implicit-goal reasoning), CRN v2 corrects 53.3% of base-model errors (reworded variant: 43.3%) while showing no degradation on tested capability benchmarks (MMLU/BoolQ N=200; car-wash N=8). A LoRA baseline at the matched CRN v1 budget (6.6M params, rank 19) achieves 83.3% correction but suffers 30-75% capability loss on the same benchmarks -- the correction-capability tradeoff. An ablation shows that the KL preservation term (lambda=0.1) is critical: lowering it to 0.01 degrades correction to 35.0%. A hidden-state injection variant at earlier layers (1.6M params, SFT-only) reaches 50.0%/55.8% but does not exceed logit correction; shallower injection (layer 4) drops to 30.0%/28.3%; multi-depth logit correction (~35M) reaches only 40%; and longer training (5,000 SFT + 2,000 DPO) stays at 53.3% -- none of the alternative configurations we tested exceeded the rank-128 logit result, consistent with a best-achieved result of ~53% rather than a floor. This is a study of a design principle (frozen base + logit correction + KL anchoring), not a claim of architectural novelty. All code, main-result weights, and evaluation scripts are released (deep variant as code only -- no trained deep checkpoints).