Fine-Tuning Language Models

Follow this topic Ask about this topic

Latest papers

34 papers2 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models
Mar 12, 2026Samy Jelassi, Mujin Kwun, Rosie Zhao +5Fine-Tuning Language Models LLM Fine-Tuning
Harvard University, Kempner Institute · MBZUAI · Microsoft Research New England
Addressing the Ecological Fallacy in Larger LMs with Human Context
Mar 6, 2026Nikita Soni, Dhruv Vijay Kunjadiya, Pratham Piyush Shah +3Language Modeling LLM Training
Department of Computer Science, Stony Brook University · College of Connected Computing, Vanderbilt University, USA
Building a Strong Instruction Language Model for a Less-Resourced Language
Mar 2, 2026Domen Vreš, Tjaša Arčon, Timotej Petrič +3Low-Resource Language Processing LLM Fine-Tuning
University of Ljubljana, Faculty of Computer and Information Science, Ljubljana, Slovenia
APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
May 26, 2025Javier MarínLLM Fine-Tuning Selective Fine-Tuning

Mar 12, 2026J/K move · Enter open · S save

Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models

Samy Jelassi, Mujin Kwun, Rosie Zhao, Yuanzhi Li+4

Harvard University, Kempner Institute · MBZUAI · Microsoft Research New England

Fine-Tuning Language Models LLM Fine-Tuning RL Fine-Tuning

Open paper AskPDF arXiv

Cross-entropy (CE) training provides dense and scalable supervision for language models, but it optimizes next-token prediction under teacher forcing rather than sequence-level behavior under model rollouts. We introduce a feature-matching objective for language-model fine-tuning that targets sequence-level statistics of the completion distribution, providing dense semantic feedback without requiring a task-specific verifier or preference model. To optimize this objective efficiently, we propose energy-based fine-tuning (EBFT), which uses strided block-parallel sampling to generate multiple rollouts from nested prefixes concurrently, batches feature extraction over these rollouts, and uses the resulting embeddings to perform an on-policy policy-gradient update. We present a theoretical perspective connecting EBFT to KL-regularized feature-matching and energy-based modeling. Empirically, across Q&A coding, unstructured coding, and translation, EBFT matches RLVR and outperforms SFT on downstream accuracy while achieving a lower validation cross-entropy than both methods.

Latest papers

Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models

Addressing the Ecological Fallacy in Larger LMs with Human Context

Building a Strong Instruction Language Model for a Less-Resourced Language

APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation