LLM Fine-Tuning

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

42 papers in the last four weeks, up 83% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 509

All topics

Sub-Billion, Super-Frontier: Small Language Models Rival Zero-Shot Frontier LLMs on General and Literary Relation Extraction
Jun 21, 2026Despina Christou, Grigorios TsoumakasRelation Extraction LLM Evaluation
School of Informatics, Aristotle University of Thessaloniki, 54124, Greece · Archimedes, Athena Research Center, Greece
First-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in Transformers
Jun 21, 2026Arjun Pillai, Christian Hoang, Anjelo Jann LarozaMultilingual Language Models LLM Interpretability
1Irvington High School · 2GenAI4E · 3Mapua University
Fine-Tuning Large Language Models for Quantum Reasoning
Jun 20, 2026Katherine Ip, Casey R. Myers, Udaya Parampalli +2LLM Fine-Tuning RL for Language Model Reasoning
School of Computing and Information Systems, The University of Melbourne, Melbourne, Australia · School of Physics, Mathematics and Computing, The University of Western Australia, 35 Stirling Hwy, Crawley WA, 6009, Australia · Pawsey Supercomputing Centre, 1 Bryce Avenue, Kensington WA, 6151, Australia +1
CAT-Translate: Building Compact Open-Source Models for Japanese-English Translation
Jun 19, 2026Yuu JinnaiLLM Fine-Tuning Small Language Models
CyberAgent / Tokyo, Japan
Finetuning with Scientific Data Increases Hallucinations: A Multi-domain Factuality Evaluation of LLMs
Jun 19, 2026Raia Abu Ahmad, Nikolas Rauscher, Ekaterina Borisova +3LLM Evaluation LLM Fine-Tuning
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) · Technische Universität Berlin · Humboldt-Universität zu Berlin
What Shapes Emergent Misalignment? Insights from Training Dynamics, Model Priors, and Data
Jun 18, 2026Yuchen Zhang, Anietta Weckauff, Diego Garcia-Olano +1LLM Evaluation LLM Alignment
Max Planck Institute for Intelligent Systems, ELLIS Institute Tübingen, Tübingen AI Center, Tübingen, Germany · Meta.
Online Dynamic Batching with Formal Guarantees for LLM Training
Jun 18, 2026Dian Li, Zekun Wang, Yaoru Wang +1LLM Fine-Tuning Distributed Training
Tencent
Tracking Representation Dynamics in Large Language Models with Persistent Homology
Jun 17, 2026Naman Malhotra, Jay Ambadkar, Abhinav Gupta +4LLM Alignment LLM Fine-Tuning
Department of Mathematics, Imperial College London, UK · Department of Computing, Imperial College London, UK
Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices
Jun 17, 2026Hassan Dbouk, Matthias Reisser, Prathamesh Mandke +2Memory-Efficient Fine-Tuning LLM Fine-Tuning
Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc. · work done while at Qualcomm
Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA
Jun 17, 2026Ikram Belmadani, Oumaima El Khettari, Carlos Ramisch +3LLM Fine-Tuning Domain-Adaptive Pretraining
Aix-Marseille Univ., CNRS, LIS UMR 7020, 13000 Marseille, France · Nantes Univ., École Centrale Nantes, CNRS, LS2N UMR 6004, 44000 Nantes, France · Grenoble Alpes Univ., CNRS, INRIA, Grenoble INP, LIG UMR 5217, 38000 Grenoble, France
Towards Scalable Customization and Deployment of Multi-Agent Systems for Enterprise Applications
Jun 16, 2026Paresh Dashore, Shreyas Kulkarni, Uttam Gurram +5Multi-Agent LLM Systems Multi-Agent System Optimization
AI Foundations, Capital One
From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning
Jun 16, 2026Lingjing Kong, Xin Liu, Guangyi Chen +9LLM Fine-Tuning RL for Language Model Reasoning
Carnegie Mellon University · Mohamed bin Zayed University of Artificial Intelligence · Institute of Foundation Models +1
Fine-tuning LLMs for Passive Depression Severity Estimation from AI Mental Health Dialogue
Jun 16, 2026Olivier Tieleman, Ziyi Zhu, Ting Su +3Depression Detection LLM Fine-Tuning
From Drift to Coherence: Stabilizing Beliefs in LLMs
Jun 16, 2026SongEun Kim, Seungyoo Lee, Edwin Fong +2LLM Fine-Tuning Bayesian Inference
Department of Statistics, Seoul National University · Korea Advanced Institute of Science & Technology · The University of Hong Kong +1
TuneAhead: Predicting Fine-tuning Performance Before Full Training Begins
Jun 16, 2026Yuxiang Luo, Haonan Long, Chen Wang +6Fine-Tuning LLM Fine-Tuning
The Hong Kong University of Science and Technology (Guangzhou) · Huawei Technologies Ltd.
A Risk Decomposition Framework for Pre-Hoc Fine-Tuning Prediction
Jun 16, 2026Yuxiang Luo, Chen Wang, Nan TangFine-Tuning LLM Fine-Tuning
The Hong Kong University of Science and Technology (Guangzhou).
LiFT: Local Search via Linear Programming for Overfitting-Controlled Transformers
Jun 15, 2026Abhishek Shukla, Anikeit Khanna, Ankur Sinha +1Neural Network Generalization LLM Fine-Tuning
Department of Management Sciences, Indian Institute of Technology Kanpur, Kanpur-208016, Uttar Pradesh, India · Brij Disa Centre for Data Science and AI, Indian Institute of Management Ahmedabad, Ahmedabad-380015, Gujarat, India · Department of Civil Engineering, Indian Institute of Technology Kanpur, Kanpur-208016, Uttar Pradesh, India +1
From Tokens to Regions: CUDA-Sensitive Instruction Tuning for GPU Kernel Generation
Jun 15, 2026Wentao Chen, Jiace Zhu, Xing Zhe Chai +4GPU Kernel Optimization LLM Fine-Tuning
1Shanghai Jiao Tong University · 2Biren Technology
PreLort: Prefix-Nested LoRA for Federated Fine-Tuning under Rank Heterogeneity
Jun 14, 2026Muhammad Waseem, Nurbek Tastan, Andrej Jovanovic +4Federated Learning Aggregation LLM Fine-Tuning
1MBZUAI, UAE · University of Cambridge, UK · 3Flower Labs, UK +1
Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Experts
Jun 14, 2026Yijun Lu, Zihan Fang, Pengpeng Qiao +6Federated Learning Aggregation LLM Fine-Tuning
Waseda University, Tokyo, Japan · City University of Hong Kong, China · Institute of Science Tokyo, Tokyo, Japan +4
Integrating Reasoning and Generalization in Text-to-SQL via Self-Enhanced Fine-Tuning
Jun 14, 2026Feng Lyu, Jinfeng Cen, Sijing Duan +4Self-Training for Language Models LLM Fine-Tuning
Central South University · Tsinghua University · Nanjing University +1
Data-Centric Benchmarking of Exploit Generation in LLMs: Understanding the Impact of Fine-Tuning
Jun 13, 2026Yiwei Chen, Lichi Li, Kai Cheung +2LLM Evaluation LLM Fine-Tuning
Cisco Systems, Inc., San Francisco, CA, USA · Michigan State University, East Lansing, MI, USA
Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning
Jun 12, 2026Dmitriy Bystrov, Daniil Medyakov, Dmitry Bylinkin +1Gradient Descent Zeroth-Order Optimization
1Moscow Independent Research Institute of Artificial Intelligence · 2Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · 3Innopolis University
CoRe: A Continuously Reward-Finetuned LLM Query Rewriter for Multi-Stage Context-Aware Relevance in Web-Scale Video Search
Jun 12, 2026Yilin Wen, Rong Yang, Xiaojia Chang +11LLM Fine-Tuning
TikTok
Direct Preference Optimization for Chatbot Fine-Tuning: An Empirical Study
Jun 11, 2026Dezhi Yu, Yvonne Qiu, ShuoJia FuLLM Alignment Direct Preference Optimization
Small LLMs for Biomedical Claim Verification: Cost-Effective Fine-Tuning, Structural Dataset Shortcuts, and Cross-Domain Generalization
Jun 11, 2026Gaurav KumarClaim Verification LLM Fine-Tuning
Moveworks AI · University of California San Diego
System Report for CCL25-Eval Task 5: New Dataset and LoRA-Fine-Tuned Qwen2.5
Jun 10, 2026Haotao XieLLM Fine-Tuning Machine Translation
The Hangzhou International Innovation Institute · Beihang University
Categorical Prior Lock-in: Why In-Context Learning Fails for Structured Data
Jun 10, 2026Antonio Pelusi, Stefano Braghin, Alberto TrombettaLLM Fine-Tuning In-Context Learning
University of Insubria, Italy · IBM Research, Ireland
Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training
Jun 10, 2026Michal Chudoba, Sergey Alyaev, Petra Galuscakova +1LLM Fine-Tuning Multimodal Large Language Models
University of Stavanger · Stavanger, Norway · NORCE Research +1
Lius: Translation Model Based Instructional Lingustic Using Continual Instruction Tuning In Kupang Malay
Jun 10, 2026Joanito Agili Lopo, Yunita Sari, Guntur Budi HerwantoLLM Fine-Tuning Low-Resource MT
Department of Computer Science and Electronics Universitas Gadjah Mada

Jun 21, 2026, cs.CLJ/K move · Enter open · S save

Sub-Billion, Super-Frontier: Small Language Models Rival Zero-Shot Frontier LLMs on General and Literary Relation Extraction

Despina Christou, Grigorios Tsoumakas

School of Informatics, Aristotle University of Thessaloniki, 54124, Greece · Archimedes, Athena Research Center, Greece

Relation Extraction LLM Evaluation LLM Fine-Tuning Small Language Models

Open paper AskPDF arXiv

Large language models (LLMs) achieve strong relation extraction (RE), but their computational demands and reliance on proprietary APIs limit deployment in resource-constrained or privacy-sensitive settings. We investigate how far small language models (SLMs) can close this gap across general-domain and literary text. We evaluate five models from 360M to 3B parameters under three domain-composition regimes and two prompt-conditioned tuning styles (30 configurations), comparing them with zero-shot frontier LLMs and a discriminative RoBERTa baseline. Across nine benchmarks, the best sub-billion model, Qwen2.5-0.5B fine-tuned on pooled general-domain data, achieves a general-domain positive-class micro-F1 of 0.83, versus 0.69 for GPT-5.4 and 0.66 for Claude Sonnet 4.6 evaluated zero-shot. This does not imply that SLMs are intrinsically stronger; rather, targeted task adaptation enables 4-bit models deployable on a single consumer GPU to outperform general-purpose frontier systems under this protocol. An in-domain RoBERTa baseline also exceeds both frontier models, indicating that the gain stems from task adaptation rather than generative decoding. On literary RE, tuned SLMs reach 0.92 on the human-annotated Biographical benchmark versus 0.83 for GPT-5.4, and 0.833 versus 0.578 on the two-benchmark literary average. A targeted domain-adaptive pretraining case study yields no practically meaningful gain over supervised fine-tuning, while the cleanest within-family scale comparison shows only marginal improvement. These results show that, when task-specific data are available, compact task-adapted models can provide accurate, private, and hardware-efficient RE.

Momentum

Related topics

Latest papers 509