LLM Fine-Tuning

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

42 papers in the last four weeks, up 83% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 509

All topics

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
May 20, 2026Hao Chen, Qi Zhang, Liyao Li +7LLM Alignment LLM Fine-Tuning
Zhejiang University · Ant Group · Eastern Institute of Technology
Federated LoRA Fine-Tuning for LLMs via Collaborative Alignment
May 20, 2026Shuaida He, Liwen Chen, Long FengLLM Fine-Tuning Low-Rank Adaptation
School of Computing & Data Science, The University of Hong Kong
ChunkFT: Byte-Streamed Optimization for Memory-Efficient Full Fine-Tuning
May 20, 2026Yongkang Liu, Zijing Wang, Mengjie Zhao +7Fine-Tuning Memory-Efficient Fine-Tuning
Northeastern University, China · Shanghai Jiao Tong University, China · CIS, LMU Munich, Germany +2
Automated ICD Classification of Psychiatric Diagnoses: From Classical NLP to Large Language Models
May 20, 2026Fernando Ortega, Raúl Lara-Cabrera, Jorge Dueñas-Lerín +3LLM Fine-Tuning Clinical Information Extraction
Department of Sistemas Informáticos, Universidad Politécnica de Madrid, Spain · KNODIS Research Group, Universidad Politécnica de Madrid, Spain · CIBERSAM ISCIII, Spain +8
SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
May 20, 2026Yongkang Liu, Xing Li, Mengjie Zhao +7LLM Fine-Tuning Adapter Tuning
Northeastern University, China · Shandong University, China · CIS, LMU Munich, Germany +1
FuRA: Full-Rank Parameter-Efficient Fine-Tuning with Spectral Preconditioning
May 19, 2026Yequan Zhao, Ruijie Zhang, Liyan Tan +3LLM Fine-Tuning Parameter-Efficient Fine-Tuning
University of California at Santa Barbara · Amazon Lab126
Security Document Classification with a Fine-Tuned Local Large Language Model: Benchmark Data and an Open-Source System
May 19, 2026Ivan DobrovolskyiLLM Fine-Tuning Text Classification
MS in AI & ML Engineering, Independent Researcher, Sunnyvale, CA, USA
Fine-Tuning Without Forgetting via Loss-Adaptive Learning Rates
May 19, 2026Parjanya Prajakta Prashant, Jiongli Zhu, Aldan Creo +1Learning Rate Scheduling LLM Fine-Tuning
University of California San Diego
AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning
May 19, 2026Ziye Chen, Hongbin Lin, Chenyu Zhang +3Memory-Efficient Fine-Tuning Zeroth-Order Optimization
Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining
May 19, 2026Aarash Abro, Muhammad TahirLLM Fine-Tuning Spectral Methods
Zeta Labs · Lahore University of Management Sciences
Are Rationales Necessary and Sufficient? Tuning LLMs for Explainable Misinformation Detection
May 19, 2026Bing Wang, Rui Miao, Ximing Li +6Fake News Detection LLM Fine-Tuning
College of Computer Science and Technology, Jilin University Changchun, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, Jilin University · School of Artificial Intelligence, Jilin University Changchun, China +3
Learning When to Adapt
May 18, 2026Ali Zindari, Xiaowen Jiang, Rotem Mulayoff +1LLM Fine-Tuning Low-Rank Adaptation
1Universität des Saarlandes · 2CISPA Helmholtz Center for Information Security
Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models
May 18, 2026Spyridon Mavromatis, Sokratis Sofianopoulos, Prokopis Prokopidis +1LLM Fine-Tuning Low-Resource MT
National and Kapodistrian University of Athens, Department of Informatics and Telecommunications · Institute for Language and Speech Processing, Athena RC
Alignment Dynamics in LLM Fine-Tuning
May 18, 2026Yuhan Huang, Huanran Chen, Yinpeng DongLLM Alignment LLM Fine-Tuning
Shanghai Qi Zhi Institue & University of Tokyo · College of AI, Tsinghua University · Shanghai Qi Zhi Institue
FedSDR: Federated Self-Distillation with Rectification
May 18, 2026Ziheng Ren, Zhanming Shen, Hao Wang +2LLM Fine-Tuning Hallucination in Language Models
Beijing University of Aeronautics and Astronautics · Zhejiang University · Stevens Institute of Technology +1
Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective
May 18, 2026Junpeng Zhang, Lei Cheng, Guoxi Zhang +3Supervised Fine-Tuning LLM Fine-Tuning
Shanghai Jiao Tong University · Beijing Institute for General Artificial Intelligence · UniDT +1
DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models
May 17, 2026Haichao Sha, Zihao Wang, Yuncheng Wu +2Fine-Tuning Privacy-Preserving Language Models
Renmin University of China · Nanyang Technological University
Towards Human-Level Book-Writing Capability
May 16, 2026Jan Zierstek, Matteo Batelic, Maya Medjad +1Long-Form Document Generation LLM Fine-Tuning
Pageshift Entertainment
Why Do Reasoning Models Lose Coverage? The Role of Data and Forks in the Road
May 16, 2026Ngoc-Hieu Nguyen, Parshin Shojaee, Phuc Minh Nguyen +4LLM Fine-Tuning Reasoning Diversity
The Pennsylvania State University · Virginia Tech · VinUniversity
LoCO: Low-rank Compositional Rotation Fine-tuning
May 15, 2026An Nguyen, Jaesik Choi, Anh TongFine-Tuning LLM Fine-Tuning
1Korea University · 2KAIST · 3INEEJI
Strategic Over-Parameterization for Generalizable Low-Rank Adaptation
May 15, 2026Jing Gao, Zhong-Yi Lu, Pan Zhang +1LLM Fine-Tuning Low-Rank Adaptation
School of Fundamental Physics and Mathematical Sciences, Hangzhou Institute for Advanced Study, UCAS, Hangzhou 310024, China · School of Physical Sciences, University of Chinese Academy of Sciences, No. 19A Yuquan Road, Beijing 100049, China · CAS Key Laboratory of Theoretical Physics, Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing 100190, China +1
Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language
May 15, 2026Vinayshekhar Bannihatti Kumar, Disha Makhija, Manoj Ghuhan Arivazhagan +1LLM Fine-Tuning Code Language Models
AWS AI Labs
Representation Without Reward: A JEPA Audit for LLM Fine-Tuning
May 14, 2026Biswa SenguptaLLM Auditing LLM Fine-Tuning
LLM Suite Team, JPMorgan Chase & Co.
Training ML Models with Predictable Failures
May 14, 2026Will Schwarzer, Scott NiekumLanguage Model Safety Evaluation LLM Fine-Tuning
University of Massachusetts Amherst
GPart: End-to-End Isometric Fine-Tuning via Global Parameter Partitioning
May 14, 2026Paolo Mandica, Michał Brzozowski, Zuzanna Dubanowska +1LLM Fine-Tuning Low-Rank Adaptation
Samsung AI Center, Warsaw, Poland · University of Warsaw, Poland
Teaching Large Language Models When Not to Know: Learning Temporal Critique for Ex-Ante Reasoning
May 14, 2026Chenlu Ding, Jiancan Wu, Yanchen Luo +3LLM Prompting LLM Fine-Tuning
University of Science and Technology of China · The Hong Kong Polytechnic University · University of Notre Dame
EndPrompt: Efficient Long-Context Extension via Terminal Anchoring
May 14, 2026Han Tian, Luxuan Chen, Xinran Chen +10Long-Context Language Modeling LLM Fine-Tuning
1Nankai University · 2Baidu Inc. · 3Shanghai Jiao Tong University +1
GradShield: Alignment Preserving Finetuning
May 13, 2026Zhanhao Hu, Xiao Huang, Patrick Mendoza +4LLM Fine-Tuning LLM Safety Alignment
University of California, Berkeley · 2HUMAIN · 3King Abdulaziz City for Science and Technology +1
PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Prompts
May 13, 2026Anjir Ahmed Chowdhury, Syed Zawad, Xiaolong Ma +2LLM Fine-Tuning Multi-Task Learning
Department of Computer Science University of Houston · IBM Research · Argonne National Laboratory
Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning
May 13, 2026Daniel M. Jimenez-Gutierrez, Enrique Zuazua, Georgios Kellaris +3LLM Fine-Tuning Non-IID Federated Learning

May 20, 2026, cs.LGJ/K move · Enter open · S save

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment

Hao Chen, Qi Zhang, Liyao Li, Zhanming Shen+6

Zhejiang University · Ant Group · Eastern Institute of Technology

LLM Alignment LLM Fine-Tuning Training Data Selection Attention Head Pruning Attention Head Specialization Data-Efficient Learning

Open paper AskPDF arXiv

Adapting Large Language Models (LLMs) to specialized domains typically incurs high data and computational overhead. While prior efficiency efforts have largely treated data selection and parameter-efficient fine-tuning as isolated processes, our empirical analysis suggests they may be intrinsically coupled. We posit the Strong Map Hypothesis: a sparse subset of attention heads plays a dominant role in task-specific adaptation, acting as keys that unlock specific data patterns. Building on this observation, we propose From Parameters to Data (P2D), a unified framework that leverages these task-sensitive attention heads as a dual compass for both sample mining and structural pruning. To rigorously quantify the total pipeline cost, we introduce the Alignment Efficiency Ratio (AER) metric for both selection latency and training time. Mechanistically, P2D identifies critical heads via a lightweight proxy and uses them as a functional filter to curate high-affinity data, establishing a synergistic pipeline. Empirically, by updating merely 10% of attention heads on 10% of the data, P2D achieves an 8.3 pp performance gain over strong baselines and delivers a 7.0x end-to-end time speedup. These results validate that precise parameter-data synchronization eliminates redundancy, offering a new paradigm for efficient alignment.

Momentum

Related topics

Latest papers 509