Fine-Tuning Language Models

Follow this topic Ask about this topic

Latest papers

63 papers21 papers in the last four weeks, up 600% on the four weeks before. 0.1% of all new papers.

Eco-MobiLLM: Efficient LLM Fine-Tuning on the Mobile Device via Server Assisted Side-Tuning
Jul 1, 2025Xingke Yang, Liang Li, Zhiyi Wan +6LLM Fine-Tuning Fine-Tuning Language Models
University of Houston, USA · Pengcheng Laboratory, China · Beijing University of Posts and Telecommunications, China +2
APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
May 26, 2025Javier MarínLLM Fine-Tuning Selective Fine-Tuning
Typhoon: Gradient-Based Task-Specific Masking for Masked Language Models
Mar 27, 2023Bruce Changlong Xu, Muhammed Shahir Abdurrahman, Hashem ElezabiMasked Language Modeling Fine-Tuning Language Models
Department of Computer Science, Stanford University

Jul 1, 2025J/K move · Enter open · S save

Eco-MobiLLM: Efficient LLM Fine-Tuning on the Mobile Device via Server Assisted Side-Tuning

Xingke Yang, Liang Li, Zhiyi Wan, Sicong Li+5

University of Houston, USA · Pengcheng Laboratory, China · Beijing University of Posts and Telecommunications, China+2

LLM Fine-Tuning Fine-Tuning Language Models On-Device Training

Open paper AskPDF arXiv

There is a huge gap between numerous intriguing applications fostered by on-device large language model (LLM) fine-tuning (FT) from fresh mobile data and the limited resources of a mobile device. While existing server-assisted methods (e.g., split learning or side-tuning) may enable LLM FT on the local mobile device, they suffer from heavy communication burdens of activation transmissions. In addition, their multiple on-device forward passes over the same samples across training epochs lead to substantial iterative computation on the mobile device. To address those issues, we develop Eco-MobiLLM, an efficient LLM FT method which can be deployed on the mobile device via server-assisted side-tuning. To reduce communication cost, Eco-MobiLLM develops an activation shortcut that transmits only the tokens involved in the loss calculation instead of full activation matrices to guide the side network tuning. Besides, to improve computing efficiency and further accelerate FT convergence, Eco-MobiLLM introduces a novel two-stage training paradigm, which allows the server to reuse historical activations and saves the mobile device from repeatedly computing forward passes for the recurring data samples. Extensive experimental results demonstrate Eco-MobiLLM's superiority.

Latest papers

Eco-MobiLLM: Efficient LLM Fine-Tuning on the Mobile Device via Server Assisted Side-Tuning

APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation

Typhoon: Gradient-Based Task-Specific Masking for Masked Language Models