On-Device Language Model Fine-Tuning

Latest papers

8 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

First-Order LoRA Training with Phone NPUs
Oct 17, 2026Xu Zheng, Wei Cheng, Zhuomin Chen +4On-Device Language Model Fine-Tuning On-Device Training
Florida International University, Miami, FL, USA · NEC Laboratories America, Princeton, NJ, USA · North Carolina State University, Raleigh, NC, USA +2
Fine-Tuning a 3B-Parameter LLM on a Smartphone: Characterizing Sustained Training
Oct 5, 2026Andrew Geyko, Marius Mosbach, André BrinkmannOn-Device Training LLM Fine-Tuning
Saarland University Saarbrücken, Germany
Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices
Jun 17, 2026Hassan Dbouk, Matthias Reisser, Prathamesh Mandke +2Memory-Efficient Fine-Tuning LLM Fine-Tuning
Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc. · work done while at Qualcomm
Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning
May 22, 2026Yoshihiko Fujisawa, Yuma Ichikawa, Yudai Fujimoto +2LLM Quantization Memory-Efficient Fine-Tuning
Fujitsu Limited · Institute of Science Tokyo · RIKEN Center for AIP +1
EdgeFlowerTune: Evaluating Federated LLM Fine-Tuning Under Realistic Edge System Constraints
May 9, 2026Jiaxiang Geng, Yiyi Lu, Lunyu Zhao +3LLM Evaluation LLM Fine-Tuning
Duke Kunshan University · Flower Labs · University of Cambridge
Activation-Efficient Fine-Tuning via Sequence-Compressed Low-Rank Subspaces
Apr 3, 2026Irene Tenison, Stella Ahn, Miriam Kim +2On-Device Language Model Fine-Tuning Memory-Efficient Fine-Tuning
MIT CSAIL · Harvard SEAS
Competition for attention predicts good-to-bad tipping in AI
Feb 16, 2026Neil F. Johnson, Frank Y. HuoOn-Device Language Model Fine-Tuning LLM Safety
Physics Department and Dynamic Online Networks Laboratory, George Washington University, Washington D.C. 20052, U.S.A.
End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost
Aug 21, 2025Qitao Tan, Xiaoying Song, Ningxi Cheng +11On-Device Language Model Fine-Tuning LLM Quantization
University of Georgia · University of North Texas · Northeastern University +3

Oct 17, 2026J/K move · Enter open · S save

First-Order LoRA Training with Phone NPUs

Xu Zheng, Wei Cheng, Zhuomin Chen, Dongkuan Xu+3

Florida International University, Miami, FL, USA · NEC Laboratories America, Princeton, NJ, USA · North Carolina State University, Raleigh, NC, USA+2

On-Device Language Model Fine-Tuning On-Device Training LLM Fine-Tuning Neural Network Hardware Acceleration Low-Rank Adaptation

Open paper AskPDF arXiv

On-device fine-tuning lets a large language model (LLM) learn from its user's private data without uploading it. Low-rank adaptation (LoRA) reduces the trainable parameters, but each first-order training step still multiplies by the frozen base-model weights during the forward pass, activation recomputation, and input-gradient computation. A phone's neural processing unit (NPU) already holds these weights for inference; using it for training also requires computing input gradients, which its inference interface does not support. We present a system that runs all three frozen-weight products on the NPU while keeping the training graph, adapters, and optimizer on the CPU. Its input-gradient kernels reuse the resident weight tiles and the forward dequantizer, avoiding a transposed weight copy and ensuring forward and backward use the same effective weights. To preserve usable gradients, the system selects precision by gradient sensitivity and applies a per-row range guard for half-precision (FP16) outputs. Its runtime fuses the remaining CPU operators and overlaps NPU execution with adapter computation, while resumable execution and thermal pacing let training proceed as a background job. We evaluate it on two Qualcomm Snapdragon platforms using four LLMs from the Qwen and Llama families. Our workloads cover nightly personalization with labels supplied on the phone and the public LaMP-2 benchmark; comparisons include an optimized CPU trainer, a phone-GPU trainer, forward-only training, and GPU-trained references.