Organizations: Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA, USA · Department of Computer Science, University of California, Santa Barbara, CA, USA
Large language models (LLMs) on personal devices keep user data local, but limited memory restricts their general capability. Since most user-specific tasks do not require the breadth of frontier models, a practical solution is to adapt a shared on-device model into lightweight specialists. However, backpropagation is prohibitive in memory, and users rarely provide labeled data. We present MobileSpec, a system that trains specialists with forward passes alone and learns from automatically checked task outcomes, as in reinforcement learning with verifiable rewards (RLVR). Because forward-only updates are built from random perturbations, training is efficient only when these perturbations resemble the weight changes of RLVR, which largely preserve the singular values of pretrained weights and rotate their singular vectors. The core algorithm of MobileSpec, FuRA-S, produces such perturbations while training only the block-spectral coordinates of pretrained weights, namely the singular values of their column blocks. As we show theoretically and empirically, rescaling these blocks independently shifts their relative contributions to each weight matrix and thereby rotates its singular vectors. Since this update can be rewritten as a transformation of the layer input, FuRA-S also adapts INT4 models without modifying their weights. With specialists of only 1-5 MB that update 0.015-0.05% of the parameters, MobileSpec raises the success rate of a phone-use agent from 6-23% up to 89-100% and trains a personalized 8B model on an iPhone 16 Pro overnight. On RLVR benchmarks, FuRA-S is competitive with full-parameter evolution strategies (ES) and outperforms ES over LoRA adapters and quantized ES on INT4 models.
Figures & tables
Figure 1: Memory needed to train on a phone. Serving the model alone occupies most of the memory available to an app, so training with backpropagation or with full-parameter ES exceeds it, whereas FuRA-S adds only 1–5 MB of trainable parameters.
Figure 2: On-device training of a specialist with MobileSpec. When the generalist agent fails at a task that the user repeats, MobileSpec trains a specialist for it on the phone overnight. Perturbed models attempt the task in sandboxed apps, and forward-only ES moves the block singular values toward those of the better perturbed models (Section 3.2 ). The resulting specialist is a 2 MB vector δ that the serving engine applies on top of the shared INT4 generalist.
Figure 3: FuRA-S. (a) Only the block singular values are trained, and the adapted block reuses the original, possibly INT4, weights. (b) FuRA-S restricts its perturbations to the region where more of them improve the model, whereas LoRA perturbs in unconstrained, less effective directions. (c) The trained singular vectors alone retain over 95% of the gain.
Figure 4: ES on Countdown with Qwen2.5-1.5B-Instruct in four search spaces: (a) accuracy on 500 test problems, (b) fraction of the 30 perturbations per iteration that improve on the unperturbed model, and (c) KL divergence from the base policy per response token.
Figure 5: Fraction of perturbations that improve on the unperturbed model on Countdown, (a) around the base model and (b) around the FuRA-S model at iteration 100.
BP-based
BP-free
Base model
Original
PPO
GRPO
Dr. GRPO
Full ES
LoRA-ES
VeRA-ES
FuRA-S (ours)
Qwen2.5-0.5B-Instruct
0.1
0.3
13.0
13.5
14.4
12.6
13.5
14.4 (0.0)
Llama-3.2-1B-Instruct
0.4
11.2
14.9
13.9
15.6
11.7
13.7
15.8 (+0.2)
Qwen2.5-1.5B-Instruct
0.7
14.2
27.8
31.0
36.7
34.7
28.1
37.0 (+0.3)
Qwen2.5-3B-Instruct
10.0
20.1
37.8
43.8
58.2
54.3
55.2
55.8 ( − 2.4)
Qwen2.5-7B-Instruct
31.2
55.1
57.0
57.5
58.9
55.5
54.2
56.2 ( − 2.7)
Table 1: Countdown accuracy (%) on 2,000 test problems. The BP-based policy-gradient baselines are from ES at Scale, Full ES is our reproduction of ES at Scale (Appendix C.1 ), and LoRA-ES uses rank-4 adapters, and VeRA-ES uses rank-1024 VeRA adapters. Best per row in bold, and parentheses give the difference from Full ES.
Method
AIME24
AMC
MATH500
Minerva
Olympiad
Avg-5
Avg-3
Full ES
26.7
57.8
77.6
26.8
37.0
45.2
47.2
LoRA-ES
16.7
53.0
74.0
22.8
35.4
40.4
44.1
FuRA-S (ours)
23.3
54.2
76.8
26.5
40.0
44.2 ( − 1.0)
47.8 (+0.6)
Table 2: Greedy pass@1 (%) of Qwen2.5-Math-7B at the final checkpoint, with FuRA-S minus Full ES in parentheses.
Task
Base
Episode reward
Step-wise (ours)
MastodonReply
23.4
79.7
89.1
CheckEventTime
9.4
92.2
100.0
RequestCarpooling
6.2
14.1
98.4
MastodonFollow
23.4
64.1
64.1
MastodonPostPoll
9.4
20.3
68.8
Table 3: Success rate (%) of MAI-UI-2B on MobileWorld tasks, for the generalist and its specialists after up to 15 ES iterations.
Figure 6: LaMP with Llama-3.1-8B-Instruct. (a) User-based setting, without and with the retrieved profile. (b) Time-based setting, with per-user specialists trained by ES (ZO in the legend) and per-user LoRA trained by backpropagation (FO in the legend).
Figure 7: Training a specialist for a LaMP-2M user on an iPhone 16 Pro: (a) training and validation loss, (b) memory use.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: (a) Rotation of the leading singular vectors of eight Qwen2.5-1.5B-Instruct matrices under four updates with the norm of the update learned on Countdown, and (b) its first-order prediction at small sizes. (c, d) Per-layer ρΣ and κ of Qwen2.5-Math-7B after MATH training, with the values reported by ISO for supervised fine-tuning in gray.
Updated factor
Values
ρΣ
κ
∠U
∠V
Block sk (FuRA-S)
1.5k
5.3%
7.5
8.6 ∘
8.8 ∘
Rk , additive
135k
3.0%
1.2
6.1 ∘
6.4 ∘
Rk , rotation
67k
0.0%
1.2
0.0 ∘
1.9 ∘
Lk , rotation in span
67k
0.6%
1.4
1.8 ∘
2.4 ∘
Dense W
8M
2.3%
1.0
2.1 ∘
2.1 ∘
Appendix
Table A1: Random updates of different block-SVD factors (medians). ∠U , ∠V : largest principal angles between the leading 16-dimensional singular subspaces of W and W′ .
Experiment
Model
Update
μrel
Step
Pairs
Batch
Updates
Lk
Countdown
Qwen2.5 0.5B–7B, Llama-3.2-1B
ES
3×10−2
α=1
15
200
500
FP32
MATH
Qwen2.5-Math-7B
ES
3×10−2
α=1
15
1,024
200
FP32
Countdown (INT4)
Qwen2.5 1.5B, 3B
ES
3×10−2
α=1
25
200
300
FP32
SFT
Llama-3-8B
ZO-Adam
10−2
η=0.5
8
16
5,987
BF16/NF4
Sudoku
Qwen3.5-4B
ES
3×10−2
α=1
16
32
100
FP32
LaMP (U)
Llama-3.1-8B, Qwen3.5-4B
ES
3×10−2
α=1
15
128
200
BF16
Appendix
Table A2: FuRA-S settings across experiments. Pairs denotes the number of antithetic perturbation pairs per update, Batch denotes the number of examples, problems, or puzzles evaluated per population member, and Lk indicates the storage precision of the frozen left singular factors. All ES experiments use μrel=3×10−2 , with α=1 except for per-user LaMP adaptation.
Selection
Method
Iter.
AIME24
AMC
MATH500
Minerva
Olympiad
Avg-5
Avg-3
Final
Full ES
200
26.7
57.8
77.6
26.8
37.0
45.2
47.2
LoRA-ES
200
16.7
53.0
74.0
22.8
35.4
40.4
44.1
FuRA-S
200
23.3
54.2
76.8
26.5
40.0
44.2 ( − 1.0)
47.8 (+0.6)
Best Avg-5
Full ES
130
33.3
57.8
76.8
28.7
39.7
47.3
48.4
LoRA-ES
120
30.0
53.0
73.2
23.2
35.3
42.9
43.9
FuRA-S
185
23.3
56.6
77.2
29.4
40.1
45.3 ( − 1.9)
48.9 (+0.5)
Appendix
Table A3: Greedy pass@1 (%) of Qwen2.5-Math-7B on five math benchmarks under three checkpoint selections, with FuRA-S minus Full ES in parentheses. Avg-5 averages all five benchmarks, and Avg-3 averages MATH500, Minerva, and OlympiadBench.
Figure 9: Countdown accuracy of the INT4 checkpoints over iterations, with the best accuracy of QES dotted.
Iteration
0
25
50
100
150
200
250
300
Qwen2.5-1.5B
3.55
15.95
17.70
18.90
19.50
20.10
21.70
21.85
Qwen2.5-3B
3.25
27.15
39.20
43.75
46.65
48.00
49.85
50.40
Appendix
Table A4: Countdown accuracy (%) of FuRA-S on the GPTQ-INT4 checkpoints over training iterations, evaluated using reconstructed BF16 weights under the QES protocol. FuRA-S exceeds the best accuracy of our QES reproduction, 16.75% and 34.15%, within 50 iterations.
Method
Peak memory (GB)
Grouped (s)
Padded (s)
FuRA-S
17.17
0.341
0.570
FuRA-S (4-bit L )
7.52
0.358
0.579
MeZO
16.26
0.318
0.481
LOZO
16.26
0.265
0.427
GRZO
20.90
–
1.028
Sparse-MeZO
30.90
0.408
0.570
Appendix
Table A5: Peak training memory and time per 16-example forward evaluation on Llama-3-8B using one A100 40GB GPU. Runtime includes perturbation and parameter updates, with length-grouped or padded batches.
Qwen2.5-1.5B
Qwen2.5-3B
Model form
Memory
Decode
Prefill 1
Prefill 8
Memory
Decode
Prefill 1
Prefill 8
FP16
2.88
6.93
23.9
80.5
5.85
10.58
31.9
156.9
INT4
1.09
5.74
31.2
87.4
1.94
7.95
42.2
160.8
INT4 + FuRA-S, deployed
1.19
6.58
46.6
105.6
2.11
8.83
63.5
189.1
INT4 + FuRA-S, training, FP16 Rk
1.20
7.94
56.7
125.2
2.12
10.34
74.4
220.0
INT4 + FuRA-S, training, FP32 Rk
1.30
8.92
62.8
209.0
2.28
11.85
81.7
355.7
Appendix
Table A6: INT4 inference cost on one A100 40GB GPU. Memory is reported in GB, decoding latency per token in milliseconds, and prefill latency in milliseconds for one or eight prompts. The deployed FuRA-S formulation retains the original GPTQ weights and introduces approximately 9% additional memory relative to INT4.
Method
Update
LR
Perturbation
Other settings
FuRA-S
ZO-Adam
0.5
μrel=10−2
per-pair batches, BF16 Lk
FuRA-S (4-bit L )
ZO-Adam
0.5
μrel=10−2
shared batch, NF4 Lk
GRZO
GRZO
5×10−8
ϵ=10−3
per-example perturbations, FP16
LOZO
ZO-SGD
10−7
ϵ=10−3
rank 2, ν=50 , FP16
MeZO
ZO-SGD
10−7
ϵ=10−3
FP16
VeRA
ZO-SGD
5×10−3
ϵ=10−2
r=1024 , d0=0.1 , FP16
Appendix
Table A7: Optimization settings for Commonsense-170K. LR denotes the learning rate, with peak values for FuRA-S, the initial value for QuZO, and constant values otherwise. All methods use the same budget of 47,898 antithetic perturbation pairs.
Method
Trainable
Trainable %
PIQA
SIQA
HellaS.
WinoG.
ARC-e
ARC-c
OBQA
Avg
Full-parameter updates
GRZO
8.03B
100
78.7
64.4
74.3
53.0
84.2
65.7
60.6
68.70
LOZO
8.03B
100
70.8
53.2
45.0
49.3
68.8
53.0
52.0
56.01
MeZO
8.03B
100
76.2
55.2
40.5
51.5
67.0
51.8
40.8
54.71
PEFT updates
FuRA-S
1.25M
0.0155
78.0
56.2
66.3
53.4
79.7
61.3
57.6
64.64
Appendix
Table A8: Accuracy (%) on seven tasks after SFT of Llama-3-8B on Commonsense-170K with the same forward-pass budget, best per group in bold. FuRA-S is the most accurate PEFT method and leads QuZO by 16.97 points in 4 bits.
Figure 10: Agentic Sudoku with Qwen3.5-4B: (a) held-out success, final and over the last three evaluations, with standard deviations; (b) trained parameters. FuRA-S is on par with full-parameter ES while training 5,000 times fewer parameters. Results for full-parameter ES are from the released logs of Zheng et al. (2026) .
Task
Metric
Base
Base + profile
FuRA-S
FuRA-S + profile
LaMP-1 citation
accuracy / F1
0.528 / 0.528
0.665 / 0.666
0.515 / 0.515
0.681 / 0.681
LaMP-2 movie tag
accuracy / F1
0.285 / 0.267
0.347 / 0.302
0.446 / 0.404
0.452 / 0.404
LaMP-3 rating
MAE / RMSE ↓
0.383 / 0.697
0.369 / 0.748
0.272 / 0.577
0.254 / 0.564
LaMP-4 news headline
ROUGE-1 / ROUGE-L
0.093 / 0.081
0.123 / 0.109
0.165 / 0.150
0.182 / 0.166
LaMP-5 paper title
ROUGE-1 / ROUGE-L
0.118 / 0.109
0.132 / 0.120
0.438 / 0.398
0.458 / 0.418
LaMP-7 tweet paraphrase
ROUGE-1 / ROUGE-L
0.335 / 0.279
0.335 / 0.284
0.476 / 0.418
0.470 / 0.414
Appendix
Table A9: LaMP user-based split with Llama-3.1-8B-Instruct, with or without one retrieved item from the user’s profile in the prompt, for FuRA-S in both training and testing. With the profile, FuRA-S improves over the base model on all six tasks.
Task
Metric
Queries (users)
Base
Per-user LoRA
FuRA-S
LaMP-1 citation
accuracy / F1
123 (100)
0.480 / 0.478
0.488 / 0.486
0.480 / 0.470
LaMP-2M movie tag
accuracy / F1
3302 (100)
0.296 / 0.256
0.491 / 0.357
0.541 / 0.422
LaMP-3 rating
MAE / RMSE ↓
112 (101)
0.402 / 0.840
0.179 / 0.463
0.259 / 0.701
LaMP-4 news headline
ROUGE-1 / ROUGE-L
6725 (100)
0.090 / 0.078
0.164 / 0.147
0.141 / 0.125
LaMP-5 paper title
ROUGE-1 / ROUGE-L
107 (101)
0.145 / 0.131
0.493 / 0.445
0.476 / 0.415
LaMP-7 tweet paraphrase
ROUGE-1 / ROUGE-L
109 (100)
0.296 / 0.246
0.320 / 0.266
0.338 / 0.290
Appendix
Table A10: LaMP time-based split with Llama-3.1-8B-Instruct: the base model and per-user LoRA and FuRA-S checkpoints. FuRA-S leads per-user LoRA on LaMP-2M and LaMP-7.
Figure 11: LaMP with Qwen3.5-4B, with the panels of Figure 6 . With the profile, FuRA-S again improves all six tasks for new users, and its per-user checkpoints lead per-user LoRA on LaMP-1 and LaMP-2M.
Task
Metric
Base
Base + profile
FuRA-S
FuRA-S + profile
LaMP-1 citation
accuracy / F1
0.500 / 0.511
0.646 / 0.642
0.509 / 0.498
0.671 / 0.670
LaMP-2 movie tag
accuracy / F1
0.372 / 0.304
0.308 / 0.257
0.414 / 0.350
0.454 / 0.388
LaMP-3 rating
MAE / RMSE ↓
0.326 / 0.675
0.381 / 0.826
0.276 / 0.607
0.251 / 0.578
LaMP-4 news headline
ROUGE-1 / ROUGE-L
0.122 / 0.107
0.117 / 0.103
0.153 / 0.139
0.154 / 0.141
LaMP-5 paper title
ROUGE-1 / ROUGE-L
0.110 / 0.099
0.324 / 0.281
0.449 / 0.410
0.460 / 0.420
LaMP-7 tweet paraphrase
ROUGE-1 / ROUGE-L
0.452 / 0.396
0.415 / 0.363
0.489 / 0.432
0.472 / 0.419
Appendix
Table A11: LaMP user-based split with Qwen3.5-4B, with or without one retrieved item from the user’s profile in the prompt, for FuRA-S in both training and testing. With the profile, FuRA-S improves over the base model on all six tasks.
Task
Metric
Queries (users)
Base
Per-user LoRA
FuRA-S
LaMP-1 citation
accuracy / F1
123 (100)
0.431 / 0.446
0.423 / 0.425
0.455 / 0.458
LaMP-2M movie tag
accuracy / F1
3302 (100)
0.319 / 0.248
0.486 / 0.334
0.512 / 0.385
LaMP-3 rating
MAE / RMSE ↓
112 (101)
0.411 / 0.732
0.196 / 0.567
0.241 / 0.620
LaMP-4 news headline
ROUGE-1 / ROUGE-L
6725 (100)
0.117 / 0.104
0.144 / 0.129
0.128 / 0.115
LaMP-5 paper title
ROUGE-1 / ROUGE-L
107 (101)
0.502 / 0.422
0.514 / 0.457
0.489 / 0.432
LaMP-7 tweet paraphrase
ROUGE-1 / ROUGE-L
109 (100)
0.471 / 0.421
0.468 / 0.421
0.467 / 0.419
Appendix
Table A12: LaMP time-based split with Qwen3.5-4B: the base model and per-user LoRA and FuRA-S checkpoints. FuRA-S leads per-user LoRA on LaMP-1 and LaMP-2M.
There is a huge gap between numerous intriguing applications fostered by on-device large language model (LLM) fine-tuning (FT) from fresh mobile data and the limited resources of a mobile device. While existing server-assisted methods (e.g., split learning or side-tuning) may enable LLM FT on the local mobile device, they suffer from heavy communication burdens of activation transmissions. In addition, their multiple on-device forward passes over the same samples across training epochs lead to substantial iterative computation on the mobile device. To address those issues, we develop Eco-MobiLLM, an efficient LLM FT method which can be deployed on the mobile device via server-assisted side-tuning. To reduce communication cost, Eco-MobiLLM develops an activation shortcut that transmits only the tokens involved in the loss calculation instead of full activation matrices to guide the side network tuning. Besides, to improve computing efficiency and further accelerate FT convergence, Eco-MobiLLM introduces a novel two-stage training paradigm, which allows the server to reuse historical activations and saves the mobile device from repeatedly computing forward passes for the recurring data samples. Extensive experimental results demonstrate Eco-MobiLLM's superiority.
Xingke Yang, Liang Li, Zhiyi Wan +6
University of Houston, USA · Pengcheng Laboratory, China · Beijing University of Posts and Telecommunications, China +2
Mixture-of-Experts (MoE) has become the de facto architecture for hundred-billion-parameter language models, yet its advantages at sub-billion scales for on-device deployment remain largely unexplored. To close this gap, we present MobileMoE, a family of on-device MoE language models with sub-billion active parameters (0.3-0.9B active and 1.3-5.3B total) that establish a new Pareto frontier for on-device LLMs. We first formulate an on-device MoE scaling law that jointly optimizes MoE architecture under mobile memory and compute constraints, identifying an on-device sweet spot - moderate sparsity with fine-grained and shared experts - that is simultaneously memory and compute-optimal. Building on the derived architectures, we train MobileMoE with a four-stage recipe covering pre-training, mid-training, instruction fine-tuning, and quantization-aware training, all on open-source datasets. Across 14 benchmarks, MobileMoE matches or exceeds leading on-device dense LLMs with 2-4× fewer inference FLOPs, and matches or surpasses the state-of-the-art MoE OLMoE-1B-7B with up to 60% fewer parameters. To bridge the last mile to mobile deployment, we provide the first efficient MoE inference on commodity smartphones with comprehensive on-device profiling. At comparable INT4 weight memory, MobileMoE-S delivers 1.8-3.8× faster prefill and 2.2-3.4× faster decode than the dense baseline MobileLLM-Pro.
Multi-billion-parameter LLMs now run on phones for inference, and training them on the device would personalize them without user data leaving the phone. Prior work has measured individual training steps of such models on phones, but not complete training runs, and not whether adapters trained on the device improve personalization. We present the first systematic characterization of a multi-billion-parameter LLM fine-tuned on a mobile device, covering memory, per-step time, thermal behavior, and energy. An iPhone 17 Pro can fine-tune a 3B-parameter LLM to a typical user within one battery charge, and the resulting adapters improve personalization as much as adapters trained on a server. Sustained training throttles the phone to about half its initial throughput, and none of the pausing or burst schedules we tested recovers it. Nearly all of each training step is spent in the frozen base model, most of it in the backward pass, which nine of the ten other runtimes we audited do not accelerate. Apple's MLX had a kernel for it that was never dispatched and was incorrect, and our repair, now merged upstream, trains an adapter 1.47x faster on a third less energy. On-device fine-tuning is feasible on current phones, and making it efficient requires runtimes and operating systems to treat training as a first-class workload.