HUOZIIME: An On-Device LLM-enhanced Input Method for Deep Personalization

Baocai Shan; Yuzhuang Xu; Wanxiang Che

cs.CLMar 23, 2026

HUOZIIME: An On-Device LLM-enhanced Input Method for Deep Personalization

Authors: Baocai Shan, Yuzhuang Xu, Wanxiang Che

Organizations: Harbin Institute of Technology, Harbin, China

Abstract

Mobile input method editors (IMEs) are the primary interface for text input, yet they remain constrained to manual typing and struggle to produce personalized text. While lightweight large language models (LLMs) make on-device auxiliary generation feasible, enabling deeply personalized, privacy-preserving, and real-time generative IMEs poses fundamental challenges.To this end, we present HUOZIIME, a personalized on-device IME powered by LLM. We endow HUOZIIME with initial human-like prediction ability by post-training a base LLM on synthesized personalization data. Notably, a hierarchical memory mechanism is designed to continually capture and leverage user-specific input history. Furthermore, we perform systemic optimizations tailored to on-device LLMbased IME deployment, ensuring efficient and responsive operation under mobile constraints.Experiments demonstrate efficient on-device execution and high-fidelity memory-driven personalization. Code and package are available at https://github.com/Shan-HIT/HuoziIME.

Figures & tables

Figure 1: Overview of HuoziIME. We illustrate the end-to-end workflow, from user input and memory retrieval to LLM-based prediction, highlighting the full-process context management of LLM in IME.

Panel of Table 1 — Table 1: Comparison of representative LLM-enhanced Chinese IMEs across functionality, personalization, memory, and privacy. We briefly list the key features of different IMEs.

Figure 2: Stylized post-training pipeline. We construct persona-specific conversational data through curation, augmentation, filtering, and iterative refinement.

Figure 3: Interaction pipeline between HuoziIME and HuoziIME-Chat in a daily conversation scenario.

Pipeline Stage	Success / Total	Rate (%)
Memory Trigger	342 / 343	99.7
Processing (Normal)	163 / 169	96.4
Processing (Refusal)	87 / 122	71.3
Retrieval@4	179 / 200	89.5
Grounded Generation	156 / 179	87.2

Table 2: Performance of the memory triggering and retrieval effectiveness.

Panel of Figure 4 — Figure 4: On-device inference performance across context lengths up to 512 tokens: (a) prefill throughput, (b) decode throughput, (c) TTFC latency, and (d) extra peak RAM.

Personalized Language Models

Personalized language models adapt responses to individual users’ preferences, histories, writing styles, and changing needs.

See all 89 papers

LoRA-generating hypernetworks for efficient on-device LLM generative personalization
Sep 21, 2026Sean Augenstein, Li Ding, Jihwan Lee +2On-Device Language Model Inference Low-Rank Adaptation
Google · Formerly at Google, work done while at Google
MobileMoE: Scaling On-Device Mixture of Experts
May 26, 2026Yanbei Chen, Hanxian Huang, Ernie Chang +5Mixture-of-Experts Language Models On-Device Language Model Inference
Meta AI
POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
Oct 17, 2025Yizhuo Chen, Xin Liu, Ruijie Wang +7LLM Personalization Personalized Language Models
University of Illinois Urbana-Champaign · Amazon · University of Notre Dame

Sep 21, 2026J/K move · Enter open · S save

LoRA-generating hypernetworks for efficient on-device LLM generative personalization

Sean Augenstein, Li Ding, Jihwan Lee, Keith Rush+1

Google · Formerly at Google, work done while at Google

On-Device Language Model Inference Low-Rank Adaptation Hypernetworks Personalized Language Models Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

On-device large language models (

LLMs'), e.g. running on mobile phones, are ripe for improvement via personalization. The limited compute resources of mobile devices impose limits on model scale and thus model quality, making any realizable quality gains highly impactful. At the same time, their personal nature (i.e., the close coupling to a particular user) means that a given on-device LLM tends to be used in similar, predictable patterns over the course of time. This paper presents a novel method for personalizing on-device LLMs. It trains a hypernetwork to map a user's context tokens to a low-rank adaptation (

LoRA') well-suited to that user. Once the trained common artifacts are deployed to users' devices, each user uses the hypernetwork to synthesize (entirely on device) a personalized LoRA. This approach blends the benefits while avoiding the drawbacks of two existing approaches to LLM customization: in-context learning (ICL') and parameter-efficient fine-tuning (PEFT'). Like ICL (and unlike PEFT), the on-device phase of our approach is computationally feasible, requiring only forward passes through neural networks. Like PEFT (and unlike ICL), our approach modifies the `target' base LLM via weights (the LoRA), avoiding negative consequences (e.g. increased latency) associated with extending the input sequence. Our approach is particularly well-suited to the mobile device regime. Apart from the on-device compute and latency benefits mentioned, it also requires minimal additional storage, as internally its architecture partly leverages the same LLM weights as belong to the target LLM to be personalized. We demonstrate the benefits of LoRA-generating hypernetworks on several representative personalization datasets, comparing against baselines like ICL and PEFT. Of note, our personalization experiments focus on more challenging and less studied long-form text generation tasks.

Abstract

Personalized Language Models

Explore similar work

LoRA-generating hypernetworks for efficient on-device LLM generative personalization

MobileMoE: Scaling On-Device Mixture of Experts

POPI: Personalizing LLMs via Optimized Natural Language Preference Inference