Is Human-Readable Text Necessary for Effective LLM Fine-Tuning?

Jinhao Zhang; Zeyu Liu; Zicheng Yan; Yunquan Zhang; Daning Cheng; Song Tang

cs.AISep 26, 2026

Is Human-Readable Text Necessary for Effective LLM Fine-Tuning?

Authors: Jinhao Zhang, Zeyu Liu, Zicheng Yan, Yunquan Zhang, Daning Cheng, Song Tang

Organizations: Beijing University of Posts and Telecommunications · Institute of Computing Technology, Chinese Academy of Sciences · University of Science and Technology of China · University of Shanghai for Science and Technology

Abstract

Is human readability necessary for effective fine-tuning of large language models? We investigate whether model-conditioned training representations can preserve or improve adaptation utility without requiring a human-readable textual form. We propose Desired-Update-Aligned Synthetic Data (DASA), which uses activation-gradient feedback from a frozen reference model to guide the optimization of continuous synthetic input embeddings. Inspired by the role of activation gradients in local risk reduction, DASA targets useful adaptation updates rather than source-text reconstruction or linguistic fluency. The resulting embeddings are used directly for downstream fine-tuning; discrete token projections are employed only for qualitative inspection. Experiments on six models from the Llama and Qwen families, ranging from 1B to 32B parameters, cover six benchmarks spanning knowledge, mathematical reasoning, code generation, and commonsense reasoning. Under matched LoRA adaptation settings, DASA achieves performance comparable to the source natural-language data and surpasses it in multiple configurations, while outperforming GRADMM in most comparisons. Further experiments cover general-domain and task-specialized source data. Under the evaluated synthesis settings, DASA provides a $3.6$ -- $4.9\times$ speedup over GRADMM with comparable peak GPU memory.

Figures & tables

	Token Budget: 50 $\times$ 2048
	Llama-3.2-1B						Llama-3.2-3B						Llama-3.1-8B
Method	MMLU	G8K	MATH	MBPP	ARC	CSQA	MMLU	G8K	MATH	MBPP	ARC	CSQA	MMLU	G8K	MATH	MBPP	ARC	CSQA
Base	45.79	46.32	32.70	49.6	54.61	59.13	58.62	82.03	51.40	68.4	73.72	73.87	65.41	85.90	50.70	75.4	82.08	75.92
Real-text	45.91	46.55	32.96	50.0	54.86	59.30	58.76	82.18	51.60	68.8	73.81	74.20	65.45	86.20	50.88	75.6	82.34	76.00
GRADMM	46.10	47.08	33.40	50.4	54.35	59.54	58.98	82.56	51.20	69.0	74.06	74.20	65.72	86.15	51.16	76.0	82.42	76.17
Ours	46.46	46.63	34.20	50.8	54.69	59.95	59.22	82.87	51.66	69.6	74.32	74.04	65.56	86.05	51.56	76.4	82.68	76.41

Table 1: Main results across models and tasks. Base, Real-text, GRADMM, and DASA under three C4 calibration budgets; all adaptation methods use identical LoRA and optimization settings.

	Qwen3-8B			Qwen3-14B
Method	MMLU $\uparrow$	GSM8K $\uparrow$	MBPP $\uparrow$	MMLU $\uparrow$	GSM8K $\uparrow$	MBPP $\uparrow$
Random	$71.8\pm 0.35$	$87.9\pm 0.48$	$55.2\pm 0.8$	$77.9\pm 0.31$	$90.1\pm 0.41$	$62.6\pm 0.6$
Real-text	$76.97\pm 0.13$	$\mathbf{93.93\pm 0.21}$	$61.8\pm 0.6$	$80.89\pm 0.17$	$95.91\pm 0.22$	$\mathbf{66.8\pm 0.4}$
GRADMM	$77.41\pm 0.15$	$93.63\pm 0.24$	$62.2\pm 0.6$	$80.51\pm 0.16$	$96.29\pm 0.22$	$66.4\pm 0.4$
Ours	$\mathbf{78.09\pm 0.16}$	$93.78\pm 0.33$	$\mathbf{63.0\pm 0.6}$	$\mathbf{81.15\pm 0.29}$	$\mathbf{97.12\pm 0.23}$	$66.6\pm 0.6$

Table 2: Robustness across random seeds. Mean

\pm

standard deviation over three independent runs under the

200\times 2048

data budget. Random uses Gaussian inputs matched to the mean and standard deviation of DASA’s initialization embeddings.

Specialized Data
Method	Qwen3-8B						Qwen3-14B
Method	MMLU	GSM8K	MATH	MBPP	ARC-C	CSQA	MMLU	GSM8K	MATH	MBPP	ARC-C	CSQA
Base	74.90	91.89	83.98	59.6	64.33	81.98	78.85	93.86	84.76	64.6	66.98	81.00
Real-text	78.02	94.62	87.18	64.0	67.49	85.01	81.82	96.36	88.00	69.0	70.39	84.03
GRADMM	78.41	94.39	87.54	62.6	67.83	84.77	82.17	96.13	88.38	68.6	70.73	83.78
Ours	79.26	94.54	88.36	62.8	68.77	85.42	82.91	96.29	89.18	68.8	71.59	84.52

Table 3: Results with task-specialized calibration data. Calibration data are drawn from the corresponding training split for each benchmark: MMLU auxiliary train, GSM8K train, MATH train, MBPP train, ARC-Challenge train, and CommonsenseQA train. All adaptation methods use identical LoRA and optimization settings.

	Generation Time (min) $\downarrow$			Peak Memory (GB) $\downarrow$
Method	$\mathbf{50\times 2048}$	$\mathbf{100\times 2048}$	$\mathbf{200\times 2048}$	$\mathbf{50\times 2048}$	$\mathbf{100\times 2048}$	$\mathbf{200\times 2048}$
GRADMM	28.3	57.4	84.2	17.3	17.7	17.9
DASA	5.8	11.6	23.3	17.3	17.4	17.9

Table 4: Computational efficiency. Generation time and peak GPU memory under three calibration budgets, measured on a single NVIDIA A800 GPU using FP16. GRADMM and DASA use matched sequence counts, sequence lengths, and synthetic-data optimization steps.

Figure 1: Sensitivity to LoRA rank and downstream optimization steps. Validation accuracy (%) under different LoRA ranks and numbers of downstream LoRA optimization steps for synthetic and real-text adaptation.

Figure 2: Qualitative example of DASA synthetic data. A human-readable solution is shown alongside a decoded visualization of an optimized continuous synthetic sample.

LLM Fine-Tuning

LLM fine-tuning adapts pretrained large language models to tasks, domains, languages, and behaviors through additional training.

See all 758 papers

FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
Jan 29, 2026Ajay Patel, Colin Raffel, Chris Callison-BurchLanguage Model Pretraining Synthetic Data Pretraining
University of Pennsylvania · University of Toronto · Vector Institute +1
Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation
May 8, 2026Haozhan Tang, Xiuqi Zhu, Xinyin Zhang +3LLM Fine-Tuning Low-Rank Adaptation
Carnegie Mellon University · Tsinghua University · Infinigence AI
Synthetic Data for any Differentiable Target
Apr 9, 2026Tristan Thrush, Sung Min Park, Herman Brunborg +5Synthetic Data Generation for Language Models RL for Generative Models
Stanford University

Jan 29, 2026J/K move · Enter open · S save

FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale

Ajay Patel, Colin Raffel, Chris Callison-Burch

University of Pennsylvania · University of Toronto · Vector Institute+1

Language Model Pretraining Synthetic Data Pretraining Synthetic Data Generation Synthetic Instruction Generation Instruction Tuning Synthetic Data Generation for Language Models

Open paper AskPDF arXiv

Due to limited supervised training data, large language models (LLMs) are typically pre-trained via a self-supervised "predict the next word" objective on a vast amount of unstructured text data. To make the resulting model useful to users, it is further trained on a far smaller amount of "instruction-tuning" data comprised of supervised training examples of instructions and responses. To overcome the limited amount of supervised data, we propose a procedure that can transform the knowledge in internet-scale pre-training documents into billions of synthetic instruction and answer training pairs. The resulting dataset, called FineInstructions, uses ~18M instruction templates created from real user-written queries and prompts. These instruction templates are matched to and instantiated with human-written source documents from unstructured pre-training corpora. With "supervised" synthetic training data generated at this scale, an LLM can be pre-trained from scratch solely with the instruction-tuning objective, which is far more in-distribution with the expected downstream usage of LLMs (responding to user prompts). We conduct controlled token-for-token training experiments and find pre-training on FineInstructions outperforms standard pre-training and other proposed synthetic pre-training techniques on standard benchmarks measuring free-form response quality. Our resources can be found at https://huggingface.co/fineinstructions .

Abstract

LLM Fine-Tuning

Explore similar work

FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale

Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation

Synthetic Data for any Differentiable Target