LLM Fine-Tuning

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

42 papers in the last four weeks, up 83% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 509

Apr 18, 2026cs.CL

Efficient Task Adaptation in Large Language Models via Selective Parameter Optimization

Large Language Models (LLMs) have demonstrated excellent performance in general language understanding, generation and other tasks. However, when fine-tuning for specific domain tasks, the general knowledge accumulated in the pre-training phase is often partially overwritten or forgotten due to parameter updates, which severely limits the generalization ability and transferability of LLMs. Traditional fine-tuning strategies mostly train on the entire parameter space, ignoring the heterogeneity of model parameters, that is, some parameters are extremely important for general tasks, while other parameters are more sensitive to specific tasks. To alleviate the above problems, this paper innovatively proposes a parameter element importance evaluation method, which divides parameters into "core parameters" and "non-core parameters" by distinguishing the importance of parameters for general language ability tasks and specific domain tasks, and fixes the core parameters during fine-tuning, and only fine-tunes the non-core parameters. Extensive experiments on scientific, medical and physical tasks using GPT-J and LLaMA-3 show that our method can mitigate catastrophic forgetting while enhancing the adaptability of the model.

Weijie Wan, Jiangjiang Zhao

Shortest Path Technology · China Mobile Communications Group Co.,Ltd

Continual Learning for LLMs LLM Fine-Tuning Domain Adaptation for LLMs Selective Fine-Tuning Parameter-Efficient Fine-Tuning

Apr 18, 2026cs.LG

Learning to Trade Like an Expert: Cognitive Fine-Tuning for Stable Financial Reasoning in Language Models

Recent deployments of large language models (LLMs) as autonomous trading agents raise questions about whether financial decision-making competence generalizes beyond specific market patterns and how it should be trained and evaluated in noisy markets lacking ground truth. We propose a structured framework for training and evaluating such models. Central to our approach is a curated, multiple-choice question (MCQ) dataset derived from classic textbooks and historical markets, verified by an AI committee, enriched with structured reasoning traces, and augmented to reduce shortcut learning. To evaluate whether performance on isolated MCQs generalizes to real-world trading, we introduce a two-stage protocol combining test-set evaluation with an MCQ-based chronological trading simulation. Extensive evaluations across market regimes provide statistically robust evidence that open models trained with our framework exhibit competitive, risk-aware behavior over time, outperform open-source baselines, and approach frontier-model performance at smaller scale. We release the dataset and evaluation framework to support further research.

Yuchen Pan, Soung Chang Liew

Department of Information Engineering, The Chinese University of Hong Kong

LLM Evaluation LLM Fine-Tuning Finance Algorithmic Trading

Apr 17, 2026cs.CR

Benign Fine-Tuning Breaks Safety Alignment in Audio LLMs

Fine-tuning on benign data is known to degrade safety alignment in text and vision LLMs, but whether distinct input properties drive this vulnerability differently remains unclear. Audio introduces a richer problem where benign samples can neighbor harmful content through what is said or how it sounds. We present the first systematic study of benign fine-tuning safety in Audio LLMs, evaluating three state-of-the-art models with a proximity-based framework that decomposes embedding-space distance into semantic, acoustic, and mixed axes. We find that the dominant vulnerability axis is architecture-conditioned, determined by how each model's encoder and projector transform audio into the backbone LLM's input space. Across three models, benign fine-tuning elevates Jailbreak Success Rate (JSR) from single digits to as high as 87%, with the most damaging axis shifting from semantic to acoustic proximity depending on encoder design. Mechanistically, fine-tuning selectively suppresses late-layer refusal circuits while frozen encoders preserve upstream representations: the model still detects harmful content but stops refusing, a recognition-refusal dissociation. Two practical defenses, filtering training data to maximize distance from harmful embeddings and a textual system prompt at inference, reduce JSR to near-zero without architectural modification. These findings show that safety evaluation should account for modality and architecture, while highlighting Audio LLMs as a useful testbed for understanding alignment fragility.

Jaechul Roh, Virat Shejwalkar, Amir Houmansadr

University of Massachusetts Amherst

Language Model Safety Evaluation LLM Fine-Tuning LLM Safety Alignment Audio-Language Models LLM Safety

Apr 16, 2026cs.CL

LLM attribution analysis across different fine-tuning strategies and model scales for automated code compliance

Existing research on large language models (LLMs) for automated code compliance has primarily focused on performance, treating the models as black boxes and overlooking how training decisions affect their interpretive behavior. This paper addresses this gap by employing a perturbation-based attribution analysis to compare the interpretive behaviors of LLMs across different fine-tuning strategies such as full fine-tuning (FFT), low-rank adaptation (LoRA) and quantized LoRA fine-tuning, as well as the impact of model scales which include varying LLM parameter sizes. Our results show that FFT produces attribution patterns that are statistically different and more focused than those from parameter-efficient fine-tuning methods. Furthermore, we found that as model scale increases, LLMs develop specific interpretive strategies such as prioritizing numerical constraints and rule identifiers in the building text, albeit with performance gains in semantic similarity of the generated and reference computer-processable rules plateauing for models larger than 7B. This paper provides crucial insights into the explainability of these models, taking a step toward building more transparent LLMs for critical, regulation-based tasks in the Architecture, Engineering, and Construction industry.

Jack Wei Lun Shi, Minghao Dang, Wawan Solihin +1

Department of Civil and Environmental Engineering, National University of Singapore, 117576, Singapore · School of Transportation Science and Engineering, Harbin Institute of Technology, Harbin 150090, China · Research and Innovation, NovaCITYNETS Pte.Ltd., 348615, Singapore

Feature Attribution LLM Interpretability LLM Fine-Tuning

Apr 16, 2026cs.CL

Think Multilingual, Not Harder: A Data-Efficient Framework for Teaching Reasoning Models to Code-Switch

Recent developments in reasoning capabilities have enabled large language models to solve increasingly complex mathematical, symbolic, and logical tasks. Interestingly, while reasoning models are often trained to generate monolingual text, these models have also been observed to code-switch (i.e., mix languages). Prior works have either viewed code-switching as an undesirable error, attempted to control code-switching through modifications to input prompts or the output decoding process, or focus on narrow subsets of languages, domains, tasks, and models. We address these gaps by introducing the first linguistically and behaviorally motivated fine-tuning framework for identifying beneficial code-switched reasoning behaviors in large language models and teaching these models to code-switch more effectively for reasoning. First, we create and systematically analyze a dataset of reasoning traces from diverse models, languages, tasks, and domains to understand the types of code-switching behaviors found in existing reasoning models. Then, we develop fine-tuning interventions that teach reasoning models to code-switch based on our observations of helpful behaviors in existing models. We find that our framework can significantly increase beneficial code-switched reasoning behaviors in a data-efficient manner. Interestingly, we also find that code-switching behaviors in reasoning models can be modified by fine-tuning for tasks that do not directly demonstrate code-switching in reasoning (e.g., machine translation). Our work suggests that data-efficient interventions can instill helpful forms of code-switching behavior in reasoning models.

Eleanor M. Lin, David Jurgens

University of Michigan

Multilingual Language Models Code-Switching Cross-Lingual Reasoning in Language Models LLM Fine-Tuning Large Reasoning Models

Apr 16, 2026cs.AR

Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation

Recent advances in large language models have improved code generation, but their use in hardware description languages is still limited. Moreover, training data and testbenches for these models are often scarce. This paper presents a workflow that uses multi-agent models to generate testbenches for high-quality fine-tuning data. By automating testbench creation, the fine-tuned model for the specification-to-Verilog task achieves performance comparable to state-of-the-art methods on the refined VerilogEval v2 benchmark while using less training data. This study provides a basis for future work on LLM-based HDL generation and automated verification.

Mu-Chi Chen, Po-Hsuan Huang, Yu-Hung Kao +6

Academia Sinica, Taipei, Taiwan · National Taiwan University, Taipei, Taiwan · National Cheng Kung University, Tainan, Taiwan

LLM Fine-Tuning Code Generation

Apr 16, 2026cs.CL

Fact4ac at the Financial Misinformation Detection Challenge Task: Reference-Free Financial Misinformation Detection via Fine-Tuning and Few-Shot Prompting of Large Language Models

The proliferation of financial misinformation poses a severe threat to market stability and investor trust, misleading market behavior and creating critical information asymmetry. Detecting such misleading narratives is inherently challenging, particularly in real-world scenarios where external evidence or supplementary references for cross-verification are strictly unavailable. This paper presents our winning methodology for the "Reference-Free Financial Misinformation Detection" shared task. Built upon the recently proposed RFC-BENCH framework (Jiang et al. 2026), this task challenges models to determine the veracity of financial claims by relying solely on internal semantic understanding and contextual consistency, rather than external fact-checking. To address this formidable evaluation setup, we propose a comprehensive framework that capitalizes on the reasoning capabilities of state-of-the-art Large Language Models (LLMs). Our approach systematically integrates in-context learning, specifically zero-shot and few-shot prompting strategies, with Parameter-Efficient Fine-Tuning (PEFT) via Low-Rank Adaptation (LoRA) to optimally align the models with the subtle linguistic cues of financial manipulation. Our proposed system demonstrated superior efficacy, successfully securing the first-place ranking on both official leaderboards. Specifically, we achieved an accuracy of 95.4% on the public test set and 96.3% on the private test set, highlighting the robustness of our method and contributing to the acceleration of context-aware misinformation detection in financial Natural Language Processing. Our models (14B and 32B) are available at https://huggingface.co/KaiNKaiho.

Cuong Hoang, Le-Minh Nguyen

Japan Advanced Institute of Science and Technology

LLM Fine-Tuning Automated Fact-Checking Finance

Apr 9, 2026cs.LG

Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domains

Large language models are increasingly deployed in high-stakes domains, where confident yet incorrect inferences may cause severe real-world harm, bringing the long-overlooked issue of confidence faithfulness to the forefront. A promising solution jointly optimizes unsupervised Reinforcement Learning from Internal Feedback (RLIF) with reasoning-trace-guided Reasoning Distillation (RD), yet it faces three persistent challenges, namely the scarcity of high-quality training corpora, factually unwarranted overconfidence, and erroneous updates amplified by indiscriminate fusion. Inspired by how human confidence accumulates from uncertainty to certainty, we propose Progressive Reasoning Gain (PRG) to measure whether reasoning steps progressively strengthen confidence in the final answer. Building on PRG, we introduce HyTuning, a hybrid post-training framework that adaptively reweights RD and RLIF, using scarce supervised reasoning traces as a stable anchor while exploiting abundant unlabeled queries for scalability. Experiments on several domain-specific and general benchmarks demonstrate that HyTuning improves accuracy while achieving confidence faithfulness under limited supervision, supporting a practical ``Less Data Approximates More'' effect. Our code will be released upon acceptance.

Haokai Ma, Lee Yan Zhen, Gang Yang +4

National University of Singapore, Singapore · Singapore Management University, Singapore

RL for Language Models Reinforcement Learning Reasoning Distillation Confidence Estimation in Language Models LLM Fine-Tuning Language Model Calibration

Apr 2, 2026cs.CL

Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Models

Retrieval-Augmented Language Models (RALMs) have shown strong potential in knowledge-intensive tasks, yet they remain vulnerable when retrieved contexts are noisy or irrelevant. Robustness against such contexts requires two distinct capabilities: abstention when contexts are uninformative, and selective extraction when relevant evidence is buried in noise. Yet existing methods face two key limitations: they do not train separately for these two capabilities, and they adapt the model at a coarse layer- or module-level granularity, overlooking that only a small subset of neurons is strongly activated for a given input. We propose NeuRIT, a Neuron-guided Robust Instruction-Tuning framework built on a localization-first perspective. NeuRIT mines context-aware neurons associated with relevant and irrelevant context processing, and uses them as anchors to selectively adapt both the identified neuron groups and the layers in which they concentrate. NeuRIT then performs two-stage instruction tuning that teaches complementary behaviors: suppress generation when there is nothing to extract, and extract relevant evidence when there is. NeuRIT consistently outperforms strong baselines across diverse QA benchmarks and generator backbones. Our code is available at https://github.com/HYU-ARK-Lab/NeuRIT.

Jae O Lee, Jaemin Kim, Sumyeong Ahn +1

1Hanyang University · 2KENTECH

Retrieval-Augmented Generation Fine-Tuning Knowledge Augmentation for Language Models LLM Fine-Tuning Neural Network Robustness Retrieval Robustness+2

Apr 2, 2026cs.CL

Adam's Law: Textual Frequency Law on Large Language Models

While textual frequency has been validated as relevant to human cognition in reading speed, its relatedness to Large Language Models (LLMs) is seldom studied. We propose a novel research direction in terms of textual data frequency, which is an understudied topic, to the best of our knowledge. Our framework is composed of three units. First, this paper proposes Textual Frequency Law (TFL), which indicates that frequent textual data should be preferred for LLMs for both prompting and fine-tuning. Since many LLMs are closed-source in their training data, we propose using online resources to estimate the sentence-level frequency. We then utilize an input paraphraser to paraphrase the input into a more frequent textual expression. Next, we propose Textual Frequency Distillation (TFD) by querying LLMs to conduct story completion by further extending the sentences in the datasets, and the resulting corpora are used to adjust the initial estimation. Finally, we propose Curriculum Textual Frequency Training (CTFT) that fine-tunes LLMs in an increasing order of sentence-level frequency. Experiments are conducted on our curated dataset Textual Frequency Paired Dataset (TFPD) on math reasoning, machine translation, commonsense reasoning and agentic tool calling. Results show the effectiveness of our framework.

Hongyuan Adam Lu, Z. L., Victor Wei +5

FaceMind Corporation · The Chinese University of Hong Kong

LLM Prompting LLM Fine-Tuning LLM Training

Apr 2, 2026cs.LG

Contrast Matters: Understanding Robustness of In-Context Fine-Tuning to Target-Context Relatedness

In-context fine-tuning (IC-Train), training an LLM with labeled examples in-context, is increasingly used in place of standard fine-tuning for domain adaptation and continual absorption of labeled data. We study the robustness of the in-context learning ability that emerges from such training: does the fine-tuned model perform well across test inputs whose in-context examples range from unrelated to nearly identical? Across 32 configurations spanning four open-source LLMs and eight test sets over machine translation, Text-to-SQL, and multilingual semantic parsing, we show that robustness hinges on an overlooked design choice: how in-context examples are selected relative to the target during training. The two prevailing strategies turn out to be accurate over complementary parts of this spectrum: random contexts yield a model that gains little from related examples even when they are placed in its context, while retrieved similar contexts weaken accuracy on targets lacking close neighbors and raise the propensity to copy labels from context. Probes tracking in-weights learning, in-context learning, and copying trace these failures to distinct training dynamics, and show that introducing contrast in target-context similarity both within a context and across batches, restores robustness across the entire spectrum.

Deeptanshu Malu, Deevyanshu Malu, Aditya Nemiwal +1

IIT Bombay

In-Context Example Selection LLM Fine-Tuning In-Context Learning Language Model Robustness

Mar 21, 2026cs.CL

Alignment Whack-a-Mole : Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models

Frontier LLM companies have repeatedly assured courts and regulators that their models do not store copies of training data. They further rely on safety alignment strategies via RLHF, system prompts, and output filters to block verbatim regurgitation of copyrighted works, and have cited the efficacy of these measures in their legal defenses against copyright infringement claims. We show that finetuning bypasses these protections: by training models to expand plot summaries into full text, a task naturally suited for commercial writing assistants, we cause GPT-4o, Gemini-2.5-Pro, and DeepSeek-V3.1 to reproduce up to 85-90% of held-out copyrighted books, with single verbatim spans exceeding 460 words, using only semantic descriptions as prompts and no actual book text. This extraction generalizes across authors: finetuning exclusively on Haruki Murakami's novels unlocks verbatim recall of copyrighted books from over 30 unrelated authors. The effect is not specific to any training author or corpus: random author pairs and public-domain finetuning data produce comparable extraction, while finetuning on synthetic text yields near-zero extraction, indicating that finetuning on individual authors' works reactivates latent memorization from pretraining. Three models from different providers memorize the same books in the same regions (

r \ge 0.90

), pointing to an industry-wide vulnerability. Our findings offer compelling evidence that model weights store copies of copyrighted works and that the security failures that manifest after finetuning on individual authors' works undermine a key premise of recent fair use rulings, where courts have conditioned favorable outcomes on the adequacy of measures preventing reproduction of protected expression.

Xinyue Liu, Niloofar Mireshghallah, Jane C. Ginsburg +1

Stony Brook University · Carnegie Mellon University · Columbia Law School

LLM Alignment Memorization in Language Models LLM Fine-Tuning Memorization in Generative Models

Mar 17, 2026cs.DC

An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU

Fine-tuning Large Language Models (LLMs) has become essential for domain adaptation, but its memory-intensive property exceeds the capabilities of most GPUs. To address this challenge and democratize LLM fine-tuning, we present SlideFormer, a novel system designed for single-GPU environments. Our innovations are: (1) A lightweight asynchronous engine that treats the GPU as a sliding window and overlaps GPU computation with CPU updates and multi-tier I/O. (2) A highly efficient heterogeneous memory management scheme significantly reduces peak memory usage. (3) Optimized Triton kernels to solve key bottlenecks and integrated advanced I/O. This collaborative design enables fine-tuning of the latest 123B+ models on a single RTX 4090, supporting up to 8x larger batch sizes and 6x larger models. In evaluations, SlideFormer achieves 1.40x to 6.27x higher throughput while roughly halving CPU/GPU memory usage compared to baselines, sustaining >95% peak performance on both NVIDIA and AMD GPUs.The code is available at https://github.com/RegiaYoung/SlideFormer.

Ruijia Yang, Zeyi Wen

Fine-Tuning Memory-Efficient Fine-Tuning GPU Kernel Optimization GPU Acceleration LLM Fine-Tuning Heterogeneous Computing

Mar 14, 2026cs.LG

Aligning the Query Space: Greedy Information Projection for Language Model Data Selection

Data selection for language models is often framed as balancing example quality and diversity. We argue that both are consequences of a more fundamental principle: selected examples should preserve the downstream query space induced by instructions, task signals, or retrieval needs. We present Greedy Information Projection (GIP), a query-aligned method that requires only candidate embeddings and a score signal from LLM judgments, metadata, or intrinsic geometry. GIP greedily selects examples whose embedding span explains the largest residual component of task/query scores, yielding a fast matching-pursuit selector. A Gaussian projection view connects this update to maximizing mutual information, equivalently minimizing the residual volume of the query subspace left unexplained by selected data; this explains how quality and diversity emerge from one objective. Empirically, GIP matches or surpasses full-data fine-tuning using small instruction and reasoning subsets, while pretraining and RAG passage-selection experiments show that the same residual principle transfers beyond supervised fine-tuning.

Victor Ye Dong, Kuan-Yun Lee, Jiamei Shuai +3

Microsoft

LLM Fine-Tuning Instruction-Tuning Data Selection Data Selection Training Data Selection Mutual Information Maximization

Mar 3, 2026cs.AI

TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learning

Large language models (LLMs) are increasingly used to assist scientists across diverse workflows. A key challenge is generating high-quality figures from textual descriptions, often represented as TikZ programs that can be rendered as scientific images. Prior research has proposed a variety of datasets and modeling approaches for this task. However, existing datasets for Text-to-TikZ are too small and noisy to capture the complexity of TikZ, causing mismatches between text and rendered figures. Moreover, prior approaches rely solely on supervised fine-tuning (SFT), which does not expose the model to the rendered semantics of the figure, often resulting in errors such as looping, irrelevant content, and incorrect spatial relations. To address these issues, we construct DaTikZ-V4, a dataset more than four times larger and substantially higher in quality than DaTikZ-V3, enriched with LLM-generated figure descriptions. Using this dataset, we train TikZilla, a family of small open-source Qwen models (3B and 8B) with a two-stage pipeline of SFT followed by reinforcement learning (RL). For RL, we leverage an image encoder trained via inverse graphics to provide semantically faithful reward signals. Extensive human evaluations with over 1,000 judgments show that TikZilla improves by 1.5-2 points over its base models on a 5-point scale, surpasses GPT-4o by 0.5 points, and matches GPT-5 in the image-based evaluation, while operating at much smaller model sizes.

Christian Greisinger, Steffen Eger

University of Technology Nuremberg

LLM Fine-Tuning

Feb 20, 2026cs.LG

SeedFlood: A Step Toward Scalable Decentralized Fine-Tuning of LLMs

This work presents SeedFlood, a new approach to decentralized LLM fine-tuning designed to scale across large models, large collaborations, and complex network topologies while achieving global consensus with negligible communication overhead. Traditional methods suffer from high communication costs that grow with model size, while information decay over network hops renders global consensus inefficient. SeedFlood takes a significant departure from these practices by exploiting the seed-reconstructible structure of zeroth-order gradients and effectively making the messages to transmit near-zero in size, allowing them to be flooded to every client in the network, and thereby enhancing scalability of decentralized training. Consequently, SeedFlood enables training in regimes previously considered impractical, such as billion-parameter scale models or distributed across hundred of clients. Our experiments on decentralized LLM fine-tuning demonstrate that SeedFlood consistently outperforms the standard zeroth-order baselines in both communication efficiency and generalization performance, and even achieves results comparable to first-order gossip-based methods in large-scale settings, while requiring orders-of-magnitude less communication cost. We also provide theoretical analysis to formalize that SeedFlood avoids topology-dependent consensus terms in the convergence bound while retaining the acceleration enabled by increased client participation.

Jihun Kim, Dongyeop Lee, Namhoon Lee

POSTECH, pohang, Republic of Korea.

Communication-Efficient Distributed Training Zeroth-Order Optimization LLM Fine-Tuning Decentralized Learning

Feb 16, 2026cs.CV

How to Train Your Long-Context Visual Document Model

We present the first comprehensive, large-scale study of training long-context vision language models up to 344K context, targeting long-document visual question answering with measured transfer to long-context text. While several such strong are open-weight, namely Qwen3 VL and GLM 4.5/6V, their training recipes and data pipelines are not reproducible. We systematically study continued pretraining, supervised finetuning, and preference optimization for 24B and 32B parameter models, backed by extensive LC evaluations and ablations to bridge this gap, and achieve state-of-the-art performance on MMLongBenchDoc for both parameter scales. In addition to this, our key findings include: (i) training on context lengths that match evaluation context lengths outperforms training on longer contexts, (ii) training and evaluating with page indices provides a simple, high-impact boost to long-document performance, (iii) our synthetic data pipelines enable self-improvement via continued pretraining and supervised finetuning, and (iv) we extend the known text-to-visual long context transfer to the reverse, showing that visual long context training transfers to long-context text performance. We also release MMLBD-C, a manually corrected version of MMLongBenchDoc to reduce erroneous and low quality examples in the benchmark.

Austin Veselka

LightOn

Visual Question Answering Vision-Language Models LLM Fine-Tuning Long-Context Modeling Vision-Language Pretraining Long-Context VLMs

Feb 16, 2026cs.CL

Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets

Large Language Models (LLMs) have seen remarkable advancements, achieving state-of-the-art results in diverse applications. Fine-tuning, an important step for adapting LLMs to specific downstream tasks, typically involves further training on corresponding datasets. However, a fundamental discrepancy exists between current fine-tuning datasets and the token-level optimization mechanism of LLMs: most datasets are designed at the sentence-level, which introduces token-level noise, causing negative influence to final performance. In this paper, we propose XTF, an explainable token-level noise filtering framework. XTF decomposes the complex and subtle contributions of token-level data to the fine-tuning process into three distinct and explicit attributes (reasoning importance, knowledge novelty, and task relevance), which can be assessed using scoring methods, and then masks the gradients of selected noisy tokens accordingly to optimize the performance of fine-tuned LLMs. We conduct extensive experiments on three representative downstream tasks (math, code and medicine) across 7 mainstream LLMs. The results demonstrate that XTF can significantly improve downstream performance by up to 13.7% compared to regular fine-tuning. Our work highlights the importance of token-level dataset optimization, and demonstrates the potential of strategies based on attribute decomposition for explaining complex training mechanisms.

Yuchen Yang, Wenze Lin, Enhao Huang +6

1The State Key Laboratory of Blockchain and Data Security, Zhejiang University · 2Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China · 3Qiuzhen College, Tsinghua University +2

LLM Fine-Tuning Training Data Curation Training Data Selection

Feb 12, 2026cs.LG

The Appeal and Reality of Recycling LoRAs with Adaptive Merging

The widespread availability of fine-tuned LoRA modules for open pre-trained models has led to an interest in methods that can adaptively merge LoRAs to improve performance. These methods typically include some way of selecting LoRAs from a pool and tune merging coefficients based on a task-specific dataset. While adaptive merging methods have demonstrated improvements in some settings, no past work has attempted to recycle LoRAs found "in the wild" on model repositories like the Hugging Face Hub. To address this gap, we consider recycling from a pool of nearly 1,000 user-contributed LoRAs trained from the Llama 3.1 8B-Instruct language model. Our empirical study includes a range of adaptive and non-adaptive merging methods in addition to a new method designed via a wide search over the methodological design space. We demonstrate that adaptive merging methods can improve performance over the base model but provide limited benefit over training a new LoRA on the same data used to set merging coefficients. We additionally find not only that the specific choice of LoRAs to merge has little importance, but that using LoRAs with randomly initialized parameter values yields similar performance. This raises the possibility that adaptive merging from recycled LoRAs primarily works via some kind of regularization effect, rather than by enabling positive cross-task transfer. To better understand why past work has proven successful, we confirm that positive transfer is indeed possible when there are highly relevant LoRAs in the pool. We release the model checkpoints and code online.

Haokun Liu, Gyung Hyun Je, Marco Ciccone +3

University of Toronto, Canada · Vector Institute, Toronto, Canada · Mistral AI (work done before joining) +2

LLM Fine-Tuning Language Model Merging Model Merging Parameter-Efficient Fine-Tuning

Feb 11, 2026cs.LG

Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT

Large language models are commonly adapted to downstream tasks through supervised fine-tuning (SFT), but substantial distribution mismatch between downstream supervision and a model's generation distribution can intensify catastrophic forgetting. Data rewriting offers a data-centric way to narrow this mismatch before SFT. Existing methods, however, typically sample rewrites from a prompt-induced conditional distribution, which need not align with the backbone's natural question-answering generation distribution, and fixed templates can reduce output diversity. We formulate data rewriting as a policy-learning problem and train a lightweight LoRA rewriting policy with reinforcement learning. The policy optimizes question-answering-style distributional alignment and semantic diversity under a hard task-consistency gate, producing verified supervision for downstream SFT. Across three instruction-tuned backbones, the resulting models attain downstream gains broadly comparable to standard SFT while reducing degradation on non-downstream benchmarks in every evaluated setting. Additional experiments on logical reasoning and medical question answering provide preliminary evidence that a rewriting policy can be reused across domains for the same backbone.

Jiacheng Wang, Zhijie Liu, Ping Jian +4

Beijing Institute of Technology, Beijing, China · Tsinghua University, Beijing, China

Supervised Fine-Tuning Reinforcement Learning LLM Fine-Tuning Catastrophic Forgetting

Feb 1, 2026cs.CL

The Role of Dataset Linguistic Structure in the Cultural Awareness of Large Language Models

The global deployment of large language models (LLMs) has raised concerns about cultural misalignment, yet the linguistic properties of fine-tuning datasets used for cultural adaptation remain poorly understood. We adopt a dataset-centric view of cultural alignment and investigate which properties of post-training data are associated with cultural performance, whether they can guide data selection before fine-tuning, and how their effects vary across languages and model families. We compute lightweight linguistic, semantic, and structural metrics for Arabic, Chinese, and Japanese datasets and apply principal component analysis (PCA) separately within each language. The resulting components form broadly interpretable axes: PC1 is generally dominated by semantic structure, PC2 captures diversity and lexical variation, and PC3 reflects more language-specific organization. We fine-tune LLaMA, Mistral, and DeepSeek models and evaluate them on benchmarks of cultural knowledge, values, and norms. Although the PCA-derived dataset descriptors are associated with downstream performance, the strongest relationships vary across models, benchmarks, and languages, indicating that no single component serves as a universal predictor. Controlled, size-matched subset interventions further show that PCA-guided selection can improve cultural performance when the relevant component and direction are validated against random sampling. PC3 provides the strongest signal for Arabic, while High-PC1 is most effective for Japanese, particularly for LLaMA. Chinese results are weaker and more model-specific and remain exploratory because of smaller subset sizes. Overall, our findings show that lightweight dataset descriptors can support pre-training data diagnostics, but effective cultural adaptation requires language- and architecture-aware selection rather than a universal linguistic criterion.

Reem I. Masoud, Chen Feng, Shunta Asano +3

University College London · King Abdulaziz University · Queen’s University Belfast +3

LLM Fine-Tuning Cultural Alignment Cultural Knowledge in Language Models

Feb 1, 2026cs.CL

Distilling Token-Trained Models into Byte-Level Models

Byte Language Models (BLMs) have emerged as a promising direction for scaling language models beyond tokenization. However, existing BLMs typically require training from scratch on trillions of bytes, making them prohibitively expensive. In this paper, we propose an efficient distillation recipe that converts existing token-trained LLMs into BLMs while retaining comparable capabilities. Our recipe follows a two-stage curriculum: (1) Progressive Knowledge Distillation, which aligns byte-level representations with the embeddings of the token-trained teacher model; and (2) Byte-Level Supervised Fine-Tuning, which enables end-to-end generation entirely in the byte space. We validate our approach across multiple model families, including Llama, Qwen, and OLMo, and demonstrate that the distilled BLMs retain most of the teacher models' performance using only approximately 125B bytes.

Zishuo Bao, Jiaqi Leng, Junxiong Wang +2

Fuzhou University · NYU Shanghai · Fudan University +2

LLM Fine-Tuning Byte-Level Language Model Language Model Distillation Language Modeling

Jan 28, 2026cs.AI

Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models

Large Language Models usually put more emphasis on accuracy and therefore, will guess even when not certain about the prediction, which is especially severe when fine-tuned on small datasets due to the inherent tendency toward miscalibration. In this work, we introduce Bayesian-LoRA, which reformulates the deterministic LoRA update as a probabilistic low-rank representation inspired by Sparse Gaussian Processes. We identify a structural isomorphism between LoRA's factorization and Kronecker-factored SGP posteriors, and show that LoRA emerges as a limiting case when posterior uncertainty collapses. We conduct extensive experiments on various LLM architectures across commonsense reasoning benchmarks. With only approximately 0.42M additional parameters and

{\approx}1.2{\times}

training cost relative to standard LoRA, Bayesian-LoRA significantly improves calibration across models up to 30B, achieving up to 84% ECE reduction and 76% NLL reduction while maintaining competitive accuracy for both in-distribution and out-of-distribution (OoD) evaluations.

Moule Lin, Shuhao Guan, Andrea Patane +2

School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland and Lero the Research Ireland Centre for Software, Ireland · School of Computer Science, University College Dublin, Dublin, Ireland

Bayesian Neural Networks Confidence Estimation in Language Models LLM Fine-Tuning Low-Rank Adaptation Language Model Calibration LLM Reliability+2

Jan 21, 2026cs.LG

Counterfactual Modeling with Fine-Tuned LLMs for Health Intervention Design and Sensor Data Augmentation

Counterfactual explanations (CFEs) provide human-centric interpretability by identifying the minimal, actionable changes required to alter a machine learning model's prediction. Therefore, CFs can be used as (i) interventions for abnormality prevention and (ii) augmented data for training robust models. We conduct a comprehensive evaluation of CF generation using large language models (LLMs), including GPT-4 (zero-shot and few-shot) and two open-source models-BioMistral-7B and LLaMA-3.1-8B, in both pretrained and fine-tuned configurations. Using the multimodal AI-READI clinical dataset, we assess CFs across three dimensions: intervention quality, feature diversity, and augmentation effectiveness. Fine-tuned LLMs, particularly LLaMA-3.1-8B, produce CFs with high plausibility (up to 99%), strong validity (up to 0.99), and realistic, behaviorally modifiable feature adjustments. When used for data augmentation under controlled label-scarcity settings, LLM-generated CFs substantially restore classifier performance, yielding an average 20% F1 recovery across three scarcity scenarios. Compared with optimization-based baselines such as DiCE, CFNOW, and NICE, LLMs offer a flexible, model-agnostic approach that generates more clinically actionable and semantically coherent counterfactuals. Overall, this work demonstrates the promise of LLM-driven counterfactuals for both interpretable intervention design and data-efficient model training in sensor-based digital health. Impact: SenseCF fine-tunes an LLM to generate valid, representative counterfactual explanations and supplement minority class in an imbalanced dataset for improving model training and boosting model robustness and predictive performance

Shovito Barua Soumma, Asiful Arefeen, Stephanie M. Carpenter +2

College of Health Solutions, Arizona State University, Phoenix, AZ 85004, USA. · School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ 85281, USA. · School of Nutritional Sciences and Wellness, University of Arizona, Tucson, AZ 85721, USA.

Counterfactual Explanations LLM Fine-Tuning Counterfactual Reasoning in Language Models Counterfactual Data Augmentation Medical LLMs Counterfactual Learning

Jan 8, 2026cs.CL

Thunder-KoNUBench: A Corpus-Aligned Benchmark for Korean Negation Understanding

Although negation is known to challenge large language models (LLMs), benchmarks for evaluating negation understanding-especially in Korean-are scarce. We conduct a corpus-based analysis of Korean negation and show that LLM performance degrades under negation. We then introduce Thunder-KoNUBench, a sentence-level negation understanding benchmark that reflects the empirical distribution of Korean negation phenomena. Evaluating 47 LLMs on Thunder-KoNUBench, we analyze the effects of model size and instruction tuning, and perform error analysis to better understand model behavior. We further show that fine-tuning on Thunder-KoNUBench improves negation understanding and broader contextual comprehension in Korean.

Sungmok Jung, Yeonkyoung So, Joonhak Lee +3

Graduate School of Data Science, Seoul National University · Dept. of Computer Science and Engineering, Seoul National University

LLM Evaluation LLM Fine-Tuning

Dec 19, 2025quant-ph

Shuttling Compiler for Trapped-Ion Quantum Computers Based on Fine-Tuned Large Language Models

In trapped-ion quantum computers, qubits must be shuttled between segments to interact. The routing logic that schedules these movements is written by hand for every new trap architecture. We present shuttling compilers based on five large language models (LLMs). Each LLM is fine-tuned on shuttling schedules produced by hand-coded heuristics for linear and branched one-dimensional trap architectures. We investigate how the shuttling operation counts of their schedules compare with those of the heuristics and how far they generalize to unseen architectures. For circuits of up to 16 qubits, the fine-tuned LLMs generate valid schedules on both training architectures, more often the fewer qubits a circuit has. In 12% of the compilations yielding a schedule, the best of ten runs needs up to 21% fewer operations than the heuristic baselines, after a rule-based post-processing step. A single run of one fine-tuned LLM produces a valid shuttling schedule for a previously unseen four-way branched architecture. This is preliminary evidence of cross-architecture generalization. On two other unseen architectures no LLM produces a valid schedule. Thus, LLM-learned shuttling compilation is feasible, and we show how far it currently reaches.

Fabian Kreppel, Reza Salkhordeh, Ferdinand Schmidt-Kaler +1

Institute of Computer Science, Johannes Gutenberg University, Staudingerweg 9, 55128 Mainz, Germany · Institute of Physics, Johannes Gutenberg University, Staudingerweg 7, 55128 Mainz, Germany · Department of Computer Science, Saarland University, Saarland Informatics Campus, E1 1, 66123 Saarbrücken, Germany

Quantum Circuit Compilation LLM Fine-Tuning Large Language Model-Guided Optimization

Nov 26, 2025cs.LG

Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning

Engineering LLMs to accelerate life sciences research requires a robust alignment with biomedical knowledge. We observe that biomedical text exhibits a fundamentally different uncertainty structure from general text: dense low-confidence runs encode epistemic knowledge gaps (dense causal chains, rare entities) rather than the sparse aleatoric stylistic variation typical of general text. Based on this discovery, we propose Balanced Fine-Tuning (BFT), a dual-scale post-training method that combines group-normalized token reweighting with sequence-level reallocation toward knowledge-dense samples exhibiting dense epistemic uncertainty. Across medical evaluation, biological reasoning, sparse-reward RL, and biological representation tasks, BFT provides more consistent gains than SFT and DFT under a shared training setup. When replacing the default closed-source backbones in GeneAgent (GPT-4o) and VCWorld (Gemini-2.5-Flash), the BFT-aligned 70B model delivers stronger performance across biological process reasoning and chemical perturbation prediction. Critically, all BFT variants further improve after subsequent GRPO with sparse rewards, while SFT and DFT degrade, suggesting that epistemic-aware post-training provides a more robust policy initialization. Beyond text generation, BFT-aligned LLMs produce more accurate and professional biomedical profile texts; after encoding these profiles with a text embedding model, the resulting representations support gene-level, cell-level, and perturbation-response tasks, suggesting that BFT-enhanced generation can facilitate biological representation and, in turn, broader biomedical downstream tasks.

Zhenchao Tang, Fang Wang, Haohuai He +13

Tencent AI for Life Sciences Lab, Shenzhen, China. · Sun Yat-sen University, Shenzhen, China. · The Hong Kong Polytechnic University, Hong Kong SAR, China. +2

LLM Alignment LLM Fine-Tuning LLM Uncertainty Estimation LLM Post-Training Medical LLMs

Nov 26, 2025cs.LG

Bilevel Data Curation for LLM Fine-tuning: Offline Selection and Online Self-Refining Generation

Supervised fine-tuning (SFT) datasets are critical to the downstream performance of large language models, yet they often contain low-quality or harmful question-response pairs. To improve SFT data quality, we develop a unified bilevel framework that combines offline data selection with the online self-refining generation. In the offline setting, bilevel data selection (BDS) selects question-response pairs from the offline SFT dataset to maximize the validation performance. We theoretically show that the optimal model given by BDS outperforms direct data mixing approach in useful data coverage. Moreover, we provide a global convergence analysis for gradient-based BDS approach for one-layer Transformer, showing that the epsilon-global optimum of offline BDS is achievable in finite time. Although efficient, offline BDS discards potentially harmful questions together with responses, thereby reducing question diversity. We address this limitation by refining the responses to selected questions using online self-refining generation framework. However, BDS is inefficient to update the response weights when responses are regenerated online. To address this issue, we introduce bilevel multi-objective optimization (BMO) for response-level weighting. We show that BMO recovers the same validation-aligned solution as BDS, but admits a closed-form importance-ratio weight that adapts to regenerated responses. Experiments on LLM quality enhancement and safety-aware fine-tuning demonstrate that the proposed framework consistently improves both data quality and downstream fine-tuning performance.

Quan Xiao, Yutong Xuan, Gaowen Liu +2

Cornell University · Cisco Research

Supervised Fine-Tuning LLM Self-Refinement LLM Fine-Tuning Bilevel Optimization

Nov 23, 2025cs.LG

Fine-Tune, Then Rectify

Driven by recent advances in artificial intelligence, a growing literature has demonstrated the potential of using large language models (LLMs) as scalable surrogates to generate human-like responses. Two common approaches to improve the performance of LLMs include: fine-tuning, which aligns the LLM more closely with human responses, and rectification, which corrects biases in LLM outputs. In this paper, we develop a two-stage framework that combines fine-tuning and rectification, and optimally allocates limited labeled samples across the two stages. A key insight is that the conventional fine-tuning objective of minimizing mean squared prediction error is generally not aligned with the downstream rectification stage. For mean estimation, we propose to minimize the variance of the prediction errors; for general M-estimation, we propose to minimize a scalarized variance metric as the fine-tuning objective. Building on this insight, we leverage the scaling law of fine-tuning to optimally allocate the limited labeled human data between the fine-tuning and rectification stages. Our empirical analysis validates the fine-tuning scaling law and confirms that our proposed optimal allocation rule reliably identifies the optimal sample allocation. We demonstrate substantial efficiency gains in estimation and inference performance relative to fine-tuning or rectification alone, or to employing the conventional mean squared error objective within the fine-tuning then rectification framework. Such efficiency gains translate to significant cost savings for making reliable decisions.

Zikun Ye, Jinglong Zhao, Lei Wang

Foster School of Business, University of Washington, Seattle, WA · Questrom School of Business, Boston University, Boston, MA

LLM Fine-Tuning

Nov 8, 2025cs.CL

Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation

Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs). Meanwhile, fine-tuning remains the default operation for adapting LLMs to new domains and tasks. Despite their widespread adoption, these two post-training interventions have been studied in isolation, leaving open a crucial question: if we fine-tune an edited model, do the edits survive? This question is motivated by practical objectives: removing covert or malicious edits, and preserving beneficial edits. If fine-tuning impairs edits (Fig.1), current KE methods become less efficient, as a newly fine-tuned model requires re-editing; if edits persist, fine-tuned models risk propagating hidden malicious edits, raising serious safety concerns. To this end, we systematically quantify edit decay after fine-tuning across 254 experimental configurations. Our results show that in general, edits decay substantially after subsequent fine-tuning. AlphaEdit exhibits the greatest decay on the zsRE benchmark when applied to GPT-J, where 25.27% of previously successful edits become unsuccessful after fine-tuning. We further find that fine-tuning only the edited layers is sufficient to effectively remove edits, while incurring only modest degradation in downstream performance. Surprisingly, fine-tuning non-edited layers leads to greater edit decay than all-layer fine-tuning. Besides, our activation space analysis reveals that fine-tuning produces a larger and more coherent representational shift, both in magnitude and direction, than KE. Overall, our study underscores the necessity of evaluating KE within the broader LLM application pipeline.

Yinjie Cheng, Paul Youssef, Christin Seifert +2

School of Computer Science University of Sheffield Sheffield, UK · Mathematics and Computer Science Marburg University Marburg, Germany

Knowledge Editing LLM Fine-Tuning Model Editing Language Model Editing