LLM Fine-Tuning

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

42 papers in the last four weeks, up 83% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 509

All topics

HPC-LLM: Practical Domain Adaptation and Retrieval-Augmented Generation for HPC Support
May 8, 2026Nourin Shahin, Izzat AlsmadiRetrieval-Augmented Generation Domain Adaptation
Texas A&M University San Antonio
PSK@EEUCA 2026: Fine-Tuning Large Language Models with Synthetic Data Augmentation for Multi-Class Toxicity Detection in Gaming Chat
May 8, 2026Srikar Kashyap PulipakaLLM Fine-Tuning Toxicity Detection
Independent Researcher
The Position Curse: LLMs Struggle to Locate the Last Few Items in a List
May 8, 2026Zhanqi Zhang, Hua-Dong Xiong, Robert C. Wilson +3LLM Evaluation LLM Fine-Tuning
UC San Diego · Georgia Tech · New York University
Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation
May 8, 2026Haozhan Tang, Xiuqi Zhu, Xinyin Zhang +3LLM Fine-Tuning Low-Rank Adaptation
Carnegie Mellon University · Tsinghua University · Infinigence AI
Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
May 7, 2026Yuxing Liu, Jianyu Wang, Tong ZhangSupervised Fine-Tuning LLM Fine-Tuning
1UIUC∗ · 2Apple
FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning
May 7, 2026Su Zhang, Junfeng Guo, Heng HuangPrivacy-Preserving Language Models LLM Fine-Tuning
Department of Computer Science, University of Maryland, College Park
PACZero: PAC-Private Fine-Tuning of Language Models via Sign Quantization
May 7, 2026Murat Bilgehan Ertan, Xiaochen Zhu, Phuong Ha Nguyen +2Privacy-Preserving Language Models LLM Fine-Tuning
CWI Amsterdam · ∗Affiliated with Vrije Universiteit Amsterdam. · MIT +1
Fine-Tuning Small Language Models for Solution-Oriented Windows Event Log Analysis
May 7, 2026Siraaj Akhtar, Saad Khan, Simon ParkinsonLLM Fine-Tuning Small Language Models
LLM Translation of Compiler Intermediate Representation
May 7, 2026Andrea Valenzuela Ramirez, Cristian Gutierrez-Gomez, Marta Barroso +2Code Translation LLM Fine-Tuning
Barcelona Supercomputing Center, Universitat Politècnica de Catalunya · Barcelona Supercomputing Center
Rethinking Adapter Placement: A Dominant Adaptation Module Perspective
May 7, 2026Suoxin Zhang, Run He, Di Fang +3LLM Fine-Tuning Adapter Tuning
South China University of Technology, China · Zhejiang University, China
One Algorithm, Two Goals: Dual Scoring for Parameter and Data Selection in LLM Fine-Tuning
May 7, 2026Xinrui Chen, Liu Yang, Ou WuSupervised Fine-Tuning LLM Fine-Tuning
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences · College of Intelligence and Computing, Tianjin University
BoostLLM: Boosting-inspired LLM Fine-tuning for Few-shot Tabular Classification
May 7, 2026Yi-Siang Wang, Kuan-Yu Chen, Yu-Chen Den +1Ensemble Learning LLM Fine-Tuning
SinoPac Holdings
Safety Anchor: Defending Harmful Fine-tuning via Geometric Bottlenecks
May 7, 2026Guoxin Lu, Letian Sha, Qing Wang +4LLM Fine-Tuning LLM Safety Alignment
Nanjing University of Posts and Telecommunications, Nanjing, China.
Quantum-enhanced Large Language Models on Quantum Hardware via Cayley Unitary Adapters
May 7, 2026Borja Aizpurua, Sukhbinder Singh, Augustine Kshetrimayum +2LLM Inference LLM Fine-Tuning
Multiverse Computing, Parque Científico y Tecnológico de Gipuzkoa, Paseo May 7 de Miramón 170, 20014, Donostia / San Sebastián, Spain · Department of Basic Sciences, Tecnun – University of Navarra, San Sebastián, Spain · Multiverse Computing, Centre for Social Innovation, 192 Spadina Avenue, ON M5T 2C2, Toronto, Canada +2
BioTool: A Comprehensive Tool-Calling Dataset for Enhancing Biomedical Capabilities of Large Language Models
May 7, 2026Xin Gao, Ruiyi Zhang, Meixi Du +2LLM Fine-Tuning LLM Tool Use
UC San Diego · MBZUAI
HyperLens: Quantifying Cognitive Effort in LLMs with Fine-grained Confidence Trajectory
May 7, 2026Chengda Lu, Xiaoyu Fan, Wei XuConfidence Estimation in Language Models LLM Interpretability
1IIIS, Tsinghua University · 2Shanghai Qi Zhi Institute.
Decomposing the Basic Abilities of Large Language Models: Mitigating Cross-Task Interference in Multi-Task Instruct-Tuning
May 7, 2026Bing Wang, Ximing Li, Changchun Li +3LLM Fine-Tuning Gradient Interference
College of Computer Science and Technology, Jilin University · Key Laboratory of Symbolic Computation and Knowledge Engineering, Ministry of Education, Jilin University · RIKEN Center for Advanced Intelligence Project +1
I'm Sorry, but I Can't Help with Braille: Revealing Accessibility Failures in State-of-the-Art LLMs
May 7, 2026Abdullah AbdullahLLM Evaluation LLM Fine-Tuning
orinu Inc. Hwaseong, Republic of Korea
On Semantic Loss Fine-Tuning Approach for Preventing Model Collapse in Causal Reasoning
May 6, 2026Pratik Deshmukh, Atirek GuptaLLM Fine-Tuning Model Collapse
Technical University of Vienna, Austria · HCLTech, Noida, India
Attribution-Guided Continual Learning for Large Language Models
May 6, 2026Yazheng Liu, Yuxuan Wan, Rui Xu +3Continual Learning for LLMs LLM Fine-Tuning
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Beijing University of Posts and Telecommunications, Beijing, China
Adapting Large Language Models to a Low-Resource Agglutinative Language: A Comparative Study of LoRA and QLoRA for Bashkir
May 6, 2026Mullosharaf K. Arabov, Svetlana S. KhaybullinaLLM Fine-Tuning Low-Resource Language Processing
Institute of Computational Mathematics and Information Technologies, Kazan Federal University, Kazan, Russia
Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs
May 6, 2026Santosh Premi Adhikari, Radu Timofte, Dmitry IgnatovLarge Language Model-Guided NAS LLM Fine-Tuning
Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany
From Parameter Dynamics to Risk Scoring : Quantifying Sample-Level Safety Degradation in LLM Fine-tuning
May 6, 2026Xiao Wang, Yifei Zhang, YongKang Liu +4LLM Fine-Tuning LLM Safety
School of Computer Science and Engineering, Northeastern University, Shenyang, China · School of Computer and Communication Engineering, Northeastern University, Qinhuangdao, China
On Adaptivity in Zeroth-Order Optimization
May 5, 2026Hassan Dbouk, Nidham Gazagnadou, Matthias Reisser +1Zeroth-Order Optimization LLM Fine-Tuning
Qualcomm AI Research
Natural Language Processing: A Comprehensive Practical Guide from Tokenisation to RLHF
May 5, 2026Mullosharaf K. ArabovLLM Fine-Tuning Low-Resource Language Processing
KAZAN (VOLGA REGION) FEDERAL UNIVERSITY · INSTITUTE OF COMPUTATIONAL MATHEMATICS AND INFORMATION TECHNOLOGIES
Benchmarking Parameter-Efficient Fine-Tuning of Large Language Models for Low-Resource Tajik Text Generation with the Tajik Web Corpus
May 5, 2026Mullosharaf K. ArabovLLM Fine-Tuning Low-Resource Language Processing
Institute of Computational Mathematics and Information Technologies Kazan Federal University Kazan, Russia
From Code to Prediction: Fine-Tuning LLMs for Neural Network Performance Classification in NNGPT
May 5, 2026Mahmoud Hanouneh, Radu Timofte, Dmitry IgnatovLLM Fine-Tuning Automated Machine Learning
Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany
Echo-LoRA: Parameter-Efficient Fine-Tuning via Cross-Layer Representation Injection
May 5, 2026Yihang Peng, Peng Jin, Jie Gong +4LLM Fine-Tuning Low-Rank Adaptation
School of Computer Science and Software Engineering, Southwest Petroleum University · School of Electronic Information and Artificial Intelligence, Leshan Normal University
Learning Dynamics of Zeroth-Order Optimization: A Kernel Perspective
May 5, 2026Zhe Li, Bicheng Ying, Zidong Liu +1Zeroth-Order Optimization LLM Fine-Tuning
Department of Computing and Information Sciences Ph.D., Rochester Institute of Technology, Rochester, NY, USA · Google Inc., Los Angeles, CA, USA · ComboCurve Inc., Houston, TX, USA
The Right Answer, the Wrong Direction: Why Transformers Fail at Counting and How to Fix It
May 5, 2026Gabriel GarciaNumerical Reasoning in Language Models Transformer Interpretability
Independent Researcher

May 8, 2026, cs.LGJ/K move · Enter open · S save

HPC-LLM: Practical Domain Adaptation and Retrieval-Augmented Generation for HPC Support

Nourin Shahin, Izzat Alsmadi

Texas A&M University San Antonio

Retrieval-Augmented Generation Domain Adaptation High-Performance Computing LLM Fine-Tuning Domain Adaptation for LLMs

Open paper AskPDF arXiv

Modern scientific research increasingly depends on High-Performance Computing (HPC) infrastructures, yet many researchers face significant operational barriers when interacting with cluster environments, job schedulers, GPU resources, and parallel computing frameworks. General-purpose large language models (LLMs) provide useful coding assistance but often lack the domain-specific operational knowledge required for reliable HPC support. This paper presents HPC-LLM, a retrieval augmented and domain-adapted assistant designed to support common HPC workflows including Slurm scheduling, MPI execution, GPU utilization, filesystem management, and cluster troubleshooting. The proposed framework integrates automated documentation ingestion, dense retrieval, lightweight domain adaptation using QLoRA, and local inference within a modular orchestration pipeline. To support domain adaptation, we construct an HPC-oriented corpus from publicly available university HPC documentation, curated operational examples, and synthetic instruction-answer pairs generated from retrieved HPC content. The resulting dataset contains approximately 9,000 to 24,000 HPC-focused training examples spanning job scheduling, GPU computing, distributed training, storage systems, and cluster administration topics. We fine-tune Llama 3.1 8B using QLoRA and evaluate the resulting model against several open weight baselines under retrieval-augmented settings on JetStream2 infrastructure. Experimental results indicate that the adapted 8B model achieves performance comparable to substantially larger general-purpose models while operating under significantly lower GPU memory requirements and inference latency. In particular, the adapted model approaches the performance of Qwen 2.5 14B while requiring substantially fewer computational resources.

Momentum

Related topics

Latest papers 509