Synthetic Data Generation for Language Models

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 110% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 160

All topics

Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation
Apr 8, 2026Qian Ma, Sarah RajtmajerPrivacy-Preserving Language Models Synthetic Data Generation
Information Sciences and Technology, The Pennsylvania State University
QAQ: Bidirectional Semantic Coherence for Selecting High-Quality Synthetic Code Instructions
Mar 12, 2026Jiayin Lei, Ming Ma, Yunxi Duan +2Instruction-Tuning Data Selection Code Generation
Beijing University of Technology, Beijing, 100124, China · Institute of Neuroscience, State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China · University of Chinese Academy of Sciences, Beijing, 100049, China +1
MAPLE: Metadata Augmented Private Language Evolution
Feb 26, 2026Eli Chien, Yuzheng Hu, Ryan McKenna +3Privacy-Preserving Language Models Synthetic Data Generation
National Taiwan University · University of Illinois Urbana-Champaign · Google Research +1
Synthetic Interaction Data for Scalable Personalization in Large Language Models
Feb 12, 2026Yuchen Ma, Yue Huang, Wenjie Wang +3RL for Language Models LLM Personalization
Munich Center for Machine Learning & LMU Munich · University of Notre Dame · Sichuan University
FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
Jan 29, 2026Ajay Patel, Colin Raffel, Chris Callison-BurchLanguage Model Pretraining Synthetic Data Pretraining
University of Pennsylvania · University of Toronto · Vector Institute +1
LuxIT: A Luxembourgish Instruction Tuning Dataset from Monolingual Seed Data
Oct 28, 2025Julian Valline, Cedric Lothritz, Siwen Guo +1LLM Fine-Tuning Synthetic Data Generation
Luxembourg Institute of Science and Technology 5 Av. des Hauts-Fourneaux L-4362 Esch-sur-Alzette
Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence
Oct 18, 2025Bingji Yi, Qiyuan Liu, Yuwei Cheng +1Synthetic Data Pretraining Model Collapse
Independent Researcher. Work done while visiting UChicago CS. · Work done while visiting UChicago CS. · Department of Statistics, University of Chicago. +1
Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation
Oct 13, 2025Hengyuan Zhang, Shiping Yang, Xiao Liang +8Synthetic Data Generation Language Model Distillation
The University of Hong Kong · Simon Fraser University · University of California, Los Angeles +3
RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data
May 25, 2025Zhengkang Guo, Wenhao Liu, Mingchen Xie +13LLM Fine-Tuning Synthetic Data Generation
College of Computer Science and Artificial Intelligence, Fudan University · Xiaohongshu Inc.
A Factorial Study of Synthetic Data Generation for Low-Resource Machine Translation using Grammar Books
Date pendingVarun Ghat Ravikumar, Sina Ahmadi, Lena Jäger +1LLM Fine-Tuning Synthetic Data Generation
Department of Informatics, University of Zurich · Department of Computational Linguistics, University of Zurich

Apr 8, 2026, cs.CRJ/K move · Enter open · S save

Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation

Qian Ma, Sarah Rajtmajer

Information Sciences and Technology, The Pennsylvania State University

Privacy-Preserving Language Models Synthetic Data Generation Synthetic Data Generation for Language Models

Open paper AskPDF arXiv

Large language models (LLMs) have emerged as a powerful tool for synthetic data generation. A particularly important use case is producing synthetic replicas of private text, which requires carefully balancing privacy and utility. We propose Realistic and Privacy-Preserving Synthetic Data Generation (RPSG), which uses private seeds and integrates privacy-preserving strategies, including a formal differential privacy (DP) mechanism in the candidate selection, to generate realistic synthetic data. Comprehensive experiments against state-of-the-art private synthetic data generation methods demonstrate that RPSG achieves high fidelity to private data while providing strong privacy protection.

Momentum

Related topics

Latest papers 160