Synthetic Data Generation for Language Models

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 110% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 160

All topics

Synthetic Worlds for Temporal Evaluation and Knowledge Updating in LLMs
Aug 31, 2026Jonathan Zheng, Zirui Shao, Alan Ritter +1Synthetic Data Pretraining Knowledge Editing
Georgia Institute of Technology · Zhejiang University
REER-PT: Reverse-Engineered Reasoning for Perplexity-Guided Pre-training Data Augmentation
Aug 31, 2026Haoran Que, Jiajun Shi, Ting Huang +7Language Model Pretraining Synthetic Data Pretraining
Peking University · ByteDance Seed · Work done at ByteDance Seed +1
From Final Artifacts to Trajectories: Retrospective Process Supervision for Evidence-Grounded Long-Form Generation
Aug 31, 2026Junjie Huang, Jiarui Qin, Di Yin +4LLM Agent Trajectory Synthesis Process Supervision
Shanghai Jiao Tong University · Tencent Youtu Lab
SPARK: Skeleton-Guided Reasoning Synthesis from Large-Scale Scientific Literature
Aug 31, 2026Yu Li, Wei Li, Xin Gao +4Evidence-Grounded Reasoning Scientific Reasoning
Shanghai AI Laboratory · University of Science and Technology of China · 1Shanghai AI Laboratory 2University of Science and Technology of China +5
Generating Clinical Vignettes that Preserve Cognitive Formulations
Aug 30, 2026Amit Oren, Nimrod Hertz-Palmor, Dean Ariel +1Constrained Generative Modeling Mental Health
Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Beer Sheva, Israel · 2MRC Cognition and Brain Sciences Unit, University of Cambridge, United Kingdom · 3Clalit Health Services, Israel +3
Unfolding Scientific Papers into Multi-Turn Generation Trajectories for Continued Pre-Training
Aug 26, 2026Qiankai Xu, Qiguang Chen, Zixin Su +4Long-Form Document Generation Synthetic Benchmark Generation
ByteDance Seed · Nanjing University · Evolvent AI
Simulate, record, verify: A language-portable framework for muscle-grounded articulatory QA (extended version)
Aug 24, 2026Seungho Eum, Unsang ParkSynthetic Data Generation for Language Models
Department of Computer Science and Engineering, Sogang University, Seoul, Korea · Department of Artificial Intelligence, Sogang University, Seoul, Korea
CARD: Controlled Agentic Reddit Discussions for Credit Card Simulation
Aug 10, 2026Yaoning Yu, Kai-Min Chang, Ye Yu +3Financial Services Controllable Text Generation
University of Illinois Urbana-Champaign United States · U.S. Bank United States · Stanford University United States +1
Ask-E: An Environment for Calibrated Question Generation
Aug 7, 2026Sarah Pratt, Jae Sung Park, Scott Geng +1Mathematical Reasoning Benchmarks Synthetic Benchmark Generation
University of Washington · 2Allen Institute for AI
Clinical Communication Processing with Models Trained on LLM-Generated Synthetic Data: A Structured Survey and Novel Application Case Studies
Aug 6, 2026Alexander Apartsin, Yehudit ApersteinSynthetic Data Generation for Language Models Clinical NLP
School of Computer Science, Faculty of Sciences, Holon Institute of Technology (HIT), Holon, Israel · Intelligent Systems, Afeka Academic College of Engineering, Tel-Aviv, Israel
Reasoning Core: Designing Broad Procedural Data for Completion-Supervised Reasoning Training
Aug 5, 2026Damien Sileo, Valentin Lacombe, Dimitri KachlerSupervised Fine-Tuning Synthetic Data Generation for Language Models
Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 - CRIStal, F-59000 Lille, France
ODRA: Synthesizing Cognitive Behavioral Therapy Sessions with Structured Chain-Of-Thought and Dynamic Patient Resistance
Aug 5, 2026Javier Rodriguez-Juan, Hiba Arnaout, Jose Garcia-Rodriguez +2Mental Health Counseling Human Behavior Simulation
13DPLab, Department of Computer Science and Technology, University of Alicante · 2UKP Lab, Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt
SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation
Aug 3, 2026Zelin Tan, Yiqun Zhang, Hao Li +11LLM Agent Skill Learning LLM Agent Trajectory Synthesis
Shanghai Artificial Intelligence Laboratory
Data Turnstile: A Scalable Open Framework for Function-Calling Data Generation
Jul 31, 2026Goutham Ramakrishnan, Megha SharmaTool-Augmented Language Model Agents Small Language Models
Amazon AGI Bengaluru, India
Beyond Rephrasing: Book-Level Organization Improves Synthetic Textbook Data for Mid-Training
Jul 30, 2026Jiawen Tao, Miao Peng, Yaoming Li +7Language Model Pretraining Synthetic Data Pretraining
Hunyuan Team, Tencent · Peking University · The Hong Kong University of Science and Technology (Guangzhou)
HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs
Jul 29, 2026Ru Peng, Tianyu Zhao, Xijun Gu +9LLM Fine-Tuning Instruction Tuning
Zhejiang University · Inclusion AI, Ant Group · Ant Group +2
BridgeAlign: Bridging Preference Alignment for Humanities and Social Sciences
Jul 29, 2026Ru Peng, Haokai Xu, Xijun Gu +11Pairwise Preference Learning Preference Optimization
Zhejiang University · Inclusion AI, Ant Group · Qwen Team, Alibaba Group +2
SciDataSailor: Deep Scientific Data Exploring
Jul 29, 2026Jiyong Rao, Yicheng Qiu, Chi Zhang +2Scientific QA AI Agent Benchmarks
Shanghai Artificial Intelligence Laboratory · Tongji University
DuplexGen: Adaptive Synthesis of Human-AI Turn-Taking Dialogues
Jul 28, 2026Takyoung Kim, Kang-wook Kim, Sang Hoon Woo +3Human-in-the-Loop AI Synthetic Data Generation for Language Models
University of Illinois Urbana-Champaign · Seoul National University · University of California, Berkeley +2
Distilling Temporal Search and Reasoning: Evolving LLMs for Future Prediction via Harness-Assisted Efficient Data Synthesis
Jul 28, 2026Wanxu Cai, Zhengyu Chen, Huaisheng Zhu +3Data Leakage Temporal Reasoning
1The Chinese University of Hong Kong · 2Meituan LongCat Team
EmoTrace: An Emotion Trajectory-Centered Framework for Psychological Support Dialogue Generation
Jul 26, 2026Kaitong Weng, Lixin Liu, Zihao Liu +2Mental Health Counseling Emotional Support Conversation
Shenzhen International Graduate School, Tsinghua University
IndicTalk: A Large-Scale Persona-Based Multilingual Conversational Corpus for Indic Languages
Jul 25, 2026Sahil Deepak Gawande, Mayank SinghSynthetic Data Generation for Language Models Low-Resource Language Processing
Lingo Research Group · IIT Gandhinagar
RIME: Enabling Large-Scale Agentic Music Post-Production
Jul 21, 2026Noah Schaffer, Nikhil SinghAudio Editing Multimodal Large Language Models
Dartmouth College
OntoBook: Ontology-Grounded Synthetic Textbooks for Medical Encoder Pretraining
Jul 21, 2026Rian Touchent, Éric de la ClergerieLanguage Model Pretraining Automated Medical Coding
ALMAnaCH · Inria, Sorbonne Université · rue Barrault 75013 Paris, 21 rue de l’école de médecine 75006 Paris
Learning from Synthetic Data without Model Collapse in Iterative Instruction Tuning
Jul 19, 2026Xiaonan Luo, Yue Huang, Kehan Guo +4Self-Training for Language Models Model Collapse
University of Notre Dame · Vanderbilt University · University of Pennsylvania
PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window
Jul 17, 2026Ryan Thornton, Mir Mehedi Ahsan Pritom, Maanak GuptaAdversarial Attacks Privacy Leakage in Language Models
Department of Computer Science Tennessee Tech University Cookeville, TN, USA
Distribution-First Population Simulation: Collapse, Calibration, and Recall in Non-WEIRD LLM Persona Modeling
Jul 17, 2026Gurkan OzkanHuman Behavior Simulation Large Language Model-Based Social Simulation
Istanbul Technical University
FormalAnalyticGeo: A Neural-Symbolic Based Framework for Multimodal Analytic Geometry Problem Generation
Jul 14, 2026Ruoran Xu, Wending Gao, Qiufeng WangMultimodal Reasoning Multimodal Generation
Xi’an Jiaotong-Liverpool University
Filtering Harmful Actions Isn't Enough: Phantom Transfer in Agentic SDF
Jul 12, 2026Chinmayi DixitSynthetic Data Generation for Language Models LLM Agent Safety
ERA Fellowship
IdeaTrail: Full-Process Agent Trajectories for Scientific Ideation
Jul 11, 2026Hengquan GuoAI Agents for Scientific Discovery Scientific Hypothesis Generation
ShanghaiTech University

Aug 31, 2026, cs.CLJ/K move · Enter open · S save

Synthetic Worlds for Temporal Evaluation and Knowledge Updating in LLMs

Jonathan Zheng, Zirui Shao, Alan Ritter, Wei Xu

Georgia Institute of Technology · Zhejiang University

Synthetic Data Pretraining Knowledge Editing Temporal Reasoning in Language Models Synthetic Benchmark Generation Synthetic Data Generation for Language Models

Open paper AskPDF arXiv

Large language models (LLMs) rely on static pretraining corpora, causing their knowledge to become outdated over time. Existing approaches for evaluating knowledge edits either suffer from rapid contamination or rely on counterfactual edits that conflict with rigid existing knowledge. In this work, we propose a synthetic, simulation-driven framework for studying knowledge insertion in LLMs. We introduce {\sc ParallelEvents}, a benchmark of fictional yet realistic future worlds that generates coherent event trajectories for controlled evaluation, avoiding contamination while preserving consistency. Building on this dataset, we develop {\sc Synapse}, a training framework that uses model-generated data to update model parameters via mid-training and instruction tuning. This synthetic pipeline enables scalable knowledge integration without costly human-curated data. Empirically, {\sc Synapse} outperforms existing methods by 14.23%, demonstrating that simulation-based synthetic training leads to robust and coherent knowledge insertions.

Momentum

Related topics

Latest papers 160