Synthetic Data Generation for Language Models

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 110% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 160

All topics

Learning First Integrals via Backward-Generated Data and Guided Reinforcement Learning
May 20, 2026Jingfeng Zhong, Zhengxiang Liu, Zhijie Wang +1Synthetic Data Generation for Language Models LLM Mathematical Reasoning
Shanghai Jiao Tong University China
Thinking-while-speaking: A Controlled, Interleaved Reasoning Method for Real-Time Speech Generation
May 20, 2026Xuan Du, Qiangyu Yan, Wenshuo Li +4RL for Language Model Reasoning Controllable Speech Generation
Huawei Technologies
PlanningBench: Generating Scalable and Verifiable Planning Data for Evaluating and Training Large Language Models
May 20, 2026Ziliang Zhao, Zenan Xu, Shuting Wang +7LLM Evaluation LLM Planning
Gaoling School of Artificial Intelligence, Renmin University of China · LLM Department, Hunyuan Team, Tencent · Beijing Academy of Artificial Intelligence +1
Synthesis and Evaluation of Long-term History-aware Medical Dialogue
May 19, 2026Hebin Hu, Renke Dai, Ah-Hwee Tan +1Healthcare AI Agent Benchmarks
South-Central Minzu University Wuhan, China · Singapore Management University Singapore, Singapore
EmbGen: Teaching with Reassembled Corpora
May 19, 2026Arun K Lenin, Kai Rouse, Andrea Nicastro +1Domain Adaptation for LLMs Synthetic Data Generation for Language Models
Commonwealth Bank of Australia Sydney, Australia
FOL2NS: Generating Natural Sentences from First-Order Logic
May 18, 2026Mei JiaSynthetic Data Generation for Language Models Neuro-Symbolic AI
University of Manchester
Generating Pretraining Tokens from Organic Data for Data-Bound Scaling
May 18, 2026Zichun Yu, Chenyan XiongLanguage Model Pretraining Synthetic Data Pretraining
Language Technologies Institute, Carnegie Mellon University · Xlue
Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs
May 17, 2026Yuxuan Lu, Ziyi Wang, Yingzhou Lu +12Tool-Augmented Language Model Agents Synthetic Data Generation
Northeastern University · Independent Researcher
FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale
May 14, 2026Runyuan He, Qiuyang Mang, Shang Zhou +14Synthetic Benchmark Generation Synthetic Data Generation for Language Models
UC Berkeley · UC San Diego · University of Washington +4
NodeSynth: Socially Aligned Synthetic Data for AI Evaluation
May 14, 2026Qazi Mamunur Rashid, Xuan Yang, Zhengzhe Yang +5Language Model Generation Evaluation Language Model Safety Evaluation
Google Research · Google · Google Deepmind
Know When To Fold 'Em: Token-Efficient LLM Synthetic Data Generation via Multi-Stage In-Flight Rejection
May 13, 2026Anjir Ahmed Chowdhury, Syed Zawad, Feng YanSynthetic Data Generation LLM Inference Acceleration
Department of Computer Science University of Houston · IBM Research
AcquisitionSynthesis: Targeted Data Generation using Acquisition Functions
May 13, 2026Ishika Agarwal, Sofia Stoica, Emre Can Acikgoz +4Synthetic Data Generation Synthetic Data Generation for Language Models
University of Illinois Urbana-Champaign1 · Amazon2
An Information-Theoretic Criterion for Efficient Data Synthesis
May 11, 2026Hanyu Li, Zhengqi Sun, Xiaotie DengSynthetic Data Pretraining Synthetic Data Generation
CFCS, School of Computer Science, Peking University, Beijing, China · Department of Information Management, Peking University, Beijing, China
LLM-Driven Performance-Space Augmentation for Meta-Learning-Based Algorithm Selection
May 10, 2026Darren Zhu, Daren LerSynthetic Data Augmentation Algorithm Selection
Department of Statistics and Data Science National University of Singapore · Department of Computer Science National University of Singapore
CauSim: Scaling Causal Reasoning with Increasingly Complex Causal Simulators
May 9, 2026Nicolás Astorga, Anita Kriz, Mihaela van der SchaarStructural Causal Models Knowledge Augmentation for Language Models
DAMTP, University of Cambridge, Cambridge, UK
Generating Leakage-Free Benchmarks for Robust RAG Evaluation
May 9, 2026Jiayi Liu, Jiaxing Zhang, Bowen Jin +1Benchmark Contamination Synthetic Benchmark Generation
Department of Computer Science, Purdue University · New Jersey Institute of Technology · University of Illinois at Urbana-Champaign +1
Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Models
May 8, 2026Aswin RRV, Jacob Dineen, Divij Handa +4Synthetic Data Generation RL for Language Model Reasoning
Arizona State University · Google Cloud AI Research · Google DeepMind
Curated Synthetic Data Doesn't Have to Collapse: A Theoretical Study of Generative Retraining with Pluralistic Preferences
May 8, 2026Ali Falahati, Mohammad Mohammadi Amiri, Kate Larson +1Synthetic Data Pretraining Model Collapse
University of Waterloo · 2Rensselaer Polytechnic Institute.
Generating training datasets for legal chatbots in Korean
May 8, 2026Changhoe Hwang, Jee-Sun Nam, Eric LaporteLegal NLP Synthetic Data Generation
Hankuk University of Foreign Studies, DICORA · Univ Gustave Eiffel, CNRS, LIGM
When2Speak: A Dataset for Temporal Participation and Turn-Taking in Multi-Party Conversations for Large Language Models
May 7, 2026Vihaan Nama, Shreya Mendi, Zian Ye +1Synthetic Data Generation Synthetic Data Generation for Language Models
Pratt School of Engineering Duke University
DataArc-SynData-Toolkit: A Unified Closed-Loop Framework for Multi-Path, Multimodal, and Multilingual Data Synthesis
May 2, 2026Zhichao Shi, Cehao Yang, Hao Zhou +6Synthetic Data Generation Synthetic Data Generation for Language Models
DataArc Tech Ltd. · IDEA Research, International Digital Economy Academy · School of Advanced Interdisciplinary Sciences, UCAS +2
BARRED: Synthetic Training of Custom Policy Guardrails via Asymmetric Debate
Apr 28, 2026Arnon Mazza, Elad LeviLLM Guardrails LLM Fine-Tuning
Plurai Inc.
STELLAR-E: a Synthetic, Tailored, End-to-end LLM Application Rigorous Evaluator
Apr 27, 2026Alessio Sordo, Lingxiao Du, Meeka-Hanna Lenisa +2LLM Evaluation Synthetic Data Generation
1Deutsche Bank, Berlin Technology Center, Berlin, Germany
LegalDrill: Diagnosis-Driven Synthesis for Legal Reasoning in Small Language Models
Apr 26, 2026Tianchun Li, Haochen Liu, Vishwa Pardeshi +5Synthetic Data Generation Small Language Models
Purdue University, West Lafayette, IN, USA · Fidelity Investments, Boston, MA, USA
AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use
Apr 23, 2026Yuanjie Lyu, Chengyu Wang, Haonan Zheng +4Tool-Augmented Language Model Agents Agentic RL
Alibaba Group, Hangzhou, China
Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs
Apr 22, 2026Aishik Mandal, Hiba Arnaout, Clarissa W. Ong +5Mental Health Counseling Mental Health
UKP Lab, Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt · Zuse School · National Research Center for Applied Cybersecurity ATHENE +4
Model in Distress: Sentiment Analysis on French Synthetic Social Media
Apr 20, 2026Pierre-Carl Langlais, Pavel Chizhov, Yannick Detrois +3Synthetic Data Augmentation Synthetic Data Generation
PleIAs, Paris, France · CAIRO, THWS, Würzburg, Germany · EPFL, Lausanne, Switzerland +1
Reverse Constitutional AI: A Framework for Controllable Toxic Data Generation via Probability-Clamped RLAIF
Apr 20, 2026Yuan Fang, Yiming Luo, Aimin Zhou +1Language Model Safety Evaluation Adversarial Scenario Generation
East China Normal University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China
LACE: Lattice Attention for Cross-thread Exploration
Apr 16, 2026Yang Li, Zirui Zhang, Yang Liu +1LLM Inference Attention Mechanisms
Department of Computer Science Rutgers University · Amazon AGI Labs
Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation
Apr 13, 2026Lester James V. Miranda, Ivan Vulić, Anna KorhonenSupervised Fine-Tuning Multilingual Language Models
Language Technology Lab, University of Cambridge

May 20, 2026, cs.LGJ/K move · Enter open · S save

Learning First Integrals via Backward-Generated Data and Guided Reinforcement Learning

Jingfeng Zhong, Zhengxiang Liu, Zhijie Wang, Shuai Li

Shanghai Jiao Tong University China

Synthetic Data Generation for Language Models LLM Mathematical Reasoning

Open paper AskPDF arXiv

The discovery of first integrals is of fundamental scientific importance for understanding conservation laws in dynamical systems. However, existing symbolic computation tools and Large Language Models (LLMs) remain limited on this task because high-quality training data are scarce and successful solutions often depend on mathematical intuition. This paper presents FISolver, an LLM-based solver developed to address this challenge. First, we introduce a "Backward Generation" algorithm that systematically builds large-scale datasets of (differential equation, first integral) pairs by deriving differential equations from sampled integrals, thereby alleviating the data scarcity bottleneck. Second, we apply supervised fine-tuning to a compact mathematical model and further improve its performance through reinforcement learning with a Levenshtein Distance-based shaped reward. In addition, we design data synthesis and blending strategies that support effective adaptation to difficult problem families from sparse examples. Experiments show that FISolver, while requiring substantially lower computational cost, significantly outperforms larger mathematical LLMs and commercial solvers such as Mathematica on challenging benchmarks, indicating a new data-driven route for automated discovery of first integrals.

Momentum

Related topics

Latest papers 160