Synthetic Instruction Generation

Latest papers

11 papers1 paper in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Instruction Mining From Images: Constructing a Synthetic Dataset for Multimodal Learning ILSP DCVLR Competition Submission
Oct 14, 2026Eleni E. Kasoura, Dimitrios Damianos, Georgios Paraskevopoulos +1Visual Instruction Tuning Synthetic Data Generation for Language Models
Institute of Language and Speech Processing Athena Research Center Athens, GR 151 25
PuzzleMate: Benchmarking MLLMs for Egocentric Puzzle Assistance
Sep 13, 2026Avijit Dasgupta, Shayon Dasgupta, Zakaria Laskar +2Synthetic Instruction Generation Jigsaw Puzzle Solving
IIIT Hyderabad, India · IIT BHU, India · IISER Thiruvananthapuram, India +1
Templated or fully synthetic? Prompt construction as a confound in measuring LLM political stance beyond writing assistance
Aug 11, 2026Ilias ChalkidisLLM Prompting Political Bias in Language Models
The National Center for AI in Society (CAISA) University of Copenhagen Denmark
UNSPECIFIC: General Constraint Synthesis for Breaking Copy-and-Paste Shortcut in LLM Instruction Following
Aug 10, 2026Jeet Sharma, Balpreet Kaur, Jeremiah Hong +2Instruction Following Constraint Satisfaction
University of Massachusetts, Amherst, USA
HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs
Jul 29, 2026Ru Peng, Tianyu Zhao, Xijun Gu +9LLM Fine-Tuning Instruction Tuning
Zhejiang University · Inclusion AI, Ant Group · Ant Group +2
Learning Complementary Action Modeling from Automotive Maintenance Instructions
Jun 26, 2026Jiaqi Wu, Bai Li, Jochen Hartmann +2Controllable Text Generation Synthetic Instruction Generation
Eindhoven University of Technology, Eindhoven, The Netherlands · Chemnitz University of Technology, Chemnitz, Germany · BMW Group, Munich, Germany
Uni-Edit: Intelligent Editing Is A General Task For Unified Model Tuning
May 20, 2026Dian Zheng, Manyuan Zhang, Hongyu Li +4Text-Guided Image Editing Unified Multimodal Models
CUHK MMLab · Meituan · TJU +1
EmbGen: Teaching with Reassembled Corpora
May 19, 2026Arun K Lenin, Kai Rouse, Andrea Nicastro +1Synthetic Instruction Generation Domain Adaptation for LLMs
Commonwealth Bank of Australia Sydney, Australia
MAFIG: Multi-agent Driven Formal Instruction Generation Framework
Apr 13, 2026Shixing Zhao, Zheng Si, Pengpeng Ouyang +5Synthetic Instruction Generation Multi-Agent LLM Systems
School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou, 450001, Henan, China · Engineering Research Center of Intelligent Swarm Systems, Ministry of Education, Zhengzhou, 450001, Henan, China · National Supercomputing Center in Zhengzhou, Zhengzhou, 450001, Henan, China +1
Enhancing Linguistic Generalization of VLA: Fine-Tuning OpenVLA via Synthetic Instruction Augmentation
Mar 17, 2026Dongik ShinSynthetic Instruction Generation Vision-Language-Action Models
FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
Jan 29, 2026Ajay Patel, Colin Raffel, Chris Callison-BurchLanguage Model Pretraining Synthetic Data Pretraining
University of Pennsylvania · University of Toronto · Vector Institute +1

Oct 14, 2026J/K move · Enter open · S save

Instruction Mining From Images: Constructing a Synthetic Dataset for Multimodal Learning ILSP DCVLR Competition Submission

Eleni E. Kasoura, Dimitrios Damianos, Georgios Paraskevopoulos, Vassilios Katsouros

Institute of Language and Speech Processing Athena Research Center Athens, GR 151 25

Visual Instruction Tuning Synthetic Data Generation for Language Models Synthetic Instruction Generation Multimodal Instruction Tuning

Open paper AskPDF arXiv

This work presents a compact and reproducible pipeline for synthesizing and curating image-based instruction-response pairs for DCVLR. We draw from the Cauldron collection and use Magpie-style instruction and response synthesis with Qwen2.5-VL-7B Instruct to create concise, image-conditioned instruction pairs. The pipeline consists of instruction synthesis, conditioned on input images and controlled via a system prompt, sanitization, tagging, and filtering of the instructions based on the score obtained by a reward model (DeBERTa-v3). We retain samples with positive scores to maximize the quality of the generated instructions. We observe that score distributions have a median and mean reward score of ~ 1.53 and ~ 1.75 respectively, indicating that most generated instructions are of good quality. The resulting dataset contains diverse instructions in the math, spatial, and social reasoning domains.