Jagle: Building a Large-Scale Japanese Multimodal Post-Training Dataset for Vision-Language Models

Issa Sugiura; Keito Sasagawa; Keisuke Nakao; Koki Maeda; Ziqi Yin; Zhishen Yang; Shuhei Kurita; Yusuke Oda

cs.CVApr 2, 2026

Jagle: Building a Large-Scale Japanese Multimodal Post-Training Dataset for Vision-Language Models

Authors: Issa Sugiura, Keito Sasagawa, Keisuke Nakao, Koki Maeda, Ziqi Yin, Zhishen Yang, Shuhei Kurita, Yusuke Oda, +3 more

Organizations: Institute of Science Tokyo · NII LLMC · Waseda University · NII · Aichi Institute of Technology · RIKEN · AIST

Abstract

Developing vision-language models (VLMs) that generalize across diverse tasks requires large-scale training datasets with diverse content. In English, such datasets are typically constructed by aggregating and curating numerous existing visual question answering (VQA) resources. However, this strategy does not readily extend to other languages, where VQA datasets remain limited in both scale and domain coverage. In this work, we introduce Jagle, the largest Japanese multimodal post-training dataset to date, comprising approximately 9.2 million instances across diverse tasks. Rather than relying on existing VQA datasets, we collect heterogeneous source data, including images, image-text pairs, and PDF documents, and generate VQA pairs through multiple strategies such as VLM-based QA generation, translation, and text rendering. Experiments demonstrate that a 2.2B model trained with Jagle surpasses InternVL3.5-2B in average score across ten Japanese evaluation tasks and comes within five points of Qwen3-VL-2B-Instruct. Moreover, combining Jagle with FineVision improves English performance over training with FineVision alone. These benefits generalize across model backbones and scales: LLM-jp-4-VL 9B beta, a 9B-scale model built on the Japanese-centric LLM-jp-4-8B-instruct, achieves Japanese performance comparable to Qwen3-VL-8B-Instruct. To facilitate reproducibility and future research, we release the dataset, models, and code.

Figures & tables

Dataset	Language	Categories	Subsets	Examples
Cambrian-7B ( Tong et al., 2024 )	English	9	70	7.1M
FineVision ( Wiedmann et al., 2025 )	English	9	185	24.2M
DEJIMA ( Katsube et al., 2025 )	Japanese	2	2	3.9M
LLM-jp-3 VILA ( Sasagawa et al., 2025 )	Japanese	3	4	0.4M
Jagle (Ours)	Japanese	5	17	9.2M

Table 1: Comparison of open multimodal post-training datasets. Task categories follow the Eagle2 taxonomy ( Li et al., 2025b ) . Jagle is the largest Japanese multimodal post-training dataset.

Figure 1: Construction pipeline of Jagle. Our pipeline leverages diverse data sources, including images, image-text pairs, and PDF corpora, and integrates multiple QA generation strategies such as VLM-based QA generation, translation, OCR-based text extraction, text rendering, and direct utilization of existing data to produce VQA samples.

Category	Subset Name	Samples	Data Source	Method
General VQA	japanese-photos-VQA	1,163	japanese-photos ( ThePioneer, 2024 )	VLM-based
	JSQuAD-Vision-filterd-15k	14,790	JSQuAD ( Kurihara et al., 2022 )	Text rendering
	ja-vg-vqa	99,202	ja-vg-vqa ( Shimizu et al., 2018 )	Direct use
	llava-instruct-ja-qwen3vl	155,657	llava-instruct ( Liu et al., 2023 )	VLM-based
	WIKI-JA-VQA	936,871	Wiki-JA-Pair (ours)	VLM-based
	WAON-VQA	1,912,226	WAON ( Sugiura et al., 2025 )	VLM-based

Table 2: Statistics of each subset in Jagle: samples, data source, and QA construction method.

Dataset	Samples	Unique Images	Turns	Answer Tokens
FineVision ( Wiedmann et al., 2025 )	24,210,611	15,606,828	88,356,316	10,146,129,067
Jagle (Ours)	9,154,524	9,051,232	58,906,972	3,510,910,782

Table 3: Comparison of dataset sizes between Jagle and FineVision ( Wiedmann et al., 2025 ) in terms of samples, unique images, turns, and answer tokens.

Figure 2: Category distribution of Jagle across four metrics: number of samples, unique images, turns, and answer tokens.

Figure 3: Representative VQA examples from each category in Jagle. The dataset covers a wide variety of visual content, including natural images, charts and tables, document images, and presentation slides.

Figure 4: Training dynamics under each data setting for the macro-averaged score over all 20 tasks (Avg), 10 Japanese tasks (JA Avg), and 10 English tasks (EN Avg). The model trained on Jagle outperforms the model trained on FineVision by over 20 points on JA Avg.

Model	Base LLM	Training Data	Steps	JA Avg	EN Avg	Avg
2B-scale models
Jagle-VL-2.2B (ours)	Qwen3-1.7B	FineVision	60k	29.6 $\pm$ 0.0	57.7	43.7 $\pm$ 0.0
		Jagle	60k	51.3 $\pm$ 0.1	52.3	51.8 $\pm$ 0.0
		Jagle+FineVision	60k	50.1 $\pm$ 0.0	59.6	54.9 $\pm$ 0.0
Qwen3-VL-2B-Instruct	Qwen3-1.7B			56.0 $\pm$ 0.2	67.0	61.5 $\pm$ 0.1
InternVL3.5-2B	Qwen3-1.7B			47.0 $\pm$ 0.2	68.0	57.5 $\pm$ 0.1

Table 4: Comparison of models trained with Jagle against existing VLMs. We report the mean and standard deviation over three evaluation runs (omitted for EN Avg, where evaluation is deterministic). Bold indicates the best score in each column within each scale group. Per-task results of all models are provided in Appendix E .

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Subset Name	Samples	Unique Images	Turns	Answer Tokens
General VQA	japanese-photos-VQA	1,163	1,163	5,775	227,515
	JSQuAD-Vision-filterd-15k	14,790	14,790	39,319	833,581
	ja-vg-vqa	99,202	99,199	793,616	15,247,119
	llava-instruct-ja-qwen3vl	155,657	80,958	386,229	60,802,343
	WIKI-JA-VQA	936,871	936,871	4,679,651	178,987,472
	WAON-VQA	1,912,226	1,912,213	9,541,389	302,272,463

Appendix

Table 5: Detailed statistics of each subset in Jagle, including the number of samples, unique images, turns, and answer tokens.

Figure 5: t-SNE visualization of SigLIP2 image embeddings for 5,000 images randomly sampled from Jagle. Chart & Table and Naive OCR images form distinct clusters, while General VQA, Captioning, and OCR QA images are largely intermingled.

Source Dataset	License
WAON ( Sugiura et al., 2025 )	Apache 2.0 (use limited to information analysis under Article 30-4 of the Japanese Copyright Act)
japanese-photos ( ThePioneer, 2024 )	CC0 1.0
ja-vg-vqa ( Shimizu et al., 2018 )	CC BY 4.0
llava-instruct ( Liu et al., 2023 )	CC BY 4.0
JSQuAD ( Kurihara et al., 2022 )	CC BY-SA 4.0
Wiki-JA-Pair (ours)	CC BY-SA 4.0 (derived from Japanese Wikipedia)

Appendix

Table 6: Licenses of the source datasets used to construct Jagle.

Model	License
Qwen3-VL-235B-A22B-Instruct ( Qwen Team, 2025b )	Apache 2.0
LLM-jp-3-13B-instruct ( LLM-jp, 2024 )	Apache 2.0
PaddleOCR-VL ( Cui et al., 2025 )	Apache 2.0

Appendix

Table 7: Licenses of the models used for QA generation and preprocessing.

Subset Name	Unique pHash Matches	Matching Benchmark Breakdown
japanese-photos-VQA	0	–
JSQuAD-Vision-filterd-15k	0	–
ja-vg-vqa	0	–
llava-instruct-ja-qwen3vl	0	–
WIKI-JA-VQA	141	JMMMU: 129, CVQA-JA: 10, Heron-Bench: 2
WAON-VQA	5	JMMMU: 5

Appendix

Table 8: Per-subset results of the pHash-based contamination analysis between Jagle training images and Japanese evaluation benchmark images. For each subset, we report the number of unique benchmark images with matching perceptual hashes. Since the same benchmark image can match images in multiple subsets, the Overall row counts unique benchmark images across all subsets.

2B-scale models
Model	CC-OCR-JA	JA-VLM-Bench	Heron-Bench	CVQA-JA	MECHA-JA
Jagle-VL-2.2B (FineVision)	18.2 $\pm$ 0.4	34.7 $\pm$ 0.0	31.8 $\pm$ 0.0	45.0	44.1
Jagle-VL-2.2B (Jagle)	55.2 $\pm$ 0.0	63.3 $\pm$ 0.0	53.4 $\pm$ 0.0	46.0	43.6
Jagle-VL-2.2B (Jagle+FineVision)	49.7 $\pm$ 0.0	61.2 $\pm$ 0.0	51.1 $\pm$ 0.0	51.0	46.1
Qwen3-VL-2B-Instruct	54.3 $\pm$ 1.1	55.1 $\pm$ 0.0	54.5 $\pm$ 0.0	51.5	48.5
InternVL3.5-2B	52.2 $\pm$ 0.4	43.5 $\pm$ 1.2	41.3 $\pm$ 0.7	40.0	44.3

Appendix

Table 9: Per-task results of all evaluated models on the 10 Japanese benchmarks (split into two column blocks). We report the mean and standard deviation over three evaluation runs; standard deviations are omitted for tasks with deterministic evaluation. Bold indicates the best score in each column within each scale group; within each group, our models and existing VLMs are separated by a horizontal line.

2B-scale models
Model	TextVQA	OK-VQA	MMMU	ScienceQA	DocVQA
Jagle-VL-2.2B (FineVision)	69.4	32.7	44.2	83.4	79.2
Jagle-VL-2.2B (Jagle)	66.6	31.8	44.7	71.0	78.7
Jagle-VL-2.2B (Jagle+FineVision)	72.4	35.0	43.0	83.6	83.8
Qwen3-VL-2B-Instruct	80.6	52.9	43.9	80.5	92.7
InternVL3.5-2B	75.3	50.5	51.5	91.7	88.9

Appendix

Table 10: Per-task results of all evaluated models on the 10 English benchmarks (split into two column blocks), where evaluation is deterministic. Bold indicates the best score in each column within each scale group; within each group, our models and existing VLMs are separated by a horizontal line.

Figure 6: Training dynamics under each data setting on each of the 10 Japanese benchmarks.

Figure 7: Training dynamics under each data setting on each of the 10 English benchmarks.

Explore similar work

JAMMEval: A Refined Collection of Japanese Benchmarks for Reliable VLM Evaluation
Apr 1, 2026Issa Sugiura, Koki Maeda, Shuhei Kurita +3VLM Evaluation Benchmark Construction
Kyoto University · NII LLMC · Institute of Science Tokyo +2
Multilingual Training and Evaluation Resources for Vision-Language Models
Apr 20, 2026Daniela Baiamonte, Elena Fano, Matteo Gabburo +3VLM Evaluation Vision-Language Models
Aithlas
JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding
Mar 30, 2026Koki Maeda, Naoaki OkazakiVLM Evaluation Scene Text Recognition
Institute of Science Tokyo, Tokyo, Japan · Research and Development Center for Large Language Models, National Institute of Informatics, Tokyo, Japan

Apr 1, 2026J/K move · Enter open · S save

JAMMEval: A Refined Collection of Japanese Benchmarks for Reliable VLM Evaluation

Issa Sugiura, Koki Maeda, Shuhei Kurita, Yusuke Oda+2

Kyoto University · NII LLMC · Institute of Science Tokyo+2

VLM Evaluation Benchmark Construction Visual Question Answering Multilingual VLM Evaluation

Open paper AskPDF arXiv

Reliable evaluation is essential for the development of vision-language models (VLMs). However, Japanese VQA benchmarks have undergone far less iterative refinement than their English counterparts. As a result, many existing benchmarks contain issues such as ambiguous questions, incorrect answers, and instances that can be solved without visual grounding, undermining evaluation reliability and leading to misleading conclusions in model comparisons. To address these limitations, we introduce JAMMEval, a refined collection of Japanese benchmarks for reliable VLM evaluation. It is constructed by systematically refining seven existing Japanese benchmark datasets through two rounds of human annotation, improving both data quality and evaluation reliability. In our experiments, we evaluate open-weight and proprietary VLMs on JAMMEval and analyze the capabilities of recent models on Japanese VQA. We further demonstrate the effectiveness of our refinement by showing that the resulting benchmarks yield evaluation scores that better reflect model capability, exhibit lower run-to-run variance, and improve the ability to distinguish between models of different capability levels. We release our dataset and code to advance reliable evaluation of VLMs.

Abstract

Explore similar work

JAMMEval: A Refined Collection of Japanese Benchmarks for Reliable VLM Evaluation

Multilingual Training and Evaluation Resources for Vision-Language Models

JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding