SciGen-Verifier: A Multimodal Reasoner for Explainable Verification in Scientific Image Generation

Jiali Chen; Zhengteng Lin; Zuqi Wang; Shirong Lin; Xi Yu; Xusen Hei; DingBa Fu; Jiayuan Xie

cs.CVSep 27, 2026

SciGen-Verifier: A Multimodal Reasoner for Explainable Verification in Scientific Image Generation

Authors: Jiali Chen, Zhengteng Lin, Zuqi Wang, Shirong Lin, Xi Yu, Xusen Hei, DingBa Fu, Jiayuan Xie, +1 more

Organizations: South China University of Technology, The Hong Kong Polytechnic University · South China University of Technology · The Hong Kong Polytechnic University

Abstract

In realistic education, a solution is often expressed not only in words but in a drawing--a circuit, a geometric construction, a function plot--and a teacher must grade the drawing as carefully as the text. Recent advances in unified multimodal models have enabled scientific image generation, yet verifying the correctness of these specialized visual outputs remains a critical bottleneck: errors often arise from intricate domain knowledge, structural reasoning, and multi-step instruction rather than surface-level artifacts. Existing verifiers mainly target natural images and compress judgement into scalar scores, leaving scientific coverage and explainable feedback for error correction underexplored. To bridge this gap, we make three main contributions. (1) We construct SciGen-Verify, a benchmark dedicated to explainable verification of scientific image generation, spanning instruction following, multidisciplinary reasoning, and world knowledge domains. It contains a three-tier hierarchical protocol over the binary judgement, supporting explanation, and corrective editing instruction. (2) We develop SciGen-Verifier, a reasoning-driven multimodal verifier trained via cold-start supervised fine-tuning followed by a curriculum-based two-stage reinforcement learning pipeline. The rubric-guided process rewards first strengthen scientific reasoning exploration and outcome rewards subsequently align output with ground-truth annotation. (3) On SciGen-Verify, SciGen-Verifier achieves competitive performance against much larger proprietary models. It further serves as a practical online critic for iterative image rectification.

Figures & tables

Figure 1: Overview of SciGen-Verify benchmark across instruction following, multidisciplinary reasoning and world knowledge domains. The context image is the input image of the problem.

Figure 2: Construction workflow of our SciGen-Verify benchmark, including data collection, negative sample construction, and annotation with expert review.

Models	Instruction			Reasoning			Knowledge			Overall
Models	Acc a	Acc e	Acc i	Acc a	Acc e	Acc i	Acc a	Acc e	Acc i	Acc a	Acc e	Acc i
GPT-6-Astra	78.16	71.84	58.74	78.65	76.78	59.55	70.33	69.86	42.48	75.71	72.83	53.59
Gemini-3-Flash	72.66	66.50	52.30	87.46	81.47	65.06	84.47	82.65	53.73	81.53	76.87	57.03
Gemini-3.1-Flash-Lite	63.55	54.19	41.39	77.35	68.64	50.79	78.39	72.60	34.31	73.10	65.14	42.16
Gemini-3.1-Pro	75.04	68.45	51.21	85.93	79.94	55.39	84.62	82.93	57.98	81.86	77.11	54.86
Gemini-3.5-Flash	72.12	69.57	51.69	86.55	84.00	68.12	85.47	82.69	49.80	81.38	78.75	56.54

Table 1: Evaluation results of different MLLMs on the SciGen-Verify . The best and second-best performance in each column are highlighted in bold and underlined , respectively. The overall denotes the macro-average accuracy.

$\mathcal{S}$	$\mathcal{R}_{1}$	$\mathcal{R}_{2}$	Instruction			Reasoning			Knowledge			Overall
$\mathcal{S}$	$\mathcal{R}_{1}$	$\mathcal{R}_{2}$	Acc a	Acc e	Acc i	Acc a	Acc e	Acc i	Acc a	Acc e	Acc i	Acc a	Acc e	Acc i
✓	✗	✗	65.35	60.63	35.34	82.25	75.36	43.45	92.66	89.97	61.36	80.09	75.32	46.72
✓	✓	✗	71.18	63.72	40.35	86.06	77.35	47.94	93.61	90.42	63.11	83.62	77.16	50.47
✓	✗	✓	69.86	65.23	42.78	84.93	79.14	51.22	93.24	91.02	64.35	82.68	78.46	52.78
✓	✓	✓	75.62	70.38	48.93	89.02	83.47	57.84	94.21	91.53	65.48	86.28	81.79	57.42

Table 2: Evaluation on training strategies.

\mathcal{S}

denotes SFT.

\mathcal{R}_{1}

and

\mathcal{R}_{2}

denote the stage 1 and 2 RL.

Figure 3: Effect of CoT reasoning.

Figure 4: Case study. (a) Comparison with a strong baseline. (b) Iterative test-time scaling for image rectification with SciGen-Verifier as the online critic.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Agreement	Acc. (%)	F1 (%)	Cohen’s $\kappa$
Human–Human (IAA)	98.3	99.2	0.83
$\text{Acc}_{e}$ judge (GLM-4.7) vs. Human	95.3	97.2	0.79
$\text{Acc}_{i}$ judge (Doubao-Seed-2.0-Lite) vs. Human	94.7	96.8	0.76

Appendix

Table 3: Agreement between the automated judges and human experts on 300 sampled instances. IAA denotes the human–human inter-annotator agreement.

Model	Doubao judge	GPT-5-mini judge	$\Delta$
SciGen-Verifier-8B	57.42	57.07	$-$ 0.35
Gemini-3.1-Pro	54.86	55.21	$+$ 0.35
Seed-2.0-Pro	53.95	53.30	$-$ 0.65
Seed-1.8	44.28	43.98	$-$ 0.30
MiMo-V2-Omni	41.58	41.02	$-$ 0.56

Appendix

Table 4: Overall

\text{Acc}_{i}

under the default judge (Doubao-Seed-2.0-Lite) and a third-party judge (GPT-5-mini).

\Delta

denotes the difference between the two judges.

Category	Test			Train
Category	C/S	P/C	P/S	C/S	P/C	P/S
Instruction Following	3.55	4.45	15.82	3.41	4.61	15.68
World Knowledge	4.97	6.84	33.98	5.67	6.31	35.81
Multi. Reasoning	4.21	6.64	27.98	4.14	6.79	28.11
Average	4.57	6.43	29.40	4.35	6.12	26.58

Appendix

Table 5: Annotation statistics of SciGen-Rubric across training and test splits. C/S: criteria per sample; P/C: points per criterion; P/S: points per sample.

Model	Instruction			Reasoning			Knowledge			Overall
Model	RR	NHR	SF1	RR	NHR	SF1	RR	NHR	SF1	RR	NHR	SF1
Seed-2.0-Pro	65.25	73.74	68.56	54.91	68.56	59.72	57.00	78.77	64.98	59.05	73.69	64.42
Gemini-3-Flash	59.79	72.90	64.99	46.29	65.51	53.11	48.99	76.84	58.86	51.69	71.75	58.99
Qwen3.6-Plus	57.72	72.43	63.26	50.24	70.34	57.34	51.42	78.07	61.12	53.13	73.61	60.57
Qwen3.5-397B	56.65	75.53	63.87	43.27	67.92	51.19	48.26	80.99	59.34	49.39	74.81	58.13
MiMo-V2-Omni	53.71	68.52	59.04	44.68	66.53	52.15	47.88	75.86	57.76	48.76	70.30	56.32

Appendix

Table 6: Performance comparison on SciGen-Rubric across three capability domains. RR, NHR, and SF1 denote RubricRecall , Non-HallucinationRate , and StructuralF1 , respectively. Instruction, Reasoning, and Knowledge correspond to Instruction Following, Multidisciplinary Reasoning, and World Knowledge. The overall scores are macro-averaged across domains. The best and second-best performance in each column are highlighted in bold and underlined , respectively.

Figure 5: Overview of the SciGen-Verifier training pipeline. Starting from the cold-start SFT checkpoint, the verifier policy $\pi_{\theta}$ samples $N$ rollouts $\tau_{1},\dots,\tau_{N}$ ( $N{=}K{=}8$ ) for each problem statement and candidate image, each consisting of a <think> trace, a binary answer, and an explanation with an editing instruction. In Stage 1, the rubric generator produces instance-specific checkpoints $r_{1},\dots,r_{n}$ , a coverage checker $\pi_{\phi}$ measures how many of them are addressed in the reasoning trace ( $\mathcal{R}_{\text{rubric}}$ ), and this process reward is blended with the rule-based judgement reward $\mathcal{R}_{\text{judge}}$ through the mastery-driven coefficients $\alpha_{t}$ and $\beta_{t}$ (Eq. 3 ). In Stage 2, $\mathcal{R}_{\text{judge}}$ is combined with the LLM-judged explanation-editing reward $\mathcal{R}_{\text{exp-edit}}$ (Eq. 4 ). Both stages update $\pi_{\theta}$ with GRPO.

Open-Source Models
MLLMs	Role	Link
Qwen3.5-397B	Large-scale open-source baseline	Qwen/Qwen3.5-397B-A17B
Qwen3-VL-8B-Thinking	Same-scale thinking baseline	Qwen/Qwen3-VL-8B-Thinking
Qwen3-VL-235B-Thinking	Large-scale thinking baseline	Qwen/Qwen3-VL-235B-A22B-Thinking
OmniVerifier-7B	Generated-image verification baseline	comin/OmniVerifier-7B
Proprietary Models

Appendix

Table 7: The details of the baseline models in our comparison.

Verifier	Step 1	Step 2	Step 3	Step 4	Step 5	Step 6	Step 7	Step 8	Step 9	Step 10
Gemini-3.5-Flash	59.4	58.9	61.9	62.6	63.5	63.2	64.1	64.5	64.4	64.9
Qwen3.8-Max	59.0	58.3	61.5	62.0	62.9	63.4	63.2	63.9	64.1	64.4
Seed-2.0-Pro	59.2	55.6	60.8	61.5	59.4	62.6	62.3	61.9	63.0	62.6
SciGen-Verifier-8B	58.7	57.7	61.3	61.0	62.4	63.0	63.0	63.1	63.2	63.5

Appendix

Table 8: Iterative test-time scaling (TTS) results with different verifiers serving as the online critic. The value denotes the relaxed evaluation score of GenExam benchmark Wang et al. (2025b) . We choose the Wan2.6-Image for TTS generation.

Figure 6: Sequential TTS case demonstrations with SciGen-Verifier feedback.

Explore similar work

Jun 23, 2026cs.CV

S1-Omni-Image: A Unified Model for Scientific Image Understanding, Generation, and Editing

We present S1-Omni-Image, an open-weight unified multimodal model for scientific image understanding, generation, and editing. Unlike general-purpose image generation models, scientific image tasks require not only high-fidelity synthesis, but also robust understanding of scientific semantics, structural relations, domain knowledge, and task intent. To this end, S1-Omni-Image builds on the scientific multimodal reasoning backbone S1-VL-32B and couples its understanding capability with an image generation module under a unified think-before-generate paradigm. Given a user instruction, the model first produces a task-oriented reasoning trace, a textual answer, and a task special token; their hidden states are then injected into the generation module to condition image generation or editing. S1-Omni-Image supports scientific image understanding, generation, and editing in a unified framework. For generation, it focuses on scientific illustrations and text rendering, including logical diagrams, relational comparisons, data charts, and realistic scientific visualizations. For editing, it casts segmentation and other domain-specific vision tasks as native image editing problems, enabling multi-turn illustration editing, medical and geographic image segmentation, medical image translation, and scientific image super-resolution. We construct SciGenEdit, a 314K-sample training dataset, and release the model weights, inference code, and SciGenEdit-10K. Experiments show that S1-Omni-Image substantially improves scientific image generation and editing while preserving the scientific image understanding capability inherited from S1-VL-32B. It outperforms open-source models on GenExam and TechImage-Bench, achieves state-of-the-art results on four editing benchmarks including MSD, cigRockSEM, SynthRAD2025, and IXI, and maintains stable performance on scientific image understanding evaluations.

Qingxiao Li, Zikai Wang, Qingli Wang +1

XScience Lab, Wenge AI

Scientific Figure Multimodal Generation Omni

Ask PDF

Jun 24, 2026cs.LG

Can AI Draw Science? A Benchmark for Evaluating Scientific Figure Generation by Text-to-Image and Multimodal Models

Text-to-image and multimodal generative models are increasingly used to produce scientific figures such as mechanism diagrams, experimental-design schematics, conceptual frameworks, and graphical abstracts. Yet existing image-generation benchmarks (e.g., GenEval, T2I-CompBench, DPG-Bench) evaluate natural images and measure compositionality, object counting, or photorealism. None of them measure what makes a generated scientific figure usable: correct and legible text labels, faithful depiction of entities and their relations, coherent diagrammatic structure, and adherence to disciplinary drawing conventions. We introduce SciDraw-Bench, a benchmark of 32 structured scientific-figure generation tasks spanning eight figure types and ten disciplines, where each task pairs a natural-language prompt with a machine-checkable specification of required labels, relations, components, conventions, and negative constraints. We propose a four-dimensional evaluation protocol: Text Fidelity (OCR-based label recall and character error rate), Semantic Correctness (vision-language-model judging against the specification), Structural Quality, and Convention Adherence, together with a meta-evaluation protocol and a preliminary inter-judge reliability analysis (human-rating validation is ongoing). We evaluate a domain-specific system, SciDraw AI, against representative general-purpose text-to-image models, and outline a code-to-figure baseline as a planned extension. In a pilot over all eight figure types, the domain-specific system substantially outperforms the general-purpose baselines on every dimension and figure type, with the largest gaps on semantic correctness and convention adherence; text fidelity remains the hardest dimension for all systems.

Davie Chen

Scientific Figure Text-To-Image Multimodal Model Large Language Model Judges Fidelity

Ask PDF

Jun 29, 2026cs.CV

SciIR: A Large-scale Training Dataset and Benchmark for Scientific Image Reasoning Generation

While Text-to-Image (T2I) models have shown remarkable success in generating photorealistic visual content, they still struggle with the rigorous semantic alignment and logical reasoning required for scientific imagery. Inspired by Peirce's Semiotic Triad, we introduce Scientific Image Reasoning (SciIR), a comprehensive resource for training and evaluation of scientific image generation. We formalize scientific reasoning into three core dimensions: Entity Structure (Icon), Scientific Process (Index), and Scientific Law (Symbol). Specifically, to overcome the scarcity of training data in scientific image generation, we elaborately create SciIR-82k, a large-scale dataset containing over 80,000 high-quality scientific image-text pairs from cutting-edge publications. The dataset is hierarchically organized according to the semiotic dimensions and incorporates a Scientific Reasoning Chain-of-Thought (Sci-RCoT) to explicitly model underlying visual logic. For evaluation, we propose SciIR-Bench, which aligns with these three semiotic levels and employs an Atomic Checklist to convert the outcome-oriented scientific accuracy into process-oriented, verifiable, fine-grained questions. Our extensive experiments reveal significant deficiencies in current models' scientific reasoning capabilities. Furthermore, by fine-tuning on the SciIR-82k dataset, we developed the Qwen-Image-SciIR model, which achieves a substantial improvement on the SciIR-Bench, increasing the final score from 35% to 43%, laying a solid foundation for future advances in scientific image generation.

Zhiyuan Ma, Zhengfeng Shi, Yuning An +6

School of Computer Science and Technology, Huazhong University of Science and Technology, China · School of Airspace Science and Engineering, Shandong University, China · Department of Electronic Engineering, Tsinghua University, China

Scientific Figure Image-Text Pairs Text-To-Image Vision Datasets

Ask PDF