K-OPSD: Verifiable On-Policy Self-Distillation for Post-Training Vision-Language Models on AEC Drawings

Yunfei Bai; Enrico Chionna; Akash Amol; Kawaljit Singh KC; Joern Tinnemeyer

cs.AISep 28, 2026

K-OPSD: Verifiable On-Policy Self-Distillation for Post-Training Vision-Language Models on AEC Drawings

Authors: Yunfei Bai, Enrico Chionna, Akash Amol, Kawaljit Singh KC, Joern Tinnemeyer

Organizations: Amazon Web Services

Abstract

Interpreting architecture, engineering, and construction (AEC) drawings is hard for general Multimodal Large Language Models (MLLMs) and vision-language models (VLMs). We introduce K-OPSD, a VLM post-training methodology for improving AEC drawing understanding. Building on On-Policy Self-Distillation (OPSD) with verifiable supervision, we construct a teacher from the model's own best-of-N generations, certified by a process-level verifier, and rescue failed prompts by resampling under a hint that exposes the verified answer. We then perform an on-policy model update by training on verified completions with a cross-entropy inner-loss, outperforming the bounded token-wise generalized Jensen-Shannon divergence (JSD) used by on-policy distillation. Using K-OPSD, we fine-tune Qwen3-VL models on the AECV-Bench dataset. The resulting models attain the top average judge score (0.819) and combined accuracy (0.738), achieving competitive results against open-source baseline models. The recipe transfers to the out-of-domain ArchCAD dataset, where the 8B model gains most. We present the verifier suite and the continual learning and self-improving pipeline, our results provide preliminary evidence that verifier-guided self-distillation is a promising route toward more reliable machine reading of architecture drawings.

Figures & tables

Figure 1: K-OPSD Training Paradigm. First, SFT establishes a format-conformant base on SME-validated, correctness-filtered ground truth, which is then frozen for all subsequent training. Each round of verifiable on-policy self-distillation samples best-of- $N$ completions from the current student, keeps only those a process-level verifier certifies, and rescues unsolved prompts by resampling under a hint that is stripped and re-verified before the trace becomes a target. Then a fresh LoRA adapter is fit on the frozen SFT-base over the verified corpus by cross-entropy inner loss and evaluated on a held-out split. After $R$ rounds the best-scoring adapter is promoted as the final model.

		AECV-Bench		ArchCAD
		Avg Judge Score	Combined Accuracy	Avg Judge Score	Combined Accuracy
Open-source model	Qwen3-VL-235B	0.698	0.667	0.538	0.268
	Pixtral-Large-2502	0.687	0.643	0.514	0.293
	Kimi-K2.5	0.765	0.714	0.619	0.415
	Gemma-3-27B	0.731	0.714	0.543	0.268
	Llama4-Maverick-17B	0.795	0.738	0.478	0.171

Table 1: Fine-tuned VLMs versus frontier and open-source MLLM/VLMs , on the in-domain AECV-Bench and out-of-domain ArchCAD sets. FT-Qwen3-VL-4B and -8B are models post-trained with our K-OPSD recipe. The FT-Qwen3VL-4B leads AECV-Bench accuracy and FT-Qwen3VL-8B leads ArchCAD accuracy, at a fraction of the baselines’ parameters.

Figure 2: K-OPSD training dynamics across rounds. Training loss (left), held-out eval loss (middle), and held-out generation entropy (right) over training rounds for Qwen3-VL-2B, 4B, and 8B. All three quantities decline for every scale, with step drops at round boundaries as each round fits a fresh adapter on the frozen SFT base. Held-out loss and entropy fall together indicating the verified self-teacher steadily lowers error while the policy grows more confident. The entropy trace is monitored for distribution collapse across rounds.

	Qwen3-VL-2B		Qwen3-VL-4B		Qwen3-VL-8B
	Avg Judge Score	Combined Accuracy	Avg Judge Score	Combined Accuracy	Avg Judge Score	Combined Accuracy
AECV-Bench
Base model	0.417	0.619	0.771	0.738	0.738	0.643
OPD	0.665	0.595	0.780	0.714	0.792	0.738
OPSD	0.588	0.476	0.713	0.690	0.760	0.690
VS-OPSD	0.676	0.667	0.730	0.667	0.736	0.690

Table 2: Method-ladder ablation on AECV-Bench and ArchCAD for Qwen3-VL at 2B, 4B, and 8B under each training stage: base, OPD, OPSD, VS-OPSD, K-OPSD. The ladder isolates three axes: external teacher vs. self-teacher, the verification gate, and the inner loss, showing that K-OPSD (cross-entropy on the verified corpus) is competitive across the methods.

Figure 3: Verified training-target yield by OPSD vs VS-OPSD vs OPD across three rounds for Qwen3-VL-2B, 4B, and 8B. At every scale and round, VS-OPSD (with best-of- $N$ verification gate) yields far more correct rollouts than OPSD and matches or exceeds OPD, evidence that the verification gate, not a stronger teacher, drives the supply of verified self-generated targets.

Figure 4: Downstream quality by inner-loss loop: VS-OPSD by JSD vs K-OPSD by cross-entropy across three rounds for Qwen3-VL-2B, 4B, and 8B. Both methods share the identical best-of- $N$ verified teacher construction and differ only in the inner loss. For both average judge score (top) and combined accuracy (bottom), K-OPSD matches or exceeds JSD at nearly every scale and round, and is more stable and more reliable as a distillation objective.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Example of VQA on AEC drawing : given an engineering drawing (left) and a query (bottom), the model localizes the relevant elements, examines the spatial distribution, and performs visual reasoning (top) that enables the model to provide an answer (right) for complex, real-world diagrams.

Figure 6: K-OPSD train rollout : composition of the 121-prompt training pool per K-OPSD round for Qwen3-VL-2B, 4B, and 8B. Each bar splits the pool into prompts solved unaided by the best-of- $N$ rollout (success, green) vs those failed and sent to hinted rescue (orange)

Method	Teacher model	Verification Gate	Inner-loop Loss
OPD	External (Claude Fable 5)	None	generalized JSD
OPSD	On-policy self-teacher	None	generalized JSD
VS-OPSD	On-policy self-teacher	process-level verifier	generalized JSD
K-OPSD (Ours)	On-policy self-teacher	process-level verifier	Cross-entropy

Appendix

Table 3: The method ladder used in the controlled ablation , characterized along three axes: the teacher vs the model’s own on-policy self-teacher, with/without process-level verification gate with best-of-

N

, and the inner loss that internalizes the weights update using generalized JSD versus cross-entropy on the verified completion.

Explore similar work

MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding
May 29, 2026Qian Kou, Xiaofeng Shi, Yulin Li +4Visual Reasoning Multimodal Large Language Models
Beijing Academy of Artificial Intelligence (BAAI), China · Institute of Information Engineering, Chinese Academy of Sciences, China · Beijing University of Technology, China
Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation
Jul 25, 2026Shuai Wang, Daoan Zhang, Zhe Tang +2Recent Vision-Language Models Unsupervised On-Policy Self-Distillation
The Hong Kong University of Science and Technology (Guangzhou) · University of Rochester · Zhejiang University of Technology +1
OPD-V: Visual On-Policy Self-Distillation with Modality Balance
Aug 5, 2026Aniri, Jinhe Bi, Peng Liao +5Multimodal Large Language Models Recent Vision-Language Models
2Ludwig Maximilian University of Munich · 3Munich Center for Machine Learning · 1National University of Singapore +1

May 29, 2026, cs.CVJ/K move · Enter open · S save

MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding

Qian Kou, Xiaofeng Shi, Yulin Li, Xiaosong Qiu+3

Beijing Academy of Artificial Intelligence (BAAI), China · Institute of Information Engineering, Chinese Academy of Sciences, China · Beijing University of Technology, China

Visual Reasoning Multimodal Large Language Models Question-Answer Pairs

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) have demonstrated significant achievements in general visual question answering (VQA) tasks. However, they remain brittle on mechanical engineering drawings, where high annotation density and weak domain knowledge, compounded by unreliable spatial relation reasoning under strict projection rules and geometric constraints, make decisive cues easy to miss and frequently lead to wrong answers. To bridge this gap, we introduce the first comprehensive mechanical drawing understanding dataset, MechVQA, created through a semi-automated construction and quality-control pipeline. MechVQA contains 3.3k high-density pictures with 21K question-answer pairs, spanning 10 different fine-grained tasks across three capability levels: Recognition, Reasoning, and Judging, providing a testbed to evaluate and improve MLLM understanding on real-world mechanical drawings. On top of MechVQA, we then develop the MechVL model through a multi-stage training paradigm, building a strong domain-specialized baseline. Extensive experimental results demonstrate that MechVL outperforms the strongest closed-source baseline by 7.57 percentage points on the MechVQA total score, significantly enhancing mechanical drawing understanding ability and providing a reusable foundation for deploying MLLMs in mechanical design and inspection scenarios.

Abstract

Explore similar work

MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding

Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation

OPD-V: Visual On-Policy Self-Distillation with Modality Balance