Lightweight Multimodal LLM-Enabled Cost-Effective Defect Grading of Power Transmission Equipment

Tao Wang; Lipeng Zhu; Jiayong Li; Feng Gao; Siwen Liang

cs.CLMar 25, 2026

Lightweight Multimodal LLM-Enabled Cost-Effective Defect Grading of Power Transmission Equipment

Authors: Tao Wang, Lipeng Zhu, Jiayong Li, Feng Gao, Siwen Liang

Abstract

Defect grading of power transmission equipment (DGPTE) is crucial to the stability of electric energy transmission. Although existing machine learning methods exhibit strong capabilities in defect detection, they are plagued by difficulties in integrating expert experience and facing class imbalance in more refined defect grading field. To address this issue, this paper introduces a novel defect grading framework based on multimodal large language model (MLLM). Specifically, this approach maximizes the commercial MLLMs' potential of DGPTE through in-context learning and obtains the state-of-te-art (SOTA) model. By sending a secondary request to this model, a small number of chain of thought-based question-answer pairs (Q&As) are generated, which effectively reduces the cost of manual annotation. In this way, these high-quality interpretable Q&As are used to train Qwen3-VL-8B via Low-Rank Adaption-based supervised fine-tuning (SFT). Experimental results on three DGPTE tasks demonstrate that fine-tuning only the language model layer yields the SOTA performance. Furthermore, multi-task joint fine-tuning verifies the feasibility of handling multiple grading tasks within only a single lightweight MLLM.

Figures & tables

Acronym	Description
DGPTE	Defect grading of power transmission equipment
MLLM	Multimodal Large Language Model
SFT	Supervised Fine-tuning
LoRA	Low-Rank Adaptation
ICL	In-context learning
FSL	Few-shot learning

TABLE I: Commonly Used Symbols

Panel of Figure 2 — Fig. 1: The three PTEDG tasks used in the experiments of this paper

Fig. 2: The framework of proposed fine-tuning strategy

Execute the checks in steps 1

\to

4 . If a No or Not is returned, the grading outcome shall be determined directly, and the subsequent check items are exempted from further evaluation.

1. Having insulator: Does the image contain at least one insulator string?

\bm{\vdash}

Not Exists

\to

grade result is ”Else”

\bm{\vdash}

Exists

\to

2. Nest location: Is the bird’s nest located on the crossarm, away from the main body of the vertical tangent tower or concrete pole?

TABLE II: Decision Tree-based Chain of Thought with Task 3

Fig. 3: An example of Q&A for MLLM fine-tuning in Task 1

Stage	Parameter	Range / Value
Fine-tuning	Fine-tuning Model	Qwen3-VL-8B
	Fine-tuning Type	SFT
	Fine-tuning Module	{VE, MMA, LLM}
	Fine-tuning Method	LoRA
	LoRA Rank	32
	LoRA Alpha	128

TABLE III: Fine-tuning and Inference Parameters

Case	Only result	DT-based CoT result	Reference images
Case 1	$\checkmark$
Case 2		$\checkmark$
Case 3	$\checkmark$		$\checkmark$
Case 4		$\checkmark$	$\checkmark$

TABLE IV: Four prompt settings for testing the commercial MLLMs

Model	Task 1				Task 2				Task 3
Model	Case 1	Case 2	Case 3	Case 4	Case 1	Case 2	Case 3	Case 4	Case 1	Case 2	Case 3	Case 4
GPT-4.1-2025-04-14	53.24%	54.64%	74.85%	89.82%	43.49%	44.80%	46.25%	50.66%	43.86%	48.26%	50.50%	52.61%
GPT-4o-2024-11-20	54.37%	55.61%	74.47%	88.64%	30.18%	41.20%	46.98%	48.03%	42.50%	39.50%	48.95%	51.60%
GPT-5-chat	51.84%	51.24%	80.68%	90.26%	30.18%	45.72%	52.64%	56.58 %	42.12%	39.50%	46.50%	45.00%
Qwen-VL-plus	53.99%	55.81%	73.53%	84.92%	48.64%	46.90%	50.08%	54.56%	40.23%	42.54%	32.86%	33.45%
Qwen3-VL-235B-a22b	61.18%	69.38%	81.53%	88.22%	38.55%	52.64%	48.64%	52.64%	43.50%	42.65%	45.79%	48.26%

TABLE V: Accuracy comparison of different MLLMs in DGPTE

Fig. 4: Accuracy comparison of different fine-tuning modules

Fig. 5: Accuracy comparison of different fine-tuning strategies

Fig. 6: two generated defect grading examples for Task 3

Image Location	Task 1				Task 2				Task 3
	Before Fine-tuning		After Fine-tuning		Before Fine-tuning		After Fine-tuning		Before Fine-tuning		After Fine-tuning
	ACC	MF1	ACC	MF1	ACC	MF1	ACC	MF1	ACC	MF1	ACC	MF1
At the Front	42.11%	56.83%	50.23%	53.17%	32.80%	46.08%	34.66%	36.52%	23.14%	38.94%	27.50%	29.36%
At Corresponding Position	59.06%	71.03%	92.63%	92.50%	34.79%	49.51%	85.85%	88.01%	30.61%	46.02%	77.04%	77.32%
At the End	43.86%	58.02%	50.43%	53.30%	30.06%	44.40%	35.33%	37.20%	25.88%	42.12%	27.50%	30.58%

TABLE VI: Performance Comparison by Image Location

Size	ACC	MF1	Training Time (h)	Training VRAM (GB)	Inference Time (s)	Inference VRAM (GB)
p224	68.55%	67.76%	2.246	255.996	5.046	12.969
p448	72.76%	73.88%	2.956	282.395	5.140	22.495
p768	77.42%	80.02%	3.907	325.293	6.975	31.850
p1280	85.17%	85.94%	9.469	489.999	9.236	42.285
p2560	-	-	-	-	10.564	74.592

TABLE VII: Performance Overhead by Different Input Sizes

Multimodal Large Language Models

Multimodal large language models combine language models with image, video, audio, and other inputs for grounded understanding and generation.

See all 1,092 papers

Synthetic Defect Image Generation for Power Line Insulator Inspection Using Multimodal Large Language Models
Mar 9, 2026Xuesong Wang, Caisheng WangSynthetic Data Augmentation Industrial Inspection
Department of Electrical and Computer Engineering, Wayne State University, 42 W. Warren Ave., Detroit, 48201, MI, USA
LDU-Bench: Multimodal LLM Evaluation for Lithography Defect Understanding under Layout-Varying Circuit Backgrounds
Aug 4, 2026Huanglong Ji, Botong Zhao, Shujing Lv +1Semiconductor Manufacturing Multimodal Large Language Models
EAGLE: Expert-Augmented Attention Guidance for Tuning-Free Industrial Anomaly Detection in Multimodal Large Language Models
Feb 19, 2026Xiaomeng Peng, Xilang Huang, Seon Han ChoiAnomaly Detection Industrial Anomaly Detection
Ewha Womans University

Mar 9, 2026J/K move · Enter open · S save

Synthetic Defect Image Generation for Power Line Insulator Inspection Using Multimodal Large Language Models

Xuesong Wang, Caisheng Wang

Department of Electrical and Computer Engineering, Wayne State University, 42 W. Warren Ave., Detroit, 48201, MI, USA

Synthetic Data Augmentation Industrial Inspection Synthetic Anomaly Generation Industrial Anomaly Detection

Open paper AskPDF arXiv

Utility companies increasingly rely on drone imagery for post-event and routine inspection, but training accurate defect-type classifiers remains difficult because defect examples are rare and inspection datasets are often limited or proprietary. We address this data-scarcity setting by using an off-the-shelf multimodal large language model (MLLM) as a training-free image generator to synthesize defect images from visual references and text prompts. Our pipeline increases diversity via dual-reference conditioning, improves label fidelity with lightweight human verification and prompt refinement, and filters the resulting synthetic pool using an embedding-based selection rule based on distances to class centroids computed from the real training split. We evaluate on ceramic insulator defect-type classification (shell vs. glaze) using a public dataset with a realistic low training-data regime (104 real training images; 152 validation; 308 test). Augmenting the 10% real training set with embedding-selected synthetic images improves test F1 score (harmonic mean of precision and recall) from 0.615 to 0.739 (20% relative), corresponding to an estimated 4--5x data-efficiency gain, and the gains persist with stronger backbone models and frozen-feature linear-probe baselines. These results suggest a practical, low-barrier path for improving defect recognition when collecting additional real defects is slow or infeasible.

Abstract

Multimodal Large Language Models

Explore similar work

Synthetic Defect Image Generation for Power Line Insulator Inspection Using Multimodal Large Language Models

LDU-Bench: Multimodal LLM Evaluation for Lithography Defect Understanding under Layout-Varying Circuit Backgrounds

EAGLE: Expert-Augmented Attention Guidance for Tuning-Free Industrial Anomaly Detection in Multimodal Large Language Models