Enhancing Small Language Models for Power Outage Report Generation via Minimum Risk Training

Hung Phan; Waqwoya Abebe; Youssef Hussein; Supriya Chinthavali; Dalton Lunga; Ali Jannesari

cs.AISep 23, 2026

Enhancing Small Language Models for Power Outage Report Generation via Minimum Risk Training

Authors: Hung Phan, Waqwoya Abebe, Youssef Hussein, Supriya Chinthavali, Dalton Lunga, Ali Jannesari

Organizations: Iowa State University · Oak Ridge National Laboratory · University of Minnesota

Abstract

Minimum Risk Training (MRT) enables neural machine translation models to directly optimize sequence-level evaluation metrics instead of relying only on token- level maximum-likelihood objectives Shen et al. [2016]. Although introduced a decade ago, recent work shows renewed potential for risk-based optimization in modern language models Yang et al. [2024], Jinnai et al. [2025]. We apply MRT to power outage report generation for the Outage Data Initiative Nationwide (ODIN), transforming heterogeneous reports into standardized XML compliant with CIM IEC 61968-3. Our MRT approach improves Qwen2.5-7B-Instruct overall accuracy from 16.20% to 68.95%, demonstrating the effectiveness of sequence- level optimization for domain-specific structured generation

Figures & tables

Configuration	Overall	Text Acc/	Tag Acc.
Qwen2.5-7B-Instruct	16.20%	28.85%	3.56%
MRT over Qwen2.5-7B-Instruct (Ours)	68.95%	50.73%	87.17%

Table 1: Performance comparison across training objectives and baselines.

Power Systems

Power systems research studies the planning, operation, control, and protection of electrical grids and their integration with energy resources.

See all 231 papers

POTracker: Optimizing Large Language Models for Standard-Compliant Power Outage Report Generation
Jun 22, 2026Hung Phan, Aniroop Naladala, Dubey Avanindra +3LLM Fine-Tuning Structured Output Generation
Iowa State University · Oak Ridge Nat. Laboratory
Evaluating Large Language Models for Forced Outage Risk Prediction: Benefits and Comparison to Machine Learning
Sep 2, 2026Christos Petridis, Zoran Obradovic, Mladen KezunovicLLM Evaluation Zero-Shot Learning
Temple University · Texas A&M University
North Small Translate: Advanced Cost-Effective Translation (Cohere CAT+)
Sep 12, 2026Tom Kocmi, Alexandre Bérard, Phil Blunsom +20Machine Translation Language Model Post-Training
Cohere

Jun 22, 2026J/K move · Enter open · S save

POTracker: Optimizing Large Language Models for Standard-Compliant Power Outage Report Generation

Hung Phan, Aniroop Naladala, Dubey Avanindra, Supryia Chinthavali+2

Iowa State University · Oak Ridge Nat. Laboratory

LLM Fine-Tuning Structured Output Generation Power Systems

Open paper AskPDF arXiv

Recent large language models (LLMs) are good at general text generation, but it is still hard to use them for domain-specific data generation because the output must follow strict formatting and structural rules. Unlike open-ended tasks such as question answering or translation, domain-specific generation must be both semantically correct and compliant with existing guidelines and standards. In this work, we study the nationwide interoperability problem of utility power outage reports in the United States. In practice, outage reports need to be machine-readable (e.g., JSON or XML) and must strictly follow requirements from energy-sector regulatory bodies. To address this problem, we propose POTracker, an optimized LLM for power outage report generation. We fine-tune Qwen2.5-7B-Instruct using our proposed objective. The key contribution is a new loss function, POTrackerLoss, that considers both textual similarity and structural (tag) similarity between the generated report and the ground-truth report. We evaluate POTracker on a dataset of 1,000 power outage reports and compare it with five well-known fine-tuning methods and one rule-based XML conversion method. Results show that POTracker outperforms other fine-tuning approaches, improving overall accuracy by up to 51% and reaching 86.47% structural accuracy for generated power outage reports. In addition, we conduct a human study to assess the quality of the ground-truth standard reports, where domain experts assign the generated labels an average score of 4.03 on a 0--5 scale.

Abstract

Power Systems

Explore similar work

POTracker: Optimizing Large Language Models for Standard-Compliant Power Outage Report Generation

Evaluating Large Language Models for Forced Outage Risk Prediction: Benefits and Comparison to Machine Learning

North Small Translate: Advanced Cost-Effective Translation (Cohere CAT+)