cs.SENov 14, 2026

Learning from Mutants: Teaching Small Language Models to Synthesise Loop Invariants

Authors: Daragh King, Vasileios Koutavas

Organizations: School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland

Abstract

Large language models (LLMs) have proven promising for synthesising loop invariants, a long-standing obstacle to automated program verification. Yet, frontier LLMs are typically accessible only through third-party APIs, hampering reproducibility and local deployment. Small language models (SLMs) sidestep these obstacles but require task-specific fine-tuning, for which high-quality training data is needed. We present a failure-driven data curation method for fine-tuning language models to synthesise loop invariants. Inspired by mutation testing, our approach applies semantic mutation operators to ground-truth invariants, producing semantic mutants that induce verification failure. Each mutant is paired with verifier feedback and its ground-truth counterpart, yielding training examples for feedback-guided invariant refinement. Whereas the vast space of mutants is a long-standing burden in mutation testing, it becomes an asset when generating training data, multiplying the examples obtainable from a fixed set of verified programs. We implement our approach for Dafny, augmenting an existing synthetic dataset of verified programs, and introduce a framework that iteratively refines candidate invariants using verifier feedback. On 472 DafnyBench programs, our best-performing fine-tuned Gemma-4-E4B configuration verifies 50.8% of programs zero-shot, compared to 8.1% for the base model, a 6.3×\times improvement. Higher-order mutants prove more effective as training data than first-order mutants, and adding mutation-based examples to direct synthesis examples, in which invariants are removed rather than mutated, improves performance under iterative refinement. Our approach also transfers to Qwen3.5-4B, whose zero-shot verification rate rises from 9.3% to 53.6%.

Figures & tables

Explore similar work

CardsList
  1. Not All Invariants Are Equal: Curating Training Data to Accelerate Program Verification with SLMs

    Mar 16, 2026Ido Pinto, Yizhak Yisrael Elboher, Haoze Wu +2Fine-TuningTraining Data Curation

  2. ExVerus: Verus Proof Repair via Counterexample Reasoning

    Mar 26, 2026Jun Yang, Yuechun Sun, Yi Wu +5Formal Verification

  3. Automating Formal Verification with Reinforcement Learning and Recursive Inference

    May 29, 2026Max TanRL for Code GenerationInference-Time Search