cs.CVNov 3, 2026

Weight-Space Programming for Improving Image-Restoration Diffusion Models: Technical Conjectures and a Recursive Improvement Loop

Authors: Zhuoran Zheng, Doubao

Abstract

Can an LLM directly edit the weights of a diffusion-based image-restoration model to improve restoration quality---without gradients, paired data, or fine-tuning? Recent experiments show frontier LLMs can fix structural bugs by modifying as little as 0.016%0.016\% of parameters, but cannot instill new capabilities. We ask whether weight-space programming (WSP) can move from debugging to improvement in the precise-regression setting of image restoration (deblurring, super-resolution, denoising, inpainting). Restoration diffusion UNets present a middle ground: low-level signal-processing layers are concentrated enough for targeted weight-direction edits, while high-level semantic layers remain distributed and resistant to sparse modification. We propose three technical conjectures: (C1) an LLM can localize the layer behind a quality gap (over-smoothing, color shift, hallucinated texture) within a 200 forward-pass budget; (C2) editing a small set of identified weight directions (e.g., scaling high-frequency filter norms, correcting output-bias channels) can measurably improve PSNR and perceptual quality; (C3) a recursive loop---diagnose →\rightarrow edit →\rightarrow evaluate →\rightarrow re-diagnose---can compound gains across rounds, forming a minimal recursive self-improvement (RSI) loop for vision. We analyze the required AI capability (arguing targeted edits need not superintelligence, while general capability generation does) and detail the challenges of precise pixel-level regression. We provide a PyTorch-style deployment pipeline and outline a benchmark. This manuscript is conjectural, intended to open a new subtopic for image restoration.

Figures & tables

Explore similar work

CardsList
  1. ImIR: Image-Instruction Tuning for All-in-One Image Restoration

    Sep 21, 2026Süleyman Aslan, Görkay Aydemir, Mısra Yavuz +5Visual Instruction TuningAll-in-One Image Restoration

  2. Restoring without Forgetting: Filter-Level Continual Image Restoration via Parameter-Space Integrated Gradients

    Sep 29, 2026Xin Feng, Jin Zhao, Yizhen Zhang +3Integrated GradientsImage Restoration

  3. Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors

    Jul 28, 2026Jaeha Kim, Kyoung Mu LeeImage RestorationTask-Driven Image Restoration