ReState: Noise-Level-Aware State Correction for Training-Free Object Removal in Few-Step Diffusion Models
Organizations: Seoul National University · OGQ
Abstract
Training-free object removal commonly redirects self-attention away from the target region, but on few-step distilled diffusion models this often leaves object-shaped residuals. We identify a temporal mismatch: a structured prediction forms inside the mask at the first forward, and the short trajectory offers few later chances to revise it, while attention rerouting acts mainly in later steps. Across four distilled backbones, magnitude-matched single-forward interventions show that background-directed state correction affects the final result more than attention rerouting. We therefore propose ReState, which corrects the masked clean prediction toward background evidence retrieved from the input using a noise-level-aware weight that vanishes at the final forward. ReState retains object-excluding attention as a complementary constraint and adds no denoiser evaluations. Across two paired benchmarks and two few-step backbones, ReState achieves the best masked PSNR, SSIM, and LPIPS among training-free methods, improving PSNR by 3.17 dB on average over the strongest baseline. On OBER-Test, it also outperforms the 50-step attention-forcing reference on all three metrics at 6.7x lower latency. The same configuration transfers to two additional distilled backbones without retuning. Project page: https://tdrlee.github.io/ReState
Figures & tables
| Property | DesignEdit AAAI’25 | Attentive Eraser AAAI’25 | AdaEraser ICML’26 | PANDORA ICME’26 | FuLLaMa WACV’26 | ReState (Ours) |
|---|---|---|---|---|---|---|
| Writes an explicit background target into the predicted state | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ |
| Adds no forwards or external models beyond the backbone | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ |
| Operates at the native few-step schedule | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ |
| OBER-Test | RORD | |||||||
|---|---|---|---|---|---|---|---|---|
| Method | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | Lat. (s) | VRAM |
| SD-Turbo (4 steps) | 11.27 | 0.203 | 0.380 | 9.86 | 0.162 | 0.438 | 0.174 | 4.58 |
| Attentive Eraser AAAI’25 | 7.28 | 0.063 | 0.516 | 7.22 | 0.060 | 0.555 | 0.200 | 4.58 |
| DesignEdit AAAI’25 | 14.74 | 0.316 | 0.283 | 13.48 | 0.263 | 0.355 | 0.184 | 4.58 |
| AdaEraser ICML’26 | 12.27 | 0.237 | 0.347 | 10.48 | 0.177 | 0.420 | 0.843 | 5.08 |
| ReState (Ours) | 18.19 | 0.485 | 0.186 | 16.59 | 0.440 | 0.280 | 0.305 | 4.58 |
| SDXL-Turbo | SD-Turbo | SDXL-Lightning | Hyper-SDXL | |||||
|---|---|---|---|---|---|---|---|---|
| 14.6 | ||||||||
| 4.1 | ||||||||
| 1.6 | ||||||||
| 0.7 | ||||||||
| Direct correction | AF (all forwards) | AF (last only) | PSNR | SSIM | LPIPS |
|---|---|---|---|---|---|
| ✗ | ✗ | ✗ | 9.86 | 0.133 | 0.435 |
| ✗ | ✗ | ✓ | 12.15 | 0.182 | 0.370 |
| ✗ | ✓ | ✗ | 15.50 | 0.348 | 0.269 |
| ✓ | ✗ | ✗ | 17.62 | 0.477 | 0.237 |
| ✓ | ✗ | ✓ | 18.65 | 0.495 | 0.196 |
| ✓ | ✓ | ✗ | 18.75 | 0.498 | 0.194 |
| Method | Forwards | Latency (s) | PSNR | SSIM | LPIPS |
|---|---|---|---|---|---|
| SDXL-Turbo | 4 | 0.247 | 9.86 | 0.133 | 0.435 |
| Attentive Eraser AAAI’25 | 8 | 0.338 | 7.09 | 0.046 | 0.514 |
| DesignEdit AAAI’25 | 4 | 0.281 | 15.50 | 0.348 | 0.269 |
| AdaEraser ICML’26 | 8 | 1.145 | 11.71 | 0.205 | 0.372 |
| ReState (Ours) | 4 | 0.373 | 18.75 | 0.498 | 0.194 |
| SDXL DesignEdit AAAI’25 | 50 | 2.50 | 17.86 | 0.441 | 0.209 |
| Evidence | PSNR | SSIM | LPIPS |
|---|---|---|---|
| Global background mean | 18.16 | 0.481 | 0.214 |
| Retrieved (default) | 18.75 | 0.498 | 0.194 |
| Ground-truth background (oracle) | 22.32 | 0.588 | 0.133 |
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Training or adaptation data | OBER | RORD |
|---|---|---|---|
| PowerPaint ECCV’24 | OpenImages v6 (seg. subset) + LAION-Aesthetics v2 5+ | ✗ | ✗ |
| CLIPAway NeurIPS’24 | frozen base; embedding projection trained on COCO | ✗ | ✗ |
| EntityErasure CVPR’25 | Open Images V7, about 1.75M (pseudo-labeled) | ✗ | ✗ |
| SmartEraser CVPR’25 | Syn4Removal, 1M (Open Images v7, SAM, COCONut) | ✗ | ✗ |
| EraseLoRA ECCV’26 | none (dataset-free test-time adaptation) | ✗ | ✗ |
| OSOR (SDXL-Inp.) ECCV’26 | CORNE, 280K pairs (curated from NHR-Edit) | ✗ | ✗ |
| Method | Nominal | Effective | Auxiliary pass |
|---|---|---|---|
| SDXL-Turbo | 4 | 4 | none |
| Attentive Eraser AAAI’25 | 4 | 8 | SARG branch pair |
| DesignEdit AAAI’25 | 4 | 4 | none |
| AdaEraser ICML’26 | 8 | 8 | reference forward |
| ReState (Ours) | 4 | 4 | none |
| Backbone | |||||
|---|---|---|---|---|---|
| SDXL-Turbo | |||||
| SD-Turbo | |||||
| SDXL-Lightning | |||||
| Hyper-SDXL | |||||
| Original SDXL (4-step grid) |
| Forward index | Noise level | ||
|---|---|---|---|
| 1 | 13.12 | ||
| 4 | 9.34 | ||
| 8 | 6.20 | ||
| 13 | 3.95 | ||
| 19 | 2.46 | ||
| 25 | 1.62 |
| Denoising evaluations | ||||
|---|---|---|---|---|
| 4 | ||||
| 8 | ||||
| 16 |
| Placement | range | PSNR | SSIM | LPIPS |
|---|---|---|---|---|
| 8 forwards, high-noise range | 11.57 | 0.255 | 0.399 | |
| 8 forwards, late range | 12.47 | 0.284 | 0.353 | |
| Last 38 forwards | 17.17 | 0.429 | 0.221 | |
| All 50 forwards | 17.86 | 0.441 | 0.209 |
| Output | High-band ratio to ground truth |
|---|---|
| No intervention | 8.34 |
| Attention forcing only | 1.92 |
| Direct correction only | 0.30 |
| ReState | 0.51 |
| Realized | OBER-Test | |||||
|---|---|---|---|---|---|---|
| PSNR | SSIM | LPIPS | ||||
| 1 | 0.943 | 0.722 | 0.325 | 18.61 | 0.487 | 0.194 |
| 2 | 0.971 | 0.839 | 0.490 | 18.70 | 0.492 | 0.193 |
| 4 | 0.985 | 0.912 | 0.658 | 18.74 | 0.496 | 0.194 |
| 8 | 0.993 | 0.954 | 0.794 | 18.75 | 0.498 | 0.194 |
| 16 | 0.996 | 0.977 | 0.885 | 18.73 | 0.498 | 0.195 |
| Corrected forwards | PSNR | SSIM | LPIPS |
|---|---|---|---|
| None (attention forcing only) | 15.50 | 0.348 | 0.269 |
| 16.85 | 0.411 | 0.237 | |
| 18.18 | 0.469 | 0.203 | |
| 18.47 | 0.482 | 0.200 | |
| 18.35 | 0.477 | 0.198 | |
| 18.74 | 0.496 | 0.194 |
| Feature source | PSNR | SSIM | LPIPS |
|---|---|---|---|
| Shared first forward (default) | 18.75 | 0.498 | 0.194 |
| No attention suppression | 18.72 | 0.497 | 0.194 |
| Near-clean input ( ) | 17.44 | 0.474 | 0.214 |
| Second forward ( ) | 18.87 | 0.499 | 0.193 |
| Encoder block (native ) | 17.96 | 0.475 | 0.216 |
| Mid block (native ) | 18.75 | 0.499 | 0.194 |
| OBER-Test | RORD | |||||||
| Method | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | Lat. (s) | VRAM |
| Training/Optimization-based methods | ||||||||
| SDXL-Inpainting | 11.65 | 0.365 | 0.391 | 10.57 | 0.348 | 0.403 | 3.75 | 9.0GB |
| FLUX.1-Fill | 11.04 | 0.376 | 0.405 | 12.33 | 0.415 | 0.366 | 15.9 | 33.8GB |
| PowerPaint ECCV’24 | 14.89 | 0.463 | 0.255 | 14.91 | 0.438 | 0.282 | 3.86 | 5.1GB |
| CLIPAway NeurIPS’24 | 15.71 | 0.426 | 0.246 | 14.84 | 0.406 | 0.268 | 7.83 | 9.8GB |
| OBER-Test | RORD | |||||
|---|---|---|---|---|---|---|
| Method | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS |
| SDXL-Lightning (4 steps) | 10.51 | 0.393 | 0.424 | 9.60 | 0.318 | 0.445 |
| DesignEdit AAAI’25 | 15.95 | 0.517 | 0.296 | 15.08 | 0.475 | 0.325 |
| ReState (Ours) | 17.51 | 0.578 | 0.271 | 16.54 | 0.528 | 0.308 |
| Hyper-SDXL (4 steps) | 10.21 | 0.346 | 0.434 | 9.21 | 0.273 | 0.455 |
| DesignEdit AAAI’25 | 15.20 | 0.487 | 0.315 | 14.14 | 0.434 | 0.358 |
| Configuration at 50 steps | PSNR | SSIM | LPIPS |
|---|---|---|---|
| DesignEdit AAAI’25 (reference) | 17.86 | 0.441 | 0.209 |
| Attentive Eraser AAAI’25 | 18.79 | 0.462 | 0.225 |
| AdaEraser ICML’26 | 15.66 | 0.399 | 0.274 |
| ReState, every forward corrected | 17.57 | 0.479 | 0.231 |
| ReState, corrections at | 18.83 | 0.485 | 0.181 |
| ReState on SDXL-Turbo (4 steps; Table 2 ) | 18.75 | 0.498 | 0.194 |
| OBER-Test | RORD | ||||||
| Method | Configuration | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS |
| Attentive Eraser AAAI’25 | SARG ( , official port) | 7.09 | 0.046 | 0.514 | 7.35 | 0.065 | 0.538 |
| SARG ( ) | 7.61 | 0.054 | 0.509 | 7.54 | 0.056 | 0.537 | |
| SARG ( ) | 12.92 | 0.264 | 0.377 | 12.56 | 0.277 | 0.416 | |
| SARG ( , selected) | 15.34 | 0.335 | 0.277 | 14.18 | 0.310 | 0.342 | |
| DesignEdit AAAI’25 | all 4 forwards (official port, selected) | 15.50 | 0.348 | 0.269 | 14.34 | 0.326 | 0.334 |