PromptPack: Training-Free Augmentation of Diffusion Editors for Intent-Aligned Spatial and Semantic Control
Organizations: NVIDIA
Abstract
Many image-editing tasks require confining changes to intended spatial regions while exactly preserving user-designated source content. Instruction-driven editing pipelines often leave such user-intent underspecified, while inpainting editors condition generation on explicitly defined edit support but not on the spatial assignment of individual edit concepts. PromptPack addresses both limitations by equipping diffusion editors with two explicit 'Edit Guides': an edit-role partition that labels each pixel as 'Keep,' 'Permit,' or 'Modify' to specify where edits may occur, and a regional-prompt assignment that maps spatial regions to text prompts to specify which concepts apply where. These guides may be manually specified or, for source-object edits, automatically derived from a text-grounded SAM 3 mask. Without retraining, PromptPack injects independently embedded regional prompts into the editor's fixed-length text sequence and uses the Edit Guides to spatially mask attention, gate flow updates, and restore protected source pixels. We integrate PromptPack into FlowEdit FLUX, FlowEdit SD3, and FLUX Kontext and demonstrate improved requested-concept placement and protected-content preservation across all three integrations on PIE-Bench++. In a pairwise user study, PromptPack achieves an 87% preference rate over FlowEdit FLUX, establishing PromptPack as a practical, training-free interface for intent-aligned diffusion editing.
Figures & tables
| Method | SSIM | LPIPS | ESR | L-ESR | WRR | CLIP | |
|---|---|---|---|---|---|---|---|
| FLUX Kontext | .727 | .255 | 8.095 | 57.8 | 32.0 | 14.1 | .320 |
| FLUX Kontext + PromptPack | .887 | .149 | .000 | 60.2 | 50.0 | 13.3 | .328 |
| SpotEdit [ 22 ] | .756 | .235 | 7.722 | 53.1 | 29.7 | 14.8 | .320 |
| FlowEdit FLUX | .741 | .280 | 12.710 | 52.3 | 33.6 | 4.7 | .318 |
| FlowEdit FLUX + PromptPack | .821 | .198 | .002 | 58.6 | 50.0 | 1.6 | .322 |
| CannyEdit [ 31 ] | .600 | .412 | 4.150 | 64.8 | 53.1 | 9.4 | .326 |
| Conditioning | SSIM | L-ESR | s | GiB | |
|---|---|---|---|---|---|
| Ordinary joint | 512 | .836 | 45.3 | 8.21 | 37.55 |
| Packed, no routing | 512 | .824 | 43.0 | 8.13 | 37.74 |
| Expanded routed | 1536 | .813 | 46.9 | 12.96 | 47.99 |
| Packed routed (ours) | 512 | .821 | 50.0 | 9.70 | 37.74 |
| Gate / editor | SSIM | L-ESR | |
|---|---|---|---|
| Tight ( ) | .858 | .658 | 40.0 |
| Wide ( ) | .773 | 1.135 | 45.0 |
| K/P/M | .849 | .666 | 42.5 |
| FLUX inpaint | .647 | 1.116 | 28.7 |
| Setting | FlowEdit FLUX | FlowEdit SD3 | FLUX Kontext |
| Sampling steps | 28 | 50 | 28 |
| Dilation begins after | 15 | 15 | 15 |
| Magnitude threshold | 4 | 0.5 | – |
| Flow boost | 1.6 | 1.6 | – |
| Output-support kernel | 10 | 10 | 10 |
| Boundary blur | 10 | 10 | 10 |
| Integration | Grid | Dilation parameter | Late blur |
|---|---|---|---|
| FlowEdit FLUX | max-pool width | none | |
| FlowEdit SD3 | max-pool width | none | |
| FLUX Kontext | repeated dilation |
| Integration | Regional added | Total used | Remaining |
|---|---|---|---|
| FlowEdit FLUX | 9 (19) | 32 (52) / 512 | 480 (460) |
| FLUX Kontext | 9 (19) | 18 (37) / 512 | 494 (475) |
| FlowEdit SD3 | 10 (19) | 30 (48) / 333 | 303 (285) |
| Method | Time (s) | Peak alloc. (GiB) |
|---|---|---|
| FlowEdit FLUX | 37.55 | |
| + PromptPack | 37.74 | |
| Overhead | +17.6% | +0.5% |
| SAM query | Score | Components | Grounded |
|---|---|---|---|
| red apple | .965 | 1 | Yes |
| white dress | .958 | 2 | Yes |
| small brown bird | .980 | 1 | Yes |
| wooden chess pieces | .965, .969 | 2 | Yes |
| dry tree | – | – | No |
| teapot | .589 | 1 | Yes |
| Setting | FlowEdit FLUX | FlowEdit SD3 | FLUX Kontext |
|---|---|---|---|
| Steps | 28 | 50 | 28 |
| Seed | 0 | 0 | 0 |
| Source/target guidance | 1.5/5.5 | 1.5/5.5 | – |
| Flow range | 0–24 | 0–24 | – |
| Threshold | 4 | 2 | – |
| Boost | 1.3 | 1.2 | – |
| Model | PP pref. [95% CI] | Users | Judg. | |
|---|---|---|---|---|
| FlowEdit FLUX | 87.2 [83.9, 90.4] | 26 | 769 | 0.001 |
| FLUX Kontext | 67.7 [60.5, 73.5] | 26 | 768 | 0.001 |
| FlowEdit SD3 | 54.7 [50.6, 59.0] | 25 | 767 | 0.055 |
| Method | SSIM | LPIPS | ESR | L-ESR | WRR | CLIP | |
|---|---|---|---|---|---|---|---|
| FLUX Kontext | .718 | .269 | 8.276 | 58.1 | 34.4 | 14.4 | .319 |
| + PromptPack | .878 | .162 | .000 | 58.8 | 48.1 | 14.4 | .327 |
| SpotEdit [ 22 ] | .744 | .249 | 7.870 | 53.8 | 31.9 | 13.8 | .319 |
| FlowEdit FLUX | .745 | .277 | 12.769 | 47.5 | 30.0 | 3.8 | .317 |
| + PromptPack | .818 | .203 | .002 | 55.6 | 47.5 | 1.2 | .322 |
| FlowEdit SD3 | .908 | .058 | 1.564 | 30.0 | 11.9 | 4.4 | .300 |
| Metric | FLUX Kontext | FlowEdit FLUX | FlowEdit SD3 |
|---|---|---|---|
| SSIM | |||
| LPIPS | |||
| ESR | |||
| L-ESR | |||
| WRR |
| Alternative | SSIM | LPIPS | ESR | L-ESR | WRR |
|---|---|---|---|---|---|
| SpotEdit | |||||
| CannyEdit | |||||
| KV-Edit |
| Criterion | FLUX Kontext | FLUX | SD3 |
|---|---|---|---|
| Confidence | +13.8 | +17.5 | +6.3 |
| Confidence | +10.6 | +18.1 | +4.4 |
| Confidence | +12.5 | +16.3 | +5.0 |
| Confidence | +15.0 | +12.5 | +3.8 |
| Any overlap | +2.5 | +13.1 | +5.6 |
| Box 25% | +10.6 | +15.6 | +7.5 |
| Conf. | Flow | PP | Canny | PP–Flow |
|---|---|---|---|---|
| .15 | 79.7 | 88.3 | 90.6 | |
| .20 | 77.3 | 85.9 | 84.4 | |
| .25 | 71.9 | 82.8 | 82.0 | |
| .30 | 68.0 | 82.0 | 78.9 |
| Method | Inside | Outside | Absent | Uncert. | Unres. | Auto L-ESR |
|---|---|---|---|---|---|---|
| FlowEdit FLUX | 14 | 2 | 9 | 2 | 3 | 33.3 |
| PromptPack | 25 | 0 | 4 | 1 | 0 | 60.0 |
| CannyEdit | 24 | 1 | 2 | 2 | 1 | 63.3 |
| Box crop | Masked crop | |||
|---|---|---|---|---|
| Method | RPR | Margin | RPR | Margin |
| FLUX Kontext | 63.0 | .0124 | 67.5 | .0186 |
| PromptPack | 66.3 | .0178 | 72.0 | .0265 |
| FlowEdit FLUX | 65.9 | .0148 | 75.2 | .0267 |
| PromptPack | 69.1 | .0193 | 77.6 | .0335 |
| FlowEdit SD3 | 58.1 | .0095 | 62.2 | .0109 |
| Conditioning | SSIM | LPIPS | ESR | L-ESR | WRR | s / GiB | |
|---|---|---|---|---|---|---|---|
| Ordinary joint | 512 | .836 | .180 | 58.6 | 45.3 | 3.1 | 8.21 / 37.55 |
| Packed, no routing | 512 | .824 | .195 | 57.8 | 43.0 | 7.8 | 8.13 / 37.74 |
| Expanded routed | 1536 | .813 | .206 | 58.6 | 46.9 | 6.2 | 12.96 / 47.99 |
| Packed routed (ours) | 512 | .821 | .198 | 58.6 | 50.0 | 1.6 | 9.70 / 37.74 |
| Budget | Retained | SSIM | L-ESR | RPR | CLIP |
|---|---|---|---|---|---|
| 4 slots | 4.0 (48%) | .869 | 38.7 | 76.8 | .322 |
| 8 slots | 7.6 (87%) | .860 | 38.7 | 87.5 | .324 |
| Full | 9.1 (100%) | .858 | 45.2 | 89.3 | .326 |
| Condition | SSIM | ESR | L-ESR | |
|---|---|---|---|---|
| Tight ( ) | .858 | .658 | 56.2 | 40.0 |
| Wide ( ) | .773 | 1.135 | 62.5 | 45.0 |
| K/P/M | .849 | .666 | 57.5 | 42.5 |
| FLUX inpaint | .647 | 1.116 | 53.8 | 28.7 |
| Gate | Support | SSIM | ESR | L-ESR |
|---|---|---|---|---|
| Tight | 15.2 | .942 | 33.3 | 33.3 |
| Wide | 29.2 | .903 | 33.3 | 25.0 |
| Symmetric K/P/M | 29.2 | .925 | 33.3 | 33.3 |
| Directional K/P/M | 23.2 | .929 | 33.3 | 33.3 |
| Threshold | Detection (%) | SSIM |
|---|---|---|
| 0 | 43.1 7.6 | 0.8058 0.0211 |
| 2 | 41.9 7.6 | 0.8118 0.0209 |
| 4 | 40.6 7.5 | 0.8508 0.0173 |
| 6 | 35.0 7.3 | 0.8987 0.0128 |
| 8 | 28.1 6.9 | 0.9313 0.0097 |
| 12 | 18.1 5.9 | 0.9541 0.0068 |
| Boost value | Detection (%) | SSIM |
|---|---|---|
| 1.0 | 35.6 7.3 | 0.9029 0.0121 |
| 1.2 | 40.0 7.5 | 0.8681 0.0158 |
| 1.4 | 44.4 7.6 | 0.8324 0.0193 |
| 1.6 | 50.0 7.7 | 0.7934 0.0226 |
| 1.8 | 45.0 7.6 | 0.7593 0.0261 |
| 2.0 | 43.8 7.6 | 0.7246 0.0293 |
| Ablation | SSIM | LPIPS | ESR | CLIP |
|---|---|---|---|---|
| Full reference | .85 | .17 | 56.3 | .32 |
| No prompt packing | .86 | .16 | 49.4 | .32 |
| No attention redirection | .85 | .17 | 52.5 | .32 |
| No compression correction | .83 | .18 | 43.8 | .32 |
| No spatial gating | .83 | .21 | 50.0 | .32 |
| No magnitude threshold | .84 | .18 | 55.6 | .32 |
Diffusion-Based Image Editing
Diffusion-based image editing changes image content, style, or attributes while aiming to preserve the parts users want to keep.
See all 238 papers