LOCUS: Causal Support Windows for Attribute Control in Autoregressive Image Generation
Organizations: Independent Researcher
Abstract
Unified autoregressive (AR) image generators struggle to satisfy multiple attribute constraints in a single prompt. Although these attributes are decodable from image-token latents, editing those latents at test time remains difficult. Adding a fixed attribute direction to the prefix can reinforce a requested attribute but rarely overrides a conflicting one, such as turning a car red when the prompt specifies blue. We investigate how the effect of an intervention depends on where it acts in the image-token sequence. We introduce the causal support profile, which measures, for each attribute, how often patching image-token states within a given window flips the rendered attribute. Structural attributes (count and position) are most responsive in early, narrow windows, while appearance attributes (color, size, and material) respond over later, broader windows. A budget-matched comparison of early and middle windows shows that the effect of window location depends on the target attribute, and two-dimensional support maps show that the responsive regions track spatial content. Based on these profiles, we propose LOCUS (Latent Optimization in Causal Support windows), which diagnoses which attribute failed and routes test-time latent optimization to its corresponding support window. With the generator, optimizer, reward, verifier, and token budget held fixed, support-routed windows outperform late windows by 23.6 percentage points in GenEval++ macro accuracy. With the generator frozen, LOCUS raises Janus-Pro's GenEval++ accuracy from 24.6% to 46.8% overall and from 19.5% to 40.5% on multi-attribute tags. On GenEval, LOCUS repairs 48.1% of failed generations in one rollout, compared with 37.4% for prefix optimization, and needs 14% fewer rounds on average to succeed.
Figures & tables
| Measurement | Value |
| Linear probe accuracy (10-way) | 67.1% |
| Chance (random guess) | 10.0% |
| SAE reconstruction (FVU) | 0.037 |
| SAE-code color decodability | 66.3% |
| Prefix fixed-direction override | 26.7% |
| Full-trajectory fixed-direction override |
| Method | Color | Count | Color/Count | Color/Pos | Pos/Count | Pos/Size | Multi-Count | Overall |
|---|---|---|---|---|---|---|---|---|
| SD3-Medium [ 14 ] | 55.0 | 50.0 | 12.5 | 35.0 | 17.5 | 15.0 | 22.5 | 29.6 |
| FLUX.1-dev [ 4 ] | 35.0 | 62.5 | 15.0 | 27.5 | 20.0 | 37.5 | 22.5 | 31.4 |
| BAGEL [ 12 ] | 32.5 | 60.0 | 25.0 | 32.5 | 25.0 | 47.5 | 37.5 | 37.1 |
| Qwen-Image [ 50 ] | 70.0 | 90.0 | 80.0 | 52.5 | 50.0 | 70.0 | 60.0 | 67.5 |
| GPT-4o [ 37 ] | 90.0 | 67.5 | 72.5 | 62.5 | 60.0 | 80.0 | 85.0 | 73.9 |
| Echo-4o [ 54 ] | 80.0 | 57.5 | 55.0 | 77.5 | 62.5 | 80.0 | 62.5 | 67.9 |
| Arm | Color | Count | Color/Count | Color/Pos | Pos/Count | Pos/Size | Multi-Count | Macro | |
|---|---|---|---|---|---|---|---|---|---|
| GPT-4.1 | Qwen3.5 | ||||||||
| Janus-Pro (base) | 45.0 | 30.0 | 12.5 | 30.0 | 7.5 | 35.0 | 12.5 | 24.6 | 30.5 |
| Prefix | 47.5 +2.5 | 57.5 +27.5 | 47.5 +35.0 | 35.0 +5.0 | 30.0 +22.5 | 32.5 2.5 | 55.0 +42.5 | 43.6 +19.0 | 54.1 +23.6 |
| Late | 40.0 5.0 | 25.0 5.0 | 15.0 +2.5 | 25.0 5.0 | 7.5 0.0 | 25.0 10.0 | 25.0 +12.5 | 23.2 1.4 | 33.8 +3.3 |
| Mid | 47.5 +2.5 | 22.5 7.5 | 22.5 +10.0 | 30.0 0.0 | 15.0 +7.5 | 30.0 5.0 | 37.5 +25.0 | 29.3 +4.7 | 37.3 +6.8 |
| Random | 62.5 +17.5 | 47.5 +17.5 | 45.0 +32.5 | 32.5 +2.5 | 12.5 +5.0 | 27.5 7.5 | 57.5 +45.0 | 40.7 +16.1 | 50.9 +20.4 |
| Benchmark | Method | S@1 | S@3 | S@6 |
|---|---|---|---|---|
| GenEval | MILR [ 36 ] | 37.4 | 61.6 | 74.8 |
| LOCUS | 48.1 +10.7 | 64.9 +3.3 | 77.1 +2.3 | |
| GenEval++ | MILR [ 36 ] | 11.0 | 22.0 | 27.5 |
| LOCUS | 14.5 +3.5 | 25.9 +3.9 | 32.3 +4.8 |
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Arm | Donor | Window | Pairs flipped |
|---|---|---|---|
| Matched | color | full trajectory | 6/6 |
| Wrong attribute | size | full trajectory | 0/6 |
| Wrong window | color | 11-token prefix | 0/6 |
| Attribute | Flip rate (%) | |||
|---|---|---|---|---|
| Matched | Wrong attr. | Wrong window | ||
| color | 96 (23) | 21 (5) | 8 (2) | 24 |
| size | 77 (17) | 45 (10) | 14 (3) | 22 |
| material | 91 (21) | 17 (4) | 0 (0) | 23 |
| Attribute | Class | cross05 [95% CI] | / Eval. |
|---|---|---|---|
| count | structure | 48 [24, 352] | 8 / det. |
| position | structure | 84 [72, 168] | 3 / det. |
| pattern | surface texture | 154 [96, 213] | 14 / CLIP |
| age | appearance | 160 [132, 216] | 10 / CLIP |
| size | appearance | 163 [128, 327] | 12 / CLIP |
| color | appearance | 186 [156, 220] | 11 / CLIP |
| Row band | Sufficiency (%) | Necessity (pp) |
|---|---|---|
| rows 0–4 (first) | 5 | 5 |
| rows 4–8 | 37 | 11 |
| rows 8–12 | 32 | 11 |
| rows 12–16 | 16 | 13 |
| rows 16–20 | 0 | |
| rows 20–24 (last) | 0 | 0 |
| Arm | Flip rate (%) | 95% CI (%) |
|---|---|---|
| LOCUS (ours) | 34.2 | [18.4, 50.0] |
| early (prefix) | 26.3 | [13.2, 39.5] |
| random | 21.1 | [7.9, 34.2] |
| late | 0.0 | [0.0, 0.0] |
| Attribute | Family | cross05 | ||
|---|---|---|---|---|
| Original | VLM | |||
| count | structure | 120 | 120 | 4 |
| size | appearance | 134 | 128 | 12 |
| material | appearance | 240 | 208 | 10 |
| color | appearance | 202 | 216 | 12 |
| Attribute | cross05 | Ovr@16 (%) | Janus-Pro | |
|---|---|---|---|---|
| position | 34 | 16 [16, 32] (6%) | 50 | 15–24 |
| size | 52 | 48 [16, 128] (19%) | 42 | 38 |
| color | 40 | 64 [48, 96] (25%) | 7 | 32–38 |
| material | 42 | 128 [80, 128] (50%) | 10 | 47 |
| Attribute | Readout | Clean | cross05 , flip | cross05 , gated | Ovr@16 (%) |
|---|---|---|---|---|---|
| position | CLIP halves | 34/450 | 16 [16, 32] | 32 [16, 80] | 50 |
| size | CLIP | 52/112 | 48 [16, 128] | 128 [48, 256] | 42 |
| color | CLIP | 40/48 | 64 [48, 96] | 80 [48, 96] | 7 |
| count | detector | 20/450 | 80 [24, 128] | 80 [24, 128] | 20 |
| material | CLIP | 42/128 | 128 [80, 128] | 96 [80, 128] | 10 |
| presence | detector | 64/360 | 192 [160, 192] | 192 [160, 192] | 3 |
| Benchmark | Method | S@1 | S@2 | S@3 | S@4 | S@5 | S@6 |
|---|---|---|---|---|---|---|---|
| GenEval | MILR [ 36 ] | 37.4 | 56.5 | 61.6 | 67.9 | 71.0 | 74.8 |
| LOCUS | 48.1 | 58.8 | 64.9 | 71.0 | 74.0 | 77.1 | |
| GenEval++ | MILR [ 36 ] | 11.0 | 16.1 | 22.0 | 24.8 | 25.2 | 27.5 |
| LOCUS | 14.5 | 20.0 | 25.9 | 29.5 | 30.0 | 32.3 |
Autoregressive Image Generation
Autoregressive image generation predicts visual tokens or scales sequentially to synthesize images, offering controllable alternatives to diffusion.
See all 122 papers