Amortized Set Prediction for Inverse IFS Reconstruction from Density Maps
Authors: Yutaka Yamaguti
Organizations: Faculty of Information Engineering, Fukuoka Institute of Technology, 3-30-1 Wajiro-higashi, Higashi-ku, Fukuoka, 811-0295, Fukuoka, Japan
Iterated Function Systems (IFS) generate self-similar fractals from a few contractive affine maps. The forward map from parameters to images is computationally inexpensive and well understood, whereas the inverse problem of estimating maps from an image is difficult and is typically handled by per-image optimization. We replace this loop with a single forward pass of a learned estimator that predicts the affine-map set directly from a visit-frequency density map, thereby amortizing the inverse problem. The design follows two constraints. First, density maps do not uniquely identify IFS parameters, so evaluation is based on reconstruction rather than parameter recovery; unordered map sets are handled by Hungarian matching, and ground-truth parameters provide a stable training surrogate. Second, the fully known forward model lets us generate exact synthetic training pairs and also supports image-only test-time refinement. On in-distribution tests, amortized initialization plus a few refinement steps lies on a better quality--speed frontier than equal-budget random-initialized per-image optimization, and a 30-step refinement (about 0.56 s per sample) remains better than a doubled-budget baseline. Extending optimization to 1000 steps shows that the benefit is not only speed: amortized initialization reaches high-quality reconstructions more frequently than random starts. On real images (MNIST and Fashion-MNIST), it improves density metrics on average over a published per-image optimizer while being roughly 12 to 2600 times faster.
Figures & tables
Figure 1: Identifiability (all pairs of test256 ). Left: density distance versus parameter-set distance (Pearson correlation −0.146 ). The 59 near-image, far-parameter pairs that simultaneously satisfy the thresholds (density ≤0.0168 , the bottom 1% ; parameter distance ≥1.299 , the top 10% ) are shown in red. Right: parameter-set distance (median/p90/max) among image pairs with density distance ≤ε ; it does not fall off as ε shrinks.
Distance
Mean
Median
1%
Max
Density L2
0.0340
0.0325
0.0168
0.0778
(W,b) distance
1.0606
1.0559
0.6535
1.9016
Table 1: Pairwise distance distribution on test256 (density distance and parameter-set distance).
Setting
sec/sample
density SSE
Chamfer
HD95
coverage@2px
(W,b) err
base 0-step
0.0095
7.85×10−4
0.0547
0.2106
0.6650
0.405
base +10
0.192
3.85×10−4
0.0365
0.1381
0.7855
0.395
base +20
0.370
2.76×10−4
0.0295
0.1059
0.8319
0.394
base +30
0.556
2.23×10−4
0.0259
0.0913
0.8621
0.395
random-r4 0-step
0.0110
1.005×10−3
0.1546
0.5719
0.4063
0.981
random-r4 +10
0.199
7.69×10−4
0.1184
0.4566
0.5102
0.973
Table 2: Quality–speed Pareto ( test256 ). base amortized(+refinement) versus equal-budget random-from-scratch (r4). Time is seconds per sample; density SSE, Chamfer, and HD95 are better when smaller and coverage@2px when larger. The (W,b) error is the Hungarian-matched parameter-set distance (same definition as Section 5 ), reported as a diagnostic.
Figure 2: Quality–speed Pareto ( test256 ). Horizontal axis: seconds per sample. Left: Chamfer (smaller is better); right: coverage@2px (larger is better). The base amortized(+refinement) curve consistently improves over equal-budget random-from-scratch.
Figure 3: Qualitative examples of the main result ( 4 representative examples of test256 , density maps by the matched renderer). Columns: target, ours- 0 (one-shot), ours- 30 ( +30 refinement), and random-r4- 30 (equal-budget per-instance); C in each cell is the Chamfer distance. At equal budget, ours- 30 is closer to the target than random-r4- 30 in most examples (point-cloud version in Figure A.1 ).
Method
step
density SSE
CD mean
CD median
CD p95
success τ
(W,b) err
base
30
2.27×10−4
0.0262
0.0234
0.0457
0.598
0.395
base
1000
1.55×10−4
0.0195
0.0181
0.0340
0.824
0.453
random-r1
1000
2.82×10−4
0.0331
0.0314
0.0540
0.301
0.971
random-r4
1000
2.09×10−4
0.0276
0.0273
0.0417
0.445
0.928
Table 3: Convergence distribution under long optimization ( test256 , matched renderer). Converged values at 1000 steps for base (amortized initialization) and random initialization (r1 / best of r4). The CD columns are the mean, median, and p95 of Chamfer; density SSE is the mean (all better when smaller). The success rate is Pr[CD≤τ] with τ=0.0262 (mean Chamfer of base +30).
Figure 4: Convergence under long optimization ( test256 ). Left: success rate Pr[CD≤τ] ( τ=0.0262 ) versus number of steps. base (amortized initialization) reaches 0.77 by 100 steps and saturates at about 0.82 , whereas random initialization remains at 0.30 (r1) / 0.44 (r4) even at 1000 steps. Right: pairwise scatter of the 1000 -step converged Chamfer for the same samples (horizontal base , vertical random). Almost all points lie above the diagonal, so amortized initialization converges to a better solution sample by sample.
Method
step
density SSE
Chamfer
HD95
coverage@2px
(W,b) err
base
0
7.85×10−4
0.0547
0.2106
0.6650
0.405
match-only
0
8.30×10−4
0.0552
0.2121
0.6629
0.407
+aux
0
7.75×10−4
0.0527
0.2010
0.6680
0.413
base
30
2.23×10−4
0.0259
0.0913
0.8621
0.395
match-only
30
2.35×10−4
0.0263
0.0934
0.8591
0.396
+aux
30
2.26×10−4
0.0264
0.0942
0.8589
0.402
Table 4: Effect of the reconstruction auxiliary loss ( test256 ). base (no auxiliary), match-only (continuation only), and +aux (with reconstruction auxiliary) at 0 and 30 steps.
Figure 5: Quality–speed of within-family OOD (Chamfer vs seconds per sample). Left: T1 (wide translation); right: S1 (large singular values).
Method
occupancy MSE
density SSE
Chamfer
HD95
coverage@2px
ours 0 -step
0.0874
4.30×10−4
0.0571
0.2245
0.6888
ours 30 -step
0.0867
1.01×10−4
0.0221
0.0753
0.9126
ours 100 -step
0.0724
6.95×10−5
0.0193
0.0598
0.9357
ours-init + occupancy-GD 100
0.0190
7.67×10−4
0.0485
0.1931
0.7114
Tu ( n=4 )
0.0296
2.88×10−3
0.1329
0.5305
0.6154
Tu ( n=10 )
0.0214
4.91×10−3
0.1017
0.3837
0.6364
Table 6: MNIST (balanced 50 images, n=10 estimator). occupancy MSE is the minimum MSE under the occupancy condition; the rest are density metrics. density SSE, Chamfer, HD95, and occupancy MSE are better when smaller, coverage@2px when larger.
Figure 6: MNIST reconstruction (density maps, common grayscale, one example per row). Columns from left: target, ours 0 -step, ours 30 -step, ours 100 -step, ours-init + occupancy-GD 100 , Tu ( n=10 ). Under density-aware refinement, mass spreads as an area and approaches the target’s thick density as 0→30→100 steps, whereas Tu and occupancy-GD have clear outlines but concentrate mass on thin lines and isolated points.
Method
occupancy MSE
density SSE
Chamfer
HD95
coverage@2px
ours 0 -step
0.1087
1.38×10−4
0.0503
0.2102
0.7724
ours 30 -step
0.0997
3.98×10−5
0.0251
0.0738
0.8846
ours 100 -step
0.0966
3.09×10−5
0.0235
0.0652
0.8979
ours-init + occupancy-GD 100
0.0450
4.94×10−4
0.0629
0.2554
0.7645
Tu ( n=10 , 1000 iters)
0.0672
1.43×10−4
0.0453
0.1754
0.8610
Table 7: Fashion-MNIST (balanced 50 images, n=10 ). occupancy MSE is the minimum MSE under the occupancy condition; the rest are density metrics.
Figure 7: Fashion-MNIST reconstruction (density maps, common grayscale, one example per row). Columns from left: target, ours 0 -step, ours 30 -step, ours 100 -step, ours-init + occupancy-GD 100 , Tu ( n=10 ). Faithful reconstruction of filled regions requires reproducing internal density; density refinement ( 0→30→100 ) reproduces the target’s area density well, whereas occupancy-GD damages density and collapses the garment to an outline.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1: Additional examples of the main result ( 4 examples of test256 , point-cloud view). Columns: target (black), ours- 0 (gray), ours- 30 (blue), and random-r4- 30 (orange); C in each cell is the Chamfer distance. These correspond to the same 4 examples as Figure 3 .
Figure B.1: MNIST, rendered with Tu’s renderer ( 32×32 saturated occupancy) ( 10 examples). Columns are the same as Figure 6 (target / ours-0 / ours-30 / ours-100 / occupancy-GD-100 / Tu ( n=10 )).
Figure B.2: Fashion-MNIST, rendered with Tu’s renderer ( 32×32 saturated occupancy) ( 10 examples). Columns are the same as Figure 7 .
Figure B.3: MNIST, density maps (common grayscale), all balanced 50 examples. Columns are the same as Figure 6 .
Figure B.4: Fashion-MNIST, density maps (common grayscale), all balanced 50 examples. Columns are the same as Figure 7 .
Department of Electrical Engineering Sharif University of Technology Tehran, Iran · Electronics Research Institute Sharif University of Technology Tehran, Iran