Projection-based appearance editing changes selected visual attributes of a physical object through projection mapping (PM). To display a desired appearance, PM typically uses projector compensation, which computes a projector input that counteracts the geometric and photometric distortions of the scene. Compensation can only approximate this appearance within the available projection range, and it cannot revise the appearance itself. Pretrained LLM-based image editors provide a direct way to produce the desired appearance from a text prompt or reference image; projector compensation can then attempt to reproduce the edited result through projection. However, these editors do not have an accurate measure of the scene geometry or scene response. Therefore, their outputs may contain geometric misalignment, inconsistent shading, or changes outside the available projection range, which downstream projector compensation cannot correct. To address this problem, we introduce SREdit, an efficient response-guided framework for projection-based appearance editing. SREdit estimates a scene response map from the reconstructed scene that reflects how strongly projected illumination can affect different surface regions. SREdit uses the source appearance to constrain the early sampling process and preserve registration with the fixed physical object, while the scene response map adapts the amount of editing across surface regions. Experiments across two pretrained image editors show that SREdit improves projected appearance quality and reproduction fidelity. It adds less than one second per image with negligible VRAM overhead. Participants also consistently preferred the results produced with SREdit in our user study.
Figures & tables
Figure 1: Method overview. The pretrained image editor does not have an accurate measure of the scene response of the physical setup. During the first r portion of sampling, SREdit uses the source appearance to preserve registration with the physical object, while the scene response map adapts the amount of editing across surface regions. Regions with lower response retain more of the source appearance, whereas regions with higher response allow larger changes. The image editor then completes the remaining sampling steps. Finally, projector compensation uses iterative refinement to obtain a compensation image for physical projection.
Figure 2: Scene response map construction. We evaluate the channelwise direct-response factor Bc(u) . Channel averaging gives R(u) , which is normalized over the reconstructed-object support Ω to form M . The insets show representative regions with lower and higher response.
Figure 3: Response-guided sampling. At one sampling step, the Euler update produces zi+1 , while ai+1 provides the source anchor at the same noise level. The scene response map, control strength, and temporal weight determine wi(u) , which moves the updated latent toward the source anchor.
Figure 4: Real-projection comparisons. Results are shown for an oil-painted-statue edit (top) and a painted-wood edit (bottom). The upper and lower rows show generated targets and camera-captured results, respectively. Red and blue boxes indicate enlarged regions for comparing local texture and color reproduction. The prompts for these representative comparisons and additional qualitative results are provided in the supplementary material.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
ΔE↓
CLIP ↑
BLIP-ITM ↑
Time (s/image) ↓
VRAM (GiB) ↓
Nepmap [ 14 ]
19.1512
0.5962
0.3885
9.8198
29.79
59.72
8.70
7.68
LAPIG [ 13 ] †
13.4033
0.3163
0.4189
17.4810
28.41
48.06
9.08
5.60
Native-LongCat [ 43 ]
14.7231
0.3721
0.3369
14.5425
30.56
91.66
58.50
29.73
SREdit-LongCat
20.6623
0.5432
0.2316
8.2552
32.44
92.62
59.10
29.73
Native-Qwen [ 58 ]
16.5086
0.3608
0.3571
13.7814
31.61
88.61
88.37
57.94
SREdit-Qwen
21.3113
0.5690
0.2559
8.1197
32.10
94.77
89.10
57.95
Table 1: Performance and efficiency under real projection. PSNR, SSIM, LPIPS, and ΔE are computed between the intended appearance of each method and its camera-captured result. CLIP and BLIP-ITM measure agreement with the requested appearance. Results are averaged over 18 prompts from six setups. Time is the wall-clock time required to generate one image, and VRAM is the peak allocated GPU memory during image generation. We use bold and underline to highlight the best and second-best results, respectively.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
ΔE↓
CLIP ↑
BLIP-ITM ↑
SREdit-Qwen
21.3113
0.5690
0.2559
8.1197
32.10
94.77
SREdit-LongCat
20.6623
0.5432
0.2316
8.2552
32.44
92.62
Uniform-Qwen
21.9647
0.6129
0.2398
7.4097
31.07
93.63
Uniform-LongCat
20.9145
0.5487
0.2286
7.9873
31.73
90.16
Table 4: Scene response map ablation. For each image editor, the results are averaged over the same 18 cases from six setups. We use r=0.2,S=2.5 for Qwen and r=0.2,S=3.0 for LongCat. We use bold and underline to highlight the best and second-best results, respectively.
Metric
All cases
Qwen
LongCat
Text alignment
65.4%
61.4%
69.4%
Fewer artifacts
72.8%
63.9%
81.7%
Realism
75.0%
68.1%
81.9%
No misalignment
96.5%/80.4%
96.4%/90.3%
96.7%/70.6%
Overall preference
72.4%
65.3%
79.4%
Table 2: User study. Pairwise perceptual evaluation with 20 participants over 36 cases. Values for text alignment, artifacts, realism, and overall preference are the percentages of judgments favoring SREdit over the corresponding Native version. For visible misalignment, entries report the percentages judged free of misalignment for SREdit/Native, respectively.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
ΔE↓
Native-Qwen
18.5257
0.6675
0.1159
10.6841
SREdit-Qwen
21.7335
0.7358
0.0965
7.6941
Table 3: Reference-guided editing. Quantitative comparison over 12 cases from four physical setups. Metrics are computed between each intended appearance and its camera-captured result within the foreground object mask and averaged over all cases. The better result for each metric is shown in bold .
Figure 5: Reference-guided qualitative results. Appearance editing using Qwen. The first two columns show the reference and source scene. The remaining columns show the intended and camera-captured appearances produced by Native-Qwen and SREdit-Qwen. The top row uses Chinese opera face painting as the reference for makeup transfer, whereas the bottom row uses emerald as the reference material for material replacement. Marked regions are enlarged for local comparison.
Figure 6: Control-parameter ablation. All results are physical projector–camera captures. Top: complete r -by- S parameter grids for both image editors. Bottom: mean CLIP and LPIPS for SREdit-Qwen and SREdit-LongCat over 18 cases (six setups, three prompts each). Dots mark measured settings, contours show interpolated trends, and red stars mark the best measured setting in each plot.
Figure 7: Scene response map ablation. Camera-captured results under real projection. Uniform replaces the scene response map with its mean while retaining the paired r and S values. Insets show unedited patches that retain the source color under Uniform; SREdit shows fewer such patches.
Figure 8: Failure case. This result uses r=0.2 and S=2.5 . Although it remains registered with the physical object, parts of the requested wood texture are weakened or spatially uneven in the intended and camera-captured appearances.
Figure S1: Qualitative comparison for Case 1. SREdit uses (r,S)=(0.2,2.5) for Qwen and (0.2,3.0) for LongCat. The upper and lower rows show generated appearances and camera-captured projection results, respectively.
Figure S2: Parameter ablation for Case 1. Columns vary S , while rows vary r . The upper and lower matrices show generated and camera-captured results, respectively, and the bottom plots report LPIPS and CLIP over the same parameter grid.
Figure S3: Qualitative comparison for Case 2. SREdit uses (r,S)=(0.2,2.5) for Qwen and (0.2,3.0) for LongCat. The upper and lower rows show generated appearances and camera-captured projection results, respectively.
Figure S4: Parameter ablation for Case 2. Columns vary S , while rows vary r . The upper and lower matrices show generated and camera-captured results, respectively, and the bottom plots report LPIPS and CLIP over the same parameter grid.
Figure S5: Qualitative comparison for Case 3. SREdit uses (r,S)=(0.2,2.5) for Qwen and (0.2,3.0) for LongCat. The upper and lower rows show generated appearances and camera-captured projection results, respectively.
Figure S6: Parameter ablation for Case 3. Columns vary S , while rows vary r . The upper and lower matrices show generated and camera-captured results, respectively, and the bottom plots report LPIPS and CLIP over the same parameter grid.
Figure S7: Qualitative comparison for Case 4. SREdit uses (r,S)=(0.2,2.5) for Qwen and (0.2,3.0) for LongCat. The upper and lower rows show generated appearances and camera-captured projection results, respectively.
Figure S8: Parameter ablation for Case 4. Columns vary S , while rows vary r . The upper and lower matrices show generated and camera-captured results, respectively, and the bottom plots report LPIPS and CLIP over the same parameter grid.
Figure S9: Qualitative comparison for Case 5. SREdit uses (r,S)=(0.2,2.5) for Qwen and (0.2,3.0) for LongCat. The upper and lower rows show generated appearances and camera-captured projection results, respectively.
Figure S10: Parameter ablation for Case 5. Columns vary S , while rows vary r . The upper and lower matrices show generated and camera-captured results, respectively, and the bottom plots report LPIPS and CLIP over the same parameter grid.
Figure S11: Qualitative comparison for Case 6. SREdit uses (r,S)=(0.2,2.5) for Qwen and (0.2,3.0) for LongCat. The upper and lower rows show generated appearances and camera-captured projection results, respectively.
Figure S12: Parameter ablation for Case 6. Columns vary S , while rows vary r . The upper and lower matrices show generated and camera-captured results, respectively, and the bottom plots report LPIPS and CLIP over the same parameter grid.
Figure S13: Qualitative comparison for Case 7. SREdit uses (r,S)=(0.2,2.5) for Qwen and (0.2,3.0) for LongCat. The upper and lower rows show generated appearances and camera-captured projection results, respectively.
Figure S14: Parameter ablation for Case 7. Columns vary S , while rows vary r . The upper and lower matrices show generated and camera-captured results, respectively, and the bottom plots report LPIPS and CLIP over the same parameter grid.
Figure S15: Qualitative comparison for Case 8. SREdit uses (r,S)=(0.2,2.5) for Qwen and (0.2,3.0) for LongCat. The upper and lower rows show generated appearances and camera-captured projection results, respectively.
Figure S16: Parameter ablation for Case 8. Columns vary S , while rows vary r . The upper and lower matrices show generated and camera-captured results, respectively, and the bottom plots report LPIPS and CLIP over the same parameter grid.
Figure S17: Qualitative comparison for Case 9. SREdit uses (r,S)=(0.2,2.5) for Qwen and (0.2,3.0) for LongCat. The upper and lower rows show generated appearances and camera-captured projection results, respectively.
Figure S18: Parameter ablation for Case 9. Columns vary S , while rows vary r . The upper and lower matrices show generated and camera-captured results, respectively, and the bottom plots report LPIPS and CLIP over the same parameter grid.
Figure S19: Qualitative comparison for Case 10. SREdit uses (r,S)=(0.2,2.5) for Qwen and (0.2,3.0) for LongCat. The upper and lower rows show generated appearances and camera-captured projection results, respectively.
Figure S20: Parameter ablation for Case 10. Columns vary S , while rows vary r . The upper and lower matrices show generated and camera-captured results, respectively, and the bottom plots report LPIPS and CLIP over the same parameter grid.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Comparison
Methods
Prompt
Prompts used in the main paper
\cref fig:teaser(b), text-guided editing
Blue Porcelain
SREdit
Give the toy a blue-and-white porcelain-style doll appearance, with an antique ceramic glaze and classic cobalt-blue patterns, while preserving the original shape and silhouette.
Ink Wash
SREdit
Give the toy an ink-wash-style doll appearance, with pale monochrome tones and flowing ink gradients, while preserving the original shape and silhouette.
Painted Wood
SREdit
Give the toy a painted-wood doll appearance, with warm colors and bold hand-painted floral patterns, while preserving the original shape and silhouette.
Tropical Style
SREdit
Give the toy a tropical outfit-style doll appearance, with vibrant island colors and bold floral patterns, while preserving the original shape and silhouette.
Appendix
Table S1: Editing prompts. Prompts used in \cref fig:teaser and in the representative main-paper and supplementary comparisons.
Projector compensation seeks to correct geometric and photometric distortions that occur when images are projected onto nonplanar or textured surfaces. However, most existing methods are highly setup-dependent, requiring fine-tuning or retraining whenever the surface, lighting, or projector-camera pose changes. Progress has been limited by two key challenges: (1) the absence of large, diverse training datasets and (2) existing geometric correction models are typically constrained by specific spatial setups; without further retraining or fine-tuning, they often fail to generalize directly to novel geometric configurations. We introduce SIComp, the first Setup-Independent framework for full projector Compensation, capable of generalizing to unseen setups without fine-tuning or retraining. To enable this, we construct a large-scale real-world dataset spanning 277 distinct projector-camera setups. SIComp adopts a co-adaptive design that decouples geometry and photometry: A carefully tailored optical flow module performs online geometric correction, while a novel photometric network handles photometric compensation. To further enhance robustness under varying illumination, we integrate intensity-varying surface priors into the network design. Extensive experiments demonstrate that SIComp consistently produces high-quality compensation across diverse unseen setups, substantially outperforming existing methods in terms of generalization ability and establishing the first generalizable solution to projector compensation. The code and dataset are available on our project page: https://hai-bo-li.github.io/SIComp/
Projector-camera (ProCams) systems achieve active scene perception and controllable appearance manipulation via structured illumination, serving as a core infrastructure for spatial augmented reality, projection mapping, and surface reflectance acquisition. Existing inverse-rendering methods for ProCams deliver high-fidelity results but rely on time-consuming per-scene optimization, while mainstream feed-forward 3D reconstruction models produce baked appearance that cannot adapt to spatially varying projector illumination. To resolve this accuracy-efficiency trade-off, we propose FF-ProCams, a Feed-Forward 3D Gaussian inverse-rendering framework for ProCams. A hybrid Mamba2-Transformer encoder aggregates cross-view geometric and photometric cues from sparse multi-view observations, and lightweight heads predict a relightable Gaussian representation in a single forward pass. We further design a projector-aware differentiable renderer to synthesize camera observations under arbitrary active illumination and ProCams poses. To enable feed-forward training, we construct a large-scale synthetic ProCams dataset covering diverse object geometries and surface materials. Experiments show FF-ProCams achieves high-fidelity projector-aware rendering, generalizes to unseen patterns, and supports novel projector-camera poses. Using only 8 input views, it outperforms optimization-based baselines with 297 views while reducing test-time reconstruction to 0.13 seconds (a three-to-five-order-of-magnitude speedup). The code and data are available at https://github.com/CPREgroup/FF-ProCams/.
Image spatial editing performs geometry-driven transformations, allowing precise control over object layout and camera viewpoints. Current models are insufficient for fine-grained spatial manipulations, motivating a dedicated assessment suite. Our contributions are listed: (i) We introduce SpatialEdit-Bench, a complete benchmark that evaluates spatial editing by jointly measuring perceptual plausibility and geometric fidelity via viewpoint reconstruction and framing analysis. (ii) To address the data bottleneck for scalable training, we construct SpatialEdit-500k, a synthetic dataset generated with a controllable Blender pipeline that renders objects across diverse backgrounds and systematic camera trajectories, providing precise ground-truth transformations for both object- and camera-centric operations. (iii) Building on this data, we develop SpatialEdit-16B, a baseline model for fine-grained spatial editing. Our method achieves competitive performance on general editing while substantially outperforming prior methods on spatial manipulation tasks. All resources will be made public at https://github.com/EasonXiao-888/SpatialEdit.
Yicheng Xiao, Wenhu Zhang, Lin Song +10
The University of Hong Kong · JD Explore Academy · Tsinghua University +2