Generating Humanless Environment Walkthroughs from Egocentric Walking Tour Videos

Yujin Ham; Junho Kim; Vivek Boominathan; Guha Balakrishnan

Figure 2: Locations of background video clips in EgoCrowds . Training clip locations are in green, and testing locations are in red. The full list of city names are in Supplementary.

Figure 3: Overview of our data construction pipeline. Both background and foreground clips are sourced from real “ walking tour ” videos. The foreground clips were selected to ensure an approximately uniform distribution across different Crowd% levels. For each instance, we generate a soft shadow with randomized strength and angle by applying an affine transform to the human mask (red dots indicate pivot points).

Figure 4: Shadow injection with varying

\alpha

values. As

\alpha

ranges from 0.2 (a) to 0.8 (d), the shadow intensity appears stronger.

Figure 5: Qualitative comparison. Red boxes indicate failures to remove humans or their shadows and yellow boxes highlight areas where the background is over-smoothed instead of being filled with plausible content. When there are fewer people and the background is relatively simple, as in (a) Jakarta, all methods perform reasonably well. However, performance degrades as masks become larger and backgrounds more complex. Also, in particular, ProPainter [ 48 ] and DiffuEraser [ 18 ] struggle when the cast shadow is sharp (i.e., less diffused). Casper [ 17 ] is robust at associating effects, but when the mask is large it often hallucinates objects or people inside the masked region. In contrast, our method shows greater robustness for large masks, preserving background structure with fewer noticeable artifacts.

Crowd%	(0–10%)			(10–20%)			(20–30%)			(30–40%)			(40–50%)			Average
Crowd%	PSNR $\uparrow$	LPIPS $\downarrow$	DreamSim $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	DreamSim $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	DreamSim $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	DreamSim $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	DreamSim $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	DreamSim $\downarrow$
ProPainter [ 48 ]	31.81	0.058	0.011	28.87	0.088	0.019	25.50	0.128	0.056	21.98	0.192	0.114	20.93	0.238	0.134	25.82	0.141	0.067
DiffuEraser [ 18 ]	31.91	0.055	0.009	28.73	0.079	0.014	24.68	0.112	0.035	22.18	0.171	0.067	22.27	0.181	0.062	25.95	0.120	0.037
Casper [ 17 ]	31.41	0.073	0.010	28.72	0.092	0.014	23.97	0.118	0.025	19.88	0.185	0.052	19.24	0.185	0.052	24.64	0.130	0.029
Ours	32.39	0.065	0.007	29.36	0.086	0.011	26.31	0.108	0.020	22.34	0.170	0.039	23.31	0.161	0.031	26.74	0.118	0.022

Table 1: Quantitative comparison across varying crowd% levels. We evaluate the removal-inpaint quality on our synthetic dataset of 7 different cities (35 videos in total). Higher PSNR indicates better reconstruction quality, while lower LPIPS and DreamSim indicate higher perceptual similarity. Best results are highlighted in red and second-best in yellow.

Figure 6: Ablation results across temporal frames. Red boxes indicate failures in capturing shadows, and the yellow boxes highlight a zoomed-in region for inpainting comparison. (a) Vanilla Casper output (baseline). (b) Finetuned on our data without shadow simulation. (c) Trained using only smaller masks ( Crowd% 0–10). (d) Our full finetuned model with shadow injection and uniform mask distribution across different Crowd% levels.

Component Enabled		PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DreamSim $\downarrow$
Shadow	Full Crowd%	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DreamSim $\downarrow$
✗	✗	24.64	0.868	0.130	0.029
✗	✓	25.09	0.870	0.128	0.028
✓	✗	26.60	0.882	0.120	0.024
✓	✓	26.74	0.881	0.118	0.022

Table 2: Ablation study on model components during the data construction. The first row shows the original Casper model without finetuning. Without ‘Shadow’ drops all shadow injection, and without ‘Full Crowd% ’ only uses masks with lower occupancy levels ( Crowd% 0–10 ).

Figure 7: SpatialTrackerV2 [ 41 ] 4D reconstruction visualization results for three scenes. We compare results using raw walking tour video inputs (top) versus our crowd-removed versions (bottom). Each image displays the inferred 3D point clouds for the scene visualized from the camera viewpoint of the final video frame, with overlaid colored circles corresponding to point tracks in the 3D space. Points on static objects should not move. When comparing regions of static objects between the top and bottom rows (such as in the red circles), we see that point tracks exhibit greater movement, and therefore greater errors, using the raw videos. Furthermore, the CrowdEraser cleaned videos produce better background details (e.g., wall patterns in Marrakech) and denser point clouds (e.g., right side in Istanbul). We provide further scene samples in Supplementary.

Figure 8: Examples of difficult cases (limitations) for CrowdEraser . (a) For long videos with sustained large occlusions, inpainted regions lack temporal consistency across the entire video. (b) In high Crowd% scenarios, the model struggles to recover fine-grained, high frequency details, resulting in artifacts.

Scene	Birmingham		Boston		Capetown		Chicago		Dubai		Rome		Zurich
Scene	PSNR $\uparrow$	DreamSim $\downarrow$	PSNR $\uparrow$	DreamSim $\downarrow$	PSNR $\uparrow$	DreamSim $\downarrow$	PSNR $\uparrow$	DreamSim $\downarrow$	PSNR $\uparrow$	DreamSim $\downarrow$	PSNR $\uparrow$	DreamSim $\downarrow$	PSNR $\uparrow$	DreamSim $\downarrow$
ProPainter [ 48 ]	27.77	0.052	26.00	0.049	25.86	0.059	25.91	0.053	22.34	0.050	29.97	0.019	25.76	0.059
DiffuEraser [ 18 ]	27.25	0.034	25.49	0.042	25.43	0.050	25.59	0.036	22.65	0.042	29.86	0.013	25.14	0.049
Casper [ 17 ]	26.15	0.028	25.12	0.031	24.20	0.035	26.02	0.022	23.14	0.027	29.44	0.012	25.21	0.027
Ours	26.58	0.022	26.00	0.025	25.50	0.026	27.08	0.019	24.81	0.019	30.31	0.009	25.99	0.022

Table 3: Quantitative comparison across cities. Best results are highlighted in red and second-best in yellow. Our CrowdEraser consistently achieves the best DreamSim, reflecting superior perceptual quality.

Figure 12: Baseline comparison across temporal frames for the “Istanbul” scene in Figure 5 . Red boxes indicate failures in foreground removal or shadow handling. Casper [ 23 ] struggles with larger masks, producing noticeable hallucinations within masked areas, while DiffuEraser [ 18 ] has difficulty handling shadows and removing associated objects.

Figure 13: Baseline comparison across temporal frames for the “Chicago” scene in Figure 5 . Red boxes highlight failures in foreground removal or shadow handling. DiffuEraser [ 18 ] struggles to associate shadows and obejects, leading to floating shadows and objects.

Figure 14: SpatialTrackerV2 4D reconstruction results. We compare results using raw walking tour video inputs (top) versus our crowd-removed versions (bottom). Each image displays the inferred 3D point clouds for the scene visualized from the camera viewpoint of the initial, middle, and final video frames, with overlaid colored circles corresponding to point tracks in the 3D space. Tracking points remain more stable in static background regions, indicating that our crowd removal leads to more reliable and robust reconstruction. Moreover, the resulting point clouds are denser and more consistent, benefiting downstream tasks such as scene modeling and 3D novel view synthesis.

Figure 15: SpatialTrackerV2 4D reconstruction results. We compare results using raw walking tour video inputs (top) versus our crowd-removed versions (bottom). Each image displays the inferred 3D point clouds for the scene visualized from the camera viewpoint of the initial, middle, and final video frames, with overlaid colored circles corresponding to point tracks in the 3D space. Tracking points remain more stable in static background regions, indicating that our crowd removal leads to more reliable and robust reconstruction. Moreover, the resulting point clouds are denser and more consistent, benefiting downstream tasks such as scene modeling and 3D novel view synthesis.

Continent	Country	City / Area	URL(s)
Train Background Video Sources
Africa	Egypt	Cairo	https://youtu.be/TVe7Th_EfrM
Asia	China	Beijing	https://youtu.be/FNK5UEObcEg , https://youtu.be/MU-obosH1ow
Asia	China	Great Wall	https://youtu.be/cVmM6sUcdwg
Asia	China	Shanghai	https://youtu.be/Z1i0v6wsGLI , https://youtu.be/DUANyWsER_o
Asia	South Korea	Cheongju	https://youtu.be/kqkUJZllt0U

Table 4: Training dataset sources. Background and foreground video sources used to construct EgoCrowds . For cities where a single video did not provide sufficient clips, multiple videos were collected to ensure sufficient coverage.

Continent	Country	City / Area	URL(s)
Test Data Background
Africa	South Africa	Cape Town	https://www.youtube.com/watch?v=eG_SV5aSBqQ
Asia	United Arab Emirates	Dubai	https://www.youtube.com/watch?v=mElSLruob6c
Europe	Italy	Rome	https://www.youtube.com/watch?v=xpRDEoEQpwk
Europe	Switzerland	Zurich	https://www.youtube.com/watch?v=UcRW2OHqC2o
Europe	UK	Birmingham	https://www.youtube.com/watch?v=IlzFH4yE2Z8

Table 5: Quantitative test dataset sources. Background and foreground video sources used in the quantitative evaluation dataset.

Continent	Country	City	URL(s)
Crowd Walking Tour Video Sources
Asia	India	Mumbai	https://youtu.be/_2GM4gVlors
Asia	Japan	Tokyo	https://youtu.be/jeQd-n7Rot0
Asia	Japan	Kyoto	https://youtu.be/OhOlwqjt_Lg
Asia	Thailand	Bangkok	https://youtu.be/sWRoDRYi1Lk
Asia	Indonesia	Jakarta	https://youtu.be/2lSUV5KZgwI

Table 6: Qualitative test dataset sources. Real walking tour video sources used for qualitative evaluation. For cities where a single video did not provide sufficient clips, multiple videos were collected to ensure sufficient coverage.

Generating Humanless Environment Walkthroughs from Egocentric Walking Tour Videos

Abstract

Figures & tables

Image Inpainting

CityRAG: Stepping Into a City via Spatially-Grounded Video Generation

World2Motion: Turning Video World Models into 3D Human Motion Generators

VOID: Video Object and Interaction Deletion

Abstract

Image Inpainting

Explore similar work

CityRAG: Stepping Into a City via Spatially-Grounded Video Generation

World2Motion: Turning Video World Models into 3D Human Motion Generators

VOID: Video Object and Interaction Deletion