Reconstructing dynamic scenes and objects from monocular egocentric video is crucial for embodied AI, yet existing 4D Gaussian Splatting (4DGS) approaches struggle with everyday interactions involving topological changes, such as cutting, assembly, opening, or closing. Since 4DGS methods rely on deforming a fixed representation, events that split, merge, or reveal surfaces cause severe visual artifacts. To address this challenge, we introduce TopoEgo, the first benchmark dedicated to topological changes in egocentric video (covering fission, fusion, open, and close), alongside Topo-4DGS, a topology-aware reconstruction method. Topo4DGS jointly models part-level motion and time-varying surface support by assigning independent rigid transformations to interacting parts and dynamically activating or deactivating Gaussians as surfaces appear, disappear, or separate. Extensive experiments demonstrate that Topo-4DGS significantly outperforms state-of-the-art 4DGS approaches across topological changes, especially on post-event frames. Code and data will be made publicly available.
Figures & tables
Figure 2: TopoEgo examples of topological changes showing pre- and post-event ground-truth frames from EPIC-KITCHENS and Ego4D. Cyan denotes the objects undergoing the changes.
Figure 3: Method overview of Topo-4DGS. Top: at event time τ , Motion Bifurcation transitions the object from a single rigid motion to K piecewise-rigid trajectories. Bottom: the Spacetime Clustering Field combines the learned Spatial Branch with the event-gated Temporal Branch to softly route the object Gaussians to the corresponding motion branches. Topological Boundary Resolution creates support for newly formed or exposed surfaces and temporally deactivates obsolete or concealed support.
TopoEgo (EPIC-KITCHENS)
TopoEgo (Ego4D)
Rigid Objects Control
Method
fission
fusion
open
close
fission
fusion
open
close
PSNR ↑
SSIM ↑
LPIPS ↓
EgoGaussian [ 29 ]
20.48
25.17
18.32
22.20
22.97
20.76
17.70
19.69
27.34
0.804
0.186
DeGauss [ 24 ]
21.39
25.60
18.58
23.24
24.37
22.23
19.51
20.99
25.10
0.742
0.232
DeGauss (w/o hands)
20.15
21.70
16.70
19.40
21.03
19.92
17.20
18.64
23.07
0.651
0.317
Def-3DGS [ 28 ]
21.93
26.03
18.93
23.58
24.64
22.54
19.67
21.31
24.37
0.732
0.243
Def-3DGS (w/o hands)
22.70
26.70
19.05
23.35
24.45
23.10
19.40
22.05
26.44
0.764
0.197
Table 1: Reconstruction comparison on TopoEgo. Object-region PSNR ↑ is reported for each event family and source dataset. Hand-masked baselines are added for a fair comparison. The last three columns report performance on clips from [ 6 , 8 ] without topological changes.
Full Scene
Topo. Object
Post-Event Object
Other Dyn. Objects
Method
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
EgoGaussian [ 29 ]
23.82
0.729
0.249
20.91
0.638
0.247
16.79
0.464
0.300
20.51
0.602
0.302
DeGauss [ 24 ]
26.64
0.837
0.264
21.99
0.698
0.257
18.76
0.591
0.310
23.65
0.754
0.224
DeGauss (w/o hands)
23.44
0.844
0.271
19.34
0.704
0.264
16.55
0.598
0.318
20.81
0.760
0.231
Def-3DGS [ 28 ]
27.15
0.846
0.256
22.33
0.705
0.256
19.31
0.600
0.312
23.97
0.752
0.223
Def-3DGS (w/o hands)
27.80
0.858
0.263
22.60
0.711
0.266
19.83
0.611
0.320
24.69
0.764
0.230
Table 2: Additional comparison on TopoEgo. Results are averaged over all 24 clips for the full scene, the topology-changing object over all frames (reported per event type and dataset in Tab. 1 ) and post-event frames, and other dynamic objects (i.e., manipulated, rigid objects).
Figure 4: Qualitative comparison on TopoEgo. Different events shown (left) with changes highlighted in cyan in pre and post GT frames.
Figure 5: Object-only reconstruction across repeated topological changes. Comparison with EgoGaussian [ 29 ] , the only baseline that also supports reconstructing objects alone. Above, three successive fission events. Below, open and close .
Config
Topo. Object
Post-Event
MO
MB
BC
TA
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
✗
✗
✗
✗
21.11
0.664
0.278
18.45
0.579
0.286
✓
✗
✗
✗
21.72
0.681
0.270
19.07
0.607
0.274
✓
✓
✗
✗
23.84
0.745
0.190
23.05
0.698
0.222
✓
✓
✓
✗
23.92
0.755
0.179
23.27
0.709
0.199
✓
✓
✓
✓
25.32
0.786
0.158
26.77
0.780
0.151
Table 3: Topo-4DGS components. MO = Multiple Objects, MB = Motion Bifurcation, BC = Per-piece Boundary Construction, and TA = Event-Conditioned Temporal Activation. Together, BC and TA constitute Topological Boundary Resolution.
Topo. Object
Post-Event
Variant
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
UT-MLP
23.18
0.701
0.270
19.41
0.622
0.277
Fixed-KM
22.95
0.703
0.267
19.59
0.633
0.263
w/o TR
21.60
0.662
0.274
18.26
0.596
0.275
w/o JSR
21.61
0.665
0.277
18.57
0.607
0.272
w/o SR
23.25
0.693
0.293
20.80
0.628
0.293
Table 4: Topo-4DGS design choices. Each variant changes one component of Topo-4DGS: UT-MLP replaces the event-gated temporal term with an MLP; Fixed-KM freezes the initial k -means partition in the spatial branch; and w/o TR, JSR, or SR removes Time Refinement, Joint Scene Refinement, or Spatial Regularization Lsp , respectively.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Main paper section
Extended material
Sec.
Sec. 5.1 Motion Bifurcation
Spatial and Temporal Branch normalization and parameterization
Sec. D.3
Sec. 5.1 Motion Bifurcation
Spatial Branch initialization and learned partition
Sec. D.4
Sec. 5.3 Optimization
Objectives, masking, and spatial regularization
Sec. D.5
Sec. 5.2 Topological Boundary Resolution
Time Refinement
Sec. F.1
Sec. 5.3 Optimization
Missing-pose interpolation and its ablation
Secs. E and 9
Sec. 5.2 Topological Boundary Resolution
Boundary construction and branch binding
Sec. F.2
Appendix
Table 5: Guide to the supplementary material. Correspondence between sections of the main paper and the extended material provided in this document.
Category
#Actions
#Videos
Coverage
Ego4D ( 1,725 videos)
Topological (four target families)
16,132
1,187
68.8%
fission ( 1→N : cut/break/tear)
3,417
567
32.9%
fusion ( N→1 : attach/sew/weld)
4,210
743
43.1%
open (separable containers)
6,410
972
56.3%
close (separable containers)
2,095
600
34.8%
Appendix
Table 6: Prevalence of the target event families in the source annotations. Counts are obtained from Ego4D state-transition annotations and EPIC-KITCHENS timestamped action segments. Coverage is the fraction of source videos containing at least one annotation associated with the corresponding event family; a video may contribute to multiple families. Retrieved interactions are manually screened for a visible topological event as described in Sec. B.2 .
Event family
Inclusion criterion
Effect on represented surface support
Fission
A manipulated object separates into independently moving pieces through cutting, fracture, tearing, detachment, or disassembly.
Newly exposed cut or separation surfaces may become visible and require representation.
Fusion
Independently moving pieces are joined into a single coupled object.
Contacting surfaces become sealed or occluded and may be removed from the active representation.
Open
Parts of a separable assembly move apart, exposing an interior (e.g., removing a lid or opening a container).
Previously concealed interior surfaces become visible and enter the active representation.
Close
Parts of a separable assembly are rejoined, concealing its interior.
Interior and contact surfaces become concealed and leave the active representation.
Appendix
Table 7: TopoEgo event taxonomy. Events are categorized by object-part motion and their effect on the surface support represented by the model. Open and close apply to separable assemblies and need not alter material connectivity.
Clips
Event family
EPIC-K.
Ego4D
Frames
Objects
Events
fission
3
3
1,922
12
11
fusion
3
3
939
8
7
open
3
3
810
6
7
close
3
3
947
10
9
Total
12
12
4,618
36
34
Appendix
Table 8: TopoEgo benchmark composition. Each event family contains 3 clips from EPIC-KITCHENS and 3 from Ego4D. A clip may contain multiple annotated dynamic objects and topological events, so the object and event totals may exceed the number of clips.
Figure 6: Example event-tree annotation. A potato undergoes 5 successive cuts, with its observed states shown above the timeline. At each event time τm (red dashed line), one slice separates from the remaining body and begins following an independent rigid transform. Gray blocks group parts that still share a transform. The event-tree leaves are the terminal pieces: the 5 slices and the remaining body (i.e., terminal-piece count K=6).
PSNR ↑
LPIPS ↓
Method
g=1
g=5
g=10
g=1
g=5
g=10
Baselines
Def-3DGS [ 28 ]
20.27
16.43
12.46
0.252
0.267
0.389
DeGauss [ 24 ]
19.62
16.12
11.34
0.265
0.276
0.432
EgoGaussian [ 29 ]
20.12
16.53
15.71
0.238
0.263
0.392
4DGS [ 26 ]
17.25
13.20
9.45
0.385
0.491
0.546
Appendix
Table 9: Event-Aware Pose Interpolation. Reconstruction baselines and interpolation variants on the four-clip subset containing one sequence from each event family. Each split alternates g training and g evaluation frames, preserving the 1:1 ratio; larger g creates longer intervals without directly estimated poses. Metrics are computed only on those evaluation frames. PSNR uses topology-changing-object mask pixels, LPIPS uses the corresponding bounding-box crop, and hand pixels are zeroed only during scoring.
Topo. Object (region)
K
PSNR ↑
SSIM ↑
LPIPS ↓
Rate (it/s) ↑
1
20.62
0.741
0.198
43.83
2 (default)
23.06
0.781
0.173
37.24
4
22.86
0.773
0.176
34.13
8
23.43
0.786
0.176
33.62
16
23.31
0.778
0.173
33.48
Appendix
Table 10: Sensitivity to terminal-piece count K . The benchmark uses the ground-truth count; here we vary the input K on the 4-clip, all-family diagnostic subset, whose clips all have ground-truth K=2 . K=1 is under-specified, K∈{4,8,16} is moderately over-specified, and K=100 is an extreme stress test. Rate denotes optimization throughput.
Full scene
Topo. Object (region)
Event times
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Ground-truth
26.92
0.842
0.249
26.66
0.815
0.125
Automatically predicted
26.87
0.838
0.253
26.59
0.805
0.139
Offset −16 frames
26.15
0.859
0.272
26.19
0.789
0.178
Offset −8 frames
26.17
0.859
0.261
26.52
0.796
0.141
Offset +8 frames
26.19
0.860
0.263
26.58
0.798
0.149
Appendix
Table 11: Event-time sensitivity. The ground-truth row uses the manually annotated event times; the remaining rows use automatically predicted times or controlled offsets from the annotations. Negative offsets place the event earlier and positive offsets place it later. Results are averaged over the two-clip fission – open subset. Object-region metrics use mask PSNR, padded-crop SSIM, and bounding-box LPIPS; hand pixels are zeroed only during scoring.
Full scene
Topo. Object (region)
Training input
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Default training set
27.42
0.817
0.205
26.78
0.812
0.132
1/2 training frames
26.43
0.842
0.269
22.86
0.760
0.202
1/4 training frames
25.34
0.758
0.234
22.43
0.657
0.183
Appendix
Table 12: Input-frame-rate sensitivity. Starting from the benchmark’s alternating 1:1 split, the reduced settings retain every second or fourth frame of the default training set while leaving all evaluation frames unchanged. The fractions therefore refer to the number of default training frames retained. Results are averaged over the 2-clip fission–open diagnostic subset. PSNR/SSIM ↑ , LPIPS ↓ ; hand pixels are zeroed in prediction and ground truth only during scoring.
Source
Event
Method
PSNR ↑
SSIM ↑
LPIPS ↓
α -IoU ↑
EPIC-K.
fission
EgoGaussian
19.46
0.597
0.364
0.465
Topo-4DGS
21.92
0.650
0.355
0.488
fusion
EgoGaussian
13.75
0.218
0.441
0.066
Topo-4DGS
25.21
0.734
0.309
0.507
open
EgoGaussian
18.60
0.608
0.309
0.514
Topo-4DGS
21.68
0.643
0.293
0.574
Appendix
Table 13: Object-only topology-changing-object reconstruction on TopoEgo. Topo-4DGS and EgoGaussian support isolated object rendering, unlike baselines without an explicit object–background decomposition. Values are averaged over the 3 clips in each source–event cell and over all 24 TopoEgo clips in the overall row. PSNR/SSIM/ α -IoU ↑ , LPIPS ↓ ; α -IoU thresholds both rendered opacity and the ground-truth mask at 0.5 . Hand pixels are zeroed in both prediction and ground truth only during scoring.
Full scene
Topo. Object (region)
Other Dyn. Objects (region)
Clip
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
P01_105 †
HyperNeRF
19.46
0.707
0.337
12.78
0.414
0.357
11.09
0.224
0.364
Topo-4DGS
30.37
0.946
0.102
24.17
0.888
0.108
21.07
0.815
0.120
P28_105 †
HyperNeRF
19.28
0.550
0.340
16.92
0.513
0.450
–
–
–
Topo-4DGS
35.37
0.966
0.071
31.50
0.907
0.137
–
–
–
P03_120 †
HyperNeRF
23.11
0.771
0.244
16.51
0.508
0.392
14.35
0.176
0.471
Appendix
Table 14: Per-sequence comparison with HyperNeRF. Full-scene, topology-changing-object, and other-dynamic-object reconstruction on four short egocentric sequences using matched held-out frames and scoring. Hand pixels are zeroed in prediction and ground truth only during scoring. A dagger ( † ) marks the three sequences outside TopoEgo selected to keep HyperNeRF [ 19 ] training tractable. Other Dyn. Objects entries are omitted when a sequence contains no evaluated additional dynamic object.
Topo-4DGS [Ours]
EgoGaussian [ 29 ]
4DGS [ 26 ]
Def-3DGS [ 28 ]
DeGauss [ 24 ]
Clip
Type
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
EPIC-KITCHENS
P02_108
fusion
33.27
0.964
0.045
29.05
0.748
0.119
31.19
0.807
0.196
32.34
0.893
0.125
31.70
0.860
0.139
P02_122
fission
32.67
0.961
0.057
28.76
0.759
0.136
29.15
0.753
0.280
30.83
0.843
0.169
31.24
0.865
0.160
P04_04
fission
33.85
0.964
0.086
22.02
0.641
0.094
32.17
0.939
0.097
32.19
0.945
0.090
30.29
0.915
0.127
P04_118
close
26.08
0.889
0.302
20.91
0.696
0.447
22.94
0.736
0.531
23.96
0.821
0.458
23.39
0.787
0.479
Appendix
Table 15: Per-sequence full-scene reconstruction on TopoEgo. Full-frame PSNR, SSIM, and LPIPS are evaluated on held-out frames. Hand pixels are zeroed in prediction and ground truth only during scoring. PSNR/SSIM ↑ , LPIPS ↓ .
Topo-4DGS [Ours]
EgoGaussian [ 29 ]
4DGS [ 26 ]
Def-3DGS [ 28 ]
DeGauss [ 24 ]
Clip
Type
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
EPIC-KITCHENS
P02_108
fusion
28.17
0.866
0.075
25.85
0.764
0.077
26.00
0.752
0.083
27.12
0.828
0.078
26.65
0.796
0.080
P02_122
fission
20.91
0.769
0.113
19.10
0.681
0.123
18.85
0.668
0.149
20.02
0.737
0.128
19.55
0.708
0.136
P04_04
fission
25.78
0.862
0.219
18.50
0.560
0.280
18.20
0.550
0.410
21.20
0.760
0.292
20.20
0.680
0.335
P04_118
close
21.96
0.701
0.301
19.20
0.611
0.418
19.35
0.620
0.451
20.18
0.665
0.429
19.82
0.640
0.440
Appendix
Table 16: Per-sequence topology-changing-object reconstruction on TopoEgo. PSNR is computed over ground-truth mask pixels, SSIM over the masked pixels of a padded image crop, and LPIPS on the corresponding bounding-box crop. Hand pixels are zeroed in prediction and ground truth only during scoring. Results use held-out evaluation frames.
Topo-4DGS [Ours]
EgoGaussian [ 29 ]
4DGS [ 26 ]
Def-3DGS [ 28 ]
DeGauss [ 24 ]
Clip
Type
Object
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
EPIC-KITCHENS
P02_122
fission
compost_bin
36.58
0.945
0.074
29.20
0.820
0.145
29.75
0.842
0.132
32.95
0.905
0.092
31.90
0.885
0.105
P04_04
fission
knife
27.82
0.834
0.193
17.10
0.340
0.430
18.05
0.455
0.380
21.45
0.676
0.285
21.20
0.661
0.295
P04_118
close
cutting_board
24.20
0.798
0.257
19.85
0.662
0.477
20.55
0.690
0.464
24.08
0.842
0.400
23.82
0.834
0.403
P06_110
fission
knife
17.83
0.705
0.151
15.85
0.595
0.250
16.15
0.625
0.225
17.05
0.668
0.178
16.88
0.690
0.171
Appendix
Table 17: Per-sequence other-dynamic-object reconstruction on TopoEgo. For each Other Dyn. Object, PSNR is computed over its ground-truth mask pixels, SSIM over the masked pixels of its padded image crop, and LPIPS on its corresponding bounding-box crop. Hand pixels are zeroed in both prediction and ground truth only during scoring. Object scores are averaged within each eligible clip.
Clip
Type
Topo-4DGS [Ours]
EgoGaussian [ 29 ]
4DGS [ 26 ]
Def-3DGS [ 28 ]
DeGauss [ 24 ]
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
EPIC-KITCHENS
P02_108
fusion
26.04
0.809
0.105
21.16
0.571
0.132
21.70
0.597
0.160
23.34
0.737
0.137
22.38
0.661
0.144
P02_122
fission
26.94
0.824
0.083
21.14
0.578
0.146
20.75
0.564
0.171
23.19
0.737
0.143
22.07
0.655
0.151
P04_04
fission
26.28
0.872
0.351
13.52
0.356
0.388
13.24
0.426
0.553
20.78
0.580
0.417
20.20
0.535
0.426
P04_118
close
22.10
0.718
0.388
17.47
0.487
0.462
17.57
0.503
0.546
19.20
0.562
0.474
18.59
0.569
0.477
Appendix
Table 18: Per-sequence post-event reconstruction on TopoEgo. Object-region PSNR is computed over ground-truth mask pixels, SSIM over the masked pixels of a padded image crop, and LPIPS on the corresponding bounding-box crop. Hand pixels are zeroed in both prediction and ground truth only during scoring. A frame is weighted by the number of annotated events it is at or after, so frames following multiple events contribute to each corresponding post-event interval; pre-event frames are excluded.
Topo-4DGS [Ours]
EgoGaussian [ 29 ]
Def-3DGS [ 28 ]
DeGauss [ 24 ]
4DGS [ 26 ]
Clip
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
P03_03
32.24
0.939
0.139
32.73
0.940
0.125
31.92
0.931
0.146
31.55
0.925
0.153
31.70
0.928
0.150
P17_01
22.06
0.767
0.372
18.67
0.685
0.431
21.74
0.752
0.384
21.48
0.743
0.392
21.31
0.738
0.398
35fa3
30.11
0.938
0.176
31.53
0.946
0.169
29.86
0.941
0.099
30.71
0.946
0.097
22.49
0.755
0.316
P18_06
38.21
0.976
0.107
37.58
0.975
0.106
35.30
0.973
0.133
33.63
0.963
0.144
32.37
0.916
0.284
P32_01
29.57
0.890
0.197
30.49
0.922
0.172
32.33
0.947
0.132
29.30
0.922
0.163
33.92
0.944
0.151
Appendix
Table 19: Full-scene reconstruction on the Rigid Objects Control. Full-frame PSNR, SSIM, and LPIPS are evaluated on held-out frames from the six control sequences outside TopoEgo. Hand pixels are zeroed in prediction and ground truth only during scoring. PSNR/SSIM ↑ , LPIPS ↓ .
Topo-4DGS [Ours]
EgoGaussian [ 29 ]
Def-3DGS [ 28 ]
DeGauss [ 24 ]
4DGS [ 26 ]
Clip
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
P03_03
29.13
0.900
0.120
29.71
0.902
0.109
28.31
0.881
0.139
28.82
0.892
0.128
28.55
0.886
0.134
P17_01
21.79
0.657
0.269
21.58
0.583
0.285
21.18
0.631
0.289
21.42
0.642
0.280
20.96
0.623
0.296
35fa3
25.36
0.765
0.214
26.24
0.773
0.188
23.87
0.658
0.253
23.67
0.634
0.237
20.29
0.486
0.378
P18_06
33.68
0.934
0.079
32.72
0.928
0.084
19.20
0.589
0.318
19.37
0.599
0.320
20.63
0.590
0.369
P32_01
27.93
0.851
0.240
28.82
0.879
0.218
25.11
0.823
0.283
29.00
0.878
0.241
31.05
0.897
0.214
Appendix
Table 20: Evaluated-object reconstruction on the Rigid Objects Control. PSNR is computed over ground-truth object-mask pixels, SSIM over the masked pixels of a padded image crop, and LPIPS on the corresponding bounding-box crop. Hand pixels are zeroed in prediction and ground truth only during scoring. Results use held-out frames from the six control sequences.
Figure 7: Extended qualitative comparison on TopoEgo. Each row shows pre- and post-event ground truth followed by post-event renders from EgoGaussian, the hand-masked variants of DeGauss, Def-3DGS, and 4DGS, and Topo-4DGS. Changes are highlighted in cyan in the ground-truth frames.
Figure 8: Qualitative comparison on Rigid Objects Control sequences ( K=1 ). The first column shows the ground-truth frame with the evaluated object outlined; the remaining columns show the corresponding object-region renders from EgoGaussian, the hand-masked variants of DeGauss, Def-3DGS, and 4DGS, and Topo-4DGS. These sequences belong to the Rigid Objects Control and contain no target topological event.
Figure 9: Qualitative limitations. Left: strongly deformable dough violates the piecewise-rigid motion assumption. Right: a small, heavily occluded object part provides insufficient visual evidence for reliable reconstruction.
Egocentric video is crucial for next-generation 4D scene reconstruction, with applications in AR/VR and embodied AI. However, reconstructing dynamic first-person scenes is challenging due to complex ego-motion, occlusions, and hand-object interactions. Existing decomposition methods are ill-suited, assuming fixed viewpoints or merging dynamics into a single foreground. To address these limitations, we introduce DP-DeGauss, a dynamic probabilistic Gaussian decomposition framework for egocentric 4D reconstruction. Our method initializes a unified 3D Gaussian set from COLMAP priors, augments each with a learnable category probability, and dynamically routes them into specialized deformation branches for background, hands, or object modeling. We employ category-specific masks for better disentanglement and introduce brightness and motion-flow control to improve static rendering and dynamic reconstruction. Extensive experiments show that DP-DeGauss outperforms baselines by +1.70dB in PSNR on average with SSIM and LPIPS gains. More importantly, our framework achieves the first and state-of-the-art disentanglement of background, hand, and object components, enabling explicit, fine-grained separation, paving the way for more intuitive ego scene understanding and editing.
Tingxi Chen, Zhengxue Cheng, Houqiang Zhong +3
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University · Nvidia.
Egocentric video provides a unique view into human perception and interaction, with growing relevance for augmented reality, robotics, and assistive technologies. However, rapid camera motion and complex scene dynamics pose major challenges for 3D reconstruction from this perspective. While 3D Gaussian Splatting (3DGS) has become a state-of-the-art method for efficient, high-quality novel view synthesis, variants, that focus on reconstructing dynamic scenes from monocular video are rarely evaluated on egocentric video. It remains unclear whether existing models generalize to this setting or if egocentric-specific solutions are needed. In this work, we evaluate dynamic monocular 3DGS models on egocentric and exocentric video using paired ego-exo recordings from the EgoExo4D dataset. We find that reconstruction quality is consistently lower in egocentric views. Analysis reveals that the difference in reconstruction quality, measured in peak signal-to-noise ratio, stems from the reconstruction of static, not dynamic, content. Our findings underscore current limitations and motivate the development of egocentric-specific approaches, while also highlighting the value of separately evaluating static and dynamic regions of a video.
Jan Warchocki, Xi Wang, Jonas Kulhanek +1
Delft University of Technology · ETH Zurich · TUM/MCML +1
Recent 4D Gaussian Splatting (4DGS) methods achieve impressive dynamic scene reconstruction but often rely on piecewise linear velocity approximations and short temporal windows. This disjointed modeling leads to severe temporal fragmentation, forcing primitives to be repeatedly eliminated and regenerated to track complex nonlinear dynamics. This makeshift approximation eliminates the long-term temporal identity of objects and causes an inevitable proliferation of Gaussians, hindering scalability to extended video sequences. To address this, we propose TRiGS, a novel 4D representation that utilizes unified, continuous geometric transformations. By integrating SE(3) transformations, hierarchical Bezier residuals, and learnable local anchors, TRiGS models geometrically consistent rigid motions for individual primitives. This continuous formulation preserves temporal identity and effectively mitigates unbounded memory growth. Extensive experiments demonstrate that TRiGS achieves high fidelity rendering on standard benchmarks while uniquely scaling to extended video sequences (e.g., 600 to 1200 frames) without severe memory bottlenecks, significantly outperforming prior works in temporal stability.
Suwoong Yeom, Joonsik Nam, Seunggyu Choi +7
Sogang University · POSTECH · Seoul National University +2