Galaxies evolve over hundreds of millions of years, and no telescope has ever watched one move. Cosmological simulations make this evolution visible as a sequence of multi-view images, inviting the question of whether a galaxy can be recovered in four dimensions from its images. This setting defeats existing dynamic scene methods, since millions of stars span only a few pixels each, faint populations blend into diffuse light, and free deformations reproduce the images without obeying gravity. We present Cosmos4D, which rests on two components. Gravitational 4D Gaussian Splatting represents each star as a self-luminous Gaussian whose motion is integrated through a gravitational potential and a low-dimensional force field shared by all stars. Luminosity-stratified dynamics give bright stars individual orbits supervised across image scales and carry faint populations with a velocity field estimated from optical flow. We further build a 4D dataset of eight galaxies from four cosmological simulations, on which Cosmos4D outperforms dynamic Gaussian baselines in both photometric quality and physical accuracy. With these advancements, we glimpse eight hundred million years of a galaxy's life in ninety-six frames.
Figures & tables
Figure 2: Overview of Cosmos4D. (a) Anchor stars are split by luminosity into young bright stars B and old faint stars F , each a self-luminous Gaussian with evolving emission. (b) Each bright star follows its own orbit, integrated backward under the fixed potential Φa and a shared learned force fψ . (c) Faint stars circle their rings at the speed measured from optical flow, refined by a shared spin Qz(α) and an offset Dθ . (d) Stars add their light rather than occlude, and average pooling lets even drifted stars still overlap their observed targets.
Figure 3: Luminosity stratification. Bright stars B are distinct points along the spiral arms, while faint stars F blend into a glow.
Figure 4: Qualitative comparison on test views, with the boxed region magnified at the bottom-right of each tile. Baselines blur the stellar disk or break it into streaks, whereas Cosmos4D recovers star-level texture, young blue stars, and tidal features seen in GT.
Methods
Metrics
G01
G02
C001
C010
C005
C014
T300C010
T300C021
Avg.
Mask
Full
Mask
Full
Mask
Full
Mask
Full
Mask
Full
Mask
Full
Mask
Full
Mask
Full
Mask
Full
4DGS [ 55 ]
PSNR ↑
19.10
25.15
20.27
25.42
20.41
26.00
24.42
29.45
23.59
29.77
19.91
25.47
17.15
24.48
15.03
23.10
19.99
26.11
SSIM ↑
0.302
0.670
0.348
0.632
0.286
0.603
0.391
0.610
0.458
0.654
0.322
0.627
0.149
0.554
0.110
0.643
0.296
0.624
LPIPS ↓
0.896
0.576
0.857
0.561
0.870
0.594
0.763
0.590
0.671
0.489
0.812
0.575
0.792
0.679
0.805
0.690
0.808
0.594
FreeGave [ 24 ]
PSNR ↑
19.99
26.00
20.75
25.88
19.77
25.21
24.29
29.40
23.91
30.14
19.86
25.45
17.08
24.38
15.62
24.80
20.16
26.41
SSIM ↑
0.320
0.685
0.362
0.660
0.283
0.622
0.405
0.643
0.493
0.693
0.349
0.659
0.168
0.556
0.115
0.645
0.312
0.645
Table 1: Quantitative comparison of novel-view rendering on all eight scenes. For each scene, Mask reports metrics within the galaxy foreground mask and Full reports metrics on the whole image. All metrics are averaged over three test views and 96 frames at 5122 resolution. Darker cells mark the best, second best, and third best results.
Scene
4DGS
FreeGave
MoDec-GS
4C4D
RetimeGS
Ours
G01
0.984
0.921
0.960
0.550
0.468
0.121
G02
0.987
0.962
0.969
0.784
0.512
0.522
C001
0.990
0.958
0.969
0.704
0.471
0.016
C010
0.996
0.979
0.986
0.965
0.849
0.096
C005
0.995
0.982
0.973
0.910
0.689
0.677
C014
0.984
0.854
0.933
0.723
0.402
0.267
Table 2: Relative error of the rotation curve vϕ(R)↓ , measured on the 3D Gaussian centers exported from each model.
Scene
4DGS
FreeGave
MoDec-GS
4C4D
RetimeGS
Ours
G01
2.454
1.809
1.996
1.682
–
0.192
G02
2.561
2.002
1.874
1.848
–
0.501
C001
2.473
1.665
2.104
1.643
–
0.241
C010
2.043
0.656
1.763
2.271
–
0.000
C005
2.235
1.559
1.938
1.979
–
0.461
C014
2.383
1.340
1.745
1.456
–
0.718
Table 3: Angular momentum conservation error ↓ in dex. RetimeGS primitives stay active for only about four frames, so their Lz along a trajectory is undefined and is thus reported as a dash.
Scene
4DGS
FreeGave
MoDec-GS
4C4D
RetimeGS
Ours
G01
0.060
0.058
0.057
0.042
0.031
0.009
G02
0.131
0.130
0.082
0.051
0.062
0.019
C001
0.288
0.280
0.323
0.307
0.237
0.163
C010
0.250
0.180
0.233
0.232
0.218
0.194
C005
0.106
0.083
0.125
0.272
0.109
0.122
C014
0.272
0.247
0.280
0.214
0.170
0.129
Table 4: Relative error of the half-light radius R50↓ , measured on deprojected renderings from the unseen near face-on view.
Figure 5: Ablations rendered on test views of G01 (top) and T300C010 (bottom). Without additive splatting, front stars dim those behind them. Without evolving emission, the young blue stars fade to grey. Without gravitational orbits or the anchor potential, bright stars drift off the disk and blur the cluster. The insets reveal these details in a young star cluster on the outer arm of G01 and in the core of T300C010.
Setting
G01
T300C010
Mask
Full
Physics
Mask
Full
Physics
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
RMSE ↓
vϕ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
RMSE ↓
vϕ↓
w/o additive splatting
19.19
0.326
0.391
25.16
0.701
0.322
8.61
0.185
16.66
0.168
0.356
23.51
0.558
0.419
24.57
0.740
w/o evolving emission
20.32
0.368
0.405
26.33
0.705
0.350
8.58
0.125
17.02
0.178
0.337
23.78
0.560
0.404
24.58
0.751
w/o gravitational orbits
19.46
0.288
0.395
25.47
0.673
0.353
8.88
0.154
17.18
0.158
0.400
23.65
0.553
0.465
23.71
0.506
w/o anchor potential
20.00
0.338
0.379
26.02
0.692
0.342
8.92
0.232
17.24
0.175
0.359
23.64
0.553
0.429
24.00
0.286
Table 5: Ablation on the test views. Mask and Full are metrics inside the galaxy foreground mask and on the whole image. RMSE (ckpc /h ) is the position error of the modeled stars against ground-truth particles, and vϕ is the relative rotation-curve error used in Tab. 2 .
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Real disk galaxies imaged by Hubble. (a) The face-on spiral M74 with radius R and azimuth ϕ in the disk plane. (b) The edge-on spiral NGC 891 with height z above the disk.
Figure 7: Background quantities on G01. (a) Circular speed of the anchor potential Φa fitted in Appendix D and of its bulge, disk, and dark matter halo components, which add in quadrature. (b) Luminosity against age at the anchor for 1,500 sampled stars each of the bright set B and the faint set F . (c) Median emission gain of Appendix C over lookback time for 400 young stars born during the sequence and 400 old stars, normalized at the anchor.
Scene
Source
N
∣B∣
∣F∣
G01
TNG50-1 Subbox0
110,013
56,572
53,441
G02
TNG50-1 Subbox0
110,453
45,217
65,236
C001
TNG100-1 Subbox1
75,123
75,123
0
C010
TNG100-1 Subbox0
77,379
77,379
0
C005
Illustris-1 Subbox0
110,265
3,159
107,106
C014
Illustris-1 Subbox0
90,324
90,324
0
Appendix
Table 6: Scenes used in our experiments. N is the number of sampled stellar particles, split into bright and faint sets.
Figure 8: Camera layout on G01, drawn with the disk normal pointing up. Fourteen cameras lie on a Fibonacci sphere around the galaxy. Gold circles are training views, blue squares are test views, and each short tick points toward the galaxy center.
Figure 9: Blended orbital segments. (a) Overlapping segments integrated backward from learned endpoints. (b) Blending two segments within an overlap versus a hard switch. (c) Quintic blend weight β and the step weight of a hard switch.
Setting
G01
T300C010
PSNR ↑
SSIM ↑
LPIPS ↓
RMSE ↓
PSNR ↑
SSIM ↑
LPIPS ↓
RMSE ↓
Full model
20.44
0.387
0.363
8.60
17.01
0.183
0.330
24.58
Single segment
19.88
0.319
0.366
11.10
16.93
0.157
0.364
26.53
Hard switching
20.43
0.385
0.364
8.60
16.97
0.180
0.335
24.60
Appendix
Table 7: Segmented integration on G01 and T300C010. Single segment integrates each bright orbit from the anchor in one piece, and hard switching replaces the blending weight with a step. Metrics are masked, and RMSE is the 3D position error of the modeled stars in ckpc /h , used only for diagnosis.
Setting
G01
T300C010
Mask
Full
Mask
Full
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
RMSE ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
RMSE ↓
Full model
20.44
0.387
0.363
26.45
0.712
0.330
8.60
17.01
0.183
0.330
23.74
0.561
0.387
24.58
w/o additive splatting
19.19
0.326
0.391
25.16
0.701
0.322
8.61
16.66
0.168
0.356
23.51
0.558
0.419
24.57
w/o evolving emission
20.32
0.368
0.405
26.33
0.705
0.350
8.58
17.02
0.178
0.337
23.78
0.560
0.404
24.58
w/o residual force
20.44
0.386
0.362
26.45
0.711
0.330
8.63
16.97
0.182
0.331
23.72
0.561
0.385
24.57
Appendix
Table 8: Complete ablation on G01 and T300C010. Mask and Full denote metrics within the galaxy foreground mask and on the whole image. RMSE is the 3D position error of modeled stars against ground-truth particles in ckpc /h and is used only for diagnosis. The full model is listed first.
Figure 10: Trajectory error of all modeled stars against ground-truth particles over lookback time for each ablation variant on G01 (top) and T300C010 (bottom). Columns show the 3D position error and its radial and vertical components in ckpc /h .
Figure 11: Disk-plane tracks of four bright stars for each ablation variant on G01 (top) and T300C010 (bottom), with the full model first and the ablations in the order of Tab. 5 . Each row covers one third of the sequence, counted back in lookback time from the anchor. Dashed gray lines are ground-truth particle tracks, and solid lines are the modeled tracks in the color of the column title. Filled dots mark the start of each interval and open circles its end, in black or gray for the ground truth and in the variant color for the model. The cross marks the galaxy center. Ground-truth particles are used for evaluation only.
Recent progress in 4D Gaussian Splatting (4DGS) has achieved impressive dynamic scene reconstruction results. While these methods demonstrate remarkable performance, the specific factors behind their gains remain underexplored, making a systematic understanding of the underlying principles challenging. In this paper, we perform a comprehensive analysis of these hidden factors to provide a clearer perspective on the 4DGS framework. We first establish a controlled baseline, FreeTimeGS_ours, by formalizing and reproducing the heuristics of the state-of-the-art FreeTimeGS. Using this framework, we examine 4DGS along its fundamental axes and identify practical secrets, including the emergent temporal partitioning driven by Gaussian durations and the decoupling between photometric fidelity and motion behavior. Based on these insights, we propose FreeTimeGS++, a principled method that employs gated marginalization, UFM-guided initialization, and color correction to improve stability and reproducibility. Our approach yields reproducible results with reduced run-to-run variance.
We propose a novel view synthesis tool for galaxies based on Gaussian Splatting (GS). Unlike RGB data, where GS has achieved success, galaxy observations consist of multi-band signals from a massive number of bodies, contain complex patterns of cross-band interactions, and capture light propagation in highly heterogeneous media. As a result, the underlying physical process poses great challenges in model representation and optimization, which have not been investigated in existing research. To this end, we propose a physically inspired Gaussian Splatting method, GalaxyGS, which is equipped with physically meaningful components that capture both the energy emission mechanism of different astronomical bodies and the light propagation between them. In the absence of directly comparable baseline methods, we adopt a wide range of possible alternative approaches. Comparisons conclusively demonstrate that GalaxyGS is better in accuracy and physical fidelity across datasets and metrics.
Yuzhe Wu, Jialin Zhu, Doğa Yılmaz +2
University College London, UK · Baidu · AI Centre, University College London, UK
This paper tackles the challenge of recovering 4D dynamic scenes from videos captured by as few as four portable cameras. Learning to model scene dynamics for temporally consistent novel-view rendering is a foundational task in computer graphics, where previous works often require dense multi-view captures using camera arrays of dozens or even hundreds of views. We propose 4C4D, a novel framework that enables high-fidelity 4D Gaussian Splatting from video captures of extremely sparse cameras. Our key insight lies that the geometric learning under sparse settings is substantially more difficult than modeling appearance. Driven by this observation, we introduce a Neural Decaying Function on Gaussian opacities for enhancing the geometric modeling capability of 4D Gaussians. This design mitigates the inherent imbalance between geometry and appearance modeling in 4DGS by encouraging the 4DGS gradients to focus more on geometric learning. Extensive experiments across sparse-view datasets with varying camera overlaps show that 4C4D achieves superior performance over prior art. Project page at: https://junshengzhou.github.io/4C4D.
Junsheng Zhou, Zhifan Yang, Liang Han +4
School of Software, Tsinghua University · Kuaishou Technology