Gallileo-4D: Frozen Backbone Ensemble for Dynamic 4D Reconstruction

Nicolò Savioli

cs.CVAug 20, 2026

Gallileo-4D: Frozen Backbone Ensemble for Dynamic 4D Reconstruction

Authors: Nicolò Savioli

Organizations: OdaxAI Research

Abstract

We describe our entry to the PhysAI Dynamic 4D Reconstruction Challenge, which placed third of 27 teams at 0.58356 APD on the final leaderboard, without a single gradient update. This was not the plan: of thirteen fine-tuning configurations of a pre-trained 4D backbone, twelve degraded the challenge score, and eleven of those twelve improved local validation at the same time. We trace this inversion to the structure of the benchmark: only 25% of the evaluation set belongs to the data variant released for training, so updates that fit the available data damage the pre-trained features the remaining 75% relies on. Our system therefore freezes the backbone and spends its budget at inference time, fusing three decoding configurations -- temporal stride-3, horizontal-flip test-time augmentation, and dense stride-1 -- under a convex weighting. The ensemble recovers +0.041 APD over the frozen baseline, more than any training run achieved, at zero training cost.

Figures & tables

Figure 1: Gallileo-4D results. Dynamic 4D reconstruction of two challenge test sequences, office (top) and gothic (bottom). Each row shows the reconstruction at the first queried timestamp $t_{0}$ , an $\alpha$ -blend of the first and last, and the last timestamp $t_{191}$ , with the recovered 3D trajectory of every tracked point drawn over the full sequence and coloured by time. The system placed third of $27$ teams on the final leaderboard without a single gradient update . A uniform tone curve is applied to the low-light gothic panels for print legibility; no geometry or trajectory is retouched.

Figure 2: What we could train on versus what we were scored on. Top: the five scenes of the released syn4d_sim split, the only training data available to us. Middle: the evaluation set is divided evenly across four rendering variants, and only sim has a training analogue, leaving three quarters of the distribution without direct supervision. Bottom: the three calibration points we obtained between the local proxy and the challenge score, two of which invert in sign. The SlotTABA bar is truncated at the plot edge; its true value is $-0.3443$ .

	Split
Attribute	syn4d_sim (train)	challenge (test)
Frames / clip	50	192
Resolution	$1024{\times}1024$	$1280{\times}720$
Aspect ratio	$1{:}1$	$16{:}9$
Cameras / clip	8 (multi-view)	1 (monocular)
Horiz. FOV	$35$ – $84^{\circ}$	$39.8$ – $89.7^{\circ}$

Table 1: Training split versus evaluation set. The variant row determines everything else in this paper.

Intervention	local	leaderb.	Verdict
	Score change
base $\rightarrow$ geo ckpt	$+0.0078$	$+0.0325$	agrees, $4.2\times$
geo $\rightarrow$ SlotTABA	$+0.0129$	$-0.3443$	false positive
geo $\rightarrow$ fine-tune	$+0.035$	$-0.030$	false positive

Table 2: Every calibration point obtained between the local proxy and the challenge leaderboard. Two of three are sign-inconsistent; the fine-tuning row is the mean over the twelve regressing runs of Tab. 3 .

Figure 3: Thirteen training runs, one survivor. (a) Change in local validation against change in challenge

\mathrm{APD}

, both relative to the frozen baseline. Eleven runs land above the horizontal axis on the local proxy and below it on the challenge metric. Only train_v2 reaches the positive quadrant. (b) Absolute challenge score per run, annotated with learning rate; the dashed line is the frozen baseline the runs were supposed to beat, and the dotted line is the final ensemble, which uses no training at all.

Run	Adapted part	LR	ep.	local	chall.	$\Delta$ vs. frozen
Naïve fine-tuning
train_v1	full network	$5\text{\times}{10}^{-5}$	1	0.42	0.48	$-0.032$
train_v2	full network	$5\text{\times}{10}^{-6}$	1	0.44	0.54	$+0.028$
train_v3	full network	$5\text{\times}{10}^{-6}$	3	0.46	0.52	$+0.008$
train_v4	full, scale-dec.	$1\text{\times}{10}^{-5}$	1	0.45	0.49	$-0.022$
train_v4d	full, scale-dec.	$1\text{\times}{10}^{-5}$	1	0.43	0.48	$-0.032$

Table 3: All thirteen training runs, with the frozen inference-time ensemble in the last row for reference. Bars are drawn to scale; green is a gain over the frozen geofinetune baseline at

0.512

APD, red a loss. Only train_v2 and train_v3 gained, and only train_v2 by more than noise — while paper_A took the best local score of the study. Local scores are soft-APD on the

31

-sequence sim proxy and are not comparable in absolute value to the challenge column.

Figure 4: The resolution axis and its failure mode. (a) Local and challenge scores against inference resolution. (b) At

1008

px the geometry head emits predictions whose median scale is

0.117

against an expected

3.5

, a collapse that median alignment cannot repair because it is not globally consistent.

Figure 5: Model architecture of Gallileo-4D. The video is patchified into per-frame patch, camera and time tokens and encoded once by the frozen 4RC backbone into the 4D latent

\mathcal{F}

. Frozen geometry and motion heads decode a base point map plus a displacement for any (query, target time) pair; the same frozen network is evaluated under three inference-time decoding configurations, and their point maps are averaged with fixed weights. No parameter is updated at any stage: the only fitted quantities in the entire system are the three fusion weights

w

Figure 6: The three decoding configurations, and how they combine. Top: which frames each configuration admits inside one

W{=}48

window. Stride-3 admits every third frame and so reaches

144

source frames of temporal context; stride-1 admits all

48

and maximises temporal resolution instead; the TTA branch runs the stride-3 pattern on a horizontally mirrored window and mirrors the predicted point maps back. The three therefore make errors that are only weakly correlated, which is what the fusion exploits. Bottom: for each query

q

and target time

\tau

the three configurations propose three world-space positions, and the submitted prediction is their fixed convex combination. Averaging in world coordinates is meaningful only because all three share the frozen backbone, and therefore one scale convention.

Figure 7: Results. (a) Public-split score after each accepted change during development, from the organisers’ OpenD4RT baseline to our submitted system; grey bars are public baselines, blue are inference-time changes, green is the submitted entry, and the dashed line is the score that entry finally received on the private split. (b) The ensemble weight search of Eq. ( 6 ); the optimum is circled and the surface around it is flat. (c) Final leaderboard: pale bars the public $25\%$ , solid the final $75\%$ ; other entries unnamed.

		APD $\uparrow$
Rank	Entry	public	final	$\Delta$
1	anonymised	0.56496	0.62426	$+0.0593$
2	anonymised	0.56557	0.58667	$+0.0211$
3	OdaxAI (ours)	0.55513	0.58356	$+0.0284$
4	anonymised	0.54082	0.58024	$+0.0394$
5	anonymised	0.54455	0.57876	$+0.0342$

Table 4: Final leaderboard, top five. The public column is scored on approximately

25\%

of the test data and the final column on the remaining

75\%

. Identities are withheld; only our own entry is named. Our submitted system moved up one place between the two, and the configuration analysed in Sec. 5 and ablated in Tab. 5 is the one scored here.

	Fusion weights
Configuration	$w_{1}$	$w_{2}$	$w_{3}$	$\mathrm{APD}$ $\uparrow$
OpenD4RT baseline	—	—	—	0.31564
4RC, released ckpt	—	—	—	0.48419
$+$ geofinetune	—	—	—	0.51234
$+$ train_v2 heads	—	—	—	0.54001
stride-3 only	1.00	—	—	0.54256

Table 5: Ablation of the ensemble. All rows use the same frozen backbone; only the decoding configurations and their weights change. Best and second best . The four-way row adds the collapsed

1008

px branch of Sec. 4.1 and was rejected.

Figure 8: Reconstruction detail on two test sequences. Left, office : a bright cluttered interior with large lateral motion, where trajectories stay smooth and separated even as actors cross. Right, gothic : low light, self-occlusion and fast articulated motion; bundles stay coherent through the occlusion, but tracks on the fastest limb fan out, which is where the residual error concentrates.

Figure 9: Zero-shot transfer to real video. The frozen pipeline applied unchanged to the judo sequence of DAVIS [ 21 ] : reconstructed geometry with the dense 3D motion field of the throw, coloured by time. Neither the decoding heads nor the fusion weights were touched, and the system was never trained or tuned on real footage. The figure is qualitative only—DAVIS provides no metric 4D ground truth—but it shows that nothing in the recipe is specific to the synthetic benchmark.

4D Reconstruction

4D reconstruction recovers changing 3D scenes, shapes, and motion over time from images, videos, or other sparse measurements.

See all 139 papers

Aug 18, 2026

Depth Anything V4: Dynamic 4D Scene Reconstruction via Riemannian Flow Matching on 4D Gaussian Splatting

We present Depth Anything V4 (DAV4), a framework for dynamic 4D scene reconstruction from monocular video. Our key contribution is the application of Riemannian Flow Matching (RFM) to 4D Gaussian Splatting parameters, defining probability paths directly on non-Euclidean manifolds (scale, rotation, opacity), ensuring all intermediate states are valid. Through controlled experiments, we isolate RFM's contribution from test-time optimization (TTO) and pre-training. A deterministic MLP baseline with the same data, architecture, and TTO achieves F-score 0.762; RFM achieves 0.806 - the +0.044 gain is RFM's isolated contribution. We provide corrected computational cost analysis: pre-training is 360 GPU-hours, amortizing for large-scale deployment (over 10,000 scenes). Uncertainty is quantified via Negative Gaussian Log-Likelihood and Expected Calibration Error. DAV4 outperforms prior Depth Anything models and per-scene 4D-GS on dynamic reconstruction and novel-view synthesis, while using no human-annotated depth labels as training losses.

Jiaming Fan, Jian Lu, Jinling Jia +1

College of Artificial Intelligence, Nanjing University of Posts and Telecommunications · College of Automation, Nanjing University of Posts and Telecommunications · Institute of Ad-vanced Technology for Carbon Neutrality, Nanjing University of Posts and Telecommunications

4D Reconstruction Dynamic Scene Reconstruction Monocular 3D Reconstruction 3D Gaussian Splatting Novel View Synthesis Riemannian Flow Matching

Ask PDF

Apr 10, 2026

Robust 4D Visual Geometry Transformer with Uncertainty-Aware Priors

Reconstructing dynamic 4D scenes is an important yet challenging task. While 3D foundation models like VGGT excel in static settings, they often struggle with dynamic sequences where motion causes significant geometric ambiguity. To address this, we present a framework designed to disentangle dynamic and static components by modeling uncertainty across different stages of the reconstruction process. Our approach introduces three synergistic mechanisms: (1) Entropy-Guided Subspace Projection, which leverages information-theoretic weighting to adaptively aggregate multi-head attention distributions, effectively isolating dynamic motion cues from semantic noise; (2) Local-Consistency Driven Geometry Purification, which enforces spatial continuity via radius-based neighborhood constraints to eliminate structural outliers; and (3) Uncertainty-Aware Cross-View Consistency, which formulates multi-view projection refinement as a heteroscedastic maximum likelihood estimation problem, utilizing depth confidence as a probabilistic weight. Experiments on dynamic benchmarks show that our approach outperforms current state-of-the-art methods, reducing Mean Accuracy error by 13.43% and improving segmentation F-measure by 10.49%. Our framework maintains the efficiency of feed-forward inference and requires no task-specific fine-tuning or per-scene optimization.

Ying Zang, Yidong Han, Chaotao Ding +8

Huzhou University · KOKONI 3D, Moxin Technology · Univeristy of Science and Technology of China +3

UQ for 3D Reconstruction Dynamic Scene Reconstruction 4D Reconstruction 3D Reconstruction Multi-View 3D Reconstruction 3D ViTs+1

Ask PDF

May 12, 2026

4DVGGT-D: 4D Visual Geometry Transformer with Improved Dynamic Depth Estimation

Reconstructing dynamic 4D scenes from monocular videos is a fundamental yet challenging task. While recent 3D foundation models provide strong geometric priors, their performance significantly degrades in dynamic environments. This degradation stems from a fundamental tension: the inherent coupling of camera ego-motion and object motion within global attention mechanisms. In this paper, we propose a novel, training-free progressive decoupling framework that disentangles dynamics from statics in a principled, coarse-to-fine manner. Our core insight is to resolve the tension by first stabilizing the camera pose, followed by geometric refinement. Specifically, our approach consists of three synergistic components: (1) a Dynamic-Mask-Guided Pose Decoupling module that isolates pose estimation from dynamic interference, yielding a stable motion-free reference frame; (2) a Topological Subspace Surgery mechanism that orthogonally decomposes the depth manifold, safely preserving dynamic objects while injecting refined, mask-aware geometry into static regions; and (3) an Information-Theoretic Confidence-Aware Fusion strategy that formulates depth integration as a heteroscedastic Bayesian inference problem, adaptively blending multi-pass predictions via inverse-variance weighting. Extensive experiments on standard 4D reconstruction benchmarks demonstrate that our method achieves consistent and substantial improvements across principal point-cloud metrics. Notably, our approach shows competitive performance in robust 4D scene reconstruction without requiring fine-tuning, suggesting the potential of mathematically grounded dynamic-static disentanglement.

Ying Zang, Xuanyi Liu, Yidong Han +9

4Huzhou University · 2Peking University · 1KOKONI 3D, Moxin Technology +4

4D Reconstruction Depth Estimation Monocular 3D Reconstruction Representation Disentanglement Dynamic Scene Reconstruction

Ask PDF

Abstract

4D Reconstruction

Explore similar work

Depth Anything V4: Dynamic 4D Scene Reconstruction via Riemannian Flow Matching on 4D Gaussian Splatting

Robust 4D Visual Geometry Transformer with Uncertainty-Aware Priors

4DVGGT-D: 4D Visual Geometry Transformer with Improved Dynamic Depth Estimation