cs.CVAug 20, 2026
SaveGallileo-4D: Frozen Backbone Ensemble for Dynamic 4D Reconstruction
Organizations: OdaxAI Research
Abstract
We describe our entry to the PhysAI Dynamic 4D Reconstruction Challenge, which placed third of 27 teams at 0.58356 APD on the final leaderboard, without a single gradient update. This was not the plan: of thirteen fine-tuning configurations of a pre-trained 4D backbone, twelve degraded the challenge score, and eleven of those twelve improved local validation at the same time. We trace this inversion to the structure of the benchmark: only 25% of the evaluation set belongs to the data variant released for training, so updates that fit the available data damage the pre-trained features the remaining 75% relies on. Our system therefore freezes the backbone and spends its budget at inference time, fusing three decoding configurations -- temporal stride-3, horizontal-flip test-time augmentation, and dense stride-1 -- under a convex weighting. The ensemble recovers +0.041 APD over the frozen baseline, more than any training run achieved, at zero training cost.
Figures & tables
Figure 1: Gallileo-4D results. Dynamic 4D reconstruction of two challenge test sequences, office (top) and gothic (bottom). Each row shows the reconstruction at the first queried timestamp , an -blend of the first and last, and the last timestamp , with the recovered 3D trajectory of every tracked point drawn over the full sequence and coloured by time. The system placed third of teams on the final leaderboard without a single gradient update . A uniform tone curve is applied to the low-light gothic panels for print legibility; no geometry or trajectory is retouched.
Figure 2: What we could train on versus what we were scored on. Top: the five scenes of the released syn4d_sim split, the only training data available to us. Middle: the evaluation set is divided evenly across four rendering variants, and only sim has a training analogue, leaving three quarters of the distribution without direct supervision. Bottom: the three calibration points we obtained between the local proxy and the challenge score, two of which invert in sign. The SlotTABA bar is truncated at the plot edge; its true value is .
| Split | ||
| Attribute | syn4d_sim (train) | challenge (test) |
| Frames / clip | 50 | 192 |
| Resolution | ||
| Aspect ratio | ||
| Cameras / clip | 8 (multi-view) | 1 (monocular) |
| Horiz. FOV | – | – |
Table 1: Training split versus evaluation set. The variant row determines everything else in this paper.
| Score change | |||
|---|---|---|---|
| Intervention | local | leaderb. | Verdict |
| base geo ckpt | agrees, | ||
| geo SlotTABA | false positive | ||
| geo fine-tune | false positive | ||
Table 2: Every calibration point obtained between the local proxy and the challenge leaderboard. Two of three are sign-inconsistent; the fine-tuning row is the mean over the twelve regressing runs of Tab. 3 .
Figure 3: Thirteen training runs, one survivor. (a) Change in local validation against change in challenge , both relative to the frozen baseline. Eleven runs land above the horizontal axis on the local proxy and below it on the challenge metric. Only train_v2 reaches the positive quadrant. (b) Absolute challenge score per run, annotated with learning rate; the dashed line is the frozen baseline the runs were supposed to beat, and the dotted line is the final ensemble, which uses no training at all.
| Run | Adapted part | LR | ep. | local | chall. | vs. frozen |
| Naïve fine-tuning | ||||||
| train_v1 | full network | 1 | 0.42 | 0.48 | ||
| train_v2 | full network | 1 | 0.44 | 0.54 | ||
| train_v3 | full network | 3 | 0.46 | 0.52 | ||
| train_v4 | full, scale-dec. | 1 | 0.45 | 0.49 | ||
| train_v4d | full, scale-dec. | 1 | 0.43 | 0.48 | ||
Table 3: All thirteen training runs, with the frozen inference-time ensemble in the last row for reference. Bars are drawn to scale; green is a gain over the frozen geofinetune baseline at APD, red a loss. Only train_v2 and train_v3 gained, and only train_v2 by more than noise — while paper_A took the best local score of the study. Local scores are soft-APD on the -sequence sim proxy and are not comparable in absolute value to the challenge column.
Figure 4: The resolution axis and its failure mode. (a) Local and challenge scores against inference resolution. (b) At px the geometry head emits predictions whose median scale is against an expected , a collapse that median alignment cannot repair because it is not globally consistent.
Figure 5: Model architecture of Gallileo-4D. The video is patchified into per-frame patch, camera and time tokens and encoded once by the frozen 4RC backbone into the 4D latent . Frozen geometry and motion heads decode a base point map plus a displacement for any (query, target time) pair; the same frozen network is evaluated under three inference-time decoding configurations, and their point maps are averaged with fixed weights. No parameter is updated at any stage: the only fitted quantities in the entire system are the three fusion weights .
Figure 6: The three decoding configurations, and how they combine. Top: which frames each configuration admits inside one window. Stride-3 admits every third frame and so reaches source frames of temporal context; stride-1 admits all and maximises temporal resolution instead; the TTA branch runs the stride-3 pattern on a horizontally mirrored window and mirrors the predicted point maps back. The three therefore make errors that are only weakly correlated, which is what the fusion exploits. Bottom: for each query and target time the three configurations propose three world-space positions, and the submitted prediction is their fixed convex combination. Averaging in world coordinates is meaningful only because all three share the frozen backbone, and therefore one scale convention.
Figure 7: Results. (a) Public-split score after each accepted change during development, from the organisers’ OpenD4RT baseline to our submitted system; grey bars are public baselines, blue are inference-time changes, green is the submitted entry, and the dashed line is the score that entry finally received on the private split. (b) The ensemble weight search of Eq. ( 6 ); the optimum is circled and the surface around it is flat. (c) Final leaderboard: pale bars the public , solid the final ; other entries unnamed.
| APD | ||||
| Rank | Entry | public | final | |
| 1 | anonymised | 0.56496 | 0.62426 | |
| 2 | anonymised | 0.56557 | 0.58667 | |
| 3 | OdaxAI (ours) | 0.55513 | 0.58356 | |
| 4 | anonymised | 0.54082 | 0.58024 | |
| 5 | anonymised | 0.54455 | 0.57876 | |
Table 4: Final leaderboard, top five. The public column is scored on approximately of the test data and the final column on the remaining . Identities are withheld; only our own entry is named. Our submitted system moved up one place between the two, and the configuration analysed in Sec. 5 and ablated in Tab. 5 is the one scored here.
| Fusion weights | ||||
| Configuration | ||||
| OpenD4RT baseline | — | — | — | 0.31564 |
| 4RC, released ckpt | — | — | — | 0.48419 |
| geofinetune | — | — | — | 0.51234 |
| train_v2 heads | — | — | — | 0.54001 |
| stride-3 only | 1.00 | — | — | 0.54256 |
Table 5: Ablation of the ensemble. All rows use the same frozen backbone; only the decoding configurations and their weights change. Best and second best . The four-way row adds the collapsed px branch of Sec. 4.1 and was rejected.
Figure 8: Reconstruction detail on two test sequences. Left, office : a bright cluttered interior with large lateral motion, where trajectories stay smooth and separated even as actors cross. Right, gothic : low light, self-occlusion and fast articulated motion; bundles stay coherent through the occlusion, but tracks on the fastest limb fan out, which is where the residual error concentrates.
Figure 9: Zero-shot transfer to real video. The frozen pipeline applied unchanged to the judo sequence of DAVIS [ 21 ] : reconstructed geometry with the dense 3D motion field of the throw, coloured by time. Neither the decoding heads nor the fusion weights were touched, and the system was never trained or tuned on real footage. The figure is qualitative only—DAVIS provides no metric 4D ground truth—but it shows that nothing in the recipe is specific to the synthetic benchmark.
More in this topic
4D Reconstruction
4D reconstruction recovers changing 3D scenes, shapes, and motion over time from images, videos, or other sparse measurements.
See all 139 papers