4D reconstruction of equine family (e.g. horses) from monocular video is important for animal welfare. Previous mainstream 4D animal reconstruction methods require joint optimization of motion and appearance over a whole video, which is time-consuming and sensitive to incomplete observation. In this work, we propose a novel framework called 4DEquine by disentangling the 4D reconstruction problem into two sub-problems: dynamic motion reconstruction and static appearance reconstruction. For motion, we introduce a simple yet effective spatio-temporal transformer with a post-optimization stage to regress smooth and pixel-aligned pose and shape sequences from video. For appearance, we design a novel feed-forward network that reconstructs a high-fidelity, animatable 3D Gaussian avatar from as few as a single image. To assist training, we create a large-scale synthetic motion dataset, VarenPoser, which features high-quality surface motions and diverse camera trajectories, as well as a synthetic appearance dataset, VarenTex, comprising realistic multi-view images generated through multi-view diffusion. While training only on synthetic datasets, 4DEquine achieves state-of-the-art performance on real-world APT36K and AiM datasets, demonstrating the superiority of 4DEquine and our new datasets for both geometry and appearance reconstruction. Comprehensive ablation studies validate the effectiveness of both the motion and appearance reconstruction network. Project page: https://luoxue-star.github.io/4DEquine_Project_Page/.
Figures & tables
Figure 1: Two samples of the VarenPoser dataset. Each sample is a video, and we only show two frames for each sample.
Figure 2: Overview of 4DEquine. (a) AniMoFormer: A spatio-temporal transformer with post-optimization for motion recovery. (b) EquineGS: A feed-forward network to reconstruct a canonical 3D Gaussian avatar from a single image. (c) DSTG-Block: The dual-stream architecture used in EquineGS.
Figure 3: VarenTex Generation pipeline. Normal and canonical coordinate maps (CCM) rendered from VarenPoser meshes , alongside a ControlNet-generated reference image , are fed into UniTex to synthesize multi-view training images.
Figure 4: Qualitative comparison with the SOTA methods on the AiM dataset. GART ∗ : Few-shot GART. “Input” here is the middle frame of each test video clip. Note that the input image for EquineGS is the first image in the video; therefore, the results of “Ours” shown in this figure correspond to novel-pose animation.
Figure 5: Zero-shot generalization of 4DEquine on Internet images of unseen species donkey. “GT” is also the input image.
Table 1: Quantitative comparisons on the APT36K dataset, AiM dataset and VarenPoser dataset.
Method
Horse
Zebra
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
3D-Fauna
12.20
0.7205
0.2782
12.33
0.6827
0.3318
4D-Fauna
13.41
0.7550
0.2467
13.39
0.7157
0.3055
GVFDiffusion
12.68
0.8189
0.2493
12.26
0.7749
0.2897
GART ∗
15.42
0.7550
0.2452
14.31
0.6485
0.2973
GART
16.19
0.7819
0.2308
15.21
0.6752
0.2287
Table 2: Novel view/pose quantitative comparisons with SOTA methods on the horse and zebra. GART ∗ : Few-shot GART. Bold and underlined numbers indicate the best performance and the second best performance, respectively.
Table 4: Ablation study of EquineGS components and post-optimization for novel-pose appearance reconstruction on the AiM horse subset. Bold and underlined numbers indicate the best performance and the second best performance, respectively.
Figure 6: Ablation study for Sub-Division.
Figure 7: More Qualitative Results of 4DEquine. In each example, the first row displays the reference video frames at various time steps, the second row visualizes the output of AniMoFormer, and the third row presents the final reconstruction from EquineGS.
Category
Number
Category
Number
Black Shire
22
Perlino Akhal Teke
29
Black Tobiano Saddlebred
40
Palomino Quarter Horse
40
Skewbald Shetland Pony
15
White Arabian
28
Pinto Paint Horse
50
Chestnut Morgan
18
Black Friesian
23
Buckskin Tennessee Walker
18
Bay Thoroughbred
41
Dapple Gray Andalusian
48
Table 5: The number of different categories of horse images.
Figure 8: Visualization of camera trajectory settings in the VarenPoser dataset. We illustrate the three camera behaviors used during data generation: Top: Fixed camera; Middle: Orbit camera; Bottom: Dolly camera (The camera moves farther away from or closer to the object). Background images are sampled from the COCO dataset.
Figure 11: Analysis of failure cases. We illustrate a scenario where severe occlusion impacts performance. The first image: The truncated/occluded input image. The second image: The output of AniMoFormer. The third image: The output of EquineGS, showing appearance degradation at unseen areas. The fourth image: Reference frame from a different time step showing the horse’s true appearance.
Recovering articulated 4D representations of animals from monocular videos remains challenging due to the large diversity of quadruped morphologies and lack of animal 4D supervision data. Existing learning-based reconstruction methods operate on individual images and rely on synthetic or model-fitted 3D supervision, which inherits the constraints of strong parametric priors and limits generalization to out-of-distribution species. When applied to out-of-distribution animals, they often recover a plausible pose while producing inaccurate geometry because the underlying shape model cannot faithfully represent the observed instance. We present ORMA, a training-free reconstruction framework that decouples articulation from shape, using the predicted pose as reference for optimization while leveraging generative 3D priors for accurate shape reconstruction. Given a reference image, we reconstruct the animal geometry and register it to the parametric model SMAL+, yielding an articulated shape adapted to the observed instance. We then combine per-frame articulated pose estimates with globally consistent camera poses to recover animal motion in a shared world coordinate frame, and further refine the reconstruction using self-supervised DINO correspondences and temporal consistency. To enable quantitative evaluation, we introduce PAW4D, a synthetic multi-species benchmark with ground-truth 3D geometry and camera motion. Experiments on PAW4D, PFERD, and challenging in-the-wild videos demonstrate that ORMA improves reconstruction accuracy while recovering globally consistend animal motion across diverse quadruped species.
Xuyi Hu, Francesco Palandra, Shangzhe Wu +3
University of Cambridge · IMATI-CNR, Milan, Italy · Technical University of Munich, Germany +1
Reconstructing 4D animals from monocular videos is challenging due to large inter-species variation, complex articulations, and the lack of reliable templates. Existing approaches typically rely on either strict category-specific priors that restrict generalization, or unconstrained generative models that sacrifice input fidelity. To bridge this gap, we present a progressive test-time optimization framework built on 3D Gaussian Splatting for high-fidelity 4D animal reconstruction from a single video. Our key insight is that a coarse shape prior suffices when coupled with a progressive strategy that disentangles articulated pose from non-rigid deformation. Specifically, we employ a symmetry-aware temporal encoding that exploits bilateral cues while absorbing camera estimation drift and a part-conditioned deformation mechanism guided by learnable part anchors and a learnable skinning field. Extensive experiments demonstrate that our approach generalizes robustly across diverse species, achieving superior geometric accuracy, temporal consistency, and visual fidelity compared to existing baselines, even under severe prior mismatch.
Siyuan Li, Weiying Chen, Yilin Wang +3
University of Alberta, Edmonton, AB, Canada · Concordia University, Montreal, QC, Canada
Reconstructing dynamic 4D scenes remains challenging due to the presence of moving objects that corrupt camera pose estimation. Existing optimization methods alleviate this issue with additional supervision, but they are mostly computationally expensive and impractical in real-time applications. To address these limitations, we propose MoRe, a feedforward 4D reconstruction network that efficiently recovers dynamic 3D scenes from monocular videos. Built upon a strong static reconstruction backbone, MoRe employs an attention-forcing strategy to disentangle dynamic motion from static structure. To further enhance robustness, we fine-tune the model on large-scale, diverse datasets encompassing both dynamic and static scenes. Moreover, our grouped causal attention captures temporal dependencies and adapts to varying token lengths across frames, ensuring temporally coherent geometry reconstruction. Extensive experiments on multiple benchmarks demonstrate that MoRe achieves high-quality dynamic reconstructions with exceptional efficiency.
Juntong Fang, Zequn Chen, Weiqi Zhang +4
School of Software, Tsinghua University, Beijing, China · Li Auto