Video-to-4D mesh generation has made substantial progress in recovering dynamic 3D geometry and motion from monocular videos. However, commonly used feed-forward approaches remain prone to sticking artifacts, where geodesically distant vertices move together incorrectly when they become spatially close. We attribute this failure to cross-part motion leakage: spatially nearby yet structurally distant vertices may retrieve similar motion information when motion decoding fails to distinguish spatial proximity from mesh connectivity. To address this issue, we introduce TRoPE, a topology-aware relative positional encoding that makes mesh connectivity explicit in the interaction between vertex queries and mesh-supported motion sources. TRoPE derives spectral features from the graph Laplacian and constructs pairwise topological relations from distances between multi-scale diffusion features, using these relations to modulate vertex--anchor attention through rotary phases. We further introduce a vertex-wise tail-risk objective that emphasizes persistently high-error vertices to explicitly suppress sticking artifacts. To better characterize and quantify these failures, we introduce MeshSticking-4D, a dedicated benchmark for this failure mode. Experiments on MeshSticking-4D show that TRoPE suppresses sticking artifacts while improving overall motion accuracy. Compared with the baseline, it reduces the worst-vertex error by 39.4% and lowers the separation failure rate from 49.35% to 34.90%. Additional evaluations on Motion-80 and Motion324-Subset show lower tail errors on diverse objects, while transfer to Mesh4D demonstrates applicability across backbones.
Figures & tables
Figure 1: TRoPE reduces sticking artifacts in 4D generation. Given a reference mesh and a monocular video (left panel), the upper and lower platforms present the Motion 3-to-4 baseline and TRoPE, respectively. The bottom insets show zoomed-in comparisons of corresponding regions. Compared with the baseline, our method reduces sticking artifacts between nearby body parts, producing more faithful motion.
Figure 2: Overview of TRoPE for Topology-Aware Video-to-4D Mesh Generation. Given a driving video and a reference mesh, a video-to-4D backbone extracts motion tokens, which TRoPE associates with mesh-supported anchors before a decoder generates the 4D mesh sequence. The lower panel details TRoPE: PLTA encodes local vertex and anchor features, while the reference connectivity graph is transformed through the graph Laplacian and multi-scale diffusion into pairwise topology-relative relations. These relations modulate query–key compatibility through rotary attention, producing motion features that preserve mesh structure and suppress cross-part motion leakage and sticking artifacts.
Figure 3: Qualitative Comparisons on MeshSticking-4D and Motion324-Subset. The first three examples are from MeshSticking-4D; the last two are from Motion324-Subset. Each column pair shows reference (Ref) and novel views. The top row presents the driving video and novel-view ground truth; the remaining rows compare five baselines and TRoPE. Compared with the baselines, TRoPE more faithfully recovers articulated poses while preserving separation between nearby parts, reducing sticking artifacts.
Method
MeshSticking-4D
Motion324-Subset
Global Motion Fidelity
Sticking Robustness
Global Motion Fidelity
Sticking Robustness
EucD ↓
WVE ↓
FMVE ↓
EETE 25↓
SFR(%) ↓
MSD ↓
EucD ↓
WVE ↓
FMVE ↓
EETE 25↓
SFR(%) ↓
MSD ↓
Motion 3-to-4
59.657
285.420
309.116
137.872
49.349
35.849
31.129
136.390
149.661
82.078
70.341
47.572
Puppeteer
229.205
688.001
736.039
307.215
78.49
35.330
108.105
261.044
283.227
159.752
100.000
66.431
ActionMesh
122.559
305.978
338.738
188.150
43.73
27.492
85.863
211.553
236.764
127.209
56.262
45.113
R-DMesh
147.973
529.012
578.764
223.051
76.80
41.657
95.854
239.064
267.710
148.485
80.079
54.816
Table 1: Quantitative Comparison on MeshSticking-4D and Motion324-Subset. We report EucD, WVE, and FMVE for motion fidelity, and EETE 25 , SFR, and MSD for sticking robustness, with lower values indicating better performance across all metrics. Results are evaluated on 256 and 128 test samples, respectively. Bold indicates the best result, and underline indicates the second-best result.
Table 2: Quantitative Comparison on Motion-80. All methods use the same 80 sequences and frames 0–15. Distance errors are scaled by 103 ; lower is better except for F-scores. Bold and underline indicate the best and second-best results.
Figure 6
Configuration
Global Motion Fidelity
Sticking Robustness
EucD ↓
WVE ↓
FMVE ↓
EETE 25 ↓
SFR (%) ↓
MSD ↓
Without CVaR
Content-only Anchor Attention
55.609
208.456
230.146
118.312
44.517
32.999
Direct + PLTA (no anchors)
58.540
277.633
300.683
122.087
43.680
32.238
Component-only Relation
56.042
195.604
218.892
110.509
38.104
30.499
Single-scale Diffusion, τ=1
55.594
184.127
208.673
102.306
35.223
29.637
Table 4: Ablation Analysis on MeshSticking-4D. CVaR is off in the upper block and on in the lower block. Bold and underline indicate the best and second-best results within each block.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Attention weights (once)
Value aggregation (per frame)
12-frame total
Content-only
13.58
1.35
29.78
Abs. Cartesian RoPE
14.04
1.29
29.49
LapPE
13.68
1.32
29.57
Abs. HKS RoPE
14.27
1.28
29.65
Euclidean-relative RoPE
41.05
1.05
53.66
Geodesic-relative RoPE
37.73
1.05
50.34
Appendix
Table 5: Attention-routing latency. One-time attention-weight computation, cached value aggregation per frame, and total latency for 12 frames with uniform caching. Times are in ms.
Figure 5: Additional qualitative comparisons. The first three examples are from MeshSticking-4D and the last two from Motion324-Subset. Each example pairs reference (Ref) and novel views. The top row shows the driving video and novel-view ground truth; the remaining rows compare five baselines and TRoPE.
Method
Motion fidelity
Sticking robustness
EucD ↓
WVE ↓
FMVE ↓
EETE 25↓
SFR (%) ↓
MSD ↓
Mesh4D
130.209
318.117
353.858
175.858
99.759
41.610
Mesh4D + TRoPE
130.204
305.285
342.128
172.244
76.986
36.461
Appendix
Table 6: Transfer to Mesh4D on MeshSticking-4D. Both variants use the same training and native first-16-frame evaluation protocol, with 256 test clips. Lower is better for all metrics; bold marks the better result within each pair.
Figure 6: TRoPE transfer to Mesh4D. Two MeshSticking-4D examples compare GT, the fine-tuned Mesh4D backbone, and the same backbone with TRoPE. Red and green insets show the same final-frame region at 2× magnification.
Figure 7: Qualitative comparison of positional encodings. Four MeshSticking-4D examples compare the variants in Table 4 . Each example pairs reference and novel views of the same frame. The top row shows the driving video and novel-view ground truth; the remaining rows compare other positional encodings and TRoPE.
Dataset
Duplicate seam vertices (%)
Events with endpoint lacking support
MeshSticking-4D
0.058
0 / 2,152 (0.0%)
Motion-80
52.09
245 / 1,105 (22.2%)
Motion324-Subset
30.18
1,555 / 5,597 (27.8%)
Appendix
Table 7: Reference-mesh relation coverage. We report the fraction of vertices in duplicate seam groups, pooled across test clips, and events with at least one endpoint lacking spectral support.
Figure 8: Limitations. (a) In challenging cases, sticking artifacts are mitigated to some extent but still persist. (b) In some cases, although sticking artifacts are suppressed, the limbs become deformed.
4D mesh generation has recently emerged as a powerful paradigm for recovering dynamic 3D structure from videos, but existing methods remain slow, computationally expensive, and difficult to scale to longer sequences. We introduce a training-free approach that accelerates 4D mesh generation while improving temporal correspondence quality. Our key observation is that temporal correspondences emerge inside a 4D backbone long before its generated meshes become visually accurate. We exploit this with a general framework we call Spatio-Temporal Attention Chain which propagates information across space and time. Starting from vertices on an anchor mesh, the chain maps vertices to latent tokens. It then follows temporal correspondences in latent space, and recovers frame-specific vertices through latent-to-vertex attention. This design avoids expensive explicit matching while preserving anchor mesh details and thereby improving dynamic mesh geometry and temporal consistency. Compared to state-of-the-art, our method generates a 4D mesh in 9 seconds, achieving a 13× speedup while producing higher-quality results. Moreover, our approach scales to videos up to 16× longer without degrading mesh quality. Beyond generation, the improved correspondences enable competitive zero-shot performance on two downstream tasks: 2D object tracking and 4D tracking. We further show that our framework enables reliable camera estimation, a capability not supported by prior 4D mesh generation methods.
Dvir Samuel, Yuval Atzmon, Gal Chechik +1
NVIDIA Research, Tel-Aviv, Israel · Bar-Ilan University, Ramat-Gan, Israel
Current video-to-4D methods struggle with complex topology changes, transparent materials, thin structures, and inner surfaces. We present Helix4D, a dynamic mesh generation framework by inheriting the expressive representation of Trellis2, adapting it from image-to-3D to video-conditioned 4D generation. Our design arises from two key questions: (a) how to enable Trellis2's frame-local attention to share information across frames while preserving its pretrained quality on rare cases such as transparent objects and inner surfaces, and (b) how to inject temporal information into a purely 3D positional encoding without breaking pretrained capabilities. We address (a) with a sliding-window cross-frame attention and anchor on the first frame. The first frame is generated by the base Trellis2 model and injected into our model, letting it inherit Trellis2's quality in rare cases through cross-frame attention. We address (b) with a 4D temporal encoding that repurposes redundant low-frequency spatial RoPE bands for time, extending the encoding from 3D with no additional parameters. Extensive experiments show the effectiveness of Helix4D for high-quality dynamic mesh generation on ActionBench and our own challenging complex dynamics set.
4D dynamics (3D geometry evolving over time) is a fundamental representation of the physical world and plays a crucial role in world modeling (e.g., animation and games). Owing to the scarcity of large-scale, long-horizon 4D mesh data with arbitrary shapes, early text-to-4D methods rely on distillation or test-time optimization from video diffusion priors, making inference prohibitively slow. Recent feed-forward generators greatly reduce inference cost but offer limited spatiotemporal controllability, and short-horizon generation often leads to error accumulation in long-horizon sequences. We propose a novel feed-forward in-betweening framework for arbitrary 4D meshes with keyframe conditioning. Building on universal mesh-animation latents, we introduce a frame-wise mesh VAE that encodes each frame into topology-agnostic latent tokens anchored by a reference mesh for keyframe conditioning. We further introduce a keyframe-conditioned rectified flow model with an MMDiT backbone that synthesizes non-keyframe frames conditioned on sparse keyframes. Experiments show strong performance and improved controllability on both DyMesh16 and DyMesh32 benchmarks.
Hiroki Nishizawa, Hubert P. H. Shum, Yoshihiro Fukuhara +2