Dense4D: Pixel-Level 4D Human Correspondence

Rıza Alp Güler

cs.CVOct 15, 2026

Dense4D: Pixel-Level 4D Human Correspondence

Authors: Rıza Alp Güler

Organizations: Department of Computing, Imperial College London

Abstract

Reconstructing human motion in 4D requires identifying the same surface points through articulation, deformation and changes in visibility. We introduce Dense4D, to our knowledge the first feed-forward model dedicated to dense 4D correspondence across the body, clothing and hair from a set of monocular RGB images, without a body template. Trained specifically for human geometry, Dense4D jointly processes person-centred crops with known image-to-crop transforms to predict Human Point Maps (HPMs) and depth for each image. An HPM assigns each person pixel the 3D position of its observed surface point at a common reference time. These maps support amodal correspondence: points hidden in the reference image can be localised from observations in other frames. Matching HPM coordinates across crops establishes correspondence without predicting camera poses or large scene displacements; per-image depth then lifts the matches into 3D tracks. Experiments on real people demonstrate substantially higher tracking accuracy and closer agreement with scanned surfaces than the evaluated baselines. Points from different frames assemble into coherent human geometry in the reference pose, providing visible evidence of consistent dense correspondence through articulation and clothing deformation. Additional qualitative results, videos, and an interactive viewer are available at https://alpguler.com/dense4d.

Figures & tables

Panel of Figure 2 — Figure 2: Dense 4D human correspondence. An extra crop token conditions the aggregator on cropping geometry ( Eq. 3 ). The heads predict reference-time HPMs and per-image depth, visualised as HPM point clouds and unprojected depth maps. Nearest-neighbour lookup matches HPM coordinates within the target frame ( Eq. 6 ). A 3D chequerboard defined in reference space colours the union and is transferred to the images through HPM coordinates, visualising dense correspondence. Unprojecting the matched pixels with predicted depth gives 4D tracks ( Eqs. 7 and 3.4 ), shown as coloured trajectories.

Method	AJ $\uparrow$	APD $\uparrow$	OA $\uparrow$
Ours	38.4	50.3	89.6
TAPIP3D	17.0	27.1	85.2
V-DPM †	16.9	27.6	76.7
SpatialTrackerV2	15.5	24.4	78.7
4RC †	14.4	24.1	76.6
Shape of Motion	12.5	21.3	80.7

Table 4.2 and Figure 4.2 : Panoptic Studio Humans. Left: tracking on human-body queries (†: adapted to benchmark intrinsics; Appendix C.2 ). Centre: our tracks from 1.7 to 5.0 s. Far right: HPM unions combining person pixels from across the input sequence in each person’s reference pose. Despite the motion across frames, these points assemble into coherent bodies with aligned limbs and surface detail, making the quality of temporal alignment directly visible.

Figure 3: Reference-time reconstruction from sixteen observations on 4D-DRESS. Left: input frames, with the reference boxed. For each method, four viewpoints show the union coloured by source appearance (top) and distance to the scan from 0 to 5 cm (bottom). The measured scan is overlaid in blue-grey in the top row.

Table 4.3 and Figure 4.3 : Amodal 3D localisation on 4D-DRESS. Left: mean distance to scan and percentages of points more than 2 and 5 cm from the surface. Centre: error by time; right: error after dropping each method’s least confident points. Bars show standard errors; the table and temporal plot use a union-wide 5 % cut.

Figure 4: Dense correspondences and 4D tracks. Each row follows a person through three instants, with matching times shown in A and B. (A) The same reference-space 3D chequerboard colours every image, making it possible to follow surface regions through body rotation and garment deformation. (B) The reconstructed person and coloured 3D trajectories, integrated with VGGT- $\Omega$ ’s scene points. Together, the panels connect correspondence on the image with the motion of human surface points in 3D.

Figure 5: Human surface consistency on DAVIS. (A) Reference image and unions of person points from multiple frames, all expressed at frame zero. Notice the distinct body outline preserved by Dense4D, compared with the overlapping limbs and dispersed points in the baseline unions. (B) Frame-zero points transferred to the target image, with their original colours retained against a dimmed background. These colours allow direct inspection of alignment between corresponding body regions. Dense4D anchors transfers to pixels within the target person mask and uses their predicted depth, ensuring consistency with its reconstructed geometry by construction.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Training surfaces and samples. (A) BEDLAM2 images and the body, garment and hair surfaces used for supervision. (B) Training crops from BEDLAM2, THuman2.1 and MammaSyn, each from a different clip or scan.

Figure 7: Hair envelope meshes. Envelopes derived from BEDLAM2’s strand grooms, shown from a common viewpoint with consistent lighting. Labels identify the source grooms.

Figure 8: Surface supervision for 4D correspondence. A human sequence with body, garment and hair labels, reference-coordinate colours and texture transfer. The plots summarise track length and labelled-pixel composition; the histogram’s training pool includes the development tracks.

Frames	$1280\times 720$	$512\times 288$	crops, read-out	forward	Total	Peak memory
	SAM 3	VGGT- $\Omega$	Dense4D, $512\times 512$ crops
100	10.6 s	1.8 s	0.8 s	4.7 s	17.9 s	19.5 GB
150	18.2 s	3.3 s	1.4 s	10.0 s	32.9 s	21.9 GB
300	33.0 s	11.0 s	2.6 s	36.6 s	83.2 s	29.1 GB

Appendix

Table 1: Inference time and memory of the full pipeline on one GH200, for a

1280\times 720

internet video, with each stage at its own input size. SAM 3 tracks the person on the full frames; VGGT-

\Omega

gives the cameras and scene depth from the full frames at

512\times 288

; Dense4D runs on

512\times 512

person crops, which carry 1.8

\times

the tokens of VGGT-

\Omega

’s frames, so its forward pass is the longer one although the two networks use the same backbone architecture. “Crops, read-out” is CPU time for building the crops and writing the outputs. All models stay loaded in one process (loading, 27 s, is not included); peak memory is the GPU allocator maximum, including 11.4 GB of model weights.

Variant	Panoptic Studio Humans			BEDLAM2
	AJ $\uparrow$	APD $\uparrow$	OA $\uparrow$	(cm) $\downarrow$
Without crop conditioning	30.4	41.4	89.3	4.06
Centre ray fixed to $(0,0,1)$	38.4	50.3	89.6	2.99
Final model	38.4	50.3	89.6	2.99

Appendix

Table 2: Crop-geometry conditioning. The fixed-ray row modifies only the final model’s inference input. Evaluation follows Sec. 4.2 for Panoptic Studio Humans and Sec. C.4 for BEDLAM2.

Figure 9: Temporal extent and input density on BEDLAM2. Pointwise reconstruction error in centimetres, with means and standard errors over tracks. Left: windows sampled at 6 fps on tracks lasting at least ten seconds (155 tracks) or sixteen seconds (25 tracks). Middle: 4 to 96 frames over the same sixteen-second window. Right: error by temporal offset within that window; the star marks the reference.

3D Point Tracking

3D point tracking estimates how scene points move through space and time, even under camera motion, occlusion, and deformation.

See all 30 papers

Point4D: Long-range 4D Motion Reconstruction
Sep 8, 2026Minsik Jeon, Jay Karhade, Deva Ramanan +1Point Tracking 3D Point Tracking
Carnegie Mellon University
Face Anything: 4D Face Reconstruction from Any Image Sequence
Apr 21, 2026Umut Kocasari, Simon Giebenhain, Richard Shaw +14D Reconstruction Deformable Shape Reconstruction
Technical University of Munich, Germany · Huawei Noah’s Ark Lab, London, UK
UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images
Feb 27, 2026Junhwa Hur, Charles Herrmann, Songyou Peng +44D Reconstruction Dynamic Scene Reconstruction
Google · Princeton University · Harvard University

Sep 8, 2026J/K move · Enter open · S save

Point4D: Long-range 4D Motion Reconstruction

Minsik Jeon, Jay Karhade, Deva Ramanan, Shubham Tulsiani

Carnegie Mellon University

Point Tracking 3D Point Tracking 4D Reconstruction Dynamic Scene Reconstruction Multi-Object Tracking

Open paper AskPDF arXiv

We introduce Point4D, a feed-forward model for 4D reconstruction of long-range video sequences. Point4D is able to reliably infer dense per-point 3D trajectories across multi-hundred-frame videos, unlike existing 4D methods that are limited to short input windows of at most a few dozen frames. A key innovation that enables this is our flexible 3D query-based motion decoder that decouples trajectory prediction from image-plane visibility. The predicted 3D endpoints are then directly re-queried in the next chunk without re-projection or matching. Furthermore, we show that extracting and reusing a visual descriptor from an arbitrary frame where the point is visible leads to better performance than relying solely on the source patch. Overall, Point4D achieves state-of-the-art performance across diverse long-video tracking benchmarks spanning over 200 frames and largely outperforms previous feed-forward 4D method. Project page: https://point-4d.github.io

Abstract

3D Point Tracking

Explore similar work

Point4D: Long-range 4D Motion Reconstruction

Face Anything: 4D Face Reconstruction from Any Image Sequence

UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images