Contrastive Multi-Modal Hypergraph Reasoning for 3D Crowd Mesh Recovery

Minghao Sun; Chongyang Xu; Yitao Xie; Buzhen Huang; Kun Li

cs.CVApr 1, 2026

Contrastive Multi-Modal Hypergraph Reasoning for 3D Crowd Mesh Recovery

Authors: Minghao Sun, Chongyang Xu, Yitao Xie, Buzhen Huang, Kun Li

Organizations: Tianjin University · Nanyang Technological University · Sichuan University

Abstract

Multi-person 3D reconstruction is pivotal for real-world interaction analysis, yet remains challenging due to severe occlusions and depth ambiguity. Current approaches typically rely on single-modality inputs, which inherently lack geometric guidance. Furthermore, these methods often reconstruct subjects in isolation, neglecting the collective group context essential for resolving ambiguities in crowded scenes. To address these limitations, we propose Contrastive Multi-modal Hypergraph Reasoning to synergize semantic, geometric, and pose cues for crowd reconstruction. We first initialize robust node representations by combining RGB features, geometric priors, and occlusion-aware incomplete poses. Additionally, we introduce a pelvis depth indicator as a global spatial anchor, aligning visual features with a metric-scale-agnostic depth ordering. Subsequently, we construct a shared-topology hypergraph that moves beyond pairwise constraints to model higher-order crowd dynamics. To improve feature fusion, we design a hypergraph-based contrastive learning scheme that jointly enhances intra-modal discriminability and enforces cross-modal orthogonality. This mechanism enables the network to propagate global context effectively, allowing it to infer missing information even under severe occlusion. Extensive experiments on the Panoptic and GigaCrowd benchmarks confirm that our method achieves new state-of-the-art performance. Code and pre-trained models are available at https://github.com/SunMH-try/CoMHR.

Figures & tables

Fig. 1: Our method reconstructs spatially consistent human meshes in extremely dense crowd scenes, capturing accurate relative depth ordering.

Fig. 2: Overview of CoMHR. We begin with (a) Multi-Modal Initialization , which lifts 2D keypoints via pseudo-depth maps and explicitly anchors RGB features with Pelvis Depth to initialize multi-modal features. These cues are fused to construct (b) a Contrastive Hypergraph , where a shared topology is regularized by contrastive learning. This structure enables (c) High-Order Reasoning to propagate collective context, driving precise SMPL regression.

Fig. 3: Qualitative comparison on GigaCrowd [ 21 ] . GroupRec [ 3 ] often shows floating feet, drifting bodies, and incorrect depth ordering, while our method maintains accurate depth, stable spatial layout, and consistent reconstruction in dense crowds.

Fig. 4: Qualitative comparison of multi-person reconstruction among Multi-HMR [ 9 ] , GroupRec [ 3 ] , and our method. Red boxes show baseline failures such as pose misalignment, missing people, and depth errors, while our method produces accurate and consistent reconstructions.

Method	Haggling	Mafia	Ultim.	Pizza	Mean
Zanfir et al. [ 27 ]	140.0	165.9	150.7	156.0	153.4
MubyNet [ 28 ]	141.4	152.3	145.0	162.5	150.3
CRMH [ 29 ]	129.6	133.5	153.0	156.7	143.2
BMP [ 30 ]	120.4	132.7	140.9	147.5	135.4
Pose2UV [ 31 ]	104.2	136.0	123.2	151.0	128.6
ROMP [ 11 ]	110.8	122.8	141.6	137.6	128.2

TABLE I: Comparison with multi-person mesh recovery methods on Panoptic dataset. MPJPE (mm); ’–’ indicates unavailable results. Best in bold , second best underlined .

Method	OKS	PA-PPDS	PCOD	RP
Method	(%) $\uparrow$	(%) $\uparrow$	(%) $\uparrow$	$\downarrow$
CRMH [ 29 ]	56.31	52.16	60.48	0.17
BEV [ 7 ]	62.47	55.41	62.38	0.22
CrowdRec [ 36 ]	65.65	73.83	81.69	0.00
GroupRec [ 3 ]	70.80	67.22	71.42	0.17
Ours	78.11	75.74	94.35	0.00

TABLE II: Comparison with state-of-the-art methods on GigaCrowd.

Configuration	Contrastive Learning	MPJPE (mm) $\downarrow$
RGB only	$\times$	110.41
Depth only	$\times$	137.40
Pose only	$\times$	184.36
RGB + Depth	$\times$	106.90
RGB + Pose	$\times$	106.89
RGB + Depth + Pose	$\times$	106.34

TABLE III: Ablation study on the Panoptic dataset. “Tz” denotes translation refinement from depth normalization.

Fig. 5: Ablation results on the Panoptic dataset. Top to bottom: (1) RGB-only baseline with depth errors, (2) RGB+Depth+Pose+ $T_{z}$ with improved 3D structure, (3) full model with contrastive learning achieving accurate depth and coherent meshes.

Configuration	OKS $\uparrow$	PA-PPDS $\uparrow$	PCOD $\uparrow$
RGB + Pose Baseline	72.93	75.27	94.03
CoMHR w/o Contrastive	69.38	76.11	92.98
Full CoMHR	78.11	75.74	94.35

TABLE IV: Ablation on Crowd-Level Metrics on GigaCrowd.

Fig. 6: Additional qualitative results showing high-order interactions and relative depth ordering.

Fig. 7: Multi-view results on GigaCrowd. Two scenes are displayed, each showing front, side, and top views to illustrate the model’s ability to handle dense crowds and maintain depth consistency.

\mathcal{L}_{\text{intra}}^{m}=-\frac{1}{N}\sum_{i}\frac{1}{|P_{i}|}\sum_{j\in P_{i}}\log\frac{\exp(\hat{r}_{i}\cdot\hat{r}_{j}/\tau)}{\sum_{k\in P_{i}}\exp(\hat{r}_{i}\cdot\hat{r}_{k}/\tau)}

Algorithm 1 Multimodal Contrastive Learning in CoMHR

3D Reconstruction

3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.

See all 930 papers

CrowdGaussian: Reconstructing High-Fidelity 3D Gaussians for Human Crowd from a Single Image
Mar 18, 2026Yizheng Song, Yiyu Zhuang, Qipeng Xu +53D Human Reconstruction Single-View 3D Reconstruction
Nanjing University, China · State Key Laboratory of Novel Software Technology, China · Fudan University, China
Feed-Forward Multi-view Multi-person Reconstruction with Contrastive Human-Aware 3D Representation
Sep 1, 2026Yuanwang Yang, Buzhen Huang, Zongxuan Ren +2Multi-View 3D Reconstruction 3D Human Reconstruction
College of Intelligence and Computing, Tianjin University, 135 Yaguan Road, Tianjin, 300350, Tianjin, China.
Human Interaction-Aware 3D Reconstruction from a Single Image
Apr 7, 2026Gwanghyun Kim, Junghun James Kim, Suh Yoon Jeon +23D Human Reconstruction Single-View 3D Reconstruction
Dept. of Electrical and Computer Engineering, Seoul National University, Republic of Korea · IPAI, Seoul National University, Republic of Korea · INMC, Seoul National University, Republic of Korea

Mar 18, 2026J/K move · Enter open · S save

CrowdGaussian: Reconstructing High-Fidelity 3D Gaussians for Human Crowd from a Single Image

Yizheng Song, Yiyu Zhuang, Qipeng Xu, Haixiang Wang+4

Nanjing University, China · State Key Laboratory of Novel Software Technology, China · Fudan University, China

3D Human Reconstruction Single-View 3D Reconstruction 3D Gaussian Splatting

Open paper AskPDF arXiv

Single-view 3D human reconstruction has garnered significant attention in recent years. Despite numerous advancements, prior research has concentrated on reconstructing 3D models from clear, close-up images of individual subjects, often yielding subpar results in the more prevalent multi-person scenarios. Reconstructing 3D human crowd models is a highly intricate task, laden with challenges such as: 1) extensive occlusions, 2) low clarity, and 3) numerous and various appearances. To address this task, we propose CrowdGaussian, a unified framework that directly reconstructs multi-person 3D Gaussian Splatting (3DGS) representations from single-image inputs. To handle occlusions, we devise a self-supervised adaptation pipeline that enables the pretrained large human model to reconstruct complete 3D humans with plausible geometry and appearance from heavily occluded inputs. Furthermore, we introduce Self-Calibrated Learning (SCL). This training strategy enables single-step diffusion models to adaptively refine coarse renderings to optimal quality by blending identity-preserving samples with clean/corrupted image pairs. The outputs can be distilled back to enhance the quality of multi-person 3DGS representations. Extensive experiments demonstrate that CrowdGaussian generates photorealistic, geometrically coherent reconstructions of multi-person scenes.

Abstract

3D Reconstruction

Explore similar work

CrowdGaussian: Reconstructing High-Fidelity 3D Gaussians for Human Crowd from a Single Image

Feed-Forward Multi-view Multi-person Reconstruction with Contrastive Human-Aware 3D Representation

Human Interaction-Aware 3D Reconstruction from a Single Image