Gaussian Mixture-Based Inverse Perception Contract for Uncertainty-Aware Robot Navigation
Authors: Bingyao Du, Joonkyung Kim, Yiwei Lyu
Organizations: The author is with the Department of Computer Science, Columbia University, New York, NY 10027, USA. · The authors are with the Department of Computer Science and Engineering, Texas A&M University, College Station, TX 77843, USA.
Reliable navigation in cluttered environments requires perception outputs that are not only accurate but also equipped with uncertainty sets suitable for safe control. An inverse perception contract (IPC) provides such a connection by mapping perceptual estimates to sets that contain the ground truth with high confidence. Existing IPC formulations, however, instantiate uncertainty as a single ellipsoidal set and rely on deterministic trust scores to guide robot motion. Such a representation cannot capture the multi-modal and irregular structure of fine-grained perception errors, often resulting in over-conservative sets and degraded navigation performance. In this work, we introduce Gaussian Mixture-based Inverse Perception Contract (GM-IPC), which extends IPC to represent uncertainty with unions of ellipsoidal confidence sets derived from Gaussian mixture models. This design moves beyond deterministic single-set abstractions, enabling fine-grained, multi-modal, and non-convex error structures to be captured with formal guarantees. A learning framework is presented that trains GM-IPC to account for probabilistic inclusion, distribution matching, and empty-space penalties, ensuring both validity and compactness of the predicted sets. We further show that the resulting uncertainty characterizations can be leveraged in downstream planning frameworks for real-time safe navigation, enabling less conservative and more adaptive robot motion while preserving safety in a probabilistic manner.
Figures & tables
Fig. 1 : Overview of GM-IPC. (a) Perception module: the robot camera captures the point cloud of environment, and the perception module predicts obstacle bounding boxes. Points inside the boxes are mapped to a 2D binary occupancy map. (b) IPC network: The map is processed by a convolutional encoder, with the robot state processed by an MLP. Their representations are then fused through one fusion layer, followed by 3 linear heads to produce the Gaussian parameters (c) Uncertainty sets: given a confidence value p , the boundaries of all Gaussians are extracted to cover the obstacle region.
Fig. 2 : Overview of navigation pipeline. At each time step, raw sensor readings Pt are processed by the perception module to produce perceptual estimates y^t . The IPC network then generates a Gaussian Mixture uncertainty set, which is passed to the MPC-CBF planner to compute a sequence of safe future states (ph,vh)h=1H . The first control output is applied to the robot, and the process repeats at the next time step. White blocks denote intermediate results, while gray blocks represent modules.
Scenario
Model
Mean Inclusion Rate
% Steps ≥95%
Compactness
Final
All
Final
All
Final ( μ±σ )
Chair
GMM
1.000
0.999
100.0
99.8
9.52±0.46
Ellip
0.999
0.989
100.0
96.6
11.74 ± 0.35
Sofa
GMM
0.997
0.993
100.0
97.8
10.50 ± 1.00
Ellip
0.994
0.988
96.7
93.7
9.48±0.39
Multi Sofa
GMM
0.999
0.998
100.0
99.0
5.58 ± 0.30
TABLE I : Validity of IPC models.
Loss Function
Mean Inclusion Rate
% Steps ≥95%
Compactness
Final
All
Final
All
Final ( μ±σ )
Full
0.997
0.993
100.0
97.8
10.50 ± 1.00
No Empty Penalty
1.0
1.0
100.0
100.0
1.00 ± 0.00
No NLL
0.997
0.993
100.0
97.0
9.43 ± 0.64
TABLE II : Ablation study on loss functions.
Fig. 3 : Ablation study on the loss function in the single-sofa case. (a) full loss function, (b) without the empty penalty term, and (c) without the NLL term.
Scenario
Ellip-IPC
GM-IPC
Mixed
0.817
0.982
Two sofas
0.773
0.935
Single sofa
0.953
0.990
Single chair
0.894
0.929
TABLE III : Average solved confidence level under equal union area of ellipsoids (30 trials per scenario; aggregated across steps).
Fig. 4 : Coverage under equal union area of ellipsoids: (a) GMM single-sofa, (b) Ellipsoid single-sofa, (c) GMM multi-sofa, (d) Ellipsoid multi-sofa.
Scenario
Model
Steps
Len
Eff.
Ctrl
Succ.
Mixed
Ellip-1
434.2
12.69
0.90
0.18
0.76
Ellip-2
408.3
11.95
0.95
0.12
0.70
GMM
403.1
11.62
0.97
0.12
0.88
Multi Sofa
Ellip-1
440.0
12.79
0.89
0.22
0.74
Ellip-2
412.4
12.08
0.94
0.13
0.86
GMM
397.2
11.42
0.99
0.13
0.90
TABLE IV : Navigation evaluation results across scenarios (50 trials each).
Fig. 5 : Navigation trajectories in multi-object scenarios using IPC. Yellow regions indicate the ground-truth obstacles, and blue points are the observations from the perception module. (a, e) Two-sofa case with GM-IPC, (b) Two-sofa case with Ellip-IPC, (c, f) Mixed case with GM-IPC, (d) Mixed case with Ellip-IPC.
Reliable robot perception requires not only predicting scene structure, but also identifying where predictions should be treated as unreliable due to sparse or missing observations. We present ContraMap, a contrastive continuous mapping method that augments kernel-based discriminative maps with an explicit uncertainty class trained using synthetic noise samples. This formulation treats unobserved regions as a contrastive class, enabling joint environment prediction and spatial uncertainty estimation in real time without Bayesian inference. Under a simple mixture-model view, we show that the probability assigned to the uncertainty class is a monotonic function of a distance-aware uncertainty surrogate. Experiments in 2D occupancy mapping, 3D semantic mapping, and tabletop scene reconstruction show that ContraMap preserves mapping quality, produces spatially coherent uncertainty estimates, and is substantially more efficient than Bayesian kernelmap baselines.
Chi Cuong Le, Weiming Zhi
School of Computer Science and the Australian Centre for Robotics, University of Sydney, Australia
Vision-Language Navigation (VLN) requires an agent to navigate 3D environments following natural language instructions. During navigation, existing agents commonly encounter perceptual uncertainty, such as insufficient evidence for reliable grounding or ambiguity in interpreting spatial cues, yet they typically ignore such information when predicting actions. In this work, we explicitly model three forms of perceptual uncertainty (i.e., geometric, semantic, and appearance uncertainty) and integrate them into the agent's observation space to enable informed decision-making. Concretely, our agent first constructs a Semantic Gaussian Map (SGM), composed of differentiable 3D Gaussian primitives initialized from panoramic observations, that encodes both the geometric structure and semantic content of the environment. On top of SGM, geometric uncertainty is estimated through variational perturbations of Gaussian position and scale to assess structural reliability; semantic uncertainty is captured by perturbing Gaussian semantic attributes to reveal ambiguous interpretations; and appearance uncertainty is characterized by Fisher Information, which measures the sensitivity of rendered observations to Gaussian-level variations. These uncertainties are incorporated into SGM, extending it into a unified 3D Value Map, which grounds them as affordances and constraints that support reliable navigation. Comprehensive evaluations across multiple VLN benchmarks show the effectiveness of our agent.
Jianzhe Gao, Rui Liu, Yuxuan Xu +6
The State Key Lab of Brain-Machine Intelligence, Zhejiang University · Department of Foundation model, 2012 Labs, Huawei · Noah’s Ark Lab, 2012 Labs, Huawei +1
Most uncertainty-aware robotic systems collapse prediction uncertainty into a single scalar score and use it to trigger uniform corrective responses. This aggregation obscures whether uncertainty arises from corrupted observations or from mismatch between the learned model and the true system dynamics. As a result, corrective actions may be applied to the wrong component of the closed loop, degrading performance relative to leaving the policy unchanged. We introduce a lightweight post hoc framework that decomposes uncertainty into aleatoric and epistemic components and uses these signals to regulate system responses at inference time. Aleatoric uncertainty is estimated from deviations in the observation distribution using a Mahalanobis density model, while epistemic uncertainty is detected using a noise robust forward dynamics ensemble that isolates model mismatch from measurement corruption. The two signals remain empirically near orthogonal during closed loop execution and enable type specific responses. High aleatoric uncertainty triggers observation recovery, while high epistemic uncertainty moderates control actions. The same signals also regulate adaptive perception by guiding model capacity selection during tracking inference. Experiments demonstrate consistent improvements across both control and perception tasks. In robotic manipulation, the decomposed controller improves task success from 59.4% to 80.4% under compound perturbations and outperforms a combined uncertainty baseline by up to 21.0%. In adaptive tracking inference on MOT17, uncertainty-guided model selection reduces average compute by 58.2% relative to a fixed high capacity detector while preserving detection quality within 0.4%. Code and demo videos are available at https://divake.github.io/uncertainty-decomposition/.