cs.ROOct 12, 2026
SaveReForesight: Occlusion-Robust Re-Identification and Future Occupancy Forecasting for Dynamic Semantic Mapping
Organizations: Skolkovo Institute of Science and Technology, Moscow, Russia · 168robotics, Moscow, Russia
Abstract
Robots sharing walkways need to know where people will be, yet semantic occupancy maps show only where they are. We present ReForesight, an open-source ROS 2 toolkit that adds instance-level future occupancy and re-identification after occlusion to a particle-based semantic mapping pipeline. A 36.6k-parameter MLP predicts each tracked person or vehicle up to 3 s ahead, guarded by a Kalman fallback and a stationary gate, and the object's own points are moved to the forecast pose, forming a "ghost" point cloud in the map frame without an occupancy decoder. An optional tracker re-associates objects after occlusion using keypoint descriptors the pipeline already computes and a metric 3D gate. On four held-out CODa sequences, the MLP lowers the mean 3 s final displacement error from 0.728 m (Kalman) to 0.662 m and beats a similar-size GRU (0.692 m) on every sequence, at 0.04 ms per instance on one CPU core. Code, all models, and a ROS-free demo are released; field demonstrations use an outdoor cleaning robot.
Figures & tables
Figure 1. ReForesight on the field robot. (a) Outdoor cleaning robot used for the recordings (logo removed for review). (b) Front-camera view in a public park. (c) Semantic occupancy map projected into the camera (teal: pedestrian instance). (d) The robot’s 3D semantic occupancy map with the pedestrian’s forecast occupancy 1 s ahead (indigo “ghost”). Four panels. Panel a shows a gray wheeled cleaning robot with side brushes on a paved park path. Panel b shows a fisheye camera view of a park walkway with a pedestrian and a small dog. Panel c shows the same view rendered as colored voxels, with the pedestrian in teal, the path in purple and trees in green. Panel d shows a 3D voxel map in purple and green with a pedestrian-shaped cluster of indigo points ahead of the robot.
Figure 2. ReForesight architecture. The unmodified mapping pipeline (left) publishes instance masks with track IDs, depth, and camera pose. The ReForesight node (right) lifts each instance into the map frame, keeps a 3 s pose history, forecasts it with the learned forecaster (MLP by default, or the Kalman fallback), applies the confirmation and stationary gates, and moves the instance’s own points to the forecast pose. Dashed: optional re-identification module. Block diagram in two panels. Left gray panel, existing mapping pipeline: input thumbnails of the robot and its front camera feed a front end (fisheye rectification, LiDAR to depth, YOLOv8n-seg with tracker IDs, SegFormer-B0, SuperPoint and LightGlue) and the Semantic DSP Map; a dashed optional ReIDTracker block sits below. A blue arrow labeled with the ROS 2 topics (masks with track IDs, depth, camera pose) leads to the right violet panel, the ReForesight node, with four blocks in a row: lift to 3D, track history, forecast (MLP, Kalman fallback, stationary gate), and ghost occupancy with a bird's-eye-view thumbnail. Below them an outputs block lists the ghost point cloud topic, the trajectory topic, and the bird's-eye-view window, next to a 3D thumbnail of an indigo ghost pedestrian.
Figure 3. ReForesight replaying the public CODa dataset (one time step of sequence 20). (a) Camera image. (b) Semantic occupancy map projected into the camera (teal: pedestrian instances). (c) Bird’s-eye view: current pedestrian points (orange), ghosts 1 s ahead (indigo), gray arrows from each current centroid to its ghost, robot at the white triangle. (d) 3D map with the ghosts (indigo) and predicted 3 s trajectories (orange). Four panels from a university campus walkway. Panel a is a sunny camera image with several pedestrians. Panel b shows the pedestrians as teal voxels on a purple ground surface. Panel c is a top-down dark view with three orange point clusters labeled Person 85, 86 and 87, each connected by a gray arrow to an indigo cluster displaced along the walking direction. Panel d is a 3D voxel map with three indigo pedestrian-shaped clusters and orange line segments showing predicted paths.
| Error at horizon | |||||
| Method | 0.5 s | 1 s | 2 s | 3 s | ADE |
| Without stationary gate | |||||
| CV (last step) | 0.337 | 0.643 | 1.262 | 1.899 | 0.986 |
| CV Kalman | 0.165 | 0.264 | 0.481 | 0.708 | 0.387 |
| MLP | 0.153 | 0.236 | 0.429 | 0.660 | 0.351 |
| GRU | 0.164 | 0.256 | 0.446 | 0.664 | 0.365 |
Table 1. Displacement error (m) by prediction horizon on the held-out CODa sequence 7 (5,812 windows, 184 tracks). The 3 s column is the FDE. Best value per column in bold.
| Without gate | With gate | |||||
|---|---|---|---|---|---|---|
| Held-out seq. | Kalman | MLP | GRU | Kalman | MLP | GRU |
| 7 (184) | 0.708 | 0.660 | 0.664 | 0.694 | 0.648 | 0.658 |
| 13 (256) | 0.895 | 0.892 | 0.949 | 0.891 | 0.889 | 0.945 |
| 16 (217) | 0.832 | 0.731 | 0.772 | 0.813 | 0.720 | 0.735 |
| 3 (42) | 0.475 | 0.367 | 0.381 | 0.466 | 0.356 | 0.375 |
| Mean | 0.728 | 0.662 | 0.692 | 0.716 | 0.653 | 0.678 |
Table 2. Cross-sequence 3 s FDE (m): each CODa sequence held out in turn (test tracks in parentheses). Learned models: lowest-validation-ADE run of three seeds. Best per row and gate setting in bold.
| S1 occlusion (%) | S2 (%) | S3 (%) | S4 | ||
|---|---|---|---|---|---|
| Variant | keep | refuse | new ID | decoy | changes |
| IoU only (no memory) | 0.0 | 100.0 | 100.0 | 0.0 | 0.54 |
| Motion, 2D gate | 100.0 | 100.0 | 0.0 | 0.0 | 0.54 |
| Motion, 3D gate | 100.0 | 100.0 | 0.0 | 100.0 | 0.54 |
| Appearance, 2D gate | 100.0 | 100.0 | 100.0 | 100.0 | 0.54 |
| Appearance, 3D gate (full) | 100.0 | 100.0 | 100.0 | 100.0 | 0.54 |
Table 3. ReIDTracker ablation on synthetic occlusion scenarios (100 seeds each). S1: identity kept after 1 to 30 missed frames / not re-used after 35 to 45. S2: a different object enters an occluded track’s gate and should get a new identity. S3: a distant decoy projects onto the predicted pixel location. S4: two objects cross (mean identity changes per run). S1, S2, and S4 use 3D positions back-projected at 8 m depth. Best in bold.