Tracking the Invisible: Self-Supervised Wheel Track Detection with Thermal-RGB Fusion

Yiming Yang; Jeremy P. Bos

cs.CVOct 9, 2026

Tracking the Invisible: Self-Supervised Wheel Track Detection with Thermal-RGB Fusion

Authors: Yiming Yang, Jeremy P. Bos

Abstract

Accurate detection of drivable area is essential for autonomous driving and advanced driver assistance systems. While state-of-the-art methods focus on general traversability estimation, they prove inadequate for snow-covered roads where following existing wheel tracks offers crucial advantages: improved traction, predictable vehicle dynamics, and verified path stability. To address this, we propose a low-cost, self-supervised approach detecting safety-critical wheel tracks using synchronized RGB-I (RGB + Long-Wave Infrared) imagery. Our method automatically generates training labels by projecting future wheel trajectories from recorded GNSS data. A neural network is designed to identify wheel tracks with the 4-channel RGB-I images from challenging, white-out road surfaces. Experimental results demonstrate the effectiveness of our approach in visually monotonous and featureless adverse environment, offering a robust and cost-effective alternative for winter autonomy. The newly introduced ThermalTrack dataset is available at: https://digitalcommons.mtu.edu/thermaltrack/.

Figures & tables

Figure 1: Left: example from WADS showing severe snow occlusion: road surfaces, lane markings, and structural boundaries are nearly indistinguishable, with only subtle terrain variations hinting at the underlying roadway. Right: the predicted probability map for wheel tracks.

Figure 2: Self-supervised wheel track labels generated by our automated procedure. One challenge in wheel track detection is that tracks appear not only in the ego vehicle’s travel path but also in the opposite path. Ignoring these can degrade training performance. The images show projected wheel track trajectories: (1) self-supervised tracks computed from GNSS data using vehicle geometry and transformations (green), and (2) manually labeled tracks from the opposite travel path (red).

Figure 3: The neural network architecture processes input images through the following sequential stages: (a) The RGB image and thermal image are downsampled to 512 x 512 pixels. Since the RGB-LWIR cameras rig has fixed geometry, we then employ a static homography transformation for image alignment. Contrast Limited Adaptive Histogram Equalization (CLAHE) is applied to enhance the contrast of aligned RGB and thermal images. (b) The intensity channel from the LWIR thermal image is integrated as the 4th channel into the RGB image, forming a 4-channel RGB-I input. (c) Our network architecture employs a ResNet-18 backbone encoder with Attention Gates and atrous (dilated) convolutional blocks. This design enables selective focus on wheel track regions while actively suppressing irrelevant areas. (d) The network outputs a pixel-wise probability map, where each value $\in$ [0,1] represents the predicted likelihood of the corresponding image pixel belonging to a wheel track.

Model	AUROC	MaxF	AP	PRE	REC	FPR	FNR
UNet	0.7068	0.5197	0.6079	0.7691	0.4192	0.0056	0.5808
PSPNet	0.7532	0.6102	0.6666	0.7978	0.5122	0.0057	0.4878
DeepLabv3+	0.7446	0.6002	0.6629	0.8074	0.4943	0.0052	0.5057
ResNet18	0.8462	0.7202	0.7337	0.7499	0.7034	0.0109	0.2966

Table I : Quantitative results of wheel track detection using baseline models, all the models are trained on RGB-I images.

Model	AUROC	MaxF	AP	PRE	REC	FPR	FNR
UNet+a2	0.7843	0.6529	0.6941	0.7929	0.5753	0.0068	0.4247
PSPNet+a1	0.8476	0.7186	0.7337	0.7471	0.7064	0.0112	0.2936
DeepLabv3+a1	0.8589	0.7460	0.7584	0.7757	0.7281	0.0102	0.2719
Ours-rgb	0.6025	0.3083	0.5164	0.6901	0.2073	0.0024	0.7927
Ours-i	0.7687	0.5803	0.6127	0.6525	0.5510	0.0137	0.4490
Ours-rgbi	0.8744	0.7579	0.7675	0.7633	0.7601	0.0114	0.2399

Table II : Ablation study for wheel track detection comparing RGB-only, Thermal-only, and RGB-I inputs, with model variants a1 (attention gate) and a2 (attention gate+atrous blocks). Ours model is ResNet18+a2. All the models are trained with RGB-I images (except Ours-rgb/Ours-i).

Figure 4: Qualitative results demonstrating the effectiveness of thermal imaging on wheel track detection. The models were trained independently on RGB, thermal, and RGB-thermal (RGB-I) images.

Figure 5: Precision-Recall curves comparison between models w/ and w/o attention gate and atrous blocks. The dashed navy-colored lines are baselines (0.043), which indicates an extreme class imbalance.

Figure 6: Qualitative wheel track detection results and probability maps visualize model performance. As illustrated in the final row, the proposed method exhibits limitations in certain scenarios: in cases where wheel track features are absent in both RGB and thermal imagery, the network produces fragmented and low-confidence predictions.

Pipeline Stage / Model Variant	Latency (ms)	Throughput
Preprocessing Pipeline
Stage 1: Downsample (RGB)	0.09	-
Stage 1: Downsample (Thermal)	0.01	-
Stage 2: CLAHE (RGB)	2.10	-
Stage 2: CLAHE (Thermal)	0.95	-
Stage 3: Merge	0.74	-

Table III : Latency and Throughput Profile of the Processing Pipeline and Models

Explore similar work

A Multi-Modal Perception Pipeline for Object Detection and Tracking in Autonomous Racing
Sep 8, 2026Davide Malvezzi, Michele Pestarino, Vittoria Cavicchioli +9Camera-Radar Fusion Camera-LiDAR Fusion
University of Modena and Reggio Emilia, Modena, Italy · HiPeRT Srl, Modena, Italy
Event-RGB Adaptive Tracking for Nighttime Highway Perception
Jul 13, 2026Haidong Wang, Hengxing Cai, Wanlei Li +2Autonomous Driving Datasets Intelligent Transportation Systems
School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China · School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen, China
Synthetic Thermal Image Generation for Real-Time Animal Detection Under Low-Visibility Conditions
Sep 26, 2026James Momoh, Khandaker Mamun AhmedInfrared Object Detection Thermal Imaging
The Beacom College of Computer and Cyber Sciences, Dakota State University

Sep 8, 2026, cs.ROJ/K move · Enter open · S save

A Multi-Modal Perception Pipeline for Object Detection and Tracking in Autonomous Racing

Davide Malvezzi, Michele Pestarino, Vittoria Cavicchioli, Valentina La Gamba+8

University of Modena and Reggio Emilia, Modena, Italy · HiPeRT Srl, Modena, Italy

Camera-Radar Fusion Camera-LiDAR Fusion Autonomous Racing Object Detection Multimodal Sensor Fusion Multi-Object Tracking

Open paper AskPDF arXiv

Object detection and tracking are fundamental components of perception systems for autonomous driving. Achieving robust performance under adverse conditions such as limited visibility, sensor noise, and failures remains an open challenge, particularly in autonomous racing, where vehicles operate at very high speeds, experience strong vibrations, and interact under small safety margins. This paper presents a multi-modal late-fusion perception pipeline for object detection and tracking in the autonomous racing domain. The proposed system extends previous work by exploiting all onboard sensors through a late-fusion approach and a dedicated multi-object tracking framework. Independent detections from cameras, LiDARs, and RADARs are combined to provide timely and robust state estimates of surrounding vehicles. The tracking method explicitly compensates for detection delays and embeds in its model prior knowledge of vehicle dynamics and track layout. Experimental evaluation on real-world data across diverse critical scenarios, representative of challenging edge cases also in urban driving, confirms the effectiveness of the proposed pipeline and its suitability to support safe and adaptive planning decisions.

Abstract

Explore similar work

A Multi-Modal Perception Pipeline for Object Detection and Tracking in Autonomous Racing

Event-RGB Adaptive Tracking for Nighttime Highway Perception

Synthetic Thermal Image Generation for Real-Time Animal Detection Under Low-Visibility Conditions