Fieldable Low-Light Multi-Task Perception via Foundation-Model-Assisted Domain Adaptation
Organizations: Beomseok Kang is with Seoul National University, Seoul, South Korea. · Niluthpol Chowdhury Mithun, Mikhail Sizintsev, Han-Pang Chiu, and Supun Samarasekera are with SRI International, Princeton, NJ, USA.
Abstract
Fieldable low-light perception systems require both semantic and geometric scene understanding under severe domain shift and embedded compute constraints. We present FAMDA, a foundation-model-assisted self-training framework for unsupervised domain adaptation (UDA) in multi-task dense prediction. The framework integrates segmentation and depth foundation models into a self-training paradigm to generate high-quality pseudo-labels for the target domain, distilling their generalization capabilities into a single lightweight student network. To connect the method to deployment, we instantiate the adapted student on a compact handheld low-light prototype with DomiNite imaging and Jetson Orin Nano embedded compute. We evaluate on standard synthetic-to-real multi-task UDA benchmarks for comparison with prior work and on our field-collected low-light dataset to validate daytime-to-low-light adaptation in the target application setting. Across these evaluations, the proposed approach achieves strong segmentation and depth accuracy with substantially lower memory use and inference latency than direct foundation-model inference, demonstrating a practical path toward embedded multi-task perception for low-light field systems.
Figures & tables
| Component | Configuration |
|---|---|
| Imaging | DomiNite low-light camera system ( HFOV) |
| Camera configuration | Calibrated two-camera setup for data collection/evaluation |
| Network input | Single low-light frame |
| Predicted outputs | Semantic segmentation and monocular depth |
| Embedded compute | Jetson Orin Nano |
| Deployment format | PyTorch ONNX TensorRT |
| Method | Outputs | mIoU | RMSE | Model Size | A5000 Lat. |
| (%) | (m) | (MB) | (ms) | ||
| SSAM-H | Seg. | 43.93 | – | 3351 | 34117 |
| DAM-L | Depth | – | 5.47 | 1279 | 60.8 |
| DaF-Seg-B5 | Seg. | 52.92 | – | 325 | 35.5 |
| DaF-Dep-B5 | Depth | – | 5.55 | 325 | 36.7 |
| DaF-Seg/Dep-B5 † | Seg.+Depth | 52.92 | 5.55 | 650 | 72.2 |
| Model | TRT Size | Lat. | FPS | Peak Mem. | Power | mIoU | RMSE |
|---|---|---|---|---|---|---|---|
| (MB) | (ms) | (Hz) | (MB) | (W) | (%) | (m) | |
| Ours-B0 | 23.4 | 123 | 8.1 | 1346 | 7.28 | 42.90 | 5.54 |
| Ours-B1 | 42.5 | 144 | 6.9 | 1361 | 7.54 | 49.23 | 5.53 |
| Ours-B2 | 66.0 | 158 | 6.3 | 1406 | 7.75 | 55.01 | 5.53 |
| Ours-B5 | 187.0 | 212 | 4.7 | 1629 | 8.80 | 55.96 | 5.52 |
| Method | Backbone | SYN CS (16 class) | VK2 CS (8 class) | |||
| mIoU | RMSE | mIoU | RMSE | |||
| w/o UDA | STL target | DeepLabV2 | 67.93 | 6.62 | 77.10 | 6.62 |
| STL source | DeepLabV2 | 35.63 | 13.79 | 58.77 | 11.99 | |
| MTL target | DeepLabV2 | 70.43 | 6.79 | 79.63 | 6.72 | |
| MTL source | DeepLabV2 | 15.32 | 14.51 | 49.50 | 12.26 | |
| w/ UDA | STL-UDA [ 35 ] | DeepLabV2 | 37.55 | 14.26 | 61.60 | 11.45 |