cs.ROOct 5, 2026
SaveDual Variational Autoencoders for Efficient Sim-to-Real Transfer in Low-Cost Robotic Navigation
Organizations: Department of Computer Science and Artificial Intelligence, University of Alicante
Abstract
Vision-based autonomous navigation for low-cost robots remains a fundamental challenge, primarily due to the significant gap between simulated training environments and real-world operational conditions. Direct policy transfer from simulation is often ineffective, while training exclusively on real data is impractical. We propose a hybrid transfer learning framework that effectively bridges the sim-to-real gap by combining domain randomization with feature-level domain adaptation. Our method employs a dual convolutional variational autoencoder architecture with a shared decoder, trained on an extensive set of 45225 simulated images and a minimal set of only 4556 real-world samples. This architecture learns a compact, common latent representation space that aligns the distributions of both domains. The adaptation process is further enhanced by two complementary data augmentation techniques designed to expand the limited real-world data. Experimental evaluation demonstrates that our method achieves an average success rate of almost 91% on image classification tasks for real-world indoor navigation, significantly outperforming both simulation-only and real-world-only training. We validate these findings through a direct, real-world deployment, where the proposed policy successfully guides a low-cost robot in a reactive exploration task. Furthermore, we validate the model's efficiency through a rigorous computational estimation, confirming its suitability for resource-constrained embedded platforms such as the Raspberry Pi 4 and NVIDIA Jetson Nano. This work presents a practical solution for developing effective and efficient navigation policies for low-cost robotic systems.
Figures & tables
Figure 1: The simulated indoor environment used for synthetic data acquisition. This is a high-fidelity representation of a specific corridor located in the Department of Computer Science and Artificial Intelligence at the University of Alicante. (a) View of the low-cost robot from the corridor’s entrance. (b) View of the low-cost from the corridor’s end.
Figure 2: Layer structure of our variational autoencoders. Each model comprises a convolutional encoder that compresses a 64x64 RGB input image into a 32-dimensional latent vector via variational inference, and a convolutional decoder that reconstructs the image from this latent representation.
Figure 3: Examples of labeled images used across our three experiments for training, fine-tuning, and testing. (a) Synthetic images from the simulated environment, used for the domain randomization training stage in Experiments (2) and (3). (b) (b) Real-world images captured in the physical environment, used for baseline training in Experiment (1), fine-tuning in Experiment (3), and evaluation in all experiments. (c) Realistically transformed images used to augment the real-world dataset for fine-tuning in Experiment (3) and evaluation in all experiments. (d) Synthesized realistic images, generated via latent-space manipulation, used to further refine the pseudo-synthetic autoencoder in Experiment (3).
| Identifier | Name | Size |
| (CV) | Cross-Validation Real-World Test | 186 |
| (T1) | Realistic Transformations Test 1 | 3400 |
| (T2) | Realistic Transformations Test 2 | 4000 |
Table 1: Evaluation datasets comprising real images previously unseen during the training and fine-tuning phases.
| Symbol | Description | Value |
| Learning rate | 0.001 | |
| Batch size | 32 | |
| Training/fine-tuning steps | 200 |
Table 2: Key hyperparameter values configured for the training and subsequent fine-tuning stages of the variational autoencoder models.
Figure 4: Reconstructions generated by the decoder of our three convolutional variational autoencoders from input real image data. (a) Test images reconstructed using the real-only autoencoder trained for Experiment (1). (b) Test images reconstructed through the sim-only autoencoder trained in Experiment (2). (c) Test images reconstructed using our proposed pseudo-synthetic autoencoder trained and fine-tuned for Experiment (3).
Figure 5: Mean accuracy rates, along with corresponding median and variance values, obtained when predicting probabilities related to the four possible ground-truth labels used to annotate input real images. We achieved the final results using three distinct classification models: the real-only image classifier (blue) trained for Experiment (1), the sim-only image classifier (orange) trained in Experiment (2), and our proposed pseudo-synthetic image classifier (green) trained and fine-tuned for Experiment (3). The reported accuracy percentages resulted from evaluating each classifier using three different test datasets, presented in decreasing order of importance.
| Real Test Dataset | Base Dual VAE | Proposed Method | Contribution |
| (CV) | 47.06% | 90.88% | +43.82% |
| (T1) | 39.15% | 66.05% | +26.90% |
| (T2) | 34.20% | 59.07% | +24.87% |
Table 3: Ablation study mean accuracy rates comparing the proposed full hybrid framework against a baseline implementation without the data augmentation strategies described in Subsection 3.3 .
Figure 6: Predictions and reconstructions obtained from labeled real images using our proposed pseudo-synthetic model trained in Experiment (3). These tests focused on reactive collision-free exploration tasks in the real world, involving reconstruction of input images and their classification into three categories: forward , turn , and stop . (a) Test image data representing scenarios requiring forward actions. (b) Test image data corresponding to situations requiring turn commands. (c) Test image data representing circumstances requiring stop actions.
| Real Test Dataset | Experiment | Mean Accuracy Rate (%) |
| (CV) | (1) | 72.57 |
| (CV) | (2) | 32.92 |
| (CV) | (3) | 90.88 |
| (T1) | (1) | 81.76 |
| (T1) | (2) | 40.85 |
| (T1) | (3) | 69.86 |
Table 4: Updated mean accuracy rates for the three models evaluated on the simplified three-category classification task.
Figure 7: Two-dimensional t-SNE visualization of the latent spaces generated from the non-augmented real-world (CV) test set. (a) Latent space from the real-only autoencoder. (b) Latent space from the sim-only autoencoder. (c) Latent space from the proposed pseudo-synthetic autoencoder.
Figure 8: Two-dimensional t-SNE visualization of the latent spaces generated from the realistically transformed (T1) test set. (a) Latent space from the real-only autoencoder. (b) Latent space from the sim-only autoencoder. (c) Latent space from the proposed pseudo-synthetic autoencoder.
Figure 9: Two-dimensional t-SNE visualization of the latent spaces generated from the realistically transformed (T2) test set. (a) Latent space from the real-only autoencoder. (b) Latent space from the sim-only autoencoder. (c) Latent space from the proposed pseudo-synthetic autoencoder.
| Real Test Dataset | Experiment | Silhouette | Davies-Bouldin | Calinski-Harabasz |
| (CV) | (1) | -0.01 | 9.48 | 0.98 |
| (CV) | (2) | -0.04 | 10.41 | 0.52 |
| (CV) | (3) | 0.10 | 6.34 | 2.51 |
| (T1) | (1) | 0.51 | 0.72 | 11578.54 |
| (T1) | (2) | 0.53 | 1.36 | 39260.67 |
| (T1) | (3) | 0.28 | 1.64 | 3895.42 |
Table 5: Quantitative analysis of 2D latent space cluster quality using three standard validity indices.
| Platform | Processor | Memory | AI Accelerator |
| Linux Laptop | Intel Core i7-8750H 6-core @ 2.20 GHz | 32 GB DDR4 | NVIDIA GeForce RTX 2080 |
| Raspberry Pi 3B+ | ARM Cortex-A53 Quad-core @ 1.40 GHz | 1 GB LPDDR2 | N/A |
| Raspberry Pi 4B | ARM Cortex-A72 Quad-core @ 1.50 GHz | 4 GB LPDDR4 | N/A |
| Jetson Nano 2 GB | ARM Cortex-A57 Quad-core @ 1.43 GHz | 2 GB LPDDR4 | 128-core NVIDIA Maxwell |
Table 6: Technical specifications of evaluated hardware platforms.
| Platform | Memory (MiB) | Cost (MFLOPs) | Latency (ms) | Throughput (FPS) |
| Linux Laptop (CPU) | 8.31 | 169.40 | 1.10 | 900 |
| Raspberry Pi 3B+ (CPU) | 8.31 | 169.40 | 257 | 4 |
| Raspberry Pi 4B (CPU) | 8.31 | 169.40 | 21.70 | 46 |
| Jetson Nano 2 GB (GPU) | 8.31 | 169.40 | 1.20 | 833 |
Table 7: Roofline-based performance estimation for the proposed pseudo-synthetic model.
Figure 10: Low-cost physical robot used for real-world deployment. The platform is a customized Makeblock mBot Ranger equipped with a single forward-facing monocular RGB camera module, which provides all perceptual information for the navigation task.
Figure 11: Real-time evaluation of the proposed image classification policy on a low-cost physical robot. The green line depicts the path taken by the differential-drive agent during an autonomous, map-less exploration task in our indoor corridor, located in the Department of Computer Science and Artificial Intelligence at the University of Alicante. The trajectory demonstrates the robot’s ability to robustly navigate and avoid distinct obstacles, analyzed at six different checkpoints. (a) A closed door followed by a column. (b) A closed door placed in a side wall. (c) A wastepaper bin located in front of a wall. (d) An open door and a chair corresponding to an office. (e) A column placed beside a window. (f) A robot stopped on a wall.
Figure 12: Detailed analysis of the robot’s reactive behavior at six different checkpoints. These examples illustrate successful collision avoidance with challenging combinations of obstacles with diverse shapes and textures under varying illumination conditions. (a) A closed door followed by a column. (b) A closed door placed in a side wall. (c) A wastepaper bin located in front of a wall. (d) An open door and a chair corresponding to an office. (e) A column placed beside a window. (f) A robot stopped on a wall.
Figure 13: Qualitative evaluation of the proposed image classification policy’s generalization, adaptation, and robustness. We compared teleoperated navigation with fully autonomous navigation on two different days with varying conditions. Our reactive system demonstrates consistent behavior in response to several unknown situations, successfully adapting to lighting changes, novel objects, and dynamic obstacles. (a) Forward scenarios. (b) Turn scenarios. (c) Stop scenarios.