Synthetic-to-Real Domain Adaptation

Ask about this topic

Overview

The field studies how to make models trained on simulated or generated data work reliably on real images, sensor readings, and physical systems, where labels may be scarce or costly. Common approaches include physics-based simulation and domain randomization, style translation, feature alignment with adversarial learning or pseudo-labels, and self-supervised or test-time adaptation. Researchers evaluate transfer in settings such as GTA-to-Cityscapes driving, LiDAR perception, robotic manipulation, and medical imaging, often comparing synthetic-only training with mixtures that include real data. A central tension is that visual realism alone may not capture the real domain’s diversity: synthetic data can provide useful supervision, but small amounts of well-matched real data and task-specific adaptation often determine whether gains transfer.

Latest papers

CardsList

137 papers14 papers in the last four weeks, down 33% on the four weeks before. 0.1% of all new papers.

Mar 24, 2026

VoDaSuRe: A Large-Scale Dataset Revealing Domain Shift in Volumetric Super-Resolution

Recent advances in volumetric super-resolution (SR) have demonstrated strong performance in medical and scientific imaging, with transformer- and CNN-based approaches achieving impressive results even at extreme scaling factors. In this work, we show that much of this performance stems from training on downsampled data rather than real low-resolution scans. This reliance on downsampling is partly driven by the scarcity of paired high- and low-resolution 3D datasets. To address this, we introduce VoDaSuRe, a large-scale volumetric dataset containing paired high- and low-resolution scans. When training models on VoDaSuRe, we reveal a significant discrepancy: SR models trained on downsampled data produce substantially sharper predictions than those trained on real low-resolution scans, which smooth fine structures. Conversely, applying models trained on downsampled data to real scans preserves more structure but is inaccurate. Our findings suggest that current SR methods are overstated - when applied to real data, they do not recover structures lost in low-resolution scans and instead predict a smoothed average. We argue that progress in deep learning-based volumetric SR requires datasets with paired real scans of high complexity, such as VoDaSuRe. Our dataset and code are publicly available through: https://augusthoeg.github.io/VoDaSuRe/

August Leander Høeg, Sophia Wiinberg Bardenfleth, Hans Martin Kjer +3

Technical University of Denmark, Kgs. Lyngby, Denmark

Image Super-ResolutionMedical Image Super-ResolutionMedical Image BenchmarksSynthetic-to-Real Domain Adaptation
AskPDF
Mar 23, 2026

RAFL: Generalizable Sim-to-Real of Soft Robots with Residual Acceleration Field Learning

Differentiable simulators enable gradient-based optimization of soft robots over material parameters, control, and morphology, but accurately modeling real systems remains challenging due to the sim-to-real gap. This issue becomes more pronounced when geometry is itself a design variable. System identification reduces discrepancies by fitting global material parameters to data; however, when constitutive models are misspecified or observations are sparse, identified parameters often absorb geometry-dependent effects rather than reflect intrinsic material behavior. More expressive constitutive models can improve accuracy but substantially increase computational cost, limiting practicality. We propose a residual acceleration field learning (RAFL) framework that augments a base simulator with a transferable, element-level corrective dynamics field. Operating on shared local features, the model is agnostic to global mesh topology and discretization. Trained end-to-end through a differentiable simulator using sparse marker observations, the learned residual generalizes across shapes. In both sim-to-sim and sim-to-real experiments, our method achieves consistent zero-shot improvements on unseen morphologies, while system identification frequently exhibits negative transfer. The framework also supports continual refinement, enabling simulation accuracy to accumulate during morphology optimization.

Dong Heon Cho, Boyuan Chen

Department of Computer Science, Duke University, 308 Research Dr., Durham, 27705, NC, USA · Department of Mechanical Engineering and Material Science, 121 Hudson Hall, Durham, 27708, NC, USA · Department of Electrical and Computer Engineering, 100 Science Dr., Durham, 27708, NC, USA

Synthetic-to-Real Domain AdaptationResidual Dynamics LearningSoft RoboticsRobot Dynamics LearningCross-Embodiment Robot LearningDifferentiable Physics
AskPDF
Mar 23, 2026

Cross-Scenario Deraining Adaptation with Unpaired Data: Superpixel Structural Priors and Multi-Stage Pseudo-Rain Synthesis

Image deraining plays a pivotal role in low-level computer vision, serving as a prerequisite for robust outdoor surveillance and autonomous driving systems. While deep learning paradigms have achieved remarkable success in firmly aligned settings, they often suffer from severe performance degradation when generalized to unseen Out-of-Distribution (OOD) scenarios. This failure stems primarily from the significant domain discrepancy between synthetic training datasets and the complex physical dynamics of real-world rain. To address these challenges, this paper proposes a pioneering cross-scenario deraining adaptation framework. Diverging from conventional approaches, our method obviates the requirements for paired rainy observations in the target domain, leveraging exclusively rain-free background images. We design a Superpixel Generation (Sup-Gen) module to extract stable structural priors from the source domain using Simple Linear Iterative Clustering. Subsequently, a Resolution-adaptive Fusion strategy is introduced to align these source structures with target backgrounds through texture similarity, ensuring the synthesis of diverse and realistic pseudo-data. Finally, we implement a pseudo-label re-Synthesize mechanism that employs multi-stage noise generation to simulate realistic rain streaks. This framework functions as a versatile plug-and-play module capable of seamless integration into arbitrary deraining architectures. Extensive experiments on state-of-the-art models demonstrate that our approach yields remarkable PSNR gains of up to 32% to 59% in OOD domains while significantly accelerating training convergence.

Kangbo Zhao, Miaoxin Guan, Xiang Chen +2

School of Information Engineering, Guangdong University of Technology, Guangzhou, 510006, China · School of Computer Science and Engineering, Nanjing University of Science and Technology

Image DerainingSynthetic-to-Real Domain AdaptationUnsupervised Domain AdaptationOOD GeneralizationImage EnhancementSynthetic Data Augmentation
AskPDF
Mar 18, 2026

From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving

Autonomous driving technologies have achieved significant advances in recent years, yet their real-world deployment remains constrained by data scarcity, safety requirements, and the need for generalization across diverse environments. In response, synthetic data and virtual environments have emerged as powerful enablers, offering scalable, controllable, and richly annotated scenarios for training and evaluation. This survey presents a comprehensive review of recent developments at the intersection of autonomous driving, simulation technologies, and synthetic datasets. We organize the landscape across three core dimensions: (i) the use of synthetic data for perception and planning, (ii) digital twin-based simulation for system validation, and (iii) domain adaptation strategies bridging synthetic and real-world data. We also highlight the role of vision-language models and simulation realism in enhancing scene understanding and generalization. A detailed taxonomy of datasets, tools, and simulation platforms is provided, alongside an analysis of trends in benchmark design. Finally, we discuss critical challenges and open research directions, including Sim2Real transfer, scalable safety validation, cooperative autonomy, and simulation-driven policy learning, that must be addressed to accelerate the path toward safe, generalizable, and globally deployable autonomous driving systems.

A. Humnabadkar, A. Sikdar, B. Cave +3

Department of Computer Science, Edge Hill University, United Kingdom.

Autonomous DrivingAutonomous Driving SimulationSynthetic-to-Real Domain AdaptationSynthetic Data GenerationDigital Twins
AskPDF
Mar 17, 2026

Structured prototype regularization for synthetic-to-real driving scene parsing

Driving scene parsing is critical for autonomous vehicles to operate reliably in complex real-world traffic environments. To reduce the reliance on costly pixel-level annotations, synthetic datasets with automatically generated labels have become a popular alternative. However, models trained on synthetic data often perform poorly when applied to real-world scenes due to the synthetic-to-real domain gap. Despite the success of unsupervised domain adaptation in narrowing this gap, most existing methods mainly focus on global feature alignment while overlooking the semantic structure of the feature space. As a result, semantic relations among classes are insufficiently modeled, limiting the model's ability to generalize. To address these challenges, this study introduces a novel unsupervised domain adaptation framework that explicitly regularizes semantic feature structures to significantly enhance driving scene parsing performance in real-world scenarios. Specifically, the proposed method enforces inter-class separation and intra-class compactness by leveraging class-specific prototypes, thereby enhancing the discriminability and structural coherence of feature clusters. An entropy-based noise filtering strategy improves the reliability of pseudo labels, while a pixel-level attention mechanism further refines feature alignment. Extensive experiments on representative benchmarks demonstrate that the proposed method consistently outperforms recent state-of-the-art methods. These results underscore the importance of preserving semantic structure for robust synthetic-to-real adaptation in driving scene parsing tasks.

Jiahe Fan, Xiao Ma, Sergey Vityazev +3

Tongji University, Shanghai 201804, China · Beijing Institute of Aerospace Control Devices, Beijing 100039, China · Ryazan State Radio Engineering University, Ryazan 390005, Russia +1

Synthetic-to-Real Domain AdaptationUnsupervised Domain AdaptationAutonomous Driving PerceptionSemantic SegmentationPrototype Learning
AskPDF
Mar 11, 2026

Synthetic Data Domain Adaptation for ASR via LLM-based Text and Phonetic Respelling Augmentation

End-to-end automatic speech recognition often degrades on domain-specific data due to scarce in-domain resources. We propose a synthetic-data-based domain adaptation framework with two contributions: (1) a large language model (LLM)-based text augmentation pipeline with a filtering strategy that balances lexical diversity, perplexity, and domain-term coverage, and (2) phonetic respelling augmentation (PRA), a novel method that introduces pronunciation variability through LLM-generated orthographic pseudo-spellings. Unlike conventional acoustic-level methods such as SpecAugment, PRA provides phonetic diversity before speech synthesis, enabling synthetic speech to better approximate real-world variability. Experimental results across four domain-specific datasets demonstrate consistent reductions in word error rate, confirming that combining domain-specific lexical coverage with realistic pronunciation variation significantly improves ASR robustness.

Natsuo Yamashita, Koichi Nagatsuka, Hiroaki Kokubo +2

Hitachi, Ltd.

Large Language Model-Based Data AugmentationDomain Adaptation for ASRSynthetic Data Augmentation for Speech RecognitionData AugmentationSynthetic-to-Real Domain Adaptation
AskPDF
Mar 9, 2026

Real-Time Monocular Scene Analysis for UAV in Outdoor Environments

In this thesis, we leverage monocular cameras on aerial robots to predict depth and semantic maps in low-altitude unstructured environments. We propose a joint deep-learning architecture, named Co-SemDepth, that can perform the two tasks accurately and rapidly, and validate its effectiveness on a variety of datasets. The training of neural networks requires an abundance of annotated data, and in the UAV field, the availability of such data is limited. We introduce a new synthetic dataset in this thesis, TopAir that contains images captured with a nadir view in outdoor environments at different altitudes, helping to fill the gap. While using synthetic data for the training is convenient, it raises issues when shifting to the real domain for testing. We conduct an extensive analytical study to assess the effect of several factors on the synthetic-to-real generalization. Co-SemDepth and TaskPrompter models are used for comparison in this study. The results reveal a superior generalization performance for Co-SemDepth in depth estimation and for TaskPrompter in semantic segmentation. Also, our analysis allows us to determine which training datasets lead to a better generalization. Moreover, to help attenuate the gap between the synthetic and real domains, image style transfer techniques are explored on aerial images to convert from the synthetic to the realistic style. Cycle-GAN and Diffusion models are employed. The results reveal that diffusion models are better in the synthetic to real style transfer. In the end, we focus on the marine domain and address its challenges. Co-SemDepth is trained on a collected synthetic marine data, called MidSea, and tested on both synthetic and real data. The results reveal good generalization performance of Co-SemDepth when tested on real data from the SMD dataset while further enhancement is needed on the MIT dataset.

Yara AlaaEldin

University of Genova; DRIM; Ph.D. Program of National Interest in Robotics and Intelligent Machines; Administrative Headquarters: Università di Genova

Monocular Depth EstimationSynthetic-to-Real Domain AdaptationAerial RoboticsSemantic SegmentationMulti-Task Learning
AskPDF
Mar 7, 2026

Bi-directional digital twin prototype anchoring with multi-periodicity learning for few-shot fault diagnosis

Intelligent fault diagnosis (IFD) has emerged as a powerful paradigm for ensuring the safety and reliability of industrial machinery. However, traditional IFD methods rely heavily on abundant labeled data for training, which is often difficult to obtain in practical industrial environments. Constructing a digital twin (DT) of the physical asset to obtain simulation data has therefore become a promising alternative. Nevertheless, existing DT-assisted diagnosis methods mainly transfer diagnostic knowledge through domain adaptation techniques, which still require a considerable amount of unlabeled data from the target asset. To address the challenges in few-shot scenarios where only extremely limited samples are available, a bi-directional DT prototype anchoring method with multi-periodicity learning is proposed. Specifically, a framework involving meta-training in the DT virtual space and test-time adaptation in the physical space is constructed for reliable few-shot model adaptation for the target asset. A bi-directional twin-domain prototype anchoring strategy with covariance-guided augmentation for adaptation is further developed to improve the robustness of prototype estimation. In addition, a multi-periodicity feature learning module is designed to capture the intrinsic periodic characteristics within current signals. A DT of an asynchronous motor is built based on finite element method, and experiments are conducted under multiple few-shot settings and three working conditions. Comparative and ablation studies demonstrate the superiority and effectiveness of the proposed method for few-shot fault diagnosis.

Pengcheng Xia, Zhichao Dong, Yixiang Huang +3

State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai 200240, PR China

Fault DiagnosisIndustrial Fault DiagnosisDigital TwinsPrototype LearningSynthetic-to-Real Domain AdaptationTest-Time Adaptation
AskPDF
Mar 5, 2026

Unsupervised domain adaptation for radioisotope identification in gamma spectroscopy

Training machine learning models for radioisotope identification using gamma spectroscopy remains an elusive challenge for many practical applications, largely stemming from the difficulty of acquiring and labeling large, diverse experimental datasets. Simulations can mitigate this challenge, but the accuracy of models trained on simulated data can deteriorate substantially when deployed to an out-of-distribution operational environment. In this study, we demonstrate that unsupervised domain adaptation (UDA) can improve the ability of a model trained on synthetic data to generalize to a new testing domain, provided unlabeled data from the target domain is available. Conventional supervised techniques are unable to utilize this data because the absence of isotope labels precludes defining a supervised classification loss. We compare a range of different UDA techniques, finding that feature alignment strategies, particularly via maximum mean discrepancy (MMD) minimization or domain-adversarial training, yield the most consistent improvement to testing scores. For instance, using a custom transformer-based neural network, we achieve a testing accuracy of 0.904±0.0220.904 \pm 0.022 on an experimental LaBr3_3 test set after performing unsupervised feature alignment via MMD minimization, compared to 0.754±0.0140.754 \pm 0.014 before alignment. Overall, our results highlight the potential of using UDA to adapt a radioisotope classifier trained on synthetic data for real-world deployment.

Peter Lalor, Ayush Panigrahy, Alex Hagen

Pacific Northwest National Laboratory, Richland, WA 99352 USA · University of Washington, Seattle, WA 98195 USA

Synthetic-to-Real Domain AdaptationUnsupervised Domain AdaptationDomain-Adversarial Learning
AskPDF
Mar 5, 2026

Engineering Regression Without Real-Data Training: Domain Adaptation for Tabular Foundation Models Using Multi-Dataset Embeddings

Predictive modeling in engineering applications has long been dominated by bespoke models and small, siloed tabular datasets, limiting the applicability of large-scale learning approaches. Despite recent progress in tabular foundation models, the resulting synthetic training distributions used for pre-training may not reflect the statistical structure of engineering data, limiting transfer to engineering regression. We introduce TREDBench, a curated collection of 83 real-world tabular regression datasets with expert engineering/non-engineering labels, and use TabPFN 2.5's dataset-level embedding to study domain structure in a common representation space. We find that engineering datasets are partially distinguishable from non-engineering datasets, while standard procedurally generated datasets are highly distinguishable from engineering datasets, revealing a substantial synthetic-real domain gap. To bridge this gap without training on real engineering samples, we propose an embedding-guided synthetic data curation method: we generate and identify "engineering-like" synthetic datasets, and perform continued pre-training of TabPFN 2.5 using only the selected synthetic tasks. Across 35 engineering regression datasets, this synthetic-only adaptation improves predictive accuracy and data efficiency, outperforming TabPFN 2.5 on 29/35 datasets and AutoGluon on 27/35, with mean multiplicative data-efficiency gains of 1.75x and 4.44x, respectively. More broadly, our results indicate that principled synthetic data curation can convert procedural generators into domain-relevant "data engines," enabling foundation models to improve in data-sparse scientific and industrial domains where real data collection is the primary bottleneck.

Lyle Regenwetter, Rosen Yu, Cyril Picard +1

Massachusetts Institute of Technology

Synthetic Data PretrainingSynthetic-to-Real Domain AdaptationTabular Foundation ModelsSynthetic Data Selection
AskPDF
Mar 3, 2026

Self-supervised Domain Adaptation for Visual 3D Pose Estimation of Nano-drone Racing Gates by Enforcing Geometric Consistency

We consider the task of visually estimating the relative pose of a drone racing gate in front of a nano-quadrotor, using a convolutional neural network pre-trained on simulated data to regress the gate's pose. Due to the sim-to-real gap, the pre-trained model underperforms in the real world and must be adapted to the target domain. We propose an unsupervised domain adaptation (UDA) approach using only real image sequences collected by the drone flying an arbitrary trajectory in front of a gate; sequences are annotated in a self-supervised fashion with the drone's odometry as measured by its onboard sensors. On this dataset, a state consistency loss enforces that two images acquired at different times yield pose predictions that are consistent with the drone's odometry. Results indicate that our approach outperforms other SoA UDA approaches, has a low mean absolute error in position (x=26, y=28, z=10 cm) and orientation (ψψ=13∘{^{\circ}}), an improvement of 40% in position and 37% in orientation over a baseline. The approach's effectiveness is appreciable with as few as 10 minutes of real-world flight data and yields models with an inference time of 30.4ms (33 fps) when deployed aboard the Crazyflie 2.1 Brushless nano-drone.

Nicholas Carlotti, Michele Antonazzi, Elia Cereda +4

Dalle Molle Institute for Artificial Intelligence (IDSIA), USI-SUPSI, Lugano, Switzerland · University of Milan, Italy · Integrated Systems Laboratory (IIS), ETHZ, Zürich, Switzerland

Relative Pose EstimationObject Pose EstimationSynthetic-to-Real Domain AdaptationUnsupervised Domain AdaptationSelf-Supervised Learning
AskPDF
Mar 3, 2026

Single Microphone Own Voice Detection based on Simulated Transfer Functions for Hearing Aids

This paper presents a simulation-based approach to own voice detection (OVD) in hearing aids using a single microphone. While OVD can significantly improve user comfort and speech intelligibility, enabling reliable OVD with a single microphone is desirable for simplifying hardware and reducing power consumption in compact hearing devices. However, most existing solutions rely on multiple microphones or additional sensors, increasing device complexity and cost. To enable ML-based OVD without requiring costly transfer-function measurements, we propose a data augmentation strategy based on simulated acoustic transfer functions (ATFs) that expose the model to a wide range of spatial propagation conditions. A transformer-based classifier is trained using analytically generated ATFs and further fine-tuned using numerically simulated ATFs with increasing geometric realism. This hierarchical adaptation enables the model to refine its spatial understanding while maintaining generalization. Experimental results show 95.52% accuracy on simulated head-and-torso test data and 90.02% accuracy for one-second speech segments, demonstrating robustness to short durations. When evaluated on real-world hearing-aid recordings, few-shot fine-tuning using a small subset achieves 91% accuracy, demonstrating that limited real-world data can effectively adapt the simulation-trained model. These results highlight the potential of simulation-based training for enabling practical single-microphone OVD systems in hearing aids.

Mathuranathan Mayuravaani, W. Bastiaan Kleijn, Andrew Lensen +1

Synthetic-to-Real Domain AdaptationSynthetic Data AugmentationSpeech Processing
AskPDF
Feb 25, 2026

Structure-to-Image: Zero-Shot Depth Estimation in Colonoscopy via High-Fidelity Sim-to-Real Adaptation

Monocular depth estimation (MDE) for colonoscopy is hampered by the domain gap between simulated and real-world images. Existing image-to-image translation methods, which use depth as a posterior constraint, often produce structural distortions and specular highlights by failing to balance realism with structure consistency. To address this, we propose a Structure-to-Image paradigm that transforms the depth map from a passive constraint into an active generative foundation. We are the first to introduce phase congruency to colonoscopic domain adaptation and design a cross-level structure constraint to co-optimize geometric structures and fine-grained details like vascular textures. In zero-shot evaluations conducted on a publicly available phantom dataset, the MDE model that was fine-tuned on our generated data achieved a maximum reduction of 44.18% in RMSE compared to competing methods. Our code is available at https://github.com/YyangJJuan/PC-S2I.git.

Juan Yang, Yuyan Zhang, Han Jia +2

College of Computer Science, Sichuan University, Chengdu, China · Department of Gastroenterology and Hepatology, West China Hospital, Sichuan University, Chengdu, China

Monocular Depth EstimationSynthetic-to-Real Domain AdaptationUnpaired Image-to-Image Translation
AskPDF
Feb 24, 2026

SynthRender and I-AsSET: Open-Source Framework and Dataset for Bidirectional Sim-Real Transfer in Industrial Object Perception

Object perception is fundamental for tasks such as robotic material handling and quality inspection. However, modern supervised deep-learning models require large annotated datasets for robust automation under semi-uncontrolled conditions; a major barrier for widespread deployment with proprietary industrial parts. We address this through an integrated framework combining synthetic data generation and structured empirical evaluation for systematic investigation of bidirectional sim-to-real transfer. Our method integrates 2D-to-3D Reality-to-Simulation techniques for 3D asset creation from physical parts with programmatic Guided Domain Randomization (GDR) via SynthRender, an open-source synthetic image generation framework. Structured ablation studies across multiple benchmarks quantify the impact of individual rendering design choices, yielding practical guidelines for data-efficient synthetic training. To support evaluation under realistic industrial conditions, we introduce Industrial Assets for Sim-to-Real Evaluation and Transfer (I-AsSET), a 32-class dataset with diverse textures, intra-class variation, strong inter-class similarities, and 19,672 annotations, providing both CAD models and reconstructed meshes for bidirectional sim-to-real benchmarking. Across three industrial benchmarks, the proposed framework achieves highly competitive performance, reaching 98.7% mAP@50 on a public robotics dataset, 97.9% mAP@50 on an automotive benchmark, and 95.1% mAP@50 on I-AsSET.

Jose Moises Araya-Martinez, Thushar Tom, Adrián Sanchis Reig +3

Technical University Berlin, Industrial Automation Technology, Berlin, Germany · Mercedes-Benz AG, Future Manufacturing Technologies, Sindelfingen, Germany · Technical University Braunschweig, Institute for Cognitive Robotics, Braunschweig, Germany

Sim-to-Real TransferSynthetic-to-Real Domain AdaptationSynthetic Data GenerationReal-to-Sim TransferObject DetectionDomain Randomization
AskPDF
Feb 23, 2026

When Pretty Isn't Useful: Investigating Why Modern Text-to-Image Models Fail as Reliable Training Data Generators

Recent text-to-image (T2I) diffusion models produce visually stunning images and demonstrate excellent prompt following. But do they perform well as synthetic vision data generators? In this work, we revisit the promise of synthetic data as a scalable substitute for real training sets and uncover a surprising performance regression. We generate large-scale synthetic datasets using state-of-the-art T2I models released between 2022 and 2025, train standard classifiers solely on this synthetic data, and evaluate them on real test data. Despite observable advances in visual fidelity and prompt adherence, classification accuracy on real test data consistently declines with newer T2I models as training data generators. Our analysis reveals a hidden trend: These models collapse to a narrow, aesthetic-centric distribution that undermines diversity and real data distribution coverage. Overall, our findings challenge a growing assumption in vision research, namely that progress in generative realism implies progress in data realism. We thus highlight an urgent need to rethink the capabilities of modern T2I models as reliable training data generators.

Krzysztof Adamkiewicz, Brian Bernhard Moser, Stanislav Frolov +3

1RPTU University Kaiserslautern-Landau, Kaiserslautern, Germany · 2German Research Center for Artificial Intelligence (DFKI), Kaiserslautern, Germany

Synthetic-to-Real Domain AdaptationDistribution ShiftSynthetic Data GenerationT2I Diffusion ModelsSynthetic Data Evaluation
AskPDF
May 26, 2025

OmniFall: From Staged Through Synthetic to Wild, A Unified Multi-Domain Dataset for Robust Fall Detection

Visual fall detection models are usually trained on small, staged datasets. Their real-world utility remains unclear; such data lacks diversity and evaluation protocols differ from paper to paper. We propose OmniFall, a unified benchmark of 15k videos (80 hours) with frame-level annotations in a single 16-class taxonomy. It spans three domains: OF-Staged unifies eight staged datasets with cross-subject and cross-view splits; OF-Synthetic adds 12k videos (17 h) with controlled demographic and environmental diversity; and OF-In-the-Wild provides a test-only set of genuine accident videos. We evaluate fine-tuned models as well as much larger zero-shot multimodal LLMs. On in-the-wild fall events, both do comparably well. The clinically critical fallen state is where they part: zero-shot models keep confusing fallen with lying, whereas models fine-tuned on synthetic data with explicit fallen-state scenes do substantially better. We release the unified annotations, the synthetic data, and the in-the-wild test set to foster the development of fall and fallen-state detectors for uncontrolled environments. Dataset: https://hf.co/datasets/simplexsigil2/omnifall

David Schneider, Zdravko Marinov, Moritz Mistol +6

Karlsruhe Institute of Technology · Mercedes-Benz Tech Innovation

Synthetic-to-Real Domain AdaptationFall DetectionSynthetic Data Evaluation
AskPDF
May 21, 2025

Synthetic Enclosed Echoes: A New Dataset to Mitigate the Gap Between Simulated and Real-World Sonar Data

This paper introduces Synthetic Enclosed Echoes (SEE), a novel dataset designed to enhance robot perception and 3D reconstruction capabilities in underwater environments. SEE comprises high-fidelity synthetic sonar data, complemented by a smaller subset of real-world sonar data. To facilitate flexible data acquisition, a simulated environment has been developed, enabling the generation of additional data through modifications such as the inclusion of new structures or imaging sonar configurations. This hybrid approach leverages the advantages of synthetic data, including readily available ground truth and the ability to generate diverse datasets, while bridging the simulation-to-reality gap with real-world data acquired in a similar environment. The SEE dataset comprehensively evaluates acoustic data-based methods, including mathematics-based sonar approaches and deep learning algorithms. These techniques were employed to validate the dataset, confirming its suitability for underwater 3D reconstruction. Furthermore, this paper proposes a novel modification to a state-of-the-art algorithm, demonstrating improved performance compared to existing methods. The SEE dataset enables the evaluation of acoustic data-based methods in realistic scenarios, thereby improving their feasibility for real-world underwater applications.

Guilherme de Oliveira, Matheus M. dos Santos, Paulo L. J. Drews-Jr

NAUTEC - Intelligent robotics and automation group, FURG - Federal University of Rio Grande, Rio Grande, RS, Brazil · ICMC - Institute of Mathematics and Computer Sciences, USP - University of S˜ao Paulo, S˜ao Carlos, SP, Brazil

Synthetic-to-Real Domain AdaptationUnderwater Imaging3D ReconstructionSynthetic Data GenerationSynthetic Benchmark GenerationUnderwater 3D Reconstruction
AskPDF