cs.ROMar 18, 2026

From Optimizable to Interactable: Mixed Digital Twin-Empowered Testing of Vehicle-Infrastructure Cooperation Systems

Authors: Jianghong Dong, Chunying Yang, Mengchi Cai, Chaoyi Chen, Qing Xu, Jianqiang Wang, Keqiang Li

Organizations: School of Vehicle and Mobility, Tsinghua University, 100084 Beijing, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Sufficient testing under corner cases is critical for the long-term operation of vehicle-infrastructure cooperation systems (VICS). However, existing corner-case generation methods are primarily AI-driven, and VICS testing under corner cases is typically limited to simulation. In this paper, we introduce an L5 ''Interactable'' level to the VICS digital twin (VICS-DT) taxonomy, extending beyond the conventional L4 ''Optimizable'' level. We further propose an L5-level VICS testing framework, IMPACT (Interactive Mixed-digital-twin Paradigm for Advanced Cooperative vehicle-infrastructure Testing). By enabling direct human interactions with VICS entities, IMPACT incorporates highly uncertain and unpredictable human behaviors into the testing loop, naturally generating high-quality corner cases that complement AI-based methods. Furthermore, the mixedDT-enabled ''Physical-Virtual Action Interaction'' facilitates safe VICS testing under corner cases, incorporating real-world environments and entities rather than purely in simulation. Finally, we implement IMPACT on the I-VIT (Interactive Vehicle-Infrastructure Testbed), and experiments demonstrate its effectiveness. The experimental videos are available at our project website: https://dongjh20.github.io/IMPACT.

Figures & tables

Figure 1: Schematics for classical DT and mixedDT. (a) In classical DT, virtual entities are typically digital counterparts of the physical ones. (b) In mixedDT, virtual entities exist independently and the physical and virtual entities are integrated into the unified mixed space, where they can coexist and interact with each other.
LevelCapabilityRequirement
L1 Visualizable [ 20 , 21 ]Mirroring the physical via the virtual (Holographic Perception)Emerging perception data, requiring holographic data governance.
L2 Diagnosable [ 22 , 23 ]Diagnosing the physical via the virtual (Causal Inference)Spatio-temporally continuous detection data, requiring causal diagnosis of the road traffic system.
L3 Predictable [ 24 , 25 ]Predicting the physical via the virtual (Decision Support)Numerous control strategies with high effect uncertainty, requiring decision support.
L4 Optimizable [ 13 , 26 ]Optimizing the physical via the virtual (Autonomic Control)Two-way data interaction, requiring autonomic closed-loop traffic control.
L5 Interactable ∗Actuating the physical via the virtual (Actionable Interaction)Bidirectional action-level interaction between physical and virtual entities, requiring direct human intervention in their control.
Note: Levels L1–L4 are adapted from Sun et al. [ 27 ] . Level L5 ( ∗ ) is newly proposed in this study.
Table 1: Digital Twin Levels of VICS
Figure 2: Schematic of I-VIT, IMPACT’s practical implementation. In the mixed testbed, the physical vehicles and infrastructure in the physical testbed (outlined in gold), the virtual vehicles and infrastructure in the virtual testbed (outlined in cyan), the virtual vehicle in the virtual environment of a driving simulator (outlined in red) and human operators in the HMI testbed operate synchronously and interact seamlessly across environments in real time. Consequently, I-VIT is an interactive VICS-DT testing environment, where human operators can directly interact with the physical and virtual entities in I-VIT. The demonstration videos are available at our project website: https://dongjh20.github.io/IMPACT .
Figure 3: System architecture of the IMPACT framework, consisting of the physical, virtual, mixed testing environments and the interactive environment.
Figure 4: Mechanisms for scenario visualization and operational intent processing via HMI devices and their corresponding implementation examples.
Figure 5: Illustration of data interactions within the IMPACT framework and its implementation, the I-VIT platform.
Figure 6: Illustration of the I-VIT architecture and interactions among its four components: physical, virtual, HMI, and mixed testbeds.
Figure 7: Physical and virtual traffic environments, vehicles and facilities. (a) The physical traffic environment, having a size of 9 m×5 m9\,\mathrm{m}\times 5\,\mathrm{m} . (b) The virtual traffic environment, developed based on the Unity game engine. (c) The serial channel-facility mapping table for physical roadside facilities. Blue denotes streetlights and traffic signals, whereas orange denotes barrier gates. (d) Physical and virtual vehicles and roadside facilities.
ComponentSetup
Physical sand table9 m×5 m9\,\mathrm{m}\times 5\,\mathrm{m} , 1:141:14 with respect to the real-world driving environment
Roadside sensors in the physical testbed16 identical cameras surround the physical sand table
Roadside facilities in the physical testbed7373 streetlights, 1111 traffic signals and 33 barrier gates
Scaled physical vehicles215 mm×190 mm×125 mm215\,\mathrm{mm}\times 190\,\mathrm{mm}\times 125\,\mathrm{mm} , powered by Raspberry Pi 4B
Host of the RSUCPU: Intel Core i9-14900KF, Memory: 32 GB, GPU: NVIDIA RTX 5080
Host of the virtual testbedCPU: Intel Core i7-10700K, Memory: 16 GB
Table 2: I-VIT Setups
Figure 8: Driving view and information board. In (a) and (b), the snapshots of physical and virtual views are presented respectively; see the screens for the specific view. In (c), the real-time speed of the preceding vehicle in the virtual driving view is displayed on the information board on the top of the preceding vehicle.
Table 3: Measurement of Communication Delays in I-VIT (Corresponding to Fig. 5(b) , Unit: ms\mathrm{ms} )
Figure 9: Illustration of the experimental setup.
Figure 10: Snapshots of the experiment. (a) Visualization of the entire platoon projected into the virtual space by Unity. The platoon configuration is presented in Fig. 9 . (b) A human operator controls the last vehicle (Vehicle 88 ) via the InnoSimulation simulator. The experimental videos are available at our project website: https://dongjh20.github.io/IMPACT .
Figure 11: Velocity and distance profiles of all vehicles in the experiments. Two braking disturbances are randomly applied to the head vehicle by the human operator through the MR HMD, which induces speed fluctuations throughout the entire platoon.

Explore similar work

CardsList

  1. Multisource human-in-the-loop digital twin testbed for connected and autonomous vehicles in mixed traffic flow

    Mar 18, 2026Jianghong Dong, Chunying Yang, Mengchi Cai +5Intelligent Transportation SystemsAutonomous Driving Simulation

    School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China. · Department of Civil and Environmental Engineering, University of Michigan, Ann Arbor, MI 48109, USA.

  2. VAST: V2X/Dynamic Map-Aware Autonomous Driving Systems Validation Toolchain

    Sep 17, 2026Shunsuke Ito, Takuya AzumiAutonomous Driving Safety EvaluationAutonomous Driving

    Graduate School of Science and Engineering Saitama University · Academic Association (Graduate School of Science and Engineering) Saitama University

  3. Closed-Loop Hybrid Digital Twin Platform for Connected and Automated Vehicle Validation

    May 19, 2026Kanglong Quan, Zhebing Xia, Linfeng Jiang +4Digital TwinsAutonomous Driving Simulation

    School of Advanced Technology Xi’an Jiaotong-Liverpool University, Jiangsu, China