DexDICE: Reconciling Safety and Performance via Dual-Expert Offline Imitation Learning

Seokin Seo; Sung Kuk Shyn; Minji Seo; Yisoo Lee; Jongmin Lee; Hongseok Yang; Kee-Eung Kim

cs.LGNov 6, 2026

DexDICE: Reconciling Safety and Performance via Dual-Expert Offline Imitation Learning

Authors: Seokin Seo, Sung Kuk Shyn, Minji Seo, Yisoo Lee, Jongmin Lee, Hongseok Yang, Kee-Eung Kim

Organizations: Korea Institute of Science and Technology (KIST) · Korea Advanced Institute of Science and Technology (KAIST) · Hyundai Motor Company · Yonsei University ·

Abstract

Ensuring high returns while providing strong safety guarantees in offline imitation learning (IL) is a fundamental challenge: existing approaches either rely on explicitly specified cost functions, which are difficult to design for complex real-world safety constraints, or require expert demonstrations that are both safe and high-performing, which are often too costly to collect. We introduce dual-expert imitation learning, which leverages two complementary data sources: (i) safety-compliant demonstrations that faithfully follow safety guidelines but may be suboptimal, and (ii) performance-oriented demonstrations that achieve high returns but may violate safety guidelines. To address this problem, we propose DexDICE (Dual-expert offline imitation learning via stationary DIstribution Correction Estimation). Instead of conventional distribution constraints, DexDICE combines DICE-style offline learning with safe support constraints, enabling agents to exploit high-return behaviors while remaining within safety-compliant regions. We prove that the ideal solution satisfying this constraint exactly has zero unsafe-region visitation with high probability. We evaluate DexDICE on a re-curated DSRL benchmark and validate it on a real-world mobile-robot navigation task trained directly from human teleoperation demonstrations, where it substantially outperforms baselines and remains robust to data scarcity and quality degradation.

Figures & tables

Figure 1: (left) Taxonomy of safe sequential decision-making problems by available performance and safety signals. (middle) Dual-expert IL instance: $\mathcal{D}_{\mathrm{P}}$ prioritizes fast goal reaching but is possibly unsafe, while $\mathcal{D}_{\mathrm{S}}$ stays safe but possibly suboptimal. (right) Prior offline IL relies on divergence regularization (e.g., behavior regularization), biasing the policy toward $\mathcal{D}_{\mathrm{S}}$ ’s suboptimal behaviors ( red path); DexDICE instead imposes a support constraint from $\mathcal{D}_{\mathrm{S}}$ ( blue path), recovering $\mathcal{D}_{\mathrm{P}}$ ’s near-optimal route within the safe support.

Panel of Figure 2 — Figure 2: Data efficiency (a) and sample quality (b) on the safe demonstration set $\mathcal{D}_{\mathrm{S}}$ across methods, mirroring counterpart plots on the performant demonstration set $\mathcal{D}_{\mathrm{P}}$ (c), (d) respectively. Results are averaged over 6 tasks. Each subplot examines: (a) varying $N_{\mathcal{D}_{\mathrm{S}}}=2k$ , $k\in\{1,3,5,7,10\}$ , with $N_{\mathcal{D}_{\mathrm{P}}}=10$ fixed; (b) varying $\textbf{quality}(\mathcal{D}_{\mathrm{S}})\in\{5\%,25\%,50\%,75\%,95\%\}$ with $N_{\mathcal{D}_{\mathrm{P}}}=N_{\mathcal{D}_{\mathrm{S}}}=20$ and $\textbf{quality}(\mathcal{D}_{\mathrm{P}})=50\%$ fixed; (c), (d) the counterparts on $\mathcal{D}_{\mathrm{P}}$ with the roles of $\mathcal{D}_{\mathrm{S}}$ and $\mathcal{D}_{\mathrm{P}}$ exchanged. The green shaded region denotes the cost constraint-satisfying area.

Panel of Figure 3 — Figure 3: Real-world mobile-robot navigation. (a) Arena: start (green) to goal (red) avoiding two obstacles. (b) Per-method results over 10 trials. Success : reached goal without collision. Time-to-goal : on successful trials. Violation rate : fraction of timesteps within an unsafe distance to obstacles.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure A : An illustration on tabular experiment. (a) The agent must reach the goal while avoiding unsafe zones. (b) Each point shows the mean return and cost (averaged over 50 seeds) for a given demonstration quality (indicated by labels). The support constraint remains in the high return, low cost region across all settings, while the distribution constraint degrades when $d_{\mathrm{S}}$ is suboptimal.

Task	Obs. Dim	Act. Dim	Cost limit
Safe Navigation Tasks
SafetyPointCircle1	28	2	25
SafetyPointCircle2	28	2	25
Safe Velocity Tasks
SafetyHopperVelocity	11	3	25
SafetyWalker2dVelocity	17	6	25

Appendix

Table G: Descriptions of Tasks in Safety Gymnasium.

Figure B : Overall data efficiency averaged over six tasks of each method by jointly varying

k

, where

N_{\mathcal{D}_{\mathrm{P}}}=N_{\mathcal{D}_{\mathrm{S}}}=2k

Figure C : Per-task analysis on data efficiency with jointly varying

k

where

N_{\mathcal{D}_{\mathrm{P}}}=N_{\mathcal{D}_{\mathrm{S}}}=2k

Figure D : Per-task analysis on data efficiency of safe demonstrations

\mathcal{D}_{\mathrm{S}}

Figure E : Per-task analysis on data efficiency of performant demonstrations

\mathcal{D}_{\mathrm{P}}

Figure F : Per-task performance for dependencies on

\textbf{quality}(\mathcal{D}_{\mathrm{S}})

Figure G : Per-task performance analysis for dependencies on

\textbf{quality}(\mathcal{D}_{\mathrm{P}})

Figure H : Sensitivity analysis of DexDICE with respect to the target FNR

\kappa

used as the threshold for

\mathds{1}_{d_{\mathrm{S}}}

. Experiments are conducted on controlled datasets of the PointCircle1 task with

N_{\mathcal{D}_{\mathrm{P}}}=N_{\mathcal{D}_{\mathrm{S}}}=10

, over 5 random seeds.

Figure I : Stability under corruption of safe demonstrations. We evaluate on the controlled dataset of the PointCircle1 task, using a sufficient number of demonstrations (

N_{\mathcal{D}_{\mathrm{P}}}=200

N_{\mathcal{D}_{\mathrm{S}}}=200

). The x-axis indicates the corruption rate of

\mathcal{D}_{\mathrm{S}}

, i.e., the proportion of safe demonstrations replaced with unsafe demonstrations that exceed the episode cost upper bound. Both DexDICE and DemoDICE-S maintain relatively high episode return and low episode cost even under moderate corruption, while other methods degrade significantly as

\mathcal{D}_{\mathrm{S}}

becomes increasingly corrupted.

Dimensions ( $L\times W\times H$ )	$750\times 665\times 510$ mm
Drive	Skid-steer / omni-directional (experiments: omni-directional)
Max speed	$3$ m/s (experiments: $1.6$ m/s, $3$ rad/s)
LiDAR	Two SICK TIM571 2D scanners

Appendix

Table P: Specifications of the SUMMIT-XL-Steel.

Figure J : Visualization of per-trial trajectories for the five evaluated methods across

10

deployment trials each. Each colored polyline represents one rollout; obstacle boxes are shown in gray.

Imitation Learning

Imitation learning trains agents to perform tasks by learning from expert demonstrations, feedback, or observed behavior.

See all 275 papers

TAIL-Safe: Task-Agnostic Safety Monitoring for Imitation Learning Policies
May 2, 2026Riad Ahmed, Momotaz BegumSafe Imitation Learning Imitation Learning
University of New Hampshire
DexPIE: Stable Dexterous Policy Improvement from Real-World Experience
Jun 8, 2026Ruizhe Liao, Wenrui Chen, Liangji Zeng +4Robot Policy Learning Robotic Manipulation
Hunan University
DISEIL: Demonstration Distillation for Sample-Efficient Imitation Learning
Sep 8, 2026Suyog Khanal, Arun Kumar A, Santu RanaImitation Learning Interactive Imitation Learning
Deakin Applied Artificial Intelligence Initiative Geelong, Australia

May 2, 2026J/K move · Enter open · S save

TAIL-Safe: Task-Agnostic Safety Monitoring for Imitation Learning Policies

Riad Ahmed, Momotaz Begum

University of New Hampshire

Safe Imitation Learning Imitation Learning Robot Imitation Learning Runtime Assurance

Open paper AskPDF arXiv

Recent imitation learning (IL) algorithms such as flow-matching and diffusion policies demonstrate remarkable performance in learning complex manipulation tasks. However, these policies often fail even when operating within their training distribution due to extreme sensitivity to initial conditions and irreducible approximation errors that lead to compounding drift. This makes it unsafe to deploy IL policies in the field where out-of-distribution scenarios are prevalent. A prerequisite for safe deployment is enabling the policy to determine whether it can execute a task the way it was learned from demonstrations. This paper presents TAIL-Safe, a principled approach to identify, for a trained IL policy, a safe set from where the policy empirically succeeds in completing the learned task. We propose a Lipschitz-continuous Q-value function that maps state-action pairs to a long-term safety score based on three short-term task-agnostic criteria: visibility, recognizability, and graspability. The zero-superlevel set of this function characterizes an empirical control invariant set over state-action pairs. When the nominal policy proposes an action outside this set, we apply a recovery mechanism inspired by Nagumo's theorem that uses gradient ascent to the Q-function to steer the policy back to safety. To learn this Q-function, we construct a high-fidelity digital twin using Gaussian Splatting that enables systematic collection of failure data without risk to physical hardware. Experiments with a Franka Emika robot demonstrate that flow-matching policies, which fail under run-time perturbations, achieve consistent task success when guided by the proposed TAIL-Safe.

Abstract

Imitation Learning

Explore similar work

TAIL-Safe: Task-Agnostic Safety Monitoring for Imitation Learning Policies

DexPIE: Stable Dexterous Policy Improvement from Real-World Experience

DISEIL: Demonstration Distillation for Sample-Efficient Imitation Learning