KlaskTron-IL: A Contact-Rich Adversarial Imitation Learning Benchmark with a Validated Sim-to-Real Bridge

Nicklas Sohm; Aswin Ramachandran; Raffaello D'Andrea

cs.RONov 2, 2026

KlaskTron-IL: A Contact-Rich Adversarial Imitation Learning Benchmark with a Validated Sim-to-Real Bridge

Authors: Nicklas Sohm, Aswin Ramachandran, Raffaello D'Andrea

Organizations: ETH Zürich, IDSC

Abstract

KlaskTron is an open-hardware physical platform for two-player adversarial reinforcement learning, with a learned actuator model that supports zero-shot sim-to-real transfer of trained policies. Building on this platform, we contribute KlaskTron-IL: a multi-tier imitation learning dataset (80k human demonstration timesteps collected directly on the physical robot, 320k symmetry-augmented timesteps, and synthetic expert trajectories up to 5M steps generated by the zero-shot-validated PPO policy), and diagnostic baselines that expose three characteristic failure modes of standard IL methods under contact-rich adversarial dynamics: BC's data scaling plateau, GAIL's interpretable but weak-gradient discriminator (Spearman $ρ= -1.0$ peg-ball proximity recovery, but a logit span under 0.1), and an asymmetric BC-prior x reward-density interaction. Because the policy interface used throughout this paper is the same one the actuator model bridges to hardware, a policy trained on this benchmark can run unchanged on the physical robot, as we demonstrate zero-shot for the strongest BC baseline, so long as its commands respect the actuator model's $\pm 0.2$ m/s identification range.

Figures & tables

Panel of Figure 1 — Figure 1: KlaskTron-IL system architecture. Dashed blocks are inherited from the platform [ 7 ] . Solid blocks are contributions of this paper. The simulator and physical robot expose the same 12D state / 2D velocity interface, and a learned actuator model lets a single trained policy run unchanged in either. Demonstrations come from two sources: the inherited PPO expert acting in simulation (Tier 3, Section 4 ) and human play recorded on the physical robot (Tiers 1–2). BC learns offline from demonstrations alone, GAIL and BC+RL additionally optimize against the environment (Section 5 ).

Tier	Source	Timesteps	Episodes	Avg. Ep. Len.	Origin
1	Human (on robot)	80,591	111	$\sim$ 727	Lab tournament
2	Augmented	322,364	444	$\sim$ 727	Symmetry 4 $\times$
3	Synthetic (HW-validated policy)	5,000,000	34,649	$\sim$ 144	PPO self-play

Table 1: KlaskTron-IL dataset tiers. All tiers share the 12D state / 4D joint-action format used on the physical robot and are released under CC-BY-4.0. The synthetic tier ships as a single 5M-step file; the scaling study (Section 5.2 ) uses prefixes.

Method	Goals/Ep.	Player-in-Goal	HW-compat.	Diagnostic Failure Mode
BC (human, 80k)	0.102	0.157	✓ ∗	Data-scaling plateau: covariate-shift floor
GAIL	0.116	0.201	$\times$	Interpretable but flat discriminator
BC+RL (sparse)	0.138	0.130	$\times$	Asymmetric prior: stabilizes sparse RL
BC+RL (dense)	0.128	0.067	$\times$	Asymmetric prior: no detectable effect under dense shaping

Table 2: Diagnostic signature of each baseline. HW-compat. indicates that a method’s commands stay within the

\pm 0.2

m/s range over which the actuator model was identified [ 7 ] , so an improved baseline would be deployable without modification. GAIL and BC+RL optimize at

\pm 1.0

m/s, five times that range; their state/action interface matches, but their commands extrapolate beyond the identification data. ∗ 4.1% of BC’s commands marginally exceed the range (max 0.331 m/s). BC and GAIL metrics are self-play; BC+RL metrics are against the frozen PPO expert over five seeds.

Figure 2: Peg occupancy: BC rollout (left) vs. human demonstrations (right). The BC policy occupies wall regions; expert demonstrations do not.

Figure 3: BC data scaling on KlaskTron-IL. Performance sits roughly $0.015$ goal differential per episode below the expert across all four scales, with overlapping $95\,\%$ CIs ( $n=5$ seeds per scale). $60\,\times$ more demonstration data does not close the gap.

Figure 4: Discriminator probing. Top: logit vs. peg–ball distance (Spearman $\rho{=}{-}1.0$ ). Bottom: data coverage. The perfect rank ordering confirms feature discovery; the narrow logit range is consistent with the weak reward signal under this configuration.

Metric	BC $\leftrightarrow$ BC	GAIL $\leftrightarrow$ GAIL	GAIL $\leftrightarrow$ BC
Goals ratio	1.053 $\pm$ 0.278	1.137 $\pm$ 0.043	1.357 $\pm$ 0.154
Goals scored/ep	0.102 $\pm$ 0.013	0.116 $\pm$ 0.008	0.144 $\pm$ 0.007
Timeout rate	0.472 $\pm$ 0.024	0.309 $\pm$ 0.005	0.379 $\pm$ 0.023
Player-in-Goal	0.157 $\pm$ 0.006	0.201 $\pm$ 0.007	0.197 $\pm$ 0.016
Ep. length	146.8 $\pm$ 4.6	103.6 $\pm$ 1.7	128.0 $\pm$ 6.8

Table 3: Policy evaluation. Mean

\pm

sample SD across 3 random seeds of a single trained policy per method (1,000 episodes per repeat).

Figure 5: Termination modes across the $2{\times}2$ ablation, five training seeds per condition with 95% CIs, evaluated against the frozen PPO expert. BC initialization is a strong stabilizer under sparse rewards. Under dense reward the two initializations are indistinguishable.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Reward term	Weight
goal_scored	$+10.0$
goal_conceded	$-10.0$
player_in_goal	$-10.0$
opponent_in_goal	$-10.0$
distance_ball_opponent_goal	$+1.0$
collision_player_ball	$+0.5$

Appendix

Table 4: Dense reward terms and weights used in the BC+RL ablation.

Robot Imitation Learning

Robot imitation learning trains robots to perform tasks by reproducing and adapting behaviors shown in demonstrations.

See all 703 papers

KlaskTron: An Open-Source Platform for Physical Adversarial Multi-Agent RL
Nov 2, 2026Aswin Ramachandran, Jona Schulz, Maurus Derungs +3Embodied Agent Benchmarks Multi-Agent Reinforcement Learning
Institute for Dynamic Systems and Control (IDSC), ETH Zürich, Switzerland
Support-Constrained RL Enables Real-World Policy Improvement without Real-World Experience
Jun 25, 2026Raymond Yu, William Huey, Mustafa Mukadam +2Robotic Manipulation RL for Robotics
University of Washington · Amazon FAR
Real-IKEA: Physical Fidelity is the Prerequisite for Robust Manipulation
Jun 7, 2026Kunqi Xu, Zhenhao Huang, Siyuan Luo +2Contact-Rich Robotic Manipulation Robotics Simulation
Human-centered Robotic Lab, National University of Singapore · School of EECS, Peking University

Nov 2, 2026J/K move · Enter open · S save

KlaskTron: An Open-Source Platform for Physical Adversarial Multi-Agent RL

Aswin Ramachandran, Jona Schulz, Maurus Derungs, Carlo Angelini+2

Institute for Dynamic Systems and Control (IDSC), ETH Zürich, Switzerland

Embodied Agent Benchmarks Multi-Agent Reinforcement Learning Adversarial MARL Sim-to-Real Transfer

Open paper AskPDF arXiv

Progress in robot learning, particularly physical multi-agent reinforcement learning (MARL), is currently challenged by a limited availability of accessible, standardized benchmarks. While simulation-based MARL has produced remarkable emergent behaviors, from coordinated team play to complex tool use, translating these advances to physical systems remains difficult. A key barrier is infrastructural: physical platforms are often prohibitively expensive, require specialized facilities, or lack open mechanisms and designs for reproducibility. We introduce KlaskTron, an open-source, low-cost (<$2500 USD), desktop-scale robotic testbed for adversarial MARL based on the dynamic tabletop game KLASK. The platform features dual CoreXY gantries with high-torque brushless motors, enabling the high-speed, precise manipulation required for competitive play. Crucially, we provide a complete ecosystem: hardware designs (CAD, BOM), a GPU-native digital twin in NVIDIA Isaac Lab for high-fidelity modeling and simulation, and a validated sim-to-real baseline using a learned neural actuator model. We demonstrate that this baseline enables zero-shot policy transfer, with trained agents exhibiting emergent adversarial behaviors. To ensure full reproducibility of our results, all project assets are released publicly: hardware CAD/BOM and assembly documentation at [https://idscethzurich.github.io/klask_hardware], inference software stack at [https://github.com/IDSCETHZurich/klask_software] and the simulation and MARL setup at [https://github.com/IDSCETHZurich/klask_rl].

Abstract

Robot Imitation Learning

Explore similar work

KlaskTron: An Open-Source Platform for Physical Adversarial Multi-Agent RL

Support-Constrained RL Enables Real-World Policy Improvement without Real-World Experience

Real-IKEA: Physical Fidelity is the Prerequisite for Robust Manipulation