We introduce TAG (Testbed for Autonomous Games), a compact, low-cost robotic platform for studying learning and control in physically coupled dynamical systems. TAG combines an actively tilted and steered ball-in-maze board with a top-mounted rotary inverted pendulum. The pendulum's reaction torque and vibration disturb the board and marble, while the board's tilt, driven by the maze-solving policy, simultaneously disturbs the pendulum's balance. The main challenge in learning control policies for each subsystem is accounting for the impact of one subsystem's dynamics on the other's learning process. To address this challenge, we present complementary reinforcement learning paradigms, leveraging model-based and model-free approaches to learn the control policies for the maze and pendulum, respectively. Our key finding is that introducing the pendulum causes an immediate, sharp drop in the maze policy's success rate, but the policy adapts online and recovers to its pre-coupling performance within a small number of learning episodes. Experimental results show that both policies, trained independently, successfully control the maze board and the pendulum despite their physical coupling.
Figures & tables
Fig. 1: TAG: A T estbed for A utonomous G ames
Fig. 2: Rotary inverted pendulum.
Fig. 3: MuJuCo Simulator for Training Pendulum.
Fig. 4: Overview of the proposed method. The pendulum policy is trained in simulation with a model-free algorithm and deployed fixed on hardware; the maze policy is trained online, directly on the physical system, using DreamerV3. Camera images are processed locally into observations, sent to a remote server that runs DreamerV3 and updates the maze policy via a replay buffer, and returned as roll/pitch commands to the board. The two subsystems are physically coupled through the shared board, with the pendulum acting as a disturbance on the maze policy.
Parameter (unit)
Nominal
Range
Motor gain KM (N ⋅ m/V)
0.0127
[0.010,0.016]
Arm damping bϕ ( 10−4 N ⋅ m ⋅ s/rad)
9.4
[3.0,10.0]
Arm Coulomb friction (mN ⋅ m)
6.0
[4.0,8.0]
Pole damping bθ ( 10−5 N ⋅ m ⋅ s/rad)
3.4
[1.4,5.4]
Pole Coulomb friction (mN ⋅ m)
0.165
[0.065,0.265]
Pole inertia (scale)
1.00
[0.92,1.08]
TABLE I: Domain randomization for pendulum training
Fig. 5: Simulation results: success rates of the trained policy under 5 ms and 10 ms action delays across a range of board-disturbance conditions. Error bars indicate 95% Wilson confidence intervals.
Fig. 6: Real experimental results: success rates of the trained policy in real hardware across a range of board movements.
Fig. 7: Incremental training of the maze policy before and after pendulum activation. Blue and orange denote training with the inactive and active pendulum, respectively, and the shaded region marks the 69-episode ( ∼ 51 minutes) adaptation window under pendulum activation at episode 9,741.
the Robert Bosch Center for Cyber-Physical Systems, Indian Institute of Science, Bengaluru. · the Robert Bosch Center for Cyber-Physical Systems and the Department of Computer Science & Automation, Indian Institute of Science, Bengaluru.