Long-horizon contact-rich robotic manipulation remains challenging due to partial observability and unstable subtask transitions under contact uncertainty. While hierarchical architectures improve temporal reasoning and bilateral imitation learning enables force-aware control, existing approaches often rely on flat policies that struggle with long-horizon coordination. We propose Bi-HIL, a bilateral control-based multimodal hierarchical imitation learning framework for long-horizon manipulation. Bi-HIL stabilizes hierarchical coordination by integrating keyframe memory with subtask-level progress rate that models phase progression within the active subtask and conditions both high- and low-level policies. We evaluate Bi-HIL on unimanual and bimanual real-robot tasks, demonstrating consistent improvements over flat and ablated variants. The results highlight the importance of explicitly modeling subtask progression together with force-aware control for robust long-horizon manipulation. For additional material, please check: https://mertcookimg.github.io/bi-hil
Figures & tables
Fig. 1: Concept of Bi-HIL
Method
Hierarchy
Bilateral
Vision
Memory
Task Progress
HAMSTER [ 9 ]
✓
✗
✓
✗
✗
SARM [ 2 ]
✓
✗
✓
✗
✓
YaY Robot [ 7 ]
✓
✗
✓
✗
✗
MemER [ 6 ]
✓
✗
✓
✓
✗
Bi-ACT [ 18 , 19 , 15 ]
✗
✓
✓
✗
✗
Bi-LAT [ 20 ] , Bi-VLA [ 21 ]
✗
✓
✓
✗
✗
TABLE I: Comparison of Long-Horizon Robot Manipulation.
Fig. 2: Overview of Bi-HIL: Bilateral Control-Based Multimodal Hierarchical Imitation Learning via Subtask-Level Progress Rate and Keyframe Memory
Fig. 3: High-Policy Architecture
Fig. 4: Definition of Representative Keyframes and Subtask-Level Progress Rate
Fig. 5: Low-Policy Architecture
Fig. 6: Data Collection of Put-Three-Balls-In-Drawer Task
Graduate School of Information Science and Technology, The University of Osaka · D3 Center, The University of Osaka · Graduate School of Maritime Sciences, Kobe University