Tutorial: Vision-Based Formation Control with Differential-Drive Robots
Organizations: SMART Laboratory Luddy School of Informatics, Computing, and Engineering Indiana University, Bloomington, IN 47405, USA
Abstract
This tutorial presents a step-by-step introduction to multi-robot formation control using differential- drive robots, an overhead camera, and Python. ArUco markers are used to estimate robot positions and orientations, while desired pairwise displacement vectors define a circular formation. By comparing measured and desired relative positions, the controller computes movement vectors that are converted into differential-drive wheel commands, with basic proximity-sensor-based obstacle avoidance. The tutorial guides readers through hardware and software setup, robot pose estimation, formation geometry, feedback control, system integration, and formation-error evaluation. Motivated by the need to bridge the gap between theoretical understanding and practical implementation in robotics education, we emphasize how mathematical concepts are translated into coordinated behaviors on physical robots. Although the implementation uses a specific robotic platform, the underlying perception and formation-control principles can be adapted to other differential-drive robots and simulation environments. The accompanying open-source GitHub repository provides the Python implementation, setup instructions, and demonstrations to support hands-on learning and further exploration of multi-robot systems.
Figures & tables
| File | Role |
|---|---|
| main.py | Initializes robots and executes the main control loop. |
| settings.py | Stores marker IDs, formation parameters, gains, and thresholds. |
| utils/camera_supervisor.py | Detects markers and updates robot positions and orientations. |
| utils/formation_control.py | Constructs formation slots and desired relative displacements. |
| utils/actuation.py | Converts a control vector into wheel commands and checks proximity sensors. |
| utils/img_overlay.py | Provides optional pose, vector, and formation-error visualization. |
| Symptom | Checks and adjustments |
|---|---|
| Camera does not open | Try another camera index, such as cv2.VideoCapture(1) , and check that another program is not using the camera. |
| No markers detected | Check the dictionary, marker IDs, lighting, focus, marker size, and visibility of the complete marker border. |
| Robot moves incorrectly | Check image coordinates, marker mounting direction, wheel signs, and the angular-error calculation. Both measured and target headings must use degrees and the same image frame. |
| Robot turns the wrong way | Check the platform’s left- and right-wheel sign conventions. Adapt the wheel mapping in apply_control() rather than changing the formation-error equations. |
| Robots oscillate near the formation | Reduce wheel speed or increase the stopping threshold. Very tight heading tolerance can also produce repeated turning. |
Multi-Robot Formation Control
Multi-robot formation control coordinates robots to maintain, change, or navigate in groups while meeting task and safety constraints.
See all 62 papers