Organizations: All authors are with the University of Illinois Urbana-Champaign, Urbana, IL, USA. · Zhexi Wang is also with Tsinghua University, Beijing, China.
Coding agents can translate language instructions into robot programs. For contact-rich manipulation, a key challenge is to generate and refine control logic that responds to physical interaction. We present FORCA, a coding agent that takes visual observations and wrist force/torque (F/T) measurements as input to write Python programs for contact probing and force-responsive motion. Between executions, the agent inspects force records alongside visual outcomes to diagnose failures and revise both motion and force-feedback control logic. FORCA requires no demonstrations, task-specific training, or simulation. Across five real-robot tasks, we demonstrate its effectiveness in three forms of contact-rich manipulation: improving manipulation accuracy, adapting to changes in object or surface properties, and regulating contact under surface variation and object motion. Compared with a vision-only variant of the same agent, FORCA improves grasp accuracy and task success and reduces contact loss in wiping and peeling.
Figures & tables
Fig. 1: Overview of Forca . The agent generates a task program that uses force feedback during execution. Recorded force, robot-state, and visual observations inform revisions between rollouts. Generated routines and notes are retained for later reuse.
Fig. 2: Predefined skill modules. The agent is initialized with these modules, assembled from open-source components under human supervision. (a) SAM 3 [ 31 ] localizes the tool named in the instruction. (b) GraspGen [ 32 ] proposes candidate grasps, and the VLM selects one. (c) PyRoKi [ 30 ] plans the trajectory. (d) The robot executes the resulting routine to acquire the tool.
Fig. 3: Illustrative program revision for cucumber peeling. The agent compares force records with visible peeling gaps and adds a condition that pauses forward motion until contact is restored. Photos are from real runs; force traces and skin patches are idealized, and the code is simplified.
Fig. 4: Example rollouts and outcome comparisons. (a) Drawer opening and (b) door-handle operation. Panels 1–3 show Forca estimating geometry visually, refining it through contact probing, and grasping to pull the drawer or turn the handle. In (a1), dashed and solid outlines mark the estimated pre-grasp and grasp poses; the axes indicate closing and approach directions. Numbered markers in (a2) identify three probe locations on the drawer knob. In (b2), orange markers indicate fingertip contacts, and blue shading marks the door-panel position confirmed by contact. Panels (a4) and (b4) compare example outcomes: the vision-only agent misses the drawer grasp and loses its grip on the door handle, while Forca opens both mechanisms. The dashed circle and red arrow in (b4) mark the fingertip-only grasp and the handle’s rebound after slipping.
Task
Method
Trans. err. (mm) ↓
Rot. err. ( ∘ ) ↓
Success (%) ↑
Drawer
Vision-only
8.3±5.4
9.6±3.2
40
Forca
5.3±1.6
1.0±0.3
100
Door
Vision-only
65.3±16.9
55.5±14.9
10
Forca
20.0±0.9
0.6±0.5
100
TABLE I: Grasp accuracy and task success for drawer opening and door-handle operation. Grasp errors: mean ± std. over 10 trials.
Fig. 5: Cube-to-target adaptation. (a) Object positions across successive attempts. The dashed outline marks the initial pose, and shading indicates the target region. (b) Object and surface configurations.
Fig. 6: Reuse across physical conditions. Distance to the target band per attempt (0: inside), starting from the program developed for the cube on the whiteboard.
Fig. 7: Vase wiping: (a) static vase and (b) zero-shot transfer to a hand-held vase. Each panel shows representative outcomes (top), execution frames (middle), and force traces (bottom). Both programs are refined in the static setting and transferred without further revision. As the vase moves, Forca maintains more consistent contact and high marker removal, whereas the vision-only program exhibits contact loss and large force variations, leaving substantial residual marks.
Vase
Method
Removal (%) ↑
Contact loss (%) ↓
Max. force (N) ↓
Static
Vision-only
73
42.6
11.8
Forca
99
0.7
11.3
Hand-held
Vision-only
21
55.9
17.5
Forca
93
1.6
12.3
TABLE II: Vase wiping results averaged over 10 trials.
Method
Contact loss (%) ↓
Excessive force (%) ↓
Peeling ratio (%) ↑
Vision-only
40.6
13.4
37.4
Forca
2.2
2.7
81.1
TABLE III: Peeling a hand-held cucumber (averaged over 10 trials)
Fig. 8: Cucumber peeling rollouts. (a) Hand-held setup and a magnified view of the boxed blade–surface contact. (b) Peeled strips after one representative pass, covering 90.6% of the intended length for Forca and 28.9% for the vision-only agent. (c) Execution frames at the marked instants. (d) Normal contact force. Shaded intervals indicate contact loss; the blue band marks the 4.5–7.5 N target range, and the dashed line marks the 30 N force limit.
Task
Iterations
Agent reasoning (min / iteration)
Robot motion (min / iteration)
Door handle
17
10
3
Drawer
8
5
1.5
Cube-to-target, from scratch
13
4
1
Cube-to-target, new condition
2
4
1
Vase wiping
20
4
1
Cucumber peeling
20
5
1.5
TABLE IV: Development iterations and mean time per iteration. For new cube configurations, the count is the mean number of revisions after the first execution, averaged over three configurations.
It.
Failed at
What happened
Revision
Forca
1–2
Approach
Camera put the door 3–4 cm too close; the probe missed it, then rejected a real touch.
Search farther; trust the touch and update the door depth.
3
Press down
First firm grasp, but at the 10 N force cap the handle turned only 18 ∘ , too little to unlatch.
Press along the handle’s arc about its axis.
4–5
Press down
Its own motion checks read slightly short moves as blocked; the handle never passed 5 ∘ .
Detect blocking by force; use the measured handle pose.
6–8
Press down
Still latched at 72.5 ∘ : grasp 21 mm and axis 15 mm off; handle twisted in the fingers.
Model the handle; press about a measured rotation axis.
9–10
Approach, press down
Setup faults: the cable plug, missing from the collision model, hit the door frame, and overly strict safety checks stopped motions; the press stopped at 49.7 ∘ .
Fix the setup; in the recorded press force, find latch release (61–65 ∘ ) and end stop (jump past 74.5 ∘ ); press to the stop.
TABLE V: Door-handle iterations of Forca and the vision-only agent.
Fig. 9: Door-handle development histories. Each marker denotes an execution at its failure stage or at task completion. Executions 18–20 repeat Forca ’s final program. Detailed failures and revisions are listed in Table V .
Contact-rich robotic manipulation studies how robots plan and control physical interactions such as grasping, insertion, pushing, and dexterous object handling.