Grip, Trace, Write: Force-Aware Imitation and Reinforcement Learning for High-DoF Robotic Handwriting
Organizations: Daxo Robotics
Abstract
Most robotic handwriting rigidly mounts the pen and reduces writing to trajectory tracking. A freely held pen instead couples grasp stability, tip motion, and contact force. We present Grip, Trace, Write, a force-aware simulation environment and learning framework for high-DoF handwriting. A two-branch actor separates pen retention from arm-wrist writing while preserving their closed-loop coupling. Training learns a grasp controller, initializes writing from inverse-kinematics demonstrations by behavior cloning, and improves writing through reinforcement learning with rewards on ink, force, time, and retention. Across diverse tasks, the policy improves completion, precision, force regulation, and efficiency over behavior cloning and inverse kinematics, and generalizes without fine-tuning to unseen English and Chinese content. These results establish force-aware handwriting as a testbed for dexterous tool use.
Figures & tables
| Controller | Metric | Primary tasks | Unseen | Mean | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Line | Circle | Shape | Digits | Letters | Word | Cursive | Chinese | word | Idiom | Chinese | |||
| IK | Completion (%) | 98.2 | 98.8 | 94.9 | 90.6 | 92.7 | 85.4 | 83.4 | 91.7 | 87.7 | 92.1 | 87.7 | 92.0 |
| Precision (%) | 95.7 | 96.4 | 82.6 | 75.8 | 82.2 | 71.5 | 61.6 | 78.7 | 71.1 | 82.1 | 71.7 | 80.6 | |
| Force MAE (N) | 0.558 | 0.537 | 0.529 | 0.531 | 0.533 | 0.582 | 0.526 | 0.497 | 0.585 | 0.580 | 0.524 | 0.537 | |
| Time (s) | 3.16 | 4.47 | 7.38 | 5.05 | 5.91 | 22.27 | 8.48 | 8.19 | 18.96 | 27.71 | 12.44 | 8.11 | |
| BC | Completion (%) | 96.7 | 97.6 | 93.6 | 90.2 | 92.5 | 85.7 | 82.4 | 91.5 | 87.2 | 91.6 | 86.9 | 91.3 |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Parameter | Hand and environment configuration |
|---|---|
| Hand and arm | High-DoF anthropomorphic robotic hand mounted on a UR5e assembly |
| Actuated coordinates | Seven arm coordinates, two wrist joints, and 25 finger joints |
| Physical action interface | Seven integrated arm-position increments; 27 speed–torque-limited wrist/finger duties |
| Control frequency | 25 Hz |
| Tool and initial grasp | Free rigid-body pen, initially seated in a tripod grasp; retained through contact |
| Grip assistance | No virtual grip assistance in the reported writing configuration |
| Parameter | Model, training, and evaluation configuration |
|---|---|
| Model architecture and observation processing | |
| Input representation | 137 state entries plus 16 trajectory samples of five entries each; samples every two trajectory indices; no image encoder |
| Actor | Separate grasp and writing MLPs, each with tanh activations; masked linear heads |
| Policy output | 34 physical-action entries plus one learned stop output; stop threshold 0.5 |
| Critic | Independent MLP |
| Normalization | Vector statistics frozen during final RL; normalized observations clipped to |
| Component | Training condition | Training and transfer |
|---|---|---|
| Grasp controller | 240 teacher episodes for BC, then pen-hold PPO under motion and contact | Transfer the selected learned grasp into the MLP |
| Writing data and BC | Traces, glyphs, words/cursive, and Chinese together | IK arm/wrist plus learned grasp supplies clean/noisy demonstrations; fit one all-task clone |
| Composed-policy RL | Full writing mixture with grasp and contact dynamics active | Both branches execute; refine outcomes while retaining grasp-mean weights |
| Weight | Value | Reward term |
|---|---|---|
| 1000 | Ordered target coverage | |
| 500 | Off-target ink | |
| 200 | Contact-force MAE | |
| 300 | Normalized elapsed time | |
| 5000 | Progress toward the guidance point |
| Task ID | Nominal duration | Requested behavior and configuration |
|---|---|---|
| L0: grasp training; arm/wrist motion supplied by the environment | ||
| L0_hold_static | 6 s | Retain the seated pen while the hand is still; no target ink strokes. |
| L0_hold_moving | 8 s | Retain the pen during imposed transport and wrist disturbance: 11 waypoints, motion scale 2.2, arm-motion box half-extents m, wrist shake amplitude . |
| L0_scribble | 10 s | Retain the pen during an environment-driven paper-contact path: 14 waypoints, 3 mm commanded press depth, wrist wobble, and 15 N impact limit. This is a contact-retention task, not glyph matching. |
| L1: geometric tracing | ||
| L1_trace_line | 8 s | A 60–120 mm straight line with randomized orientation and placement; flat force profile. |
| Field | Type / unit | Meaning |
|---|---|---|
| task_id , level | Identifier | Select the task family and level. |
| instruction | Text | Human-readable description. The policy MLP does not encode this string as language. |
| strokes | List of arrays, m | Ordered centerline points in paper coordinates relative to the sheet center. List boundaries distinguish pen-down strokes. |
| forces | List of arrays, N | Requested pressing force at each corresponding stroke point. |
| widths | List of arrays, mm | Target mark widths derived from the force arrays and the tool’s ink model. |
| duration | Seconds | Nominal schedule duration used to pace the requested strokes and transits. |
| Entry | Absolute request | Policy entry | Interpretation |
|---|---|---|---|
| 1 | Horizontal displacement, m | ||
| 2 | Horizontal displacement, m | ||
| 3 | Height error relative to the sheet, m | ||
| 4 | Requested force, N; not force error or measured contact force | ||
| 5 | Due-time offset, s; negative when the nominal due time has passed |
RL for Tool Use
Reinforcement learning for tool use trains agents to choose, execute, and learn from external tools through interaction and feedback.
See all 198 papers