Organizations: HRI2 Lab, Istituto Italiano di Tecnologia, Genoa, 16163, Italy · Ph.D. program of national interest in Robotics and Intelligent Machines (DRIM) and University of Genova, Genoa, 16126, Italy · School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an, 710049, China
Effective physical interaction control in robotic manipulation requires not only kinematically feasible motion but also sufficient force-interaction capability. Existing redundancy resolution methods often ignore task-specific force demands or maximize the force capability indiscriminately, sacrificing dexterity when large force margins are unnecessary. We propose a task-oriented force capability optimization framework for redundant mobile manipulators. A Vision-Language Model (VLM) infers object physical properties from an RGB image and a task description, generating a desired task-force sequence that captures gravitational and inertial demands. We then define a task-oriented force capability metric as the signed distance between a task-force uncertainty ball and the dynamic residual force polytope (RFP), quantifying compatibility between task demands and the robot's remaining actuation capacity. This metric is incorporated, alongside manipulability, joint-limit avoidance, trajectory smoothness, and base-oscillation suppression, into a whole-body multi-objective trajectory-optimization problem. Experiments on a mobile manipulator performing lifting and single-point-holding tasks under varying payload conditions demonstrate that the proposed method provides sufficient force capability for heavy loads while preserving high manipulability for light loads. This yields a task-adaptive balance that fixed capability-maximizing baselines (RFP inscribed radius, RFP cone) and manipulability-only optimization fail to achieve. The core implementation is publicly available at https://github.com/yeying256/FORTE.
Figures & tables
Fig. 1 : Overview of the proposed task-oriented force-capability optimization framework. Task-inefficient configurations may cause insufficient force capability for heavy payloads or limited manipulability for light payloads. The proposed framework estimates the task-force alignment, evaluates its compatibility with the robot’s residual force capability, and optimizes the whole-body configuration to achieve task-sufficient force capability while preserving manipulability.
Fig. 2 : Illustration of the construction of the dynamic Residual Force Polytope (RFP). The nominal torque τnom,k required to execute the trajectory is first computed from the joint position, velocity, and acceleration using inverse dynamics. The residual torque margins define the remaining admissible torque set Tr,k in joint space. By substituting the force-to-torque mapping τf=JvT(qk)f into the residual torque constraints, the joint-space residual actuation capability is mapped into the task-space dynamic residual force polytope Pr,k .
Fig. 3 : Overview of the proposed task-oriented force capability optimization framework. The framework consists of three main modules: VLM-based desired force estimation, force capability evaluation, and whole-body trajectory optimization. Given a task description D , an RGB image I , and a desired end-effector trajectory Xd , the VLM estimates task-related physical parameters and generates the desired force sequence Fc . The desired force at each trajectory point is modeled as the center of a task force uncertainty ball, while the Dynamic Residual Force Polytope (Dynamic RFP) characterizes the remaining force generation capability under the current robot configuration. The minimum signed distance between the task force uncertainty ball and the Dynamic RFP is used to construct the force capability cost. Finally, the force capability cost is jointly optimized with manipulability, joint-limit avoidance, trajectory smoothness, and base oscillation suppression using a constrained nonlinear programming solver to obtain the optimal whole-body trajectory.
Fig. 4 : Overview of the proposed desired force estimation module. Given a prompt-based task description D and an RGB image I , the VLM estimates the mass m^ of the manipulated object. Meanwhile, the desired end-effector trajectory Xd is processed by finite differences to obtain the end-effector acceleration wp¨e,k . The estimated mass and the trajectory-induced acceleration are then combined with gravity compensation to compute the desired task force wfkc=m^(wp¨e,k−wg) at each waypoint. The resulting desired force sequence Fc serves as the task requirement representation for the subsequent force capability evaluation and trajectory optimization.
Fig. 5 : Geometric illustration of the proposed force capability metric based on the relative position between the task force uncertainty ball and the Dynamic Residual Force Polytope (Dynamic RFP). The red circle denotes the task force uncertainty ball Bk , whose center is the desired task force wfkc and whose radius is rk . The blue polygon represents the Dynamic RFP Pr,k , which characterizes the available residual force generation capability at the k -th waypoint. (a) The task force ball is fully contained inside the Dynamic RFP, yielding a positive force capability margin dk>0 . (b) The task force ball is tangent to the boundary of the Dynamic RFP, corresponding to the critical case dk=0 . (c) The task force ball extends outside the Dynamic RFP, indicating an insufficient force capability margin with dk<0 .
Method
mALDE ↓
mAPE ↓
rls2↑
mMnRE ↑
q↑
Image2Mass (single-view)
0.465
0.685
0.691
0.675
0.766
Image2Mass (multi-view)
0.437
0.637
0.701
0.693
0.786
Ours (single-view)
0.333
0.377
0.839
0.742
0.917
Table 1: Mass estimation performance on the Household Test Set (56 items, 423 images). The Image2Mass results are taken from [ 15 ] .
Parameter
Position trajectory
Task force
Offset c
0
[25,15,−65]TN
Amplitude a
[0.10,0.10,−0.10]Tm
[80,60,80]TN
Frequency f
[0,0.20,0.10]THz
[1/6,1/4,1/12]THz
Phase ϕ
[π/2,0,π/2]T
[0.5,π/2,π/4]T
Duration
T=12s
Table 2: Parameters of the continuous trajectory and task-force simulation.
Fig. 6 : Directional force capability and arm translational manipulability under continuously varying task-force demands. Solid and dashed curves represent the RFP directional capability and manipulability, respectively. The black curve denotes the magnitude of the desired task force, and the shaded region represents the force uncertainty.
Fig. 7 : Overview of the experimental setup for the lifting and single-point holding experiments, including the mobile manipulator, payload conditions, and compared methods. The end-effector follows the indicated Cartesian trajectory in the lifting experiment and maintains a fixed desired Cartesian pose in the single-point holding experiment.
Fig. 8 : Performance comparison of different planning methods in the 2.7 L and 0.33 L beverage lifting task. The upper plot shows the directional dynamic residual force polytope capacity, defined as the force magnitude at the intersection between the ray along the desired force direction and the boundary of the dynamic residual force polytope. The lower plot shows the arm manipulability. All curves report the mean over three trials, the shaded regions denote the standard deviation, and the orange band represents the uncertainty range of the desired task force.
Fig. 9 : Single-point comparison of force output capability and manipulability under different payload conditions. The left vertical axis represents the directional residual force polytope capacity and the desired force range, while the right vertical axis represents the manipulability.
Fig. 10 : Comparison of real-world single-point optimization results obtained by different optimization methods under 0.33 L and 2.7 L payload conditions. All methods optimize the same end-effector pose, while producing different robot configurations due to their respective optimization objectives.
Fig. 11 : Single-point comparison under different payload conditions. Each column corresponds to one optimization method, and each row corresponds to one beverage load.
Fig. 12 : Single-point comparison of manipulability ellipsoids under different payload conditions. Each column corresponds to one optimization method, and each row corresponds to one beverage load.
Force-controlled robotic manipulation studies how robots regulate contact forces and compliance to grasp, assemble, and work safely with objects and surfaces.
Embodied intelligence for contact-rich manipulation has predominantly relied on position control, while explicit awareness and regulation of interaction forces remain under-explored, limiting stability, precision, and robustness in real-world tasks. We propose ForceVLA2, an end-to-end vision-language-action framework that equips robots with hybrid force-position control and explicit force awareness. ForceVLA2 introduces force-based prompts into the VLM expert to construct force-aware task concepts across stages, and employs a Cross-Scale Mixture-of-Experts (MoE) in the action expert to adaptively fuse these concepts with real-time interaction forces for closed-loop hybrid force-position regulation. To support learning and evaluation, we construct ForceVLA2-Dataset, containing 1,000 trajectories over 5 contact-rich tasks, including wiping, pressing, and assembling, with multi-view images, task prompts, proprioceptive state, and force signals. Extensive experiments show that ForceVLA2 substantially improves success rates and reliability in contact-rich manipulation, outperforming pi0 and pi0.5 by 48.0% and 35.0%, respectively, across the 5 tasks, and mitigating common failure modes such as arm overload and unstable contact, thereby actively advancing force-aware interactive physical intelligence in VLAs. The project page is available at https://sites.google.com/view/force-vla2/home.
Yang Li, Zhaxizhuoma, Hongru Jiang +11
Tongji University · Shanghai AI Laboratory · Lumos Robotics +6
Robotic foundation models (RFMs) are increasingly capable of general-purpose manipulation, yet reliable physical interaction remains challenging in contact-rich settings. We present FP2, a lightweight downstream interface that equips task-adapted RFMs with explicit force control while preserving their action-generation capability. FP2 adopts an action-regulation decomposition: the task-adapted RFM serves as a foundation policy responsible for task-level action generation, while a high-frequency force control policy focuses solely on interaction regulation. To condition force regulation on the ongoing manipulation, FP2 compresses foundation-policy contextual representations and combines them with wrench and proprioceptive histories to predict structured force-control parameters. We evaluate FP2 with four RFM backbones across four real-world contact-rich manipulation tasks. FP2 consistently improves task performance and force regulation quality over the corresponding foundation policies, while comparing favorably with representative force-aware and force-control baselines. Ablations further show that foundation-policy context and physical feedback are complementary for effective force regulation, while preserving foundation-policy action generation improves both efficiency and novel-object generalization. Project website: http://force-policy.github.io/fp2
Humanoid robots are expected to perform diverse human-level tasks in daily environments, many of which require precise regulation of interaction forces. While recent vision-language-action (VLA) models have shown promise for semantic planning and visuomotor control, existing humanoid systems primarily represent actions through geometric motion goals and rely on whole-body controllers focused on motion tracking, with limited explicit reasoning or control of interaction forces. This limitation is particularly relevant in contact-rich tasks, where geometrically similar motions may require different force regimes depending on the task context and where visual observations may become unreliable after contact. In this work, we present Opt2VLA, a force-aware VLA framework that introduces explicit force commands at the VLA-to-control interface for humanoid whole-body manipulation. A single multi-task VLA policy jointly predicts both geometric motion goals and continuous contact-force references, which are tracked by task-specific reinforcement learning (RL)-based whole-body controllers. To provide scalable and physically grounded supervision, we generate dynamically feasible and contact-consistent training data via whole-body trajectory optimization (TO) with explicit force references. We evaluate Opt2VLA on three contact-rich humanoid tasks and show that explicit force conditioning enables more accurate and consistent force regulation than motion-only control, while physically grounded torque supervision from TO further improves force tracking accuracy and stability. Closed-loop evaluations further demonstrate language-conditioned force modulation with Opt2VLA in simulation and on humanoid hardware.
Fukang Liu, Yipu Chen, Jaehwi Jang +3
Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, GA, USA