Learning Functional Specialisation for Bimanual Robotic Control

Jingwen Li; Levin Kuhlmann; Jason Friedman; Gideon Kowadlo

cs.ROOct 24, 2026

Learning Functional Specialisation for Bimanual Robotic Control

Authors: Jingwen Li, Levin Kuhlmann, Jason Friedman, Gideon Kowadlo

Organizations: Department of Data Science and AI, Monash University · Department of Physical Therapy & Sagol School of Neuroscience, Tel Aviv University · Cerenaut

Abstract

Bimanual robotic control requires simultaneous management of distinct functional demands under motor noise, energetic costs, and communication delays. We investigate how learning responsibility should be organised across modular controllers and the two arms in a simulated musculoskeletal system with two communicating recurrent controllers. Using a 2 x 2 design, we compare differential versus shared controller objectives and fixed versus balanced hand-task assignments. Task-aligned functional specialisation was strongest when differential objectives were combined with a fixed assignment aligned with contralateral control routing. This condition achieved the highest canonical-task performance and control efficiency, reaching 81.33% bimanual success. Its advantage over shared objectives increased with motor noise but narrowed under stronger energy penalties. Stronger specialisation also reduced hand-role flexibility: reversing the hand roles decreased bimanual success by 80.46 percentage points under Differential-Fixed training, compared with 1.51 percentage points under Shared-Balanced training. These results show that controller objectives and hand-task experience jointly determine functional organisation. Stronger role-specific specialisation favours performance and efficiency when roles remain stable, whereas more distributed training favours interchangeability when roles change.

Figures & tables

Figure 1: (A) Bimanual task and musculoskeletal effectors. One hand stabilises a fixed target against external force perturbations (left) while the other tracks a continuously moving target (right). The inset shows the MotorNet RigidTendonArm26 effector used for each arm, a planar two-degree-of-freedom model actuated by six Hill-type muscles. Two identical effectors are arranged symmetrically in the workspace. (B) Bilateral modular controller architecture. Two GRU-based controllers receive shared sensory observations, exchange information through delayed bidirectional connections, and contribute motor commands to the two arms. Motor commands are subject to signal-dependent noise, with motor-command magnitude determining the energetic penalty during training.

Figure 2: Experimental design for learning responsibility. (A) Training-objective structure. Differential training routes tracking and holding losses separately to the two controllers, whereas Shared training applies the combined functional loss to both controllers, with the energy term common to both schemes. (B) Task-assignment structure. Fixed training maintains the same hand–task mapping throughout training, whereas balanced training presents right-tracking/left-holding and left-tracking/right-holding mappings equally across trials.

Figure 3: Functional specialisation across task assignments and training objectives. (A) Role alignment across training conditions, where larger positive values indicate stronger alignment with the assigned tracking and holding roles, near-zero values indicate little role-aligned preference, and negative values indicate reversed alignment. (B) Functional separation across training conditions, where larger values indicate greater separation between controller task-preference distributions. Large markers show five-seed means. Small markers show seed means across 64 control conditions, with horizontal jitter for visibility. Error bars denote 95% Student’s $t$ confidence intervals across seeds.

Metric	Differential-Fixed	Shared-Fixed	Differential-Balanced	Shared-Balanced
Bimanual SR $\uparrow$	81.33 $\pm$ 3.93%	71.10 $\pm$ 6.69% ∗	28.03 $\pm$ 7.45% ∗	41.36 $\pm$ 4.92% ∗
Tracking SR $\uparrow$	92.31 $\pm$ 1.88%	89.28 $\pm$ 2.35% ∗	64.33 $\pm$ 4.17% ∗	84.38 $\pm$ 2.16% ∗
Holding SR $\uparrow$	87.08 $\pm$ 4.67%	76.77 $\pm$ 7.81% ∗	45.05 $\pm$ 10.03% ∗	46.82 $\pm$ 5.69% ∗
Trajectory error (mm) $\downarrow$	13.45 $\pm$ 0.60	14.00 $\pm$ 0.42 ∗	20.29 $\pm$ 2.70 ∗	15.12 $\pm$ 0.42 ∗
Holding error (mm) $\downarrow$	6.79 $\pm$ 0.60	7.99 $\pm$ 0.63 ∗	11.63 $\pm$ 0.80 ∗	10.81 $\pm$ 0.34 ∗
Mean control energy $\downarrow$	0.550 $\pm$ 0.135	0.683 $\pm$ 0.171 ∗	0.510 $\pm$ 0.093	0.647 $\pm$ 0.120 ∗

Table 1: Bimanual performance and control efficiency under the right-tracking/left-holding mapping. Values are mean

\pm

standard deviation across five means, each averaged over 64 control conditions defined in Section 3.5 . ∗ denotes a significant difference from Differential-Fixed after Holm correction (

p_{\mathrm{Holm}}<0.05

). Arrows indicate the preferred direction of each metric.

Figure 4: Bimanual performance and control efficiency across control conditions. (A) Bimanual SR under the canonical mapping. (B) Energy per success (log scale). Points show five-seed means for each of 64 control conditions. Horizontal and vertical bars indicate the median and interquartile range across conditions. (C) Differential–Shared differences in signed role alignment and Bimanual SR as (C) motor-noise gain or (D) energy penalty increases, averaging over the remaining factors. Positive $\Delta$ values indicate higher values under Differential training.

Figure 5: Bimanual performance under control constraints. (A) Success rate across motor-noise gain for four training conditions. (B) Differential–Shared Bimanual SR differences under fixed and balanced assignment. (C) Differential–Shared differences across noise and energy penalty under (C1) fixed and (C2) balanced assignment. Results average over undisplayed factors, with contrasts computed within matched conditions and seeds. A–B error bars show 95% Student’s $t$ confidence intervals across seeds. Positive differences favour Differential training.

Figure 6: Hand-role flexibility across changing task demands without retraining. (A) Bimanual SR under canonical and reversed hand–task mappings. (B) Hand-specific tracking and holding success rates when both hands perform the same function. (C) Perturbed-tracking success rate when either hand tracks under external perturbation. Means are averaged over 64 control conditions. Panels A and C show 95% Student’s $t$ confidence intervals across five seeds.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Condition	Batch composition	Training objective
Differential-Fixed	128 canonical	$\theta_{L}:L_{T}+\lambda E,\quad\theta_{R}:L_{H}+\lambda E$
Differential-Balanced	64 canonical + 64 reversed	$\theta_{L}:L_{T}+\lambda E,\quad\theta_{R}:L_{H}+\lambda E$
Shared-Fixed	128 canonical	$\theta_{L},\theta_{R}:L_{T}+L_{H}+\lambda E$
Shared-Balanced	64 canonical + 64 reversed	$\theta_{L},\theta_{R}:L_{T}+L_{H}+\lambda E$

Appendix

Table 2: Training implementation for the learning-responsibility conditions. Canonical denotes right-hand tracking and left-hand holding. Reversed denotes left-hand tracking and right-hand holding.

Training condition	Bimanual SR	Signed role alignment	Functional separation
Differential-Fixed	$81.33\ [76.45,86.20]$	$0.0811\ [0.0719,0.0904]$	$0.190\ [0.177,0.203]$
Shared-Fixed	$71.10\ [62.80,79.40]$	$0.0292\ [0.0201,0.0384]$	$0.120\ [0.110,0.130]$
Differential-Balanced	$28.03\ [18.78,37.28]$	$-0.0004\ [-0.0076,0.0068]$	$0.133\ [0.119,0.147]$
Shared-Balanced	$41.36\ [35.25,47.47]$	$0.0347\ [0.0204,0.0490]$	$0.118\ [0.101,0.135]$

Appendix

Table 3: Performance and controller organisation across the complete constraint grid. Values are mean [95% confidence interval] across five seed-level grid means. Each seed-level value averages the same 64 constraint settings. Bimanual SR is reported as a percentage.

		$\Delta$ Tracking SR (pp)				$\Delta$ Holding SR (pp)
Model	Lesioned controller	$\gamma=0$	$\gamma=0.95$	$\gamma=0.975$	$\gamma=0.99$	$\gamma=0$	$\gamma=0.95$	$\gamma=0.975$	$\gamma=0.99$
Differential-Fixed	Left	$-92.31\pm 1.88$	$-19.87\pm 8.33$	$-6.17\pm 3.01$	$-0.89\pm 0.91$	$-86.13\pm 4.26$	$-7.38\pm 1.36$	$-1.43\pm 0.71$	$-0.30\pm 0.19$
	Right	$-91.57\pm 1.70$	$-10.78\pm 6.14$	$-3.41\pm 2.77$	$-0.24\pm 0.23$	$-87.08\pm 4.67$	$-19.94\pm 2.62$	$-5.74\pm 1.67$	$-0.97\pm 0.63$
Shared-Fixed	Left	$-89.28\pm 2.35$	$-6.14\pm 2.84$	$-0.92\pm 0.71$	$-0.12\pm 0.32$	$-66.03\pm 4.67$	$-0.14\pm 0.24$	$+0.02\pm 0.23$	$+0.08\pm 0.19$
	Right	$-86.33\pm 2.83$	$-0.48\pm 0.45$	$-0.05\pm 0.25$	$+0.08\pm 0.16$	$-76.77\pm 7.81$	$-5.59\pm 1.36$	$-1.63\pm 0.43$	$-0.45\pm 0.25$
Differential-Balanced	Left	$-64.33\pm 4.17$	$-40.68\pm 7.19$	$-26.06\pm 3.91$	$-10.50\pm 1.97$	$-45.05\pm 10.03$	$-36.38\pm 9.11$	$-27.51\pm 8.24$	$-12.93\pm 3.16$

Appendix

Table 4: Graded controller lesion effects on canonical-task success rates. Values are mean

\pm

SD across five seed-level means, with each seed averaged equally over the 64 control settings. Changes are calculated as lesion minus intact in percentage points. Retained gain

\gamma

scales the lesioned controller’s outgoing influence, and negative values indicate reduced success after lesion.

Metric	Shared-Fixed	Differential-Balanced	Shared-Balanced
Bimanual SR	$<0.001$	$<0.001$	$<0.001$
Tracking SR	$0.001$	$<0.001$	$<0.001$
Holding SR	$<0.001$	$<0.001$	$<0.001$
Trajectory error	$0.004$	$<0.001$	$<0.001$
Holding error	$<0.001$	$<0.001$	$<0.001$
Mean control energy	$<0.001$	$0.162$	$<0.001$

Appendix

Table 5: Holm-adjusted p-values from paired Monte Carlo permutation tests for Differential-Fixed versus each alternative across the 64 prespecified constraint settings. Tests assess systematic condition differences across the fixed constraint grid and are two-sided. Correction was applied jointly across all 21 comparisons.

Metric	Delay $d$	Mean	Difference from $d=0$ [95% CI]	$p_{\mathrm{H}}$
Bimanual SR (%)	0	50.38	Reference	–
	5	53.57	$+3.19$ [ $-2.86$ , $9.24$ ]	0.434
	10	56.32	$+5.94$ [ $4.55$ , $7.32$ ]	0.003
	20	61.55	$+11.16$ [ $5.53$ , $16.80$ ]	0.037
Signed role alignment	0	0.0502	Reference	–
	5	0.0312	$-0.0190$ [ $-0.0337$ , $-0.0043$ ]	0.116

Appendix

Table 6: Performance and functional specialisation by communication-delay level. Differences compare each nonzero delay with

d=0

. Bimanual SR differences are reported in percentage points.

p_{\mathrm{H}}

denotes Holm-adjusted

p

-values.

Delay	Noise	Fixed assignment	Balanced assignment
(steps)	$k$	$\Delta$ SR [95% CI]	$\Delta$ SR [95% CI]
0	0	$6.8$ [ $1.0$ , $12.5$ ]	$-30.4$ [ $-54.6$ , $-6.1$ ]
0	0.05	$13.0$ [ $-11.0$ , $37.0$ ]	$-22.1$ [ $-45.1$ , $0.9$ ]
0	0.1	$16.5$ [ $9.3$ , $23.8$ ]	$-5.1$ [ $-17.5$ , $7.2$ ]
0	0.2	$20.4$ [ $4.7$ , $36.2$ ]	$4.3$ [ $-18.3$ , $26.9$ ]
5	0	$1.6$ [ $-0.8$ , $4.1$ ]	$-29.7$ [ $-52.2$ , $-7.2$ ]

Appendix

Table 7: Delay-stratified Differential

-

Shared Bimanual SR contrasts in percentage points.

ID	Task demands	Success criterion
M1	Right hand tracks, left hand holds under a pulse (main task)	Tracking whole-episode mean error $\leq 20$ mm and holding pulse-and-recovery mean error $\leq 10$ mm in the same trial
M2	Left hand tracks, right hand holds under a pulse	Tracking whole-episode mean error $\leq 20$ mm and holding pulse-and-recovery mean error $\leq 10$ mm in the same trial
S1	Both hands track independent moving targets	Both hands’ whole-episode mean tracking errors $\leq 20$ mm
S2	Both hands hold fixed targets under external pulses	Both hands’ pulse-and-recovery mean holding errors $\leq 10$ mm
P1	Left hand tracks and receives a pulse	Left-hand mean tracking error $\leq 20$ mm over both the full episode and pulse-and-recovery interval
P2	Right hand tracks and receives a pulse	Right-hand mean tracking error $\leq 20$ mm over both the full episode and pulse-and-recovery interval

Appendix

Table 8: Definitions of the six hand–task configurations. Success criteria follow the behavioural metrics defined in Appendix B.2 .

ID	Metric	Differential- Fixed	Shared- Fixed	Differential- Balanced	Shared- Balanced
M1	Bimanual SR	$81.33\ [76.45,86.20]$	$71.10\ [62.80,79.40]$	$28.03\ [18.78,37.28]$	$41.36\ [35.25,47.47]$
M2	Bimanual SR	$0.87\ [-0.41,2.14]$	$1.28\ [0.34,2.22]$	$1.07\ [0.70,1.45]$	$39.85\ [35.87,43.82]$
S1	Bimanual SR	$4.89\ [1.39,8.39]$	$9.78\ [5.21,14.34]$	$11.65\ [5.90,17.41]$	$63.52\ [58.81,68.22]$
S1	Left-hand SR	$5.94\ [1.72,10.17]$	$12.65\ [7.07,18.23]$	$19.51\ [11.28,27.74]$	$80.48\ [76.27,84.68]$
S1	Right-hand SR	$71.68\ [58.84,84.53]$	$69.34\ [62.89,75.80]$	$48.11\ [44.02,52.21]$	$78.01\ [74.43,81.59]$
S2	Bimanual SR	$36.67\ [25.80,47.54]$	$32.59\ [29.04,36.13]$	$9.70\ [6.68,12.71]$	$36.05\ [27.99,44.11]$

Appendix

Table 9: Performance across the six hand–task configurations. Values are success rates in percent, reported as mean [95% confidence interval] across five seed-level grid means.

Contrast	Training condition	Difference [95% CI]	$p_{\mathrm{H}}$
M1 $-$ M2	Differential-Fixed	$80.46$ [ $75.56$ , $85.36$ ]	$<0.001$
	Shared-Fixed	$69.82$ [ $60.83$ , $78.80$ ]	$<0.001$
	Differential-Balanced	$26.96$ [ $17.67$ , $36.25$ ]	$0.009$
	Shared-Balanced	$1.51$ [ $-3.29$ , $6.32$ ]	$1.000$
S1 right $-$ left	Differential-Fixed	$65.74$ [ $52.42$ , $79.06$ ]	$0.002$
	Shared-Fixed	$56.69$ [ $47.25$ , $66.13$ ]	$<0.001$

Appendix

Table 10: Within-condition hand-role asymmetry contrasts. Differences are reported in percentage points as mean [95% confidence interval].

p_{\mathrm{H}}

denotes the Holm-adjusted

p

-value across the 16 prespecified tests.

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 584 papers

Sep 20, 2026

BiRoAD: Learning Shared and Role-Adaptive Representations for Bimanual Manipulation

Bimanual manipulation requires policies that coordinate two arms while adapting their functional roles to scene geometry, object configuration, and task context. Learning such scene-conditioned role adaptation remains challenging, as demonstrations may contain uneven role distributions that limit generalization to underrepresented arm--role configurations. In addition, many bimanual policies predict actions in fixed left- and right-arm action spaces. While this provides a natural parameterization for robot control, it does not explicitly specify how behaviors should transform when functional roles are exchanged across arms. Across different scene initializations, the two arms may follow a similar coordination pattern, but the role-specific behavior assigned to each arm should change with the scene. Therefore, we propose BiRoAD, a Bimanual Role-Adaptive Decomposition framework for learning shared and role-adaptive representations in bimanual policies. Given bimanual trajectory or action-token features, BiRoAD decomposes these features into swap--symmetric and swap--antisymmetric components: the former captures coordination structure invariant to arm exchange, and the latter captures role-specific distinctions that vary consistently with functional role assignment. The two components are then recomposed as residual updates to the original paired arm representations, allowing BiRoAD to serve as a modular feature transformation without changing the policy inputs, imitation-learning objective, or requiring manually defined role labels. Across multiple bimanual manipulation tasks with balanced and imbalanced role distributions, BiRoAD improves robustness across role configurations over corresponding base policies, with notable gains on underrepresented role configurations.

Yan Shen, Yuchen Liu, Feng Jiang +5

CFCS, School of Computer Science, Peking University

Representation Learning Bimanual Robotic Manipulation Generalization in Robotic Manipulation Robot Imitation Learning Motion Representation Learning Equivariant Representation Learning

Ask PDF

Aug 14, 2026

Communication in modular robotic motor control: Bilateral controllers under realistic constraints

Robotic motor control in musculoskeletal systems requires fast, accurate movement and robust postural stabilization under signal-dependent noise (where motor command variance scales with command magnitude) and energetic cost. Modular controllers can distribute these competing demands across interacting submodules, but it remains unclear whether they outperform monolithic architectures under realistic constraints, and how inter-module communication shapes the resulting strategy. Inspired by the bilateral hemispheric organization of the brain, we introduce a recurrent controller of two GRU-based modules connected by a learnable, delayed inter-hemispheric channel, trained end-to-end in a differentiable two-arm musculoskeletal simulator. Across reaching and holding tasks, the modular architecture substantially outperforms a capacity-matched monolithic baseline. Compared to a matched modular controller without communication, learned inter-hemispheric communication reshapes the solution: improved endpoint precision, lower energetic cost in non-zero-delay regimes, and reduced muscle co-contraction. Our findings show that for robotics, biologically inspired modular controllers offer a practical route to robust movement under noise and energetic constraints, with inter-module communication providing a mechanism to tune trade-offs between precision, stability, and actuation cost.

Jingwen Li, Levin Kuhlmann, Jason Friedman +1

Department of Data Science and AI, Monash University · Department of Physical Therapy & Sagol School of Neuroscience, Tel Aviv University · Cerenaut

Robotic Control Robotics Modular Robotics Modular Neural Networks

Ask PDF

Jun 11, 2026

See Selectively, Act Adaptively: Dual-Level Structural Decomposition for Bimanual Robot Manipulation

In bimanual robotic manipulation, task-relevant visual information varies with the task stage and context, while the interaction of the two arms shifts between independent and coordinated modes, making policy learning challenging. However, existing monolithic VLA policies typically process multi-view observations and distinct bimanual interaction patterns without explicitly adapting their visual conditioning or action-generation structure. To address this issue, we propose a bimanual manipulation VLA framework based on Dual-Level Structural Decomposition. The View-Selective Visual Router dynamically adjusts wrist-view contributions to emphasize relevant visual cues, while the Interaction-Aware Action Mixture-of-Experts decomposes action generation into coordinated and arm-wise pathways to adapt to varying bimanual interaction modes. We evaluate the proposed method on six simulated bimanual manipulation tasks in RoboTwin 2.0 and three long-horizon real-world tasks. Our model improves the overall average success rate over a monolithic baseline by 27.7% in simulation and 43.3% in real-world evaluation, while consistently outperforming single-module variants across both settings. These results demonstrate that jointly considering selective visual processing and explicit decomposition of bimanual interaction structures provides an effective inductive bias for robust bimanual manipulation.

Yoon-Ji Choi, Young-Chae Son, Soo-Chul Lim

Dongguk University

Long-Horizon Robotic Manipulation Bimanual Robotic Manipulation Mixture of Experts VLMs for Robotics

Ask PDF

Abstract

Robotic Control

Explore similar work

BiRoAD: Learning Shared and Role-Adaptive Representations for Bimanual Manipulation

Communication in modular robotic motor control: Bilateral controllers under realistic constraints

See Selectively, Act Adaptively: Dual-Level Structural Decomposition for Bimanual Robot Manipulation