Autonomous Block Assembly for Boom Cranes with Passive Joint Dynamics: Integrated Vision MPC Control

Gerald Ebmer; Minh Nhat Vu; Tobias Glück; Wolfgang Kemmetmüller

cs.ROFeb 17, 2026

Autonomous Block Assembly for Boom Cranes with Passive Joint Dynamics: Integrated Vision MPC Control

Authors: Gerald Ebmer, Minh Nhat Vu, Tobias Glück, Wolfgang Kemmetmüller

Organizations: Automation & Control Institute (ACIN), TU Wien, Gusshausstrasse 27-29, 1040 Wien, Austria · Center for Vision, Automation & Control, AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria

Abstract

This paper presents an autonomous control framework for articulated boom cranes performing prefabricated block assembly in construction environments. The key challenge addressed is precise placement control under passive joint dynamics that cause pendulum-like sway, complicating the accurate positioning of building components. Our integrated approach combines real-time vision-based pose estimation of building blocks, collision-aware B-spline path planning, and nonlinear model predictive control (NMPC) to achieve autonomous pickup, placement, and obstacle-avoidance assembly operations. The framework is validated on a laboratory-scale testbed that emulates crane kinematics and passive dynamics while enabling rapid experimentation. The collision-aware planner generates feasible B-spline references in real-time on CPU hardware with anytime performance, while the NMPC controller actively suppresses passive joint sway and tracks the planned trajectory under continuous vision feedback. Experimental results demonstrate autonomous block stacking and obstacle-avoidance assembly, with sway damping reducing settling times by more than an order of magnitude compared to uncontrolled passive dynamics, confirming the real-time feasibility of the integrated approach for construction automation.

Figures & tables

Fig. 1 : Motivation: articulated boom crane with concrete block (left) and developed laboratory-scale setup (right) as a testbed for autonomous assembly.

Fig. 2 : System overview. Left: schematic of the laboratory setup with robot, passive joint, gripper, blocks, and reference frames

{0}

(base),

\mathcal{E}

(end-effector),

\mathcal{T}

(tool center), and

\mathcal{C}

(camera). Right: block diagram of the integrated framework, illustrating the interconnection of perception, path planning, MPC control, and the environment model. The task sequencer executes pickup and placement routines based on a configuration file specifying the schedule.

Target	Mode	Success	Time	Iter.	Length
$10\text{\,}\mathrm{ms}$	Cold	95%	$12.6\pm\pm 2.9\text{\,}\mathrm{ms}$	2.00	$0.780\text{\,}\mathrm{m}$
	Warm	94%	$12.5\pm\pm 3.4\text{\,}\mathrm{ms}$	2.23	$0.753\text{\,}\mathrm{m}$
$20\text{\,}\mathrm{ms}$	Cold	98%	$23.1\pm\pm 2.4\text{\,}\mathrm{ms}$	3.99	$0.685\text{\,}\mathrm{m}$
	Warm	100%	$22.7\pm\pm 2.7\text{\,}\mathrm{ms}$	5.41	$0.598\text{\,}\mathrm{m}$
$50\text{\,}\mathrm{ms}$	Cold	100%	$53.6\pm\pm 3.9\text{\,}\mathrm{ms}$	13.45	$0.497\text{\,}\mathrm{m}$
	Warm	100%	$53.3\pm\pm 3.7\text{\,}\mathrm{ms}$	12.66	$0.521\text{\,}\mathrm{m}$

TABLE I : Planner benchmark results (

N{=}100

runs). Cold: first cycle from linear initialization; Warm: continued refinement with warm start. Runtime is per cycle. All runs use ensemble size

M{=}15

, one interior via point (

K{=}1

), initial standard deviation

\sigma_{\text{init}}{=}0.2

, and

N_{c}{=}40

collision checks per path.

Panel of Figure 4 — Fig. 3 : Evolution of the stochastic ensemble (15 candidates) with an obstacle wall of three stacked blocks. Candidates start with wide variability ( ) and contract toward a feasible collision-free region ( ).

Panel of Figure 5 — Fig. 4 : Block pickup and placement. In ( ), a randomly placed block is grasped using vision-based pose estimation feedback. In ( ), the block is placed on top of an existing block at a predefined goal, consistent with nominal assembly targets.

Fig. 5 : Reference tracking and error during block pickup and placement (see Figure 4 ). Vertical dashed lines indicate replanning events (new path initialization), while dotted lines mark vision updates (goal corrections from pose estimation). From top to bottom: Cartesian position

\mathbf{p}

, yaw angle

\psi

, Cartesian position errors

\tilde{\mathbf{p}}

and yaw angle error

\tilde{\psi}

Fig. 6 : Path progress during block pickup and placement. Top: path parameter

s

. Middle: time derivative

\dot{s}

. Bottom: input

v

Fig. 7 : MPC performance metrics during block pickup and placement. Top: computation times, with

t_{\text{total}}

the overall MPC iteration time and

t_{\text{QP}}

the QP solver time. Middle: number of QP solver iterations per MPC step. Bottom: convergence residuals, where

r_{\text{stat}}

measures first-order optimality (gradient norm) and

r_{\text{eq}}

the satisfaction of system dynamics constraints.

Panel of Figure 9 — Fig. 8 : Assembly of blocks with an obstacle wall (see Figure 3 ). The wall consists of three stacked blocks separating pickup and placement locations. Two blocks are transferred across the obstacle in sequence, each consisting of pickup, traverse, and placement.

Fig. 9 : Damping performance of the MPC controller under external excitation. The controller is active from

t=

0\text{,}\mathrm{s}

to $t=$15\text{\,}\mathrm{s}

, after which the natural, lightly damped sway motion of the passive joints becomes visible. Quantitative parameters of the fitted oscillations are reported in Table II .

	Decay rate $\sigma$	Damping ratio $\zeta$	$T_{90\to 10}$
MPC on	$2.50\text{\,}\text{/}\mathrm{s}$	0.23	$0.88\text{\,}\mathrm{s}$
MPC off	$0.12\text{\,}\text{/}\mathrm{s}$	0.013	$17.8\text{\,}\mathrm{s}$

TABLE II : Damping characteristics of

q_{9}

with MPC active vs. inactive. With MPC active, the decay rate and damping ratio increase by more than an order of magnitude, reducing the settling time from

17.8\text{\,}\mathrm{s}

to below

1\text{\,}\mathrm{s}

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 584 papers

Learning to Build: Autonomous Robotic Assembly of Stable Structures Without Predefined Plans
Feb 27, 2026Jingwen Wang, Johannes Kirschner, Paul Rolland +2Robotic Construction Sequential Decision Making
Lab of Creative Computation, ´Ecole Polytechnique F´ed´erale de Lausanne · Swiss Data Science Center
CAST: Collision-Aware Assembly with Construction Robots using Simultaneous Trajectory Estimation and Planning
Sep 21, 2026Karthik Shaji, Chisung Kim, John D'Amato +2Multi-Robot Motion Planning Robotic Construction
Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta 30332, USA · School of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta 30332, USA · Institute of Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta 30332, USA
Sampling-Based MuJoCo MPC for Double-Pendulum Sway Suppression on a Shipboard Crane
Mar 17, 2026Oscar Pang, Lisa Coiffard, Paul Templier +3Robotic Control Robust Control
Adaptive and Intelligent Robotics Lab, Department of Computing, Imperial College London, Exhibition Rd, London, SW7 2BX, United Kingdom

Feb 27, 2026J/K move · Enter open · S save

Learning to Build: Autonomous Robotic Assembly of Stable Structures Without Predefined Plans

Jingwen Wang, Johannes Kirschner, Paul Rolland, Luis Salamanca+1

Lab of Creative Computation, ´Ecole Polytechnique F´ed´erale de Lausanne · Swiss Data Science Center

Robotic Construction Sequential Decision Making Robotic RL Robotics Robotic Manipulation Robotic Assembly

Open paper AskPDF arXiv

This paper presents a novel autonomous robotic assembly framework for constructing stable structures without relying on predefined architectural blueprints. Instead of following fixed plans, construction tasks are defined through targets and obstacles, allowing the system to adapt more flexibly to environmental uncertainty and variations during the building process. A reinforcement learning (RL) policy, trained using deep Q-learning with successor features, serves as the decision-making component. As a proof of concept, we evaluate the approach on a benchmark of 15 2D robotic assembly tasks of discrete block construction. Experiments using a real-world closed-loop robotic setup demonstrate the feasibility of the method and its ability to handle construction noise. The results suggest that our framework offers a promising direction for more adaptable and robust robotic construction in real-world environments.

Abstract

Robotic Control

Explore similar work

Learning to Build: Autonomous Robotic Assembly of Stable Structures Without Predefined Plans

CAST: Collision-Aware Assembly with Construction Robots using Simultaneous Trajectory Estimation and Planning

Sampling-Based MuJoCo MPC for Double-Pendulum Sway Suppression on a Shipboard Crane