Fluoroscopy-Constrained Magnetic Robot Control via Zernike-Based Field Modeling and Nonlinear MPC

Xinhao Chen; Hongkun Yao; Anuruddha Bhattacharjee; Suraj Raval; Lamar O. Mair; Yancy Diaz-Mercado; Axel Krieger

cs.ROFeb 17, 2026

Fluoroscopy-Constrained Magnetic Robot Control via Zernike-Based Field Modeling and Nonlinear MPC

Authors: Xinhao Chen, Hongkun Yao, Anuruddha Bhattacharjee, Suraj Raval, Lamar O. Mair, Yancy Diaz-Mercado, Axel Krieger

Organizations: Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD 21218, USA. · Department of Mechanical Engineering, University of Maryland, College Park, MD 20742, USA. · Division of Magnetic Manipulation and Particle Research, Weinberg Medical Physics, MD 20852, USA.

Abstract

Magnetic actuation enables surgical robots to navigate complex anatomical pathways while reducing tissue trauma and improving surgical precision. However, clinical deployment is limited by the challenges of controlling such systems under fluoroscopic imaging, which provides low frame rate and noisy pose feedback. This paper presents a control framework that remains accurate and stable under such conditions by combining a nonlinear model predictive control (NMPC) framework that directly outputs coil currents, an analytically differentiable magnetic field model based on Zernike polynomials, and a Kalman filter to estimate the robot state. Experimental validation is conducted with two magnetic robots in a 3D-printed fluid workspace and a spine phantom replicating drug delivery in the epidural space. Results show the proposed control method remains highly accurate when feedback is downsampled to 3 Hz with added Gaussian noise (sigma = 2 mm), mimicking clinical fluoroscopy. In the spine phantom experiments, the proposed method successfully executed a drug delivery trajectory with a root mean square (RMS) position error of 1.18 mm while maintaining safe clearance from critical anatomical boundaries.

Figures & tables

Fig. 1: (a) Electromagnetic coil system used in this study. The experimental workspace (yellow square) is centered between the coil stacks. (b) Spine phantom workspace imaged with an optical camera (left) and fluoroscopy X-rays (right). The robot agent is visible in both. (c) Robotic agents evaluated, including a drug-delivery capsule that can be filled with liquid medication.

Fig. 2: (a) The spine structure in sagittal (longitudinal) plane. (b) The spine structure in transverse (horizontal) plane. [ 29 ]

Fig. 3: (a) CAD model of a human vertebral segment sectioned in the sagittal plane. (b) Spine phantom fabricated from ABS, agarose gel, and a glycerin–water solution. The epidural space is indicated by purple lines.

#	$n$	$m$	Polynomial
0	0	0	$1$
1	1	1	$x$
2	1	1	$y$
3	2	0	$2(x^{2}+y^{2})-1$
4	2	2	$x^{2}-y^{2}$
5	2	2	$2xy$

TABLE I: Zernike polynomials in Cartesian coordinates [ 33 ] .

Configuration	Coil center (mm)	$\rho_{\max}$ (mm)
Small	$(0,\,57.69)$	$118.731$
Medium	$(0,\,71.295)$	$136.186$
Large	$(0,\,90.925)$	$165.341$

TABLE II: Coil centers and normalization radii

Fig. 4: (a) Mean absolute error (MAE) and $R^{2}$ score of Zernike model with different polynomial order. (b) FEA-computed magnetic-field magnitude for a small coil at unit current (coil located at the bottom of the figure). (c) Absolute fitting-error distribution for the Zernike model under the same condition.

Fig. 5: Flow chart of the proposed control method

Fig. 6: (a) The workspace for the magnet robot. The purple line shows the desired trajectory. (b) Robot actual and NMPC-predicted trajectories.

Fig. 7: (a) RMS position errors under different tracking rates. (b) RMS orientation errors under different tracking rates.

Fig. 8: (a) RMS Position errors under different tracking noise levels. (b) RMS Orientation errors under different tracking noise levels.

Fig. 9: (a) RMS position errors under different tracking rates with fixed noise level. (b) RMS orientation errors under different tracking rates with fixed noise level. (c) Violin plot at 3Hz with fixed noise level. All tests has $p<10^{-3}$ , denoted by ***.

Fig. 10: Results of spine-phantom experiment. The green curve denotes the desired trajectory; the red curve shows the actual robot trajectory; purple lines delineate the epidural space’s boundaries.

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 584 papers

Magnet-Aware Control of Legged Robots
Oct 6, 2026J. Playan Garai, S. B. Djuve, C. McGreavy +1Robotic Control Legged Robotics
Technical University of Munich (TUM), Munich, Germany · Conseil Europ´een pour la Recherche Nucl´eaire (CERN), BE/CEM/MRO, Geneva, Switzerland
MagServo: Uncertainty-Resilient Hierarchical Magnetic Servoing via Learned Latent Representations
Oct 5, 2026Yuhan Tan, Yameng Zhang, Pei Liu +2Robotic Control Robot Trajectory Tracking
Department of Bioengineering, Imperial College London, UK. · Department of Mechanical Engineering, The University of Hong Kong, Hong Kong SAR. · School of Computation, Information and Technology, Technical University of Munich, Germany.
Expanding the Workspace of Electromagnetic Navigation Systems Using Dynamic Feedback for Single- and Multi-agent Control
Nov 23, 2025Jasan Zughaibi, Denis von Arx, Maurus Derungs +5Multi-Agent Control Feedback Control
Multi-Scale Robotics Lab, ETH Zurich, 8092 Zurich, Switzerland · Learning and Dynamical Systems Group, Max Planck Institute for Intelligent Systems, 72076 Tübingen, Germany

Oct 6, 2026J/K move · Enter open · S save

Magnet-Aware Control of Legged Robots

J. Playan Garai, S. B. Djuve, C. McGreavy, M. Khadiv

Technical University of Munich (TUM), Munich, Germany · Conseil Europ´een pour la Recherche Nucl´eaire (CERN), BE/CEM/MRO, Geneva, Switzerland

Robotic Control Legged Robotics Whole-Body Control Model Predictive Control Robot Locomotion

Open paper AskPDF arXiv

Autonomous robots can increase uptime and reduce human exposure in Big Science facilities, but strong magnetic fields needed for their operation corrupt sensors and induce pose-dependent mechanical wrenches that destabilize robots and challenge conventional reactive controllers. This paper presents a control framework for modeling, estimating, and dynamically compensating for spatially varying magnetic wrenches acting on legged robots to improve robustness in these fields. We introduce a custom physics plugin for the MuJoCo simulator to model magnetic forces on rigid-body elements, alongside an inverse field-estimation framework to infer the latent magnetic field directly from quadruped dynamic responses and any number of sensor readings. Furthermore, we develop a Magnet-Aware Model Predictive Control (MPC) and Whole-Body Control (WBC) architecture that predicts and counteracts magnetic perturbations during locomotion to increase the range of magnetic fields in which the robot can operate. The effectiveness of the framework is validated through both simulation and physical hardware experiments. We show our method increases the the maximum rejectable disturbances from magnetic field in the force space by a factor of 2.5 and and between 1.6-2 times in torque space compared to a non-compensated system. Within the proposed magnetic field, this constitutes an increase in the area in which the robot can operate by 29,34% of which 10,84% would have previously caused an immediate collapse to non-compensated controllers.

Abstract

Robotic Control

Explore similar work

Magnet-Aware Control of Legged Robots

MagServo: Uncertainty-Resilient Hierarchical Magnetic Servoing via Learned Latent Representations

Expanding the Workspace of Electromagnetic Navigation Systems Using Dynamic Feedback for Single- and Multi-agent Control