IRIS: Learning-Driven Task-Specific Cinema Robot Arm for Visuomotor Motion Control

Qilong Cheng; Matthew Mackay; Ali Bereyhi

cs.ROFeb 19, 2026

IRIS: Learning-Driven Task-Specific Cinema Robot Arm for Visuomotor Motion Control

Authors: Qilong Cheng, Matthew Mackay, Ali Bereyhi

Organizations: Mechanical and Aerospace Eng. Department, New York University · MIE Department, University of Toronto · ECE Department, University of Toronto

Abstract

Robotic camera systems enable dynamic, repeatable motion beyond human capabilities, yet their adoption remains limited by the high cost and operational complexity of industrial-grade platforms. We present the Intelligent Robotic Imaging System (IRIS), a task-specific 6-DOF manipulator designed for autonomous, learning-driven cinematic motion control. IRIS integrates a lightweight, fully 3D-printed hardware design with a goal-conditioned visuomotor imitation learning framework based on Action Chunking with Transformers (ACT). The system learns object-aware and perceptually smooth camera trajectories directly from human demonstrations, eliminating the need for explicit geometric programming. The complete platform costs under $1,000 USD, supports a 1.5 kg payload, and achieves approximately 1 mm repeatability. Real-world experiments demonstrate accurate trajectory tracking, reliable autonomous execution, and generalization across diverse cinematic motions.

Figures & tables

Fig. 1: Tabletop deployment of the IRIS prototype performing a real-world demonstration. An end-effector-mounted camera captures a target object (cup) using visuomotor control.

Fig. 2: Overview of the IRIS system pipeline. Cinema task objectives guide task-specific hardware design. Training data are collected exclusively from real-world human demonstrations, while classical planner trajectories generated in simulation are used for analysis and comparison. A ROS-based low-level control stack executes inverse-dynamics control, and a goal-conditioned imitation learning policy (ACT) is trained on human data and deployed on the physical robot, enabling smooth, obstacle-aware cinematic motion via sim-to-real transfer.

Robot Arm	Reach	Speed	Acceleration	Repeatability	Payload	Ease of Use	Cost
	(mm)	(m/s)	(m/s 2 )	(mm)	(kg)		(USD)
Commercial Cinema Robots
Bolt Cinebot [ 4 ]	2000	5.0	30	$\pm$ 0.05	20	Medium	$69k+
Colossus [ 27 ]	1100	6.8	40	$\pm$ 0.05	10+	Medium	$200k+
Kira [ 5 ]	1500	3.5	20	$\pm$ 0.05	10	Low	$50k+
E-Jib Mini [ 28 ]	1800	1.0	5	$\pm$ 0.1	5	Low	$9k

TABLE I: Comparison of commercial cinema robots and 3D-printed robot arms for cinematic applications.

Fig. 3: IRIS hardware overview: lightweight task-specific architecture with relocated actuation and a differential wrist.

Fig. 4: Sim-to-real execution of planner-generated trajectories. A classical potential-field planner generates collision-free reference paths in simulation (left), which are then executed on the physical IRIS robot (right) via a ROS-based control stack for validation and comparison.

Fig. 5: IRIS policy architecture. During training (left), the model conditions on observation history, a goal image, and a CVAE-encoded latent style token $z$ derived from the ground-truth future trajectory. Visual inputs pass through a shared ResNet-18 and Spatial Softmax to preserve spatial coordinates, then fuse with proprioception into temporal tokens. A transformer decoder predicts the 15-step joint trajectory $\hat{q}_{t+1:t+H}$ . At inference (right), the CVAE branch is replaced with $z=0$ for deterministic execution.

Fig. 6: Demonstration dataset coverage. (a) End-effector workspace coverage over all demonstrations. (b) Joint-space distributions of the six joints. (c) Per-step joint displacement magnitude, indicating bounded control actions.

Parameter	Value
Observation horizon $S$	8
Prediction horizon $H$	15
Visual backbone	ResNet-18 (ImageNet-pretrained)
Image resolution	$224\times 224$
Transformer dimension $d$	256
Attention heads	8

TABLE II: Key model and training hyperparameters.

Fig. 7: Low-level evaluation. Top: End-effector XY scatter over $K=10$ trials at start and WP2–WP4. Bottom: Reference vs. executed 3D trajectories and Cartesian error. IRIS achieves sub-millimeter repeatability and centimeter-scale accuracy.

Method	Success $\uparrow$	Vis. Align. $\uparrow$	Cart. Jerk $\downarrow$	Framing $\downarrow$	SRR $\uparrow$	Latency $\downarrow$
	(%)	( $\mathcal{S}_{\text{vis}}$ )	(m $/$ s 3 )	( $\mathcal{E}_{\text{frame}}$ px)	(%)	(ms)
Baselines
Human Expert Replay	90.0	0.874	3.64	105.7	67.1	–
Classical Planner (RRT*)	10.0	0.636	0.22	219.6	10.5	–
Ablation Studies
w/ Incremental Action Space	0.0	0.636	0.83	103.1	31.9	9.1

TABLE III: Performance Comparison and Ablation Studies. Evaluation of task metrics across 10 randomized trials.

Fig. 8: Qualitative shot reproduction results. Left: Experimental setup with the goal images for unobstructed (top) and obstacle (bottom) scenes. Right: Temporal image sequences comparing the Classical Planner, Human Expert, and our IL policy. Unlike the planner, which fails against unmodeled obstacles, our policy successfully maintains the desired framing while avoiding collisions, closely matching expert demonstrations.

Explore similar work

VidAct: Learning Manipulation from In-the-Wild Videos with Object-Centric 3D Awareness
Sep 29, 2026Hang Li, Mingxin Zhang, Zihan Wu +9Object-Centric Representation Learning Robotic Manipulation
Technical University of Munich · Peking University · Huawei Heisenberg Research Center +2
Imagine2Act: Leveraging Object-Action Motion Consistency from Imagined Goals for Robotic Manipulation
Sep 21, 2025Liang Heng, Jiadong Xu, Yiwen Wang +6Robot Skill Learning Language-Conditioned Robot Manipulation
CFCS, School of Computer Science, Peking University · PrimeBot · AGIBOT
KAI: A Kinematic-Aware Interface for Data-Efficient Articulated Object Manipulation
Jul 27, 2026Yaping Li, Zhaxizhuoma, Qiaojun Yu +3Robot Skill Learning Generalization in Robotic Manipulation
The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University

Sep 29, 2026J/K move · Enter open · S save

VidAct: Learning Manipulation from In-the-Wild Videos with Object-Centric 3D Awareness

Hang Li, Mingxin Zhang, Zihan Wu, Yang Tian+8

Technical University of Munich · Peking University · Huawei Heisenberg Research Center+2

Object-Centric Representation Learning Robotic Manipulation Robot Imitation Learning Motion Representation Learning Robot Learning

Open paper AskPDF arXiv

Video demonstrations offer a scalable alternative to costly robot data for learning manipulation, yet existing reconstruction-based approaches often rely on constrained camera viewpoints or human-to-robot retargeting, while the reconstructed trajectories are difficult to adapt to new objects configurations without distorting the trajectory shape. Another key limitation is that the resulting policies often lack precise object-level 3D geometry awareness, limiting object grounding and object shape awareness critical for precise manipulation. To bridge these gaps, we propose VidAct, an efficient video-to-robot framework that learns object-centric, 3D-aware manipulation policies from a single monocular video per task and enables zero-shot real-world deployment. VidAct consists of three key components. First, VidAct reconstructs object meshes and motion from arbitrary demo videos and canonicalizes the motion in the static object frame, avoiding embodiment-specific retargeting and accommodating diverse camera viewpoints. Second, VidAct employ residual trajectory transfer for adapting the reconstructed motion to novel object configurations while preserving its motion shape. Finally, as the key policy-learning component, VidAct predicts simulation-provided privileged complete-object point clouds at each frame as an auxiliary task while retaining RGB-only deployment, providing dense object-centric supervision over both object pose and 3D geometry. Experiments on human, robot, generated, and internet videos demonstrate broad video applicability and zero-shot deployment. Per-frame complete-object 3D supervision improves policy generalization and sim-to-real success, while residual trajectory transfer enables reliable trajectory adaptation with better shape preservation.

Abstract

Explore similar work

VidAct: Learning Manipulation from In-the-Wild Videos with Object-Centric 3D Awareness

Imagine2Act: Leveraging Object-Action Motion Consistency from Imagined Goals for Robotic Manipulation

KAI: A Kinematic-Aware Interface for Data-Efficient Articulated Object Manipulation