Multiview Progress Prediction of Robot Activities

Elena Zoppellari; Federico Becattini; Marco Fiorucci; Lamberto Ballan

cs.ROFeb 25, 2026

Multiview Progress Prediction of Robot Activities

Authors: Elena Zoppellari, Federico Becattini, Marco Fiorucci, Lamberto Ballan

Organizations: University of Padova, Italy · University of Siena, Italy

Abstract

For robots to operate effectively and safely alongside humans, they must be able to understand the progress of ongoing actions. This ability, known as action progress prediction, is critical for tasks ranging from timely assistance to autonomous decision-making. However, modeling action progression in robotics has often been overlooked. Moreover, a single camera may be insufficient for understanding robot's ego-actions, as self-occlusion can significantly hinder perception and model performance. In this paper, we propose a multi-view architecture for action progress prediction in robot manipulation tasks. Experiments on Mobile ALOHA demonstrate the effectiveness of the proposed approach.

Figures & tables

Action	Start Condition	End Condition
Use Cabinet	Base velocity $>$ 0.2 m/s	Shoulder or elbow velocity $<$ 0.4 m/s
Push Chair	Base angular velocity $>$ 0.3 rad/s	Either gripper effort $<$ -500 Nm
Take Elevator	Base velocity $>$ 0.3 m/s	Base angular velocity $<$ 0.3 rad/s
Cook Shrimp	Right gripper effort $>$ 200 Nm	Left gripper effort $<$ 600 Nm
Wash Pan	Left gripper effort $>$ 150 Nm	Left gripper effort $<$ 100 Nm
Wipe Wine	Base velocity $>$ 0.25 m/s	Shoulder, elbow or wrist angle vel. $<$ 0.5 m/s

Table 1: Heuristic start and end conditions for each action in Mobile ALOHA [ 7 ] , based on robot motion signals.

Model	Segments	[0-25]%	[25-50]%	[50-75]%	[75-100]%	Whole
Avg. Index	-	4.70	22.26	46.79	71.69	10.00
Static	-	37.46	12.51	12.44	37.39	25.01
Random	-	39.69	27.15	26.88	39.45	33.87
MobileNet	✗	6.76	12.10	8.21	10.30	7.02
MobileNet	✓	7.60	4.69	7.90	19.20	6.34
ResNet152	✗	3.24	5.30	31.90	61.00	6.57

Table 2: MAE on Mobile ALOHA, evaluated on different progress intervals. Overall best result per progress interval highlighted in bold .

Camera	Segments	[0-25]%	[25-50]%	[50-75]%	[75-100]%	Whole
Left	✗	8.58	11.20	14.90	30.70	9.98
Left	✓	20.80	6.63	5.47	9.27	10.10
Right	✗	5.84	12.70	34.70	53.10	9.41
Right	✓	24.50	15.70	9.09	9.75	15.60
Central	✗	4.99	8.43	6.61	28.40	6.23
Central	✓	14.90	5.98	5.35	6.42	7.90

Table 3: MAE on Mobile ALOHA, evaluated on partial segments for different cameras with the ViT-based model. Overall best result per progress interval highlighted in bold .

Camera	Segments	Use Cabinet	Push Chair	Take Elevator	Cook Shrimp	Wash Pan	Wipe Wine
Left	✗	8.48	13.00	8.18	12.80	8.49	7.57
Left	✓	8.02	15.50	8.59	9.66	8.66	8.26
Right	✗	8.36	10.30	13.40	8.05	10.60	7.55
Right	✓	12.40	21.20	26.80	11.80	12.00	12.10
Central	✗	5.90	5.37	5.03	6.85	7.55	7.27
Central	✓	7.85	9.90	7.09	7.33	7.06	6.60

Table 4: MAE per action on Mobile ALOHA (ViT+LSTM) using different cameras. Overall best result in bold .

Panel of Figure 2 — Figure 2: Progress estimated from different cameras: left (green), central (red) and right (cyan). A perfect progress estimation would exhibit a straight diagonal line from bottom left to top right.

Panel of Figure 3 — Figure 3: Progress estimated with multiple cameras. Two versions are shown: trained on full videos (red) and trained on semgments (green). A perfect progress estimation would exhibit a straight diagonal line from bottom left to top right.

Explore similar work

UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling
Jun 19, 2026Tao Xu, Runhao Zhang, Zhijian Huang +7World Models for Robotics Multi-View Learning
1Tongji University · 2Shanghai Innovation Institute · 4Xiaomi EV +1
See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation
Mar 10, 2026Tingjun Dai, Mingfei Han, Tingwen Du +6Vision-Language-Action Models Language-Conditioned Robot Manipulation
School of Information Science and Technology, University of Science and Technology of China, Anhui, China. · University of Technology Sydney, Ultimo, NSW, Australia. · Department of Computer Vision, Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates. +3
Progress Reward Modeling for Robotic Learning: A Comprehensive Survey
Jul 22, 2026Jianshu Zhang, Keliang Wu, Haoran Lu +8Reward Modeling Process Reward Models
Northwestern University · Carnegie Mellon University · University of Wisconsin–Madison +2

Jun 19, 2026J/K move · Enter open · S save

UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling

Tao Xu, Runhao Zhang, Zhijian Huang, Jiayi Guan+6

1Tongji University · 2Shanghai Innovation Institute · 4Xiaomi EV+1

World Models for Robotics Multi-View Learning Efficient VLA Models Robotic Manipulation Vision-Language-Action Models Vision-Based Robot Control

Open paper AskPDF arXiv

Occluded tasks remain a bottleneck in robot manipulation. Existing solutions either deploy additional physical cameras requiring training-inference camera parity, or rely on explicit 3D reconstruction with high computational cost. Moreover, both approaches rely on standard agent-view and wrist-view observations, while failing to capture occlusion information and future scene evolution. To this end, we propose UniviewVLA, a unified multiview Vision-Language-Action model with world modeling, which infers multiview scene evolution for action prediction from only standard two-camera observations. We demonstrate that by leveraging generated multiview future views from the world model, UniviewVLA reveals occluded cues and models future scene evolution, improving action prediction and removing the need for extra hardware or explicit reconstruction. Besides, to accelerate inference while preserving prediction accuracy, UniviewVLA develops Motion-Informative Token Compression, which compresses each generated view from 625 to 16 tokens and reduces per-view latency from 6-7s to 0.2-0.3s. UniviewVLA also proposes training-free Action-Entropy View Selection, which dynamically identifies the most action-informative view at different inference stages. Extensive experiments show that UniviewVLA achieves 95.8% on LIBERO and 4.60 on CALVIN ABCD to D, both standard occlusion-free benchmarks. On customized occlusion-focused tasks, it improves success rate from 40.0% to 73.3%, and average real-robot success rate by 33.4 points, demonstrating stronger occlusion-focused performance without sacrificing standard occlusion-free benchmarks.

Abstract

Explore similar work

UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling

See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation

Progress Reward Modeling for Robotic Learning: A Comprehensive Survey