Learning to See through Action: Predictive Curiosity for Robots
Authors: Stelios Zarifis, Panagiotis Filntisis, George Retsinas, Petros Maragos
Organizations: HERON - Hellenic Robotics Center of Excellence, Athena Research Center, Greece · Institute of Robotics, Athena Research Center, Athens, Greece · National Technical University of Athens, School of Electrical & Computer Engineering, Greece
Humans and animals do not learn vision from a fixed set of data: they move, interact, and learn from the visual consequences of their actions. Motivated by this developmental view, we study whether robots can improve visual learning by actively collecting experience through curiosity-driven interaction. We introduce DySCOVER, a self-supervised framework for active robot visual learning. A robot observes RGB frames, learns a visual autoencoder that encodes and reconstructs them, and trains a recurrent world model to predict the next visual latent state conditioned on the current state and action. The agent's policy is trained only with intrinsic rewards: reconstruction error encourages visually novel observations, while the world model's error encourages transitions whose consequences are not yet predictable. This prediction error biases the policy toward more intricate trajectories that enrich the agent's set of experiences with observations beyond those encouraged by visual novelty alone. Our experiments conducted with robotic arms as the embodied agents show that adding predictive curiosity improves the vision model over using visual curiosity alone. Across variations in initial conditions, experience capacity, embodiment, and scene content, adding predictive curiosity consistently improved visual learning. These results show that predictive curiosity can improve active visual learning beyond reconstruction-based curiosity alone.
Figures & tables
Fig. 1: DySCOVER Overview. (1) The recurrent policy collects action–observation transitions using the visual representation as state. (2) Intrinsic curiosity is calculated using the errors of the vision and world models. (3) Policy is updated online, visual and world models are updated using replayed experience.
Fig. 2: Robot environments used in this study: UR10e arm with an anthropomorphic Shadow Hand (first), Franka arm (second), UR10e arm with a different gripper (third), and an object-augmented scene (fourth) for testing adaptability on new environments.
Fig. 3: Evaluation frame MSE during 10 million training steps for the UR10e with “Shadow Hand”. Using the WM reward term, the convergence is faster and to lower reconstruction error than visual-only curiosity.
Method
MSE ↓
PSNR (dB) ↑
MSE reduction w.r.t. baseline ↑
Random data collection
4.08×10−4
33.90
−25.9%
Visual-only curiosity (baseline)
3.24×10−4
34.89
—
World Model reward only
3.12×10−4
35.06
3.7%
DySCOVER, raw WM reward
3.02×10−4
35.20
6.8%
DySCOVER, adjusted WM reward
2.77×10−4
35.58
14.5%
TABLE I: DySCOVER performance
Fig. 4: Example reconstruction and error heatmap comparisons on a frame during training.
Fig. 5: Evaluation frame reconstruction MSE during 10 million training steps with a compact replay buffer. Lower is better. DySCOVER converges faster and reaches lower reconstruction error than visual-only curiosity despite the reduced replay buffer capacity.
Fig. 6: Evaluation frame reconstruction MSE during 10 million training steps for the Franka robot. Lower is better. DySCOVER shows similar performance improvement over the visual-only curiosity baseline.
Enviroment
Method
MSE ↓
PSNR (dB) ↑
MSE reduction w.r.t. baseline ↑
UR10e
Visual-only curiosity
3.34×10−4
34.76
—
UR10e
DySCOVER
2.83×10−4
35.49
15.3%
Franka
Visual-only curiosity
2.20×10−4
36.57
—
Franka
DySCOVER
1.99×10−4
37.02
9.5%
TABLE III: Embodiment comparison
Fig. 7: Evaluation reconstruction error during object-scene adaptation. DySCOVER reaches lower error than visual-only curiosity, and also remains below the visual-only control initialized from the same visual model.
Method
Global MSE ↓
Robot MSE ↓
Object MSE ↓
Visual-only curiosity
3.460×10−3
1.032×10−2
1.256×10−2
DySCOVER
3.265×10−3
7.809×10−3
1.043×10−2
DySCOVER Improvement
5.6%
24.3%
17.0%
TABLE IV: Object-scene reconstruction performance
Fig. 8: Qualitative reconstruction comparison on object-scene frames. DySCOVER performs better in difficult scenes with occlusions.
Metric
Visual-only
DySCOVER
Change
# Objects displaced >5cm
4.628
5.468
+18.2%
Max object displacement (cm)
12.2
17.9
+47.3%
Motion-direction entropy
0.796
0.846
+6.3%
TABLE V: Object-interaction average statistics per cycle.
Fig. 9: Reward decomposition during an example object-scene rollout. The top plot shows the reward per step for a rollout for the visual-only curiosity agent. The bottom plot decomposes the intrinsic reward into the visual-reward term and the WM-reward term; shaded regions indicate intervals where an object is moved by the agent. The WM reward tends to increase during object movement intervals while the reconstruction-based reward does not correlate with them.
Fig. 10: Example of policy rollout comparison at 4 million training steps. Each row shows sampled frames from a rollout and policy statistics for an action dimension. The blue curve is the policy mean, the shaded area the policy’s standard deviation, and the orange curve is the sampled action. DySCOVER exhibits more complex control than the baseline, producing richer robot configurations and exposing the vision model to more diverse views. Note, DySCOVER with adjusted WM term produces much smoother control, yet complex enough to collect useful frames for the vision model.