Understanding human motion processing is essential for building reliable, human-centered computer vision systems. Although deep neural networks (DNNs) achieve strong performance in optical flow estimation, they remain less robust than humans and rely on fundamentally different computational strategies. Visual motion illusions provide a powerful probe into these mechanisms, revealing how human and machine vision align or diverge. While recent DNN-based motion models can reproduce dynamic illusions such as reverse-phi, it remains unclear whether they can perceive illusory motion in static images, exemplified by the Rotating Snakes illusion. We evaluate several representative optical flow models on Rotating Snakes and show that most fail to generate flow fields consistent with human perception. Under simulated conditions mimicking saccadic eye movements, only the human-inspired Dual-Channel model exhibits the expected rotational motion, with the closest correspondence emerging during the saccade simulation. Ablation analyses further reveal that both luminance-based and higher-order color--feature--based motion signals contribute to this behavior and that a recurrent attention mechanism is critical for integrating local cues. Our results highlight a substantial gap between current optical-flow models and human visual motion processing, and offer insights for developing future motion-estimation systems with improved correspondence to human perception and human-centric AI.
Figures & tables
Figure 1: Rotating Snakes illusory and control images. Top row: illusory images in grayscale (G), blue–yellow (B-Y), and red–green (R-G) color schemes, each producing a robust perception of counterclockwise rotation in human observers. Bottom row: corresponding control images created by permuting the luminance or color sequence within each repeating unit, which fails to induce perception of motion while preserving global layout.
Figure 2: Schematic illustrations of three model architecture categories. (a) Multi-scale architectures estimate motion using pyramidal feature hierarchies. (b) Recurrent-decoding architectures iteratively refine optical flow over multiple update steps. (c) Bio-inspired architectures incorporate motion-energy units, or integration mechanisms motivated by primate visual motion pathways.
Figure 3: Visualization of normalized model-predicted optical flow for grayscale stimuli across central simulated viewing conditions. Static (top row), onset (second row), and microsaccadic 30-pixel shift (third row). Colored dots correspond to model architectures listed in Appendix 1. The fourth row shows model responses to rotated control stimuli, allowing qualitative comparison with veridical motion. Under static and stimulus-onset conditions, optical flow maps differed qualitatively from the microsaccade condition; however, all models failed to reproduce continuous motion consistent with human perception, with only Dual partially capturing counterclockwise motion. Most models accurately captured veridical motion, except LFN2-K , which predicted minimal optical flow apart from a small region of strong rightward motion. Among non–bio-inspired models, those with recurrent architectures more effectively captured veridical motion qualitatively.
Figure 4: Correlation between model-predicted and expected human illusory percepts. Color types: grayscale (G), blue–yellow (B–Y), and red–green (R–G). (a) Correlations are shown for illusion (left) and control (right) stimuli under viewing conditions: static, onset, microsaccade, and saccade (all shift magnitudes included). Among all models, the Dual model exhibits the strongest alignment with human illusory percepts for the grayscale and blue–yellow variants, but shows negative correlations for the red–green variant. For control stimuli, Dual produces little to no consistent alignment. (b) Optical flow predictions from Dual for illusion and control stimuli at three example combinations of color type and shift magnitude: (G, 15 px), (B–Y, 60 px), and (R–G, 90 px). (c) Correlation with human behavioral data from [ 3 ] , collected using an independent stimulus set in which the intermediate luminance levels of dark gray and light gray are parametrically varied to modulate the perceived direction and strength of rotation. Stimuli are sorted by perceived direction (clockwise, unclear, counterclockwise).
Figure 5: Ablation analysis of the Dual model to identify the critical components for reproducing human illusory motion percepts. (a) Schematic of the Dual architecture showing the first-order ( E1 ) and higher-order ( E2 ) motion pathways, their fusion into Em , and subsequent recurrent integration (Stage II). (b) Probing analysis. Optical flow predictions for the grayscale illusion, control, and veridical rotation stimuli under a 30-pixel microsaccade. Columns show outputs from: the first-order pathway alone ( E1 ), the first and last recurrent iterations applied to E1 (Stage II-1 and Stage II-6), and the corresponding first and last recurrent iterations applied to Em . (c) Control and ablation analysis. Correlations are shown for Dual, RAFT, and ME-Attention trained on the full dataset, as well as for Dual trained with dataset and architectural ablations, evaluated on illusion and veridical-rotation stimuli.
Figure 6: Spatial response profiles of two exemplar E1 units sensitive to rotational motion. Activations are normalized per unit to highlight spatial structure. For each unit (rows), spatial activation maps are shown for the illusion, control, and veridical rotation conditions. The preferred motion direction ( θ ) of each unit is indicated by the black arrow overlaid on each map.
Figure 7: Optical flow predictions for other anomalous-motion illusion stimuli. ( a ) Peripheral drift illusion: observers typically experience smooth clockwise rotation following a blink or small eye movement. ( b ) Central drift illusion: the white elements appear to rotate slowly in a clockwise direction. ( c ) Ouchi illusion: eye movements induce a relative sliding motion between the center and surround. Representative outputs from RAFT, ME-Attention, and Dual are shown under the microsaccade simulation.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Parameters
Input Size
Input Frames
PWC-Net [ 39 ]
8.75 M
1024×436
2
LFN2 [ 18 ]
6.42 M
1024×436
2
RAFT [ 44 ]
5.26 M
520×960
2
CCMR [ 19 ]
11.5 M
512×512
2
FlowDiffuser [ 27 ]
14.5 M
512×512
2
FFV1MT [ 37 ]
N.A.
768×768
5
Appendix
Table S1 : Summary of motion estimation models.
Figure S1 : Visualization of normalized model-predicted optical flow for blue–yellow illusion stimuli. We evaluated model predictions under three simulated viewing conditions commonly used in psychophysics: (i) static presentation, (ii) onset presentation, and (iii) microsaccade shifts (small translational displacements that approximate eye movements). For comparison, model predictions for veridical rotation of control stimuli are also shown. Colored dots above each column indicate the model architecture type (multiscale, recurrent, or bio-inspired). Optical flow direction is encoded using a circular color wheel ( Optical flow direction coding , bottom), where hue denotes direction and brightness denotes normalized magnitude. The human percept icon (bottom right) represents the expected counterclockwise rotational flow perceived by human observers.
Figure S2 : Visualization of normalized model-predicted optical flow for red–green illusion stimuli. Figure layout follow the conventions introduced in Figure S1 . Here we present the corresponding results for the red–green variant under identical simulated conditions.
Figure S3 : Visualization of normalized model-predicted optical flow for blue–yellow illusion and control stimuli across select models. This figure extends the results in Figures 2–3 by directly comparing model outputs for illusion versus control stimuli under matched viewing conditions (static, onset, and microsaccade/saccade shifts of 15–120 px). Results are shown for a subset of representative architectures (RAFT, CCMR, FlowDiffuser, ME-Attention, and Dual). Figure layout, color conventions, and direction-coding icons follow those introduced in Supplementary Figure S1 .
Figure S4 : Correlation between model-predicted and expected human illusory percepts for illusion stimuli inducing clockwise motion perception, and for corresponding control stimuli across models and viewing conditions.
Figure S5 : Visualization of normalized model-predicted optical flow for grayscale stimuli under peripheral viewing condition. Figure layout, color conventions, and direction-coding icons follow those introduced in Supplementary Figure S1 . The “Human percept” icon (bottom right) illustrates the expected counterclockwise rotational flow perceived by human observers. The flow field was defined on the same uniform field and placed at the same spatial location as the stimulus, matching spatial extent and displacement offsets.
Figure S6 : Correlation across variations in microsaccade timing for bio-inspired models. For each model, we simulated sequences in which a single microsaccadic displacement (30 px) occurred at different possible time points within the sequence (frame 2 through the final frame). Heatmaps show the correlation between model-predicited optical flow and the expected human illusory percept for grayscale (G), blue–yellow (B–Y), and red–green (R–G) variants.
Figure S7 : Effect of reversing microsaccade direction on model-predicted optical flow. We repeated the central-viewing microsaccade simulation from Fig. 3 , but applied the microsaccadic displacement in the opposite direction. Model predictions are shown for illusion and control stimuli of three color variants across all models. The bottom icons follow those introduced in Supplementary Figure S1 .
Figure S8 : Effect of microsaccade direction on Dual model predictions. Correlations are shown for illusion (left) and control (right) stimuli across eight possible microsaccade directions. The direction of the microsaccadic displacement was varied while keeping all other parameters identical to those used in Fig. 3 and Fig. S7 . Qualitative flow fields for the blue–yellow variant (bottom) illustrate how Dual’s predicted rotation varies with displacement direction. The bottom icons follow those introduced in Supplementary Figure S1 .
Figure S9 : Distribution of human-alignment scores of Dual model responses under random microsaccades or saccades. Histograms show the distribution of correlations between predicted optical flow and the expected human illusory percept for grayscale (top), blue–yellow (middle), and red–green (bottom) variants across 1000 simulations. Asterisks denote significant differences between illusion and control distributions (One-sided Wilcoxon signed rank test).
Figure S10 : Average endpoint error (EPE) across models and stimuli. Heatmaps show the mean Euclidean distance between the model-predicted flow and the human illusory-percept flow under each viewing condition (static, onset, and microsaccade/saccade) for illusion (left) and control (right) stimuli (same simulations as Fig. 4 ).
Figure S11 : Average angular error (AE) across models and stimuli. The simulations and layout follow those introduced in S10 .
Figure S12 : Visualization of decoded flow fields from representations at different processing stages. Shown are the first-order pathway alone ( E1 ), the first and last recurrent iterations applied to E1 (Stage II-1 and Stage II-6), and the corresponding first and last recurrent iterations applied to Em .
Figure S13 : Predicted flow fields of retrained models evaluated on illusion and veridical-rotation stimuli. Shown are Dual, RAFT, and ME-Attention trained on the full dataset, as well as Dual trained with data and architectural ablations.
Figure S14 : Visualization of normalized model-predicted optical flow for peripheral drift illusions (PDI) and the central drift illusion (CDI) across selected models. The layout follows the conventions introduced in Fig. S1 . This figure extends the results shown in Figure 6 by presenting the corresponding PDI variants and the CDI under the same subset of simulated conditions as Fig. S1 .
Figure S15 : Visualization of normalized model-predicted optical flow for Ouchi illusions. The layout follows the conventions introduced in Fig. S1 . This figure complements the results in Figure 6 by showing the corresponding Ouchi variants evaluated under the same subset of simulated conditions as Fig. S1 .
Deep neural networks trained on natural images are shown to produce outputs consistent with human observers for brightness illusions. While this phenomenon has been documented across architectures, all evidence, to date, is measured at the output level: restored pixels, decoded trajectories, or classification decisions. Whether these models actually represent illusions internally, and if so where and how, remains unknown. We show that denoising models develop illusion-sensitive representations at specific internal layers, across varied architectures. Specifically, we identify the layers and channels that discriminate illusory from physically matched control regions. We show that the denoising objective is a more important driver of the effect than the architecture. On domain-appropriate stimuli, these activations track a validated psychophysical model of human brightness perception (FLODOG; Spearman ρ≥0.70) and scale monotonically with parametric illusion strength. Leveraging these findings, we provide causal evidence via channel ablation showing that illusion-sensitive channels specifically and substantially affect the internal signal. Yet injecting these representations into the generation pipeline produces no measurable pixel shift across all tested architectures; we term such representations perceptual phantoms: active in internal processing yet invisible to any output-based evaluation. While related internal-output dissociations have been characterized in language models, this is the first such characterization for perceptual representations in denoising vision models.
How does an intelligent visual system combine what objects look like with how they move while remaining robust as appearance changes? We addressed this question by comparing human perception and neural activity in macaque inferior temporal cortex with representations from image- and video-based neural networks spanning recognition, segmentation, optic-flow processing and predictive world modeling. Temporal integration improved object representations, but most video recognition models generalized poorly when appearance was disrupted while motion structure was preserved. Humans and macaque IT remained robust. Notably, predictive world models combined strong cross-appearance generalization with the closest correspondence to IT, outperforming other video-modeling approaches in neural fidelity. Yet no model reproduced the cortical transformation from early appearance-dominated responses toward later appearance-invariant motion coding. These results identify progressive integration of motion into object representations as a principle of robust dynamic vision and implicate predictive learning as a promising route toward realizing this computation in artificial systems.
Matteo Dunnhofer, Christian Micheloni, Kohitij Kar
York University, Department of Biology and Centre for Vision Research, Toronto, Canada · Department of Mathematics, Computer Science, and Physics, University of Udine, Udine, Italy
Human visual perception offers valuable insights for understanding computational principles of motion-based scene interpretation. Humans robustly detect and segment moving entities that constitute independently moveable chunks of matter, whether observing sparse moving dots, textured surfaces, or naturalistic scenes. In contrast, existing computer vision systems lack a unified approach that works across these diverse settings. Inspired by principles of human perception, we propose a generative model that hierarchically groups low-level motion cues and high-level appearance features into particles (small Gaussians representing local matter), and groups particles into clusters capturing coherently and independently moveable physical entities. We develop a hardware-accelerated inference algorithm based on parallelized block Gibbs sampling to recover stable particle motion and groupings. Our model operates on different kinds of inputs (random dots, stylized textures, or naturalistic RGB video), enabling it to work across settings where biological vision succeeds but existing computer vision approaches do not. We validate this unified framework across three domains: on 2D random dot kinematograms, our approach captures human object perception including graded uncertainty across ambiguous conditions; on a Gestalt-inspired dataset of camouflaged rotating objects, our approach recovers correct 3D structure from motion and thereby accurate 2D object segmentation; and on naturalistic RGB videos, our model tracks the moving 3D matter that makes up deforming objects, enabling robust object-level scene understanding. This work thus establishes a general framework for motion-based perception grounded in principles of human vision.