cs.AIJan 20, 2026

RoSeViT: Role-Separated Vision Transformers for ARC Visual Reasoning

Authors: Zhiguang Liu, Yi Shang

Organizations: University of Missouri - Columbia

Abstract

ARC tasks can be formulated as visual transformation problems, where a model must infer a high-level transformation rule from a small set of demonstrations and apply it consistently across a grid. While dense Vision Transformers (ViTs) perform well in this setting, they entangle local visual state and global transformation logic within a shared token representation, which can hinder consistent reasoning. In this paper, we introduce RoSeViT, a role-separated Vision Transformer that explicitly decouples high-level instruction inference from visual workspace computation. RoSeViT uses controller tokens to aggregate transformation-level information and workspace tokens to represent the visual grid, with a structured attention pass in which nonlocal workspace interactions are routed through the controller. This design is motivated by our formal analysis showing that enforcing a shared instruction representation can reduce the effective complexity of visual transformation modeling. RoSeViT outperforms dense ViT baselines under the standard VARC training and evaluation protocol for ARC. In a controlled backbone replacement setting, it improves ARC-1 accuracy from 54.5 to 56.6, and with recurrent refinement reaches 58.8. When integrated into the full system, RoSeViT-Ensemble improves performance from 60.4 to 62.6 on ARC-1. These results demonstrate that explicitly separating instruction and workspace representations leads to more effective visual reasoning on ARC tasks. GitHub repository: https://github.com/lz7fd/RoSeViT.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning

    Jul 31, 2026Binnan Liu, Yechi Ma, Tian Xie +1Visual ReasoningVisually Grounded Reasoning

  2. Language-Guided Abstraction for Visual Reasoning

    Jun 11, 2026Xu-Jing Ye, Yuan-Gen Wang, Ruping WangVisual ReasoningVision-Language Alignment

  3. SPAR: Single-Pass Any-Resolution ViT for Open-vocabulary Segmentation

    Apr 2, 2026Naomi Kombol, Ivan Martinović, Siniša Šegvić +1Vision TransformerEfficient ViTs