cs.CVNov 4, 2026

Context-Aware Autoregressive Image Generation for Emerging Reasoning Properties

Authors: Jixuan Ying, Haoyu Liu, Timing Yang, Tingyu Zhu, Predrag Neskovic, Anand Bhattad, Cihang Xie, Zeyu Zheng, 

Organizations: Johns Hopkins University · University of California, Berkeley · Office of Naval Research · University of California, Santa Cruz

Abstract

Autoregressive image generation has recently emerged as a competitive visual generation paradigm, yet its generation order is often predefined or heuristic, such as next-patch prediction, random-order prediction, or one-step generation for specific scales. We observe that these strategies can cause models to overlook important structural dependencies in images, thereby limiting both structural robustness and reasoning capability. In this work, we propose a context-aware autoregressive generation framework, where the model dynamically determines the next token location based on global context and current local information. Rather than following a fixed or random order, our method allows the generation process to adapt to image structure and semantic dependencies. Extensive experiments show that context-aware generation significantly improves structural robustness across different image AR paradigms, including masked autoregressive and scale-wise autoregressive models. Beyond quantitative improvements, we further observe emerging reasoning properties: the model can better understand complex semantic instructions and generate images that more faithfully follow the given guidance. These findings suggest that generation order is a critical yet underexplored factor in image autoregressive modeling, and that context-aware decoding can unlock stronger reasoning-aware visual generation capabilities. Code is available at https://github.com/Hpesojyjx/Saccades-AR.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Rethinking Visual Autoregressive Sampling with Information-Grounding Guidance

    Sep 28, 2025Ky Dan Nguyen, Hoang Lam Tran, Anh-Dung Dinh +4Visual Autoregressive ModelsConditional Image Generation

  2. From "What" to "How": Constrained Reasoning for Autoregressive Image Generation

    Mar 3, 2026Ruxue Yan, Xubo Liu, Wenya Guo +3Spatially Grounded Image GenerationAutoregressive Image Generation

  3. OmniGen-AR: AutoRegressive Any-to-Image Generation

    Jun 8, 2026Junke Wang, Xun Wang, Qiushan Guo +4Conditional Image GenerationImage Generation