EyeWorld: A Generative World Model of Ocular State and Dynamics

Ziyu Gao; Xinyuan Wu; Xiaolan Chen; Zhuoran Liu; Ruoyu Chen; Bowen Liu; Bingjie Yan; Zhenhan Wang

cs.CVMar 14, 2026

EyeWorld: A Generative World Model of Ocular State and Dynamics

Authors: Ziyu Gao, Xinyuan Wu, Xiaolan Chen, Zhuoran Liu, Ruoyu Chen, Bowen Liu, Bingjie Yan, Zhenhan Wang, +5 more

Organizations: School of Optometry, The Hong Kong Polytechnic University, Hong Kong. · Eye Center of Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, Zhejiang, China · Zhejiang Provincial Key Laboratory of Ophthalmology, Zhejiang Provincial Clinical Research Center for Eye Diseases, Zhejiang Provincial Engineering Institute On Eye Diseases, Hangzhou, 310009, Zhejiang, China · ELLIS Institute Finland, Espoo 02150, Finland. · Aalto University, Espoo 02150, Finland. · Yong Loo Lin School of Medicine, National University of Singapore, Singapore. · Centre for Eye and Vision Research (CEVR), 17W Hong Kong Science Park, Hong Kong. · Research Centre for SHARP Vision (RCSV), The Hong Kong Polytechnic University, Hong Kong.

Abstract

Ophthalmic decision-making depends on subtle lesion-scale cues interpreted across multimodal imaging and over time, yet most medical foundation models remain static and degrade under modality and acquisition shifts. Here we introduce EyeWorld, a generative world model that conceptualizes the eye as a partially observed dynamical system grounded in clinical imaging. EyeWorld learns an observation-stable latent ocular state shared across modalities, unifying fine-grained parsing, structure-preserving cross-modality translation and quality-robust enhancement within a single framework. Longitudinal supervision further enables time-conditioned state transitions, supporting forecasting of clinically meaningful progression while preserving stable anatomy. By moving from static representation learning to explicit dynamical modeling, EyeWorld provides a unified approach to robust multimodal interpretation and prognosis-oriented simulation in medicine.

Explore similar work

Bridging the Gap in Ophthalmic AI: MM-Retinal-Reason Dataset and OphthaReason Model toward Dynamic Multimodal Reasoning
Aug 22, 2025Ruiqi Wu, Yuang Yao, Na Su +11Medical Image Analysis Multimodal Reasoning
School of Computer Science and Engineering, Southeast University, Nanjing, China · Department of Ophthalmology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China · School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies
Jun 15, 2026Ke Liu, Mengxuan Li, Yanyi Bao +4World Model Learning Clinical Decision Support
College of Computer Science, Zhejiang University, Hangzhou, China. · School of Medicine, Zhejiang University, China. · Department of Biomedical Informatics, Harvard University, USA.
A World Model of Radiologist Reading for Medical Image Representation Learning
May 17, 2026Yiwei Li, Zihao Wu, Huaqin Zhao +5World Model Learning Healthcare
University of Georgia · University of Texas at Arlington · New Jersey Institute of Technology

Aug 22, 2025J/K move · Enter open · S save

Bridging the Gap in Ophthalmic AI: MM-Retinal-Reason Dataset and OphthaReason Model toward Dynamic Multimodal Reasoning

Ruiqi Wu, Yuang Yao, Na Su, Tengfei Ma+10

School of Computer Science and Engineering, Southeast University, Nanjing, China · Department of Ophthalmology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China · School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

Medical Image Analysis Multimodal Reasoning Medical VLMs Multimodal CoT Reasoning RL for VLMs

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have recently demonstrated remarkable reasoning abilities under reinforcement learning (RL) paradigm. However, most existing multimodal medical reasoning models focus on basic reasoning, which refers to shallow inference based on visual feature matching. In contrast, real-world clinical diagnosis extends beyond basic reasoning, demanding complex reasoning that integrates heterogeneous clinical information (such as chief complaints and medical history) with multimodal medical imaging data. To bridge this gap, we introduce MM-Retinal-Reason, an ophthalmic multimodal dataset covering the full spectrum of perception and reasoning. Specifically, it is the first dataset in ophthalmology to encompass both basic and complex reasoning tasks with Chain-of-Thought (CoT) trajectories, aiming to enhance visual-centric reasoning and emulate realistic clinical decision-making. Building upon MM-Retinal-Reason, we propose OphthaReason, the first RL-enhanced ophthalmic multimodal reasoning model with step-by-step reasoning traces. To enable flexible adaptation to both basic and complex reasoning tasks, we further introduce Uncertainty-Aware Dynamic Thinking (UADT), which estimates sample-level uncertainty via entropy and dynamically modulates exploration depth through a shaped advantage mechanism. Comprehensive experiments demonstrate the effectiveness of our model on both basic and complex reasoning tasks, outperforming general-purpose MLLMs, medical MLLMs, RL-based medical MLLMs, and ophthalmic MLLMs by at least 15.47%. Project Page: link.

Abstract

Explore similar work

Bridging the Gap in Ophthalmic AI: MM-Retinal-Reason Dataset and OphthaReason Model toward Dynamic Multimodal Reasoning

Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies

A World Model of Radiologist Reading for Medical Image Representation Learning