Percept-Aware Surgical Planning for Visual Cortical Prostheses with Vascular Avoidance

Galen Pogoncheff; Alvin Wang; Jacob Granley; Michael Beyeler

cs.CVFeb 27, 2026

Percept-Aware Surgical Planning for Visual Cortical Prostheses with Vascular Avoidance

Authors: Galen Pogoncheff, Alvin Wang, Jacob Granley, Michael Beyeler

Organizations: Department of Computer Science, University of California Santa, Barbara · School of Computer Science, Carnegie Mellon University · Department of Psychological & Brain Sciences, University of California, Santa Barbara

Abstract

Cortical visual prostheses aim to restore sight by electrically stimulating neurons in early visual cortex (V1). With the emergence of high-density and flexible neural interfaces, electrode placement within three-dimensional cortex has become a critical surgical planning problem. Existing strategies emphasize visual field coverage and anatomical heuristics but do not directly optimize predicted perceptual outcomes under safety constraints. We present a percept-aware framework for surgical planning of cortical visual prostheses that formulates electrode placement as a constrained optimization problem in anatomical space. Electrode coordinates are treated as learnable parameters and optimized end-to-end using a differentiable forward model of prosthetic vision. The objective minimizes task-level perceptual error while incorporating vascular avoidance and gray matter feasibility constraints. Evaluated on simulated reading and natural image tasks using realistic folded cortical geometry (FreeSurfer fsaverage), percept-aware optimization consistently improves reconstruction fidelity relative to coverage-based placement strategies. Importantly, vascular safety constraints eliminate margin violations while preserving perceptual performance. The framework further enables co-optimization of multi-electrode thread configurations under fixed insertion budgets. These results demonstrate how differentiable percept models can inform anatomically grounded, safety-aware computer-assisted planning for cortical neural interfaces and provide a foundation for optimizing next-generation visual prostheses.

Figures & tables

Figure 1: Percept-aware optimization framework. Given 3D electrode coordinates on FreeSurfer fsaverage anatomy, target percepts, and patient retinotopy, a differentiable model of cortical prosthetic vision predicts elicited percepts. Electrode locations are iteratively updated to minimize perceptual error while enforcing vascular safety constraints. Solid arrows: forward simulation pathway. Dashed arrows: gradient signals used to update electrode positions.

Baseline	Dataset	$\Delta$ MSE $(\%)$ $\downarrow$	$\Delta$ SSIM $(\%)$ $\uparrow$	$\Delta$ Acc. $(\%)$ $\uparrow$
VF Tiling	MNIST	-67.7* [-73.6, -47.2]	11.1* [7.8, 16.7]	62.6* [28.9, 119.9]
VF Coverage	MNIST	-33.4* [-42.0, -16.3]	4.7* [3.1, 7.3]	22.4* [7.8, 34.7]
VF Tiling	CIFAR-10	-58.4* [-87.4, -24.1]	57.4* [35.3, 102.2]	10.2* [5.5, 29.2]
VF Coverage	CIFAR-10	16.0 [-25.2, 39.1]	-10.6 [-24.7, 0.2]	-2.2 [-9.1, 6.25]

Table 1: Relative change in MSE, SSIM, and downstream classification accuracy (Acc.) of the proposed method versus baselines. Values are median percent difference [IQR] across electrode counts and phosphene spreads (

\rho

). * indicates

p\leq 0.01

Figure 2: Comparing percept-aware electrode optimization and Visual Field Tiling. Left : Simulated phosphenes for reading (MNIST) and natural image (CIFAR-10) tasks. Right : Relative improvements in perceptual fidelity across experimental configurations.

Figure 3: Safety-aware electrode optimization with vascular constraints. Left : Cortical surface ( fsaverage left occipital) with high-resolution vascular map (red, displayed in V1 only) and optimized electrode locations (black). Right : Percept SSIM scores without (solid lines) and with (dashed lines) vascular avoidance.

Figure 4: Simulated phosphenes using percept-aware optimization with and without multi-electrode threads under a fixed number of cortical insertions ( $128$ ). Threaded designs improve percept quality without increasing number of cortical insertions.

Brain-Computer Interfaces

Brain-computer interfaces translate neural activity into commands or feedback for communication, movement, rehabilitation, and sensory restoration.

See all 130 papers

A Fully Differentiable Neuro-Soft-Symbolic Framework for Perceptual Task Planning
Sep 18, 2026Hongyan Wei, Wael AbdAlmageedNeuro-Symbolic Planning Robot Task Planning
Clemson University
Adaptive Cortically Constrained EEG-Vision Alignment for Zero-Shot Brain-to-Image Retrieval
Sep 21, 2026Ye Wang, Haokun Ren, Wei Wu +4Electroencephalography EEG-Based Visual Decoding
School of Artificial Intelligence, Chongqing University of Posts and Telecommunications, Chongqing, China · School of Medicine, Shanghai Jiaotong University, Shanghai, China · National Center for Applied Mathematics in Chongqing, Chongqing Normal University, Chongqing, China +1
SurgAM: Surgical Affordance Map Prediction with Multimodal Feature Fusion for Robot Autonomy
Jul 5, 2026Lei Song, Yonghao Long, Mengya Xu +3Robotic Perception Surgical Robotics
Department of Computer Science and Engineering, The Chinese University of Hong Kong · Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China · Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China +3

Sep 18, 2026J/K move · Enter open · S save

A Fully Differentiable Neuro-Soft-Symbolic Framework for Perceptual Task Planning

Hongyan Wei, Wael AbdAlmageed

Clemson University

Neuro-Symbolic Planning Robot Task Planning Neuro-Symbolic Reasoning Robotic Perception

Open paper AskPDF arXiv

Perceptual planning tasks require two key capabilities: accurately perceiving uncertain scenes and planning valid action sequences following logical rules. Conventional methods convert perception into discrete symbolic facts and then plan, discarding perceptual uncertainty and severing task-level feedback to perception. We introduce a generic, fully differentiable neuro-soft-symbolic framework that connects visual perception and task planning within a single computational graph. The framework maintains a continuous soft symbolic state, lifts domain rules into a differentiable soft-

T_P

transition operator, and optimizes action logits over a short planning horizon. Gradients from the planning objective can also update the perception parameters, allowing task-relevant perceptual representations to be refined during planning. On Blocksworld, our method solves 40/40 LatPlan-40 tasks and 596/600 PlanBench-600 tasks, compared with 33/40 for LatPlan and 587/600 for the reasoning-model baseline, while requiring substantially less computation and time. In the perceptual-uncertainty ablation, our method improves the success rate from 59% with frozen perception to 83%. We further conduct task-and-motion simulations on Blocksworld scenes, providing an execution-level validation of the compatibility between decoded task plans and downstream robotic motion execution.

Abstract

Brain-Computer Interfaces

Explore similar work

A Fully Differentiable Neuro-Soft-Symbolic Framework for Perceptual Task Planning

Adaptive Cortically Constrained EEG-Vision Alignment for Zero-Shot Brain-to-Image Retrieval

SurgAM: Surgical Affordance Map Prediction with Multimodal Feature Fusion for Robot Autonomy