AI-Driven Neural Surrogates for In Silico Design of Cognitive-Affective Neuromodulation Targets
Authors: Marco Rothermel, Madleen Stenger, Soroush Daftarian, Svenja Jule Francke, Bita Shariatpanahi, José C. García Alanis, Mohammad-Ali Nikouei Mahani, Stefan G. Hofmann, Tim Hahn, Hamidreza Jamalabadi
Organizations: Department of Psychiatry and Psychotherapy, University of Marburg, Germany. · Center for Mind, Brain, and Behavior (CMBB), University of Marburg, Germany. · Department of Psychology, University of Marburg, Germany. · Faculty of Medicine, University of British Columbia, Canada.
In neuropsychiatry, the primary goal is often not only to decode brain activity but to change it, for example to lessen a negative affective bias or an overly salient memory. Motivated by control theory, we develop an AI-driven neural-surrogate framework that proposes candidate representational changes and tests their predicted perceptual effects from snapshots of stimulus-evoked fMRI activity, without physical stimulation. The framework combines fMRI decoding, deep generative modeling, and constrained latent-space steering. Valence and memorability are used only as worked examples. Using more than 36,000 image-fMRI observations from four deeply sampled Natural Scenes Dataset participants, subject-specific models recovered coarse generative structure from visually responsive cortex (two-way identification, 0.79-0.88; chance, 0.5). Graded perturbations were reconstructed as images and evaluated with automated scorers and human ratings from 7,200 trials by 18 participants. In the primary VDVAE model, valence shifted from -0.61 to +1.03 SD and memorability from -1.34 to +1.45 SD; a later Versatile Diffusion refinement reduced or altered these effects. Across five perturbation levels, human valence ratings moved in the predicted direction under the linear time-correction model (mean slope, 0.038 SD per unit of alpha; 95 percent CI, 0.003-0.074; positive in 16 of 18 participants). Perceived memorability did not change reliably. Baseline agreement with the automated assessor was suggestive for valence (r = 0.30) and weak for memorability (r = 0.10). Extreme perturbations drifted from the original stimulus, so intended change must be weighed against loss of fidelity. These findings provide a falsifiable upstream method for designing and behaviorally testing candidate representational targets for future neuromodulation in psychiatry, while marking the limits of the present static approximation.
Figures & tables
Figure 1: Conceptual Illustration of Surrogate-Guided In Silico Cognitive-Affective Neuromodulation Design. Reconstructed visual stimuli illustrate the predicted consequences of surrogate-guided representational perturbations targeting emotional valence. Numerical scores below each image summarize automated valence estimates from EmoNet ( Goetschalckx et al., 2019 ) and independent human valence ratings; they do not represent cognitive changes directly induced through brain stimulation. Warmer hues and pleasant scenes are associated with positive-direction perturbations, whereas cooler and more aversive content is associated with negative-direction perturbations. These examples are illustrative and should not be used to infer effect size or consistency; baseline human–assessor correspondence and perturbation effects are evaluated in Sections 4.1 and 4.2 , respectively. Additional examples are available at https://www.neuralsurrogate.com/ .
Figure 2: Framework for AI-Driven In Silico Perturbation Design. Left: Subject-specific ridge regression maps stimulus-evoked 7T fMRI patterns from the Natural Scenes Dataset Allen et al. (2022) into the latent space of a pretrained VDVAE Child (2020) , enabling reconstruction of the viewed stimuli. Valence and memorability are quantified using EmoNet Goetschalckx et al. (2019) and MemNet Khosla et al. (2015) . Right: Empirical attribute directions are applied at graded strengths to fMRI-decoded latent representations. The perturbed representations are reconstructed using a CLIP-conditioned diffusion pipeline Radford et al. (2021) ; Ozcelik and VanRullen (2023) and evaluated using model-based scores, independent human ratings, and image-fidelity measures. The fMRI-space projection visualizes candidate activity-pattern directions under the learned decoder; the pipeline does not derive physical stimulation parameters.
Variable
Category
Count
Percent
Age group
21–29
12
66.7
30–39
6
33.3
Gender
Female
10
55.6
Male
8
44.4
Mental-health history
No, never
12
66.7
Yes, in the past
2
11.1
Table 1: Participant characteristics for the independent behavioral evaluation ( N=18 ).
Figure 3: Independent human evaluation of reconstructed stimuli. (a) Participants completed the PHQ-9 and a demographic survey, followed by task instructions and practice. The main experiment contained one original-image block and nine reconstruction blocks, with 40 trials per block. The reconstruction conditions comprised the unperturbed baseline ( α=0 ) and valence- or memorability-targeted perturbations at α∈{−4,−2,2,4} . Each image was shown for 350 ms and followed by ratings of arousal, valence, and perceived memorability. (b) Before each block, a fixed demonstration screen illustrated the valence–arousal space using OASIS images ( Kurdi et al., 2017 ) ; memorability was demonstrated separately. Demonstration images were not drawn from NSD, were not generated by the surrogate pipeline, and were excluded from the analyses. Participants rated only the perceptual consequences of the in silico perturbations; no neurostimulation was administered.
Property
Comparison
Evaluator
n
r [95% CI]
p
Valence
Score preservation
EmoNet
4 fMRI subjects
0.63 [0.61, 0.65]
<0.001
Human–assessor agreement
Human ratings–EmoNet
40 images
0.30 [ −0.01 , 0.56]
0.058
Memorability
Score preservation
MemNet
4 fMRI subjects
0.77 [0.75, 0.79]
<0.001
Human–assessor agreement
Human ratings–MemNet
40 images
0.10 [ −0.22 , 0.40]
0.551
Table 2: Baseline score preservation and human–assessor agreement. Automated score preservation was quantified by correlating assessor scores at the unperturbed baseline with scores from the corresponding final-diffusion image-to-image reconstructions. Correlations were Fisher- z averaged across the four fMRI subjects, with p values Holm-adjusted across the 20 preservation tests. Human–assessor agreement was calculated across the 40 source-image indices in the unperturbed behavioral condition by correlating mean human ratings with standardized α=0 Versatile Diffusion assessor scores averaged across the four fMRI subjects, rather than with the separate image-to-image reference scores. This comparison is source-index matched; the fMRI-subject provenance of the displayed images was not recorded. Human–assessor confidence intervals use the Fisher- z transformation, and their two-sided p values are unadjusted.
Figure 4: Attribute-score changes across latent perturbation strengths. (a–b) Changes relative to the unperturbed condition ( α=0 ), expressed in baseline standard-deviation units, for valence and memorability. Individual observations are shown as unconnected points; lines connect group means, and error bars denote 95% confidence intervals. The symmetric-logarithmic axes have a linear region within ±0.10 SD. (c–d) Participant- and assessor-level slopes estimated across the full range from α=−4 to α=+4 , in SD units per unit of α . Error bars denote 95% confidence intervals, and the symmetric-logarithmic axes have a linear region within ±0.01 SD per unit of α . (e–f) Descriptive kernel-density estimates and rug plots of the same full-range slopes. These distributions are shown for visualization rather than inferential comparison, particularly because the automated estimates contain four fMRI subjects, compared with 18 human participants. Human estimates incorporate the leave-one-participant-out, assessor-neutral linear fatigue correction. The shared legend identifies Human Scores, VDVAE Scores, and Versatile Diffusion Scores.
Figure 5: Trade-offs between attribute-score modulation and visual fidelity. (a–b) Pearson correlations between assessor scores at each perturbation level and the corresponding scores of the unperturbed reconstructions ( α=0 ), shown for the VDVAE and Versatile Diffusion stages. At α=0 , the correlations quantify score preservation between the unperturbed reconstruction and its image-to-image reconstruction. (c–d) PixCorr and CLIP image-embedding similarity relative to the corresponding α=0 reconstruction. Fidelity measures are oriented so that higher values indicate better preservation. Small, semi-transparent symbols show the four fMRI subjects and are not connected. Lines join only group means; large symbols and error bars show Fisher- z mean correlations with 95% confidence intervals in panels a–b , and means ± SEM in panels c–d . Color denotes the targeted attribute: valence (blue) or memorability (magenta). To improve readability, the valence and memorability series are slightly horizontally offset; this visual jitter has no numerical meaning. In the fidelity panels, filled squares denote PixCorr and open circles denote CLIP similarity.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Regression target
Output per image
Ridge penalty λ
Maximum iterations
VDVAE latents
91,168
50,000
10,000
CLIP text embeddings
77×768
100,000
50,000
CLIP vision embeddings
257×768
60,000
50,000
Appendix
Table A1: Parameters of the subject-specific fMRI-to-feature regressions. All models included an intercept and used training-set fMRI normalization.
Resolution
Coordinates
S1
S2
S5
S7
1×1
32
0.252
0.230
0.243
0.200
4×4
1,024
0.187
0.149
0.117
0.117
8×8
8,192
−0.011
−0.012
−0.015
−0.013
16×16
65,536
−0.022
−0.021
−0.019
−0.019
32×32
16,384
−0.023
−0.022
−0.019
−0.019
Appendix
Table B1: Layer-resolved held-out R2 of the fMRI-to-VDVAE mapping. Values are reported relative to a training-mean latent predictor. Coordinates denote the number of latent variables within each spatial-resolution group.
Measure
Direct θ
Neural RR⊤θ
Permuted/random routing
Cosine with θ
1.00
0.30 – 0.37
–
Energy in 1×1 and 4×4 blocks
3% – 9%
23% – 45%
<3%
PixCorr with α=0
0.60 – 0.86
0.13 – 0.24
0.92 – 0.98
Memorability: images with intended 0→+2 change
65% – 92%
0% – 16%
Not consistently observed
Subjects with monotonic memorability
4/4
0/4
0/4
Appendix
Table B2: Comparison of direct and neural-routed latent perturbations. Ranges summarize the four NSD subjects and the two cognitive attributes where applicable. PixCorr was computed relative to the corresponding unperturbed reconstruction.
Figure C1: Held-out validation of the empirical steering directions. (a) Distribution of held-out VDVAE latent projections onto the EmoNet valence direction for images in the bottom 15%, central 70%, and top 15% of held-out EmoNet scores. Dashed lines mark the mean projections of the low- and high-scoring groups. (b) Held-out EmoNet scores versus projection onto the valence direction ( r=0.812 ). (c) Corresponding projection distributions for the MemNet memorability direction. (d) Held-out MemNet scores versus projection onto the memorability direction ( r=0.834 ). The panels show fMRI subject 1; each direction was estimated from training-image latents only. Pearson correlations quantify held-out geometric correspondence with the frozen assessor; they do not constitute human or neural validation.
Figure D1: Illustrative valence and memorability perturbation sequences. (a) Two valence-targeted Versatile Diffusion reconstruction sequences from one fMRI subject. (b) Two memorability-targeted sequences from the same reconstruction pipeline. The first column shows the original NSD image; the remaining columns show reconstructions at α∈{−4,−2,0,+2,+4} . The dark border identifies the unperturbed α=0 reconstruction. Negative and positive values indicate the intended directions of decreasing and increasing automated attribute scores, respectively. The examples were selected for visibly interpretable trajectories and are not intended to represent the distribution of effects across all images. They illustrate both the intended modulation and the increasing generative drift observed at extreme perturbation strengths. Additional examples are available at https://www.neuralsurrogate.com/ .
Figure E1: Surrogate-predicted neural patterns under extreme cognitive-attribute perturbations. Predicted cortical beta-pattern differences relative to the unperturbed condition ( α=0 ). Panels a and b show negative ( α=−4 ) and positive ( α=+4 ) valence perturbations, respectively; panels c and d show low- ( α=−4 ) and high-memorability ( α=+4 ) perturbations. Yellow indicates predicted increases and cyan indicates predicted decreases relative to α=0 . These maps are descriptive outputs of the fMRI-derived surrogate and do not represent independently measured activation changes or validated stimulation targets. No inferential statistical tests were performed.
Figure F1: Behavioral time-course and negative-control diagnostics. (a,b) Changes in valence and perceived-memorability ratings across the ten fixed block positions, expressed relative to the unperturbed reconstruction block ( α=0 , block 2). Small points show individual participants; lines connect only the group means, and error bars denote 95% confidence intervals. (c) Participant-specific within-block rating slopes for valence, perceived memorability, and arousal. Slopes were estimated across the 40 trial positions within each block. (d) Arousal-rating changes across block position, used as a negative-control outcome because arousal was not directly targeted. The between-block curves are descriptive rather than causal estimates because block position, image composition, and perturbation condition were not independently randomized.
Figure F2: Sensitivity of the human effects to alternative detrending specifications. (a,b) Mean valence and perceived-memorability changes under the raw, baseline-composition, within-block-only, assessor-neutral linear-time, and assessor-neutral quadratic-time specifications. Changes are expressed relative to α=0 in participant-specific baseline-SD units; error bars denote 95% confidence intervals. (c,d) Participant-level observations after the primary leave-one-participant-out assessor-neutral linear correction. Small points are unconnected participant estimates; lines join only group means. The comparison shows that the valence inference depends principally on correction of the between-block trend, whereas perceived memorability remains close to zero under every specification.
Valence
Perceived memorability
Analysis
Mean slope [95% CI]
Positive
Mean slope [95% CI]
Positive
Raw
−0.006 [ −0.021 , −0.034 ]
11/18
−0.004 [ −0.021 , −0.013 ]
7/18
Baseline composition
−0.014 [ −0.015 , −0.044 ]
12/18
−0.004 [ −0.021 , −0.013 ]
7/18
Within-block only
−0.006 [ −0.021 , −0.034 ]
11/18
−0.004 [ −0.021 , −0.013 ]
7/18
Assessor-neutral linear (primary)
−0.038 [ −0.003 , −0.074 ]
16/18
−0.011 [ −0.028 , −0.007 ]
5/18
Assessor-neutral quadratic
−0.010 [ −0.020 , −0.040 ]
11/18
−0.009 [ −0.027 , −0.008 ]
6/18
Appendix
Table F1: Full-range participant-slope sensitivity across behavioral analyses. Slopes were estimated across α=−4,−2,0,+2,+4 and are expressed in participant-specific baseline SD per unit of α . Confidence intervals summarize the participant-level slopes. Exact-source slopes were calculated from the corresponding participant-specific matched-image condition means.
Figure F3: Exact-source matched behavioral sensitivity analysis. (a) Valence changes when each nonzero- α image was paired with the same source image in the α=0 block. (b) Corresponding analysis for perceived memorability. Cyan squares show all available exact-source pairs; magenta diamonds additionally restrict the analysis to assessor-neutral images with an absolute Versatile Diffusion assessor change no larger than 0.50 baseline SD. Small points show individual participants; lines connect only group means, and error bars denote 95% confidence intervals. Values are displayed on a signed symmetric-logarithmic scale. Exact matching controls source-image identity but does not remove the fixed-order confound; the leave-one-participant-out linear time correction was therefore retained.
Decoding visual experience from brain activity has advanced substantially, but current brain-to-text systems largely recover semantic content while discarding affect. Additionally, language models can generate emotional text when prompted with categorical labels, but such labels collapse rich inter-subject variability into coarse discrete bins. We present EmoMind, the first end-to-end pipeline for decoding affective captions directly from fMRI signals. EmoMind first retrieves a semantically grounded neutral scene description from brain-decoded visual features, then rewrites it using a continuous 34-dimensional emotion vector decoded from the same fMRI recording. To control the balance between content preservation and affective expression, we train the rewriter with classifier-free guidance against an identity-preserving null branch, enabling smooth interpolation between semantic fidelity and affective expressivity. We evaluate affective caption generation with a three-axis validation framework spanning subject-specificity, structural geometry, and causal control. We further augment this framework with a synthetic-brain substitution test that probes robustness to the measurement apparatus, and we benchmark each axis against GPT-4 prompted with brain-decoded top-5 emotion labels as a strong discrete baseline. Across two independent emotion fMRI datasets, EmoMind significantly outperforms label-prompted GPT-4 on all three axes, with the largest gains on metrics that require person-specific affective structure rather than population-level emotion aggregation. These results establish continuous brain-decoded affect as a viable control signal for individualized affective caption generation and open new directions for studying individual affective brain organisation.
Bilal A. Mohammed, Lin Gu, Ruogu Fang
Department of Biomedical Engineering Vanderbilt University Nashville, TN · Research Institute of Electrical Communication Tohoku University Sendai, Japan · Department of Biomedical Engineering University of Florida Gainesville, FL
Parkinson's disease (PD) affects over ten million people worldwide. Although temporal interference (TI) and deep brain stimulation (DBS) are promising therapies, inter-individual variability limits empirical treatment selection, increasing non-negligible surgical risk and cost. Previous explorations either resort to limited statistical biomarkers that are insufficient to characterize variability, or employ AI-driven methods which is prone to overfitting and opacity. We bridge this gap with a pretraining-finetuning framework to predict outcomes directly from resting-state fMRI. Critically, a generative virtual brain foundation model, pretrained on a collective dataset (2707 subjects, 5621 sessions) to capture universal disorder patterns, was finetuned on PD cohorts receiving TI (n=51) or DBS (n=55) to yield individualized virtual brains with high fidelity to empirical functional connectivity (r=0.935). By constructing counterfactual estimations between pathological and healthy neural states within these personalized models, we predicted clinical responses (TI: AUPR=0.853; DBS: AUPR=0.915), substantially outperforming baselines. External and prospective validations (n=14, n=11) highlight the feasibility of clinical translation. Moreover, our framework provides state-dependent regional patterns linked to response, offering hypothesis-generating mechanistic insights.
Siyuan Du, Siyi Li, Shuwei Bai +10
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · Shanghai AI Laboratory, Shanghai, China · University of Science and Technology of China, Anhui, China +6
Changes in spontaneous speech provide an early signal of cognitive dysfunction in Alzheimer's disease (AD) that large language models (LLMs) can detect. However, detection alone cannot establish whether the underlying model representations contribute functionally to behavior. We introduce an activation-guided intervention framework using Qwen3-8B. The framework identifies feed-forward neurons with higher activation rates for AD than control transcripts and modulates their output contributions during generation by scaling the corresponding down-projection weights. This yielded nine edited variants differing in intervention direction, magnitude, and scope. The original and edited models completed the same 12-turn neuropsychological battery, assessed through blinded human ratings and computational linguistic measures. Amplifying AD-associated neurons produced graded impairments in story recall, verbal fluency, working memory, procedural discourse, scene construction, and coreference resolution. Attenuation largely preserved performance and selectively improved several outcomes. Amplification also reduced lexical surprisal, idea density, syntactic complexity, and discourse quantity, broadly paralleling changes reported in human AD speech. These findings show that neurons identified solely from clinical language differences can influence behavior across multiple cognitive domains, providing proof of concept for an AD-related computational phenotype and a controlled framework for experimentally examining links between language and broader cognitive dysfunction.
Rui He, Ercong Nie, Hong Jiang +3
Grammar and Cognition Lab, Department of Translation & Language Sciences, Universitat Pompeu Fabra, Barcelona, Spain. · School of Foreign Languages, Shanghai Jiao Tong University, Shanghai, China. · Zhuhai People’s Hospital (The Affiliated Hospital of Beijing Institute of Technology, Zhuhai Clinical Medical College of Jinan University), Zhuhai, China. +4