Bayesian Inference of Psychometric Variables From Brain and Behavior in Implicit Association Tests
Authors: Christian A. Kothe, Sean Mullen, Michael V. Bronstein, Grant Hanada, Marcelo Cicconet, Aaron N. McInnes, Tim Mullen, Marc Aafjes, Scott R. Sponheim, Alik S. Widge
Organizations: Intheon, La Jolla, CA, USA · Department of Psychiatry and Behavioral Sciences, University of Minnesota, MN, USA · Institute for Health Informatics, University of Minnesota, MN, USA · Deliberate AI, New York, NY, USA · Minneapolis VA Medical Center, MN, USA
Objective. We establish a principled method for inferring mental health related psychometric variables from neural and behavioral data using the Implicit Association Test (IAT) as the data generation engine, aiming to overcome the limited predictive performance (typically under 0.7 AUC) of the gold-standard D-score method, which relies solely on reaction times. Approach. We propose a sparse hierarchical Bayesian model that leverages multi-modal data to predict experiences related to mental illness symptoms in new participants. The model is a multivariate generalization of the D-score with trainable parameters, engineered for parameter efficiency in the small-cohort regime typical of IAT studies. Data from two IAT variants were analyzed: a suicidality-related E-IAT (n=39) and a psychosis-related PSY-IAT (n=34). Main Results. Our approach overcomes a high inter-individual variability and low within-session effect size in the dataset, reaching AUCs of 0.73 (E-IAT) and 0.76 (PSY-IAT) in the best modality configurations, though corrected 95% confidence intervals are wide (±0.18) and results are marginally significant after FDR correction (q=0.10). Restricting the E-IAT to MDD participants improves AUC to 0.79 [0.62,0.97] (significant at q=0.05). Performance is on par with the best reference methods (shrinkage LDA and EEGNet) for each task, even when the latter were adapted to the task, while the proposed method was not. Accuracy was substantially above near-chance D-scores (0.50-0.53 AUC) in both tasks, with more consistent cross-task performance than any single reference method. Significance. Our framework shows promise for enhancing IAT-based assessment of experiences related to entrapment and psychosis, and potentially other mental health conditions, though further validation on larger and independent cohorts will be needed to establish clinical utility.
Figures & tables
Figure 1: ( a ) Block/trial structure used for E-IAT and PSY-IAT task designs. A “congruent” or “incongruent” block represents a block of trials where the concept-attribute pair (i.e., “trapped/me”) displayed is in agreement or conflict, respectively, with the subject type (patient / control). ( b , c ) Screenshots of PSY-IAT task showing a trial from the two block types. ( b ) depicts an incongruent trial for a person with schizophrenia (a congruent trial for a person without schizophrenia). ( c ) depicts a congruent trial for a person with schizophrenia (incongruent for a person without schizophrenia).
Figure 2: ( a ) A diagram of the data capture setup showing the positioning of the sensors relative to the subject. ( b ) A photograph of the data collection setup at the University of Minnesota used for the E-IAT task data collection. The same setup was used for the collection of the PSY-IAT task at the Minneapolis VA Medical Center.
Figure 3: Diagram for the USBL model, shown for a combination of EEG and FAU modalities. Here T is the number of trials in a session and P is the number of participants in the dataset being analyzed. The modality superscripts EEG and FAU have been shortened to E and F, respectively. XE and XF are the preprocessed observations for the current trial and participant (subscripts omitted) and WE and WF are the associated weight matrices. y is the dependent variable for the current participant (same for all trials), α is the vector of per-modality weights αm , A is the latent EEG contrast matrix, βF is the unscaled weight matrix for the FAU modality and τ and λ are the global and local (per-channel) horseshoe scales, respectively. ΣU is the model-derived EEG spatial covariance matrix and σi is the learned innovation scale for the EEG smoothing (GRW) prior. γ is the vector of per-region hyper-parameters estimated as part of the model.
Method
Modalities
AUC
Sensitivity
Specificity
Proposed
EEG
0.65±0.21 [0.43, 0.86] pBH=0.42
0.64±0.21 [0.41, 0.86]
0.65±0.25 [0.39, 0.91]
EEG, Gaze
0.73±0.17 † [0.56, 0.91] pBH=0.053
0.48±0.24 [0.23, 0.73]
0.82±0.21 [0.61, 1.04]
EEG, FAU
0.48±0.22 [0.24, 0.71] pBH=0.88
0.49±0.23 [0.24, 0.73]
0.49±0.26 [0.21, 0.77]
EEG, Dyn
0.48±0.23 [0.24, 0.72] pBH=0.88
0.42±0.22 [0.18, 0.65]
0.52±0.27 [0.23, 0.80]
Gaze
0.40±0.21 [0.17, 0.62] pBH=0.61
0.56±0.19 [0.35, 0.76]
0.33±0.25 [0.07, 0.58]
FAU, Dyn
0.43±0.24 [0.18, 0.69] pBH=0.69
0.40±0.23 [0.15, 0.64]
0.50±0.27 [0.21, 0.78]
Table 1: Performance comparison between the proposed method and a battery of reference methods across different modality combinations, on E-IAT data. Alternative hyper-prior settings are reported in table 4 . Note results on FAU and Dyn features in the E-IAT were not significantly different from chance level for the baseline methods (similarly to the proposed method), and are not reproduced in the table for conciseness. Values shown as mean ± SD across 50 CV folds; bracketed intervals are Nadeau & Bengio corrected 95% CIs on the mean. CIs were not truncated at [0, 1] to show nominal coverage rate. For the proposed method, BH-corrected p -values ( pBH ) are reported below each AUC, corrected across all 16 instantiations of the (unmodified) proposed method (entries in tables 1 , 2 , and 5 ). Results marginally significant at q=0.10 are shown in bold and marked with †.
Method
Modalities
AUC
Sensitivity
Specificity
Proposed
EEG
0.62±0.26 [0.35, 0.90] pBH=0.61
0.75±0.22 [0.51, 0.98]
0.56±0.26 [0.28, 0.83]
EEG, Gaze
0.42±0.25 [0.16, 0.69] pBH=0.69
0.48±0.30 [0.16, 0.80]
0.47±0.31 [0.15, 0.80]
EEG, FAU, Gaze
0.67±0.18 [0.48, 0.87] pBH=0.22
0.65±0.20 [0.43, 0.86]
0.66±0.28 [0.37, 0.96]
EEG, FAU
0.69±0.20 [0.48, 0.90] pBH=0.22
0.69±0.22 [0.46, 0.92]
0.60±0.27 [0.31, 0.89]
EEG, Dyn
0.60±0.21 [0.38, 0.83] pBH=0.61
0.53±0.24 [0.28, 0.78]
0.58±0.29 [0.28, 0.89]
Gaze
0.58±0.23 [0.35, 0.82] pBH=0.69
0.74±0.27 [0.46, 1.02]
0.39±0.22 [0.16, 0.62]
Table 2: Performance comparison between the proposed method and a battery of reference methods across different modality combinations, on PSY-IAT data. Values shown as mean ± SD across 50 CV folds; bracketed intervals are Nadeau & Bengio corrected 95% CIs on the mean. For the proposed method, BH-corrected p -values ( pBH ) are reported below each AUC, corrected across the 16 instantiations of the proposed method (entries in tables 1 , 2 , and 5 ). Results marginally significant at q=0.10 are shown in bold and marked with †.
Task
Modalities
Model Variant
MSE
Cross-Entropy
E-IAT
EEG, Gaze
Base
0.249 [ 0.247 , 0.250 ]
0.690 [ 0.688 , 0.693 ]
Calibrated
0.228 [ 0.190 , 0.267 ]
0.650 [ 0.562 , 0.739 ]
PSY-IAT
FAU, Dyn
Base
0.242 [ 0.236 , 0.249 ]
0.678 [ 0.665 , 0.691 ]
Calibrated
0.233 [ 0.180 , 0.285 ]
0.685 [ 0.521 , 0.850 ]
FAU, Dyn, Gaze
Base
0.242 [ 0.236 , 0.248 ]
0.677 [ 0.666 , 0.688 ]
Calibrated
0.227 [ 0.180 , 0.275 ]
0.667 [ 0.527 , 0.807 ]
Table 3: Comparison of Brier scores (MSE) and cross-entropy between base and calibrated models, for all modality combinations with highest AUC results. Values shown as mean [95% CI]. Lower scores indicate better calibration.
Task / Modalities
Variant
AUC
Sensitivity
Specificity
E-IAT EEG, Gaze
σ =0.1 (default)
0.73±0.17 [0.56, 0.91]
0.48±0.24 [0.23, 0.73]
0.82±0.21 [0.61, 1.04]
E-IAT EEG, Gaze
σ =0.01
0.77±0.17 [0.59, 0.95]
0.48±0.23 [0.23, 0.72]
0.85±0.20 [0.64, 1.05]
PSY-IAT EEG
Dugh (default)
0.62±0.26 [0.35, 0.90]
0.75±0.22 [0.51, 0.98]
0.56±0.26 [0.28, 0.83]
PSY-IAT EEG
LR
0.56±0.20 [0.35, 0.78]
0.53±0.22 [0.30, 0.76]
0.48±0.29 [0.18, 0.79]
PSY-IAT FAU, Dyn
σ =0.1 (default)
0.73±0.19 [0.53, 0.93]
0.73±0.25 [0.47, 0.99]
0.65±0.27 [0.36, 0.93]
PSY-IAT FAU, Dyn
σ =0.01
0.71±0.20 [0.49, 0.92]
0.71±0.25 [0.44, 0.97]
0.61±0.29 [0.30, 0.91]
Table 4: Supplementary parameter variations of the proposed method, comparing an alternative smoothness hyper-prior ( σ =0.01; see discussion) and a simpler Bayesian low-rank (LR) model variant for EEG. Values shown as mean ± SD across 50 CV folds; bracketed intervals are Nadeau & Bengio corrected 95% CIs on the mean.
Method
Modalities
AUC
Sensitivity
Specificity
Proposed
EEG, Gaze
0.79±0.17 * [0.62, 0.97] pBH=0.032
0.47±0.23 [0.22, 0.71]
0.82±0.21 [0.60, 1.05]
EEG, Gaze ( σ =0.01)
0.84±0.15 [0.68, 1.00]
0.44±0.21 [0.22, 0.67]
0.91±0.18 [0.72, 1.09]
Table 5: Performance of the proposed method on the E-IAT when restricting the analysis to MDD participants only ( n=33 ), excluding the 6 control subjects. This addresses the concern (see section Ground Truthing) that our model may partly be detecting an MDD/CTL distinction rather than the entrapment contrast. Values shown as mean ± SD across 50 CV folds; bracketed intervals are Nadeau & Bengio corrected 95% CIs on the mean. The default- σ configuration is included in the BH-FDR family jointly with tables 1 / 2 ; the alternative σ =0.01 variant is included for informational purposes and was not enrolled in statistical tests. AUC results significant after BH correction ( pBH<0.05 ) are shown in bold and marked with *.
Figure 4: EEG and eye-tracking weights for the best-performing E-IAT model; ( a ) spatial EEG weight maps, ( b ) eye tracking weight time course (note 0.0 is not at the center of the vertical axis).
Figure 5: Mean ERP at channel F7 (top) by group (left=low entrapment, right=high entrapment) for each condition (“trapped/me” vs “free/me”) with the scalp topographies underneath at time t=500 msec after stimulus onset.
Figure 6: Mean ERP at channel T7 (top) by group (left=control, right=psychosis) for each condition (“Schizophrenia/me” vs “NOT Schizophrenia/me”) with the scalp topographies underneath at time t=300 msec after stimulus onset.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
orig y→ new y′ (C/I)
(+) Stim. Assoc.
( − ) Stim. Assoc.
(+) Participant ( y=1 )
1→1 (C)
1→0 (I)
( − ) Participant ( y=0 )
0→0 (I)
0→1 (C)
Appendix
Table 6: Basic operation of the “recoding” trick. The label gets flipped on the right-hand column and becomes equivalent to the (crossed) participant/trial congruency factor.
Figure 7: Weights for the PSY-IAT model using Dynam+AU modalities; ( a ) AU weights over time (note the high stationarity), ( b ) matrix of Dynam weights (noisy).
Multimodal behavioral analysis offers a scalable approach to assessing depression, anxiety, and stress, yet generic fusion models often ignore the psychometric structure of questionnaire labels. In DASS-21, risk labels are derived from ordered symptom items through fixed item-to-subscale mappings. We propose DynaBridge, a dynamic summary-guided cross-task multimodal framework for DASS-structured mental health assessment. DynaBridge encodes acoustic, visual, and textual cues across multiple sessions and augments them with frozen-LLM-generated DASS-aware summaries as participant-level semantic evidence. It predicts ordinal item distributions, reconstructs depression, anxiety, and stress risk evidence from item-level soft scores, and fuses this evidence with direct multimodal risk predictions. A confidence-aware refinement strategy further incorporates high-confidence semantic cues conservatively. On the official AdoDAS validation split, DynaBridge outperforms the official baseline and representative multimodal methods, achieving 0.5012 mean F1 for D/A/S risk prediction and 0.3216 mean QWK for DASS-21 item prediction. These results show the value of bridging multimodal cues, semantic summaries, and DASS-21 psychometric structure.
Shiyu Teng, Haichen Yu, Jiaqing Liu +6
College of Information Science and Engineering Ritsumeikan University Ibaraki, Osaka, Japan · College of Computer Science and Technology Zhejiang University Hangzhou, Zhejiang, China
Psychiatry research seeks to understand the manifestations of psychopathology in behavior, as measured in questionnaire data, by identifying a small number of latent factors that explain them. While factor analysis is the canonical tool for this purpose, the resulting factors may not be interpretable, and may also be subject to confounding variables. Moreover, missing data are common, and explicit imputation is often required. To overcome these limitations, we introduce Interpretability Constrained Questionnaire Factorization (ICQF), a non-negative matrix factorization method with regularization tailored for questionnaire data. Our method aims to promote factor interpretability and solution stability. We provide an optimization procedure with theoretical convergence guarantees, and an automated procedure to determine latent dimensionality accurately. We validate these procedures using realistic synthetic data. We demonstrate the effectiveness of our method in a widely used general-purpose questionnaire, in two independent datasets (the Healthy Brain Network and Adolescent Brain Cognitive Development studies). Specifically, we show that ICQF preserves diagnostic information across a range of disorders, outperforming competing methods for smaller dataset sizes, and improves interpretability, as assessed by our clinical research collaborators and co-authors. This suggests that the regularization in our method matches domain characteristics, in addition to satisfying qualitative desiderata.
Ka Chun Lam, Francisco Pereira, Bridget W Mahony +1
Machine Learning Core National Institute of Mental Health, National Institutes of Health · Section on Developmental Neurogenomics, Human Genetics Branch National Institute of Mental Health, National Institutes of Health
Precision mental health requires treatment decisions that account for heterogeneous symptoms reflecting multiple clinical domains. However, existing methods for estimating individualized treatment effects (ITE) rely on a single summary outcome or a specific set of observed symptoms or measures, which are sensitive to symptom selection and limit generalizability to unmeasured yet clinically relevant domains. We propose DRIFT, a new maximin framework for estimating robust ITEs from high-dimensional item-level data by leveraging latent factor representations and adversarial learning. DRIFT learns latent constructs via generalized factor analysis, then constructs an anchored on-target uncertainty set that extrapolates beyond the observed measures to approximate the broader hyper-population of potential outcomes. By optimizing worst-case performance over this uncertainty set, DRIFT yields ITEs that are robust to underrepresented or unmeasured domains. We further show that DRIFT is invariant to admissible reparameterizations of the latent factors and admits a closed-form maximin solution, with theoretical guarantees for identification and convergence. In analyses of a randomized controlled trial for major depressive disorder (EMBARC), DRIFT demonstrates superior performance and improved generalizability to external multi-domain outcomes, including side effects and self-reported symptoms not used during training.
Yuying Lu, Wenbo Fei, Yuanjia Wang +1
Department of Biostatistics, Columbia Mailman School of Public Health. · Department of Biostatistics, Peking University Health Science Center. · Beijing International Center for Mathematical Research, Peking University.