Few-Shot Cross-Subject Neural--Image Retrieval via Stimulus-Anchored Latent Bias Calibration
Organizations: School of Biomedical Engineering, Health Science Center, Shenzhen University, and the Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Shenzhen, China
Abstract
In cross-subject neural--image retrieval, comparing the mean of a few stimulus-matched target observations with the global source mean conflates subject-specific representation shifts with stimulus-composition effects. To address this problem, we propose Stimulus-Anchored Latent Bias Calibration (SA-LBC), which estimates the first-order shift under fixed preprocessing and a frozen source-trained representation by averaging residuals between target responses and same-stimulus source-population anchors, then shrinks the correction using source-derived uncertainty. Under uniform sampling without replacement from a finite stimulus population, we derive exact finite-population MSE expressions for the unshrunk anchored and global-mean estimators, including residual--anchor cross-covariance, and establish a necessary and sufficient condition for anchoring to reduce MSE relative to global-mean estimation. In a 10-subject leave-one-subject-out evaluation on THINGS-EEG2, SA-LBC increases mean reciprocal rank (MRR) for the Attention--CLIP ViT-B/32 configuration from 0.1867 without calibration to 0.2013 and 0.2336 with M=2 and M=16 stimulus-level calibration samples, respectively. Across six neural--visual configurations and all calibration budgets, stimulus-matched anchors yield higher MRR than anchors with shuffled stimulus identities, but retrieval gains over no calibration remain representation-dependent. At (M=2), shrinkage improves MRR over the unshrunk anchored estimator in all six configurations, but its effects at larger budgets vary across representations; the relative ranking of translation and orientation corrections also depends on regularization strength. Thus, lower shift-estimation MSE does not guarantee higher retrieval performance. The source code is publicly available at: https://github.com/Xizhang-Yao/SA-LBC.
Figures & tables
| Method | ||||
|---|---|---|---|---|
| No adaptation | 0.1867 0.0495 | 0.1867 0.0495 | 0.1867 0.0495 | 0.1867 0.0495 |
| Global mean | 0.1673 0.0265 | 0.1994 0.0338 | 0.2213 0.0412 | 0.2347 0.0459 |
| Adaptive global mean | 0.1998 0.0459 | 0.2107 0.0460 | 0.2219 0.0475 | 0.2318 0.0489 |
| Shuffled anchor | 0.1953 0.0456 | 0.2056 0.0451 | 0.2166 0.0456 | 0.2275 0.0470 |
| Raw anchored ( ) | 0.1822 0.0341 | 0.2090 0.0401 | 0.2271 0.0450 | 0.2381 0.0481 |
| SA-LBC | 0.2013 0.0464 | 0.2120 0.0476 | 0.2239 0.0495 | 0.2336 0.0504 |
| EEG encoder | Visual backbone | |||||
|---|---|---|---|---|---|---|
| Attention | CLIP ViT-B/32 | 0.23364 | 0.16749 | 0.20697 | 0.19326 | 0.24114 |
| EEGProject | CLIP ViT-B/32 | 0.25908 | 0.17956 | 0.21624 | 0.22060 | 0.26695 |
| TSConv | CLIP ViT-B/32 | 0.21047 | 0.18012 | 0.18171 | 0.21503 | 0.21582 |
| Attention | CLIP RN101 | 0.21311 | 0.15846 | 0.19034 | 0.18487 | 0.22101 |
| Attention | DINOv2 ViT-B/14 | 0.18318 | 0.13609 | 0.16186 | 0.15945 | 0.18855 |
| TSConv | DINOv2 ViT-B/14 | 0.18264 | 0.15619 | 0.15840 | 0.18479 | 0.18641 |
| EEG encoder | Visual backbone | Baseline MRR | MRR ( ) | MRR ( ) | ||
|---|---|---|---|---|---|---|
| Attention | CLIP ViT-B/32 | 0.18675 | +0.01458 | 0.0078 | +0.04690 | 0.0078 |
| EEGProject | CLIP ViT-B/32 | 0.21393 | +0.01224 | 0.0234 | +0.04515 | 0.0078 |
| TSConv | CLIP ViT-B/32 | 0.20947 | -0.00330 | 0.0078 | +0.00100 | 0.6172 |
| Attention | CLIP RN101 | 0.18022 | +0.00944 | 0.0215 | +0.03290 | 0.0078 |
| Attention | DINOv2 ViT-B/14 | 0.15587 | +0.00800 | 0.0078 | +0.02730 | 0.0078 |
| TSConv | DINOv2 ViT-B/14 | 0.18122 | -0.00236 | 0.0547 | +0.00143 | 0.2754 |
Cross-Modal Retrieval
Cross-modal retrieval matches queries and items across different data types, such as text, images, video, audio, and sensor signals.
See all 199 papers