An In-Silico Audit of Topographic Organization in a Multimodal Brain-Response Model
Organizations: StrawAI, Krakow, Poland
Abstract
TRIBE v2 (d'Ascoli et al., 2026) predicts fMRI responses across the cortical surface from video, audio and text. Its authors validated it with category-selective localizers (faces, places, bodies, words) and language contrasts, but not with the classic topographic organization of sensory cortex: retinotopy, tonotopy, and motion selectivity in area MT+. There is a clear architectural reason to expect retinotopy to fail: TRIBE's video features are averaged over space before the response head, which should discard visual-field position. I present mapping stimuli, a fitting pipeline adapted from standard fMRI mapping analyses, and a stand-in model with known ground truth that serves as a positive and negative control, and apply them to TRIBE v2's predictions, comparing the fitted maps with human atlases using spin tests. TRIBE v2 shows a sharp split. There is no contralateral hemifield response and no polar-angle map, robustly across stimulus controls. There is no tonotopic gradient in primary auditory cortex: best frequencies span an interquartile range of only 0.14 octaves across core and belt, and the null persists with loudness-equalized tones. An eccentricity-like gradient appears only when the ring geometry changes, consistent with global image statistics rather than a positional code. In contrast, motion selectivity is strong and precisely localized: 22% of the 1% most motion-selective vertices fall in MT+, which covers 0.42% of cortex. In-silico experiments that depend on feature identity appear viable with TRIBE v2, while those that depend on visual-field or frequency position are not supported by the current model.
Figures & tables
| Prediction | Rationale | |
|---|---|---|
| H1 polar angle / contralaterality | Absent: no contralateral V1 response to hemifield stimuli, no polar-angle map | Spatial averaging of video features |
| H2 eccentricity | Partly present: an eccentricity-like gradient from global image statistics | Averaged features still change with ring size |
| H3 motion | Present: moving static dots peaks in MT+ | V-JEPA 2 is a video model; the original paper found a motion-related component |
| H4 tonotopy | Open: best-frequency gradient in A1 | Audio features are not pooled over frequency, but training data were mostly speech/film sound |
| Test | Retinotopic+tonotopic stand-in | Pooled+flat stand-in |
|---|---|---|
| R1 V1 contralateral effect ( ) | 2.73 (spin ) | 0.01 ( ) |
| R2 polar angle vs. template (circular ) | 0.997 ( ) | 0.01 ( ) |
| R2b pRFs on contralateral side | 100% | 52–57% (chance) |
| R3 eccentricity vs. template (Spearman) | 0.991 ( ) | 0.01–0.12 ∗ |
| R4 wedge power at vs. | 0.60 (position-driven) | 0.48 (orientation-driven) |
| M1 MT+ moving static ( ) | 3.95 ( ) | 3.90 (planted in both) |
| Test | Value | (vs. spin/label null) |
|---|---|---|
| R0 V1–V3 vertices with pRF fit threshold (fraction) | 1.00 | – |
| R1 V1 contralateral effect ( ) | 0.048 | 0.237 |
| R2 polar angle vs. template (circular ) | 0.024 | 0.595 |
| R2b pRFs in correct contralateral hemifield (fraction) | 0.612 | – (0.5 = chance) |
| R3 eccentricity vs. template (Spearman) | 0.129 | 0.724 |
| R4 wedge power at / ( + ), V1–V3 median | 0.744 | – ( 0.5: position-coding) |
| Pattern | Meaning |
|---|---|
| R1 0, R2 0, R2b 0.5, R4 0.5 | H1 confirmed: no positional code; responses follow orientation and stimulated area, not location. In-silico experiments depending on visual-field position are invalid with TRIBE v2. |
| R1 0 and R2b 0.5, but R2 weak | Coarse left/right information survives averaging; fine polar angle does not. |
| R3 0 while R1/R2 0 | Eccentricity from global image statistics (ring size), not true position; confirm with the area-matched ring control. |
| M1 large, well above MT+’s share of cortex | H3 confirmed: motion selectivity localizes correctly. |
| T1 0 with both -values small, T2 1 octave | H4 confirmed: a genuine tonotopic gradient, learned without explicit tone training. |
| T2 0 | Auditory vertices share one tuning: no tonotopy (as observed; Section 3.2 ). |