This paper identifies a structural limitation in current multimodal AI architectures that is topological rather than parametric. Contrastive alignment (CLIP), cross-attention fusion (GPT-4V/Gemini), and diffusion-based generation share a common geometric prior -- modal separability -- which we term contact topology. The argument rests on three pillars with philosophy as the generative center. The philosophical pillar reinterprets Wittgenstein's saying/showing distinction as a problem rather than a conclusion: where Wittgenstein chose silence, the Chinese craft epistemology tradition responded with xiang (operative schema) -- the third state emerging when saying and showing interpenetrate. A cruciform framework (dao/qi x saying/showing) positions xiang at the intersection, executing dual huacai (transformation-and-cutting) along both axes. This generates a dual-layer dynamics: chuanghua (creative transformation as spontaneous event) and huacai (its institutionalization into repeatable form). The cognitive science pillar reinterprets DMN/ECN/SN tripartite co-activation through the pathological mirror: overlap isomorphism vs. superimposition collapse in a 2D parameter space (coupling intensity x regulatory capacity). The mathematical pillar formalizes these via fiber bundles and Yang-Mills curvature, with the cruciform structure mapped to fiber bundle language. We propose UOO implementation via Neural ODEs with topological regularization, the ANALOGY-MM benchmark with error-type-ratio metric, and the META-TOP three-tier benchmark testing cross-civilizational topological isomorphism across seven archetypes. A phased experimental roadmap with explicit termination criteria ensures clean exit if falsified.
Figures & tables
Figure 1: The Three Topological Regimes of Multimodal Integration. Panel A: Contact topology (current AI)—separated manifolds with interface-only alignment. Panel B: Overlap topology (creative emergence / UOO target)—non-separable interior with persistent β1 loops maintaining structural tension. Panel C: Superimposition collapse (psychosis / computational failure)—loss of transversality and topological singularity. Panel D: Corresponding persistence diagrams as quantitative diagnostic: τ(X)=β1/β0 total persistence.
Hypothesis
β0 prediction
β1,β2 prediction
Rapid sequential processing
Two persistent independent components
Trivial
Overlap Zone (inseparable)
Single connected component
Persistent non-trivial loops and voids
Table 2
Figure 2: The Cruciform Structure. Xiang (tu-xiang, operative schema) occupies the intersection of two philosophical axes: the vertical dao / qi axis (metaphysical/physical) and the horizontal saying/showing axis (propositional/presentational). Xiang simultaneously executes dual huacai (transformation-and-cutting) along both axes. Four materials from the Chinese craft tradition each occupy a precise coordinate within this structure.
Figure 3: The UOO Computation Graph. Left: mathematical framework (base space B , fibers Eb , connection ∇ , curvature F∇ , harmonic maps). Right: computational implementation (bilinear entanglement → Neural ODE → topological regularization). Correspondence arrows map each mathematical construct to its computational realization.
Outcome pattern
Implication for theory
Cross-modal ETR significantly > single-modal ETR
Supports contact topology diagnosis
No ETR difference across conditions
Opposes reality of topological type distinction
Some architecture shows no ETR shift on cross-modal
That architecture may have partially achieved overlap topology
Table 5
Figure 4: Cognitive-Pathological Parameter Space and ANALOGY-MM Benchmark. Left: two-dimensional parameter space (coupling intensity × regulatory capacity) with four quadrants—Overlap Zone (high coupling, high regulation), Superimposition Collapse (high coupling, low regulation), Conventional Cognition (low coupling, high regulation), and Cognitive Slowing (low coupling, low regulation). τ(X) contour lines indicate predicted structural tension. Right: ANALOGY-MM cross-modal analogy workflow with 2 × 2 forced-choice design and ETR computation.
Structure correct
Structure wrong
Content domain correct
A (correct answer)
B (ETR target error)
Content domain wrong
C (structure transfer, wrong domain)
D (random / double error)
Table 7
Figure 5: META-TOP — Three-Tier Benchmark System. Left: progressive three-tier structure testing increasingly deep cross-modal topological understanding. Tier L1 (ANALOGY-MM) tests structural mapping with ETR. Tier L2 (META-TOP Simple) tests dynamical pattern recognition with TSAS. Tier L3 (META-TOP Full) tests cross-civilizational topological isomorphism across seven archetypes. Right: the seven topological archetypes, with Archetype 7 ( chuanghua , creative transformation) as the core test—detecting the dynamic emergence of topology itself.
Phase
Scope
Timeline
Termination criterion
Phase 0
White paper revision; candidate definitions for B , G , Eb ; pathological mirror integration
1–2 months
—
Phase 1
Synthetic data PoC + NS validation + synthetic TDA pipeline
1–3 months
Separable bias ≥ non-separable on zero-shot transfer → terminate
Phase 1.5
TDA pipeline pre-validation on public fMRI creativity datasets
1–2 months
TDA metrics fail to distinguish high/low creativity → redesign
Phase 2
Behavioral experiment 1 + ANALOGY-MM first round
3–6 months
No cross-dimension semantic propagation → reassess framework
Phase 3
Neuroimaging experiment 2 + dimensional pathological mirror test
6–12 months
No non-decomposable emergent component → fiber bundle needs alternative
Phase 4
Cross-domain isomorphism (Expts 3–4) + causal TMS
12–24 months
Topological signatures differ across modality pairs → isomorphism falsified
Table 9
Design dimension
Enhancement
Participant design
Dimensional individual-differences design : recruit 150–200 healthy participants, continuously measuring creativity (AUT originality + DAT + CAQ) and schizotypy (O-LIFE scale), using continuous regression instead of between-group ANOVA
Primary metric
Automated semantic distance (SemDis) as primary quantification; human ratings as validation only
Demand characteristic control
Low-level perturbation control condition (non-semantic perturbation) to exclude confounds
Temporal resolution
Continuous trajectory tracking (mouse trajectory/force-sensitive input) throughout 5-second response window
Sequential processing control
10Hz forced alternation control group: if inseparability signature collapses under high-frequency alternation, this evidences the Overlap Zone’s dependence on temporal co-presence
Network framework
DMN/ECN/SN tripartite analysis; SN interpreted as coupling regulator (§4.2)
Table 10
Layer
Method
Decision criterion
Behavioral
Continuous trajectory tracking
Rapid switching → discrete stepwise polyline; Overlap Zone → smooth global topological deformation
Electrophysiological
Zero-lag phase locking (MEG)
DMN and ECN must simultaneously fire Gamma bursts within the same low-frequency Theta phase slot—sharing a metronome, not alternating
State space
HMM microstate analysis
Must reveal a third dynamical attractor state whose Betti number characteristics cannot be reconstructed as linear superposition of “pure propositional” and “pure imagistic” states
Causal (information-theoretic)
Bidirectional transfer entropy
Perturbation must trigger instantaneous nonlinear global network reorganization (not sequential A→B causal chain)
Causal (model-based)
DCM (fMRI) + Granger Causality (MEG)
SN→DMN-ECN gating model must win Bayesian Model Selection; forward Granger causality must exceed reverse
In this paper, we propose Gestalt, a new paradigm of large multimodal model built around multimodal interplay. Despite rapid advances, large multimodal models are reaching a bottleneck: existing approaches focus primarily on accommodating additional modalities while overlooking the distinct characteristics of each modality and the relations among them. Motivated by the multistage property of human multisensory perception, we propose a multimodal interplay pyramid that organizes multimodal modeling as a progression from modality-specific processing, through cross-modal alignment, to deeper multimodal integration. Guided by this pyramid, Gestalt adopts a unified discrete diffusion framework and an interplay-partitioned architecture, with learnable interplay tokens mediating cross-modal exchange and integration. The pyramid also structures its data organization and training strategy. Strong performance across image generation, multimodal understanding, and text-only evaluation shows that Gestalt significantly improves cross-modal integration while preserving modality-specific information, effectively harnessing the strengths of diffusion-based multimodal models and offering a promising path toward unified multimodal intelligence. Project page: https://GeWu-Lab.github.io/Gestalt.
Zequn Yang, Yu Miao, Haotian Ni +8
Gaoling School of Artificial Intelligence, Renmin University of China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Beihang University +3
Nearby neurons in cortex share similar response profiles, producing systematic spatial organization across sensory and cognitive systems. Recent topographic models reproduce aspects of this structure but remain unimodal and spatially constrain each layer separately, yielding fragmented maps that capture neither the contiguity of cortical processing streams nor their integration across modalities. We introduce Topo-Omni, a topographic multimodal model in which visual, auditory, and language/cognitive processing share a single contiguous in-silico sheet. Built by fine-tuning a pretrained foundation model with a spatial smoothness objective, this architecture develops clusters across modalities that are consistent with human neuroimaging, from sensory to cognitive systems. Driving or suppressing a cluster selectively biases or impairs perception, paralleling human intervention studies. Finally, we use our model to screen for novel clusters in-silico and discover new natural landscape and animal networks which we validate in human data. A single spatial principle thus organizes representations across modalities and processing stages, yielding testable hypotheses about cortical organization.
AI researchers describe state-of-the-art models as one thing repeated at scale: the Transformer, wired identically for text, pixels, or speech. Neuroscientists describe the cortex as a mosaic - dense Layer 4 in visual cortex for spatial encoding, thick Layers 5/6 in motion cortex for temporal integration - different jobs solved by different structures. This paper argues the gap is a structural error, not a stylistic one, and is measurable. A century of cytoarchitecture, from Brodmann to single-cell Patch-seq, shows distinct cognitive functions are implemented by qualitatively different structures, not by rescaling one template. The convolutional neural network is the field's own proof: local receptive fields and hierarchical depth encoded this prior directly, reaching strong image recognition on far less data than later architectures needed. The paper traces how this lesson was discarded: the "Hardware Lottery" made the Transformer the path of least resistance, not the principled choice, and Mixture-of-Experts, often cited as diversity, in fact partitions parameters among identical experts. A functionalist analysis shows the Transformer is best understood as a functional analog of the hippocampal formation, not a general-purpose cortex - the same mistake as treating cortex as one giant Broca's area, except the field has now standardized on a giant hippocampus, applied to tasks it was never built for: audition, executive gating, working memory. The paper closes with an alternative: a Heterogeneous Topological Network, a System of Systems in which distinct modules keep the inductive bias their computation demands and communicate through standardized interfaces. This is a design discipline for AI architects, not cognitive science: specify modularity before training, using structural evidence as a design input rather than reverse-engineering architecture from a trained model's behavior.