Scalable Machines with Intrinsic Higher Mental-State Dynamics

Ahsan Adeel; M. Bilal

cs.LGMar 13, 2026

Scalable Machines with Intrinsic Higher Mental-State Dynamics

Authors: Ahsan Adeel, M. Bilal

Organizations: TREND Project: Funded by the UK Advanced Research + Invention Agency (ARIA).

Abstract

Drawing on recent breakthroughs in cellular neurobiology and detailed biophysical modeling linking neocortical pyramidal neurons to distinct mental-state regimes, this work introduces a mathematically grounded formulation showing how models (e.g., Transformers) can implement computational principles underlying awake imaginative thought to pre-select relevant information before attention is applied via triadic modulation loops among queries ( $Q$ ), keys ( $K$ ), and values ( $V$ ).~Scalability experiments on ImageNet-1K, benchmarked against a standard Vision Transformer (ViT), demonstrate significantly faster learning with reduced computational demand (fewer heads, layers, and tokens), consistent with our prior findings in reinforcement learning and language modeling. The approach operates at approximately $\mathcal{O}(N)$ complexity with respect to the number of input tokens $N$ .

Figures & tables

Panel of Figure 1 — Figure 1: Integrated view of biologically inspired mechanisms: (a) Pyramidal two-point neuron; (b) $\rm{Co}^{4}$ triadic reasoning via $Q$ – $K$ – $V$ interactions; (c–d) MOD function dynamics illustrating context-sensitive filtering; $Y$ represents the FF signal, which is separated into relevant and irrelevant streams depending on the strength of $C$ ; (e–f) MOD contours and vector fields across the $R$ – $C$ space. Surface plots are presented over a wider range of $R$ and $C$ to illustrate the global geometry of the cooperation landscape, while contour and vector-field visualizations focus on a smaller range to highlight local gradient flow and regime transitions near the origin. Variations in the strengths of $R$ and $C$ shift the system across distinct processing regimes analogous to the neurobiological AA, AD, and AD+Awake regimes, producing corresponding geometric deformations in gradient flow. By shaping representations prior to downstream readout, these modulation laws guide optimization along $R–C$ interaction manifolds, reducing propagation through noisy or irrelevant directions.

Figure 2: Early training comparison between an attention-only Vision Transformer (ViT) ( Dosovitskiy, 2020 ) , trained from scratch, and a Co 4 machine endowed with intrinsic mental-state-dependent processing regimes analogous to awake imaginative processing ( Phillips et al., 2024 ; Graham et al., 2025 ) , which pre-select relevant information before attention is applied. The task is to identify a bird from the Mini-ImageNet dataset. Brightness indicates regions emphasized. In the ViT model, this is after attention. In contrast, Co 4 rapidly forms a coherent interpretation of the input, highlighting the top- $k$ salient regions via internally generated awake imaginative regimes before attention is computed. Co 4 exhibits earlier and sharper activation over the semantically relevant object (bird), indicating more coherent internal inference. These findings raise questions about the necessity of deep attention stacks.

Figure 3: The figure visualizes the complete attention distribution over N input tokens: a single-layer Co 4 machine versus an attention-only ViT ( Dosovitskiy, 2020 ) , both trained on Mini-ImageNet for 30 epochs. The ViT exhibits more dispersed attention with less selective localization. In contrast, Co 4 demonstrates more centered, context-sensitive activation patterns, indicating stronger spatial coherence.

Figure 4: Co 4 versus Transformer on CIFAR-10, Tiny-ImageNet, and Mini-ImageNet, trained from scratch: (i–iii) performance of a single-layer model; (iv) inference runtime as a function of sequence length for a single layer (see A.6 for theoretical computational cost analysis); and (v–vi) layer-wise validation accuracy on Tiny-ImageNet and Mini-ImageNet.

Dataset	Model	L	Combination	Hidden	MLP	H	Params.	Acc.
ImageNet-1K	Co 4	8	Co 4	768	3076	1	53.4M	70.24%
	Co 4	6	Co 4	768	3076	1	40.4M	67.30%
	Co 4	6	4 Co 4 + 2 MHSA	768	3076	1	41.6M	75.01 %
	Transformer	12	MHSA	768	3076	12	86M	74.80%
Mini-ImageNet	Co 4	1	Co 4	384	3076	1	2.0M	39.20%
	Co 4	4	Co 4	384	3076	1	8M	45.40%

Table 1: Comparison between Transformer and Co 4 models across ImageNet-1K, Mini-ImageNet, Tiny-ImageNet, and CIFAR-10: L, H, Params, and Acc denote the number of layers, attention heads, model parameters, and accuracy, respectively.

Dataset	L	H	Ep.	Embedding	Acc.
Imagenet-1k	6	1	300	768	67.30%
	6	6	300	768	67.81%
	6	12	300	768	67.89%
Cifar10	4	1	30	384	80.30%
	4	4	30	384	80.36%

Table 2: Effect of varying the number of attention heads on Co 4 performance: L, H, Ep, and Acc denote the number of layers, attention heads, epochs, and accuracy, respectively.

Modulation Equations	Acc.
$\begin{aligned} Q_{m}&=Q_{X}+Q_{\text{L}}\cdot K_{X}\\ K_{m}&=K_{X}+K_{\text{L}}\cdot Q_{X}\\ V_{m}&=V_{X}^{2}+2V_{X}+Q_{m}K_{m}(1+\|V_{L}\|)\end{aligned}$	78%
$\begin{aligned} Q_{m}&=Q_{\text{L}}+Q_{\text{L}}\cdot Q_{X}\\ K_{m}&=K_{\text{L}}+K_{\text{L}}\cdot K_{X}\\ V_{m}&=V_{X}^{2}+2V_{X}+Q_{m}K_{m}(1+\|V_{L}\|)\end{aligned}$	69%
$\begin{aligned} Q_{m}&=Q_{\text{L}}+Q_{\text{L}}\cdot K_{X}\\ K_{m}&=K_{\text{L}}+K_{\text{L}}\cdot Q_{X}\\ V_{m}&=V_{X}^{2}+2V_{X}+Q_{m}K_{m}(1+\|V_{L}\|)\end{aligned}$	72%
$\begin{aligned} Q_{m}&=Q_{X}+Q_{\text{L}}\cdot K_{X}\\ K_{m}&=K_{X}+K_{\text{L}}\cdot Q_{X}\\ V_{m}&=V\end{aligned}$	68%
$\begin{aligned} Q_{m}&=Q_{X}+Q_{\text{L}}\cdot K_{X}\\ K_{m}&=K_{X}+K_{\text{L}}\cdot Q_{X}\\ V_{m}&=V_{X}+V_{L}\cdot Q_{m}\cdot K_{m}\end{aligned}$	68%

Table 3: Effect of different modulation equations on CIFAR-10 performance (4 layers, 1 head, 30 epochs, 384-dimensional embeddings).

Dataset	Model	L	Tok.	$k$	Acc.	BL
Tiny-ImageNet	Co 4	4	64	44	31%	–
	Co 4	4	64	32	33%	–
	Co 4	4	64	8	40%	–
	Co 4	4	64	3	37%	–
	ViT	4	64	8	32%	35%
	Co 4	4	64	8	40%	–

Table 4: Top-

k

token selection study comparing Co 4 with Transformer (ViT) under identical architectural settings. Performance is shown for multiple values of

k

, together with the ViT baseline ( Dosovitskiy, 2020 ) : L, Tok, k , Acc, and BL denote the number of layers, total tokens, the top- k threshold, top- k accuracy, and baseline ViT accuracy, respectively.

Figure 5: Training results across CartPole (i–iii), PyBullet Ant (iv–v), CarRacing (visual input: $96\times 96\times 4$ ) (vi), and Acrobot (vii), with heat maps (viii–ix) provided as empirical evidence. $T_{M1}$ – $T_{M4}$ denote alternative well-established TPN-inspired asynchronous MOD functions ( Kay and Phillips, 2020 ) (see A.3).

Table 10

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: An example of “thinking fast and slow” as discussed in ( Kahneman, 2011 ) , illustrates how solving a riddle can involve combining rapid, intuitive processing in high-level perceptual state (fast thinking) with more deliberate, reflective refinement in the awake thought state (slow thinking).

Figure 7: $\rm{Co}^{4}$ basic architecture: latent $Q$ , $K$ , and $V$ tokens evolve via TPN-like circuits and triadic modulation under distal, proximal, and universal context. Enables scalable, context-sensitive reasoning at an approximate cost of $\mathcal{O}(N)$ .

Figure 8: (a) Raster plots for 150 neurons. (i) standard TPNs + BDSP (ii) and context-sensitive (CS)-TPNs + BDSP with cooperative $MOD(R,C)$ dynamics. In CS-TPNS, a clear distinction in bursting probability is observed as the inputs transition from low–low (LL) to high–high (HH), corresponding to input combinations $00$ , $01$ , $10$ , and $11$ . (b) CS-TPNs fire substantially less frequently than TPNs alone and exhibit bursting primarily when $R$ and $C$ inputs are coherent.

Explore similar work

AVIS: Adaptive Test-Time Scaling for Vision-Language Models
Jun 10, 2026Ahmadreza Jeddi, Minh Ngoc Le, Amirhossein Kazerouni +8Test-Time Adaptation of VLMs Efficient VLM Inference
AI Center-Toronto, Samsung Electronics · University of Toronto · Vector Institute +1
Taking Shortcuts for Categorical VQA Using Super Neurons
Mar 11, 2026Pierre Musacchio, Jaeyi Jeong, Dahun Kim +1Efficient VLM Inference Visual Question Answering
Seoul National University · EPFL · Google Deepmind
MANAR: Memory-augmented Attention with Navigational Abstract Conceptual Representation
Mar 19, 2026Zuher Jahshan, Ben Ben Ishay, Leonid YavitsAttention Mechanisms Multi-Head Attention
Bar Ilan University, Ramat Gan, Israel

Jun 10, 2026J/K move · Enter open · S save

AVIS: Adaptive Test-Time Scaling for Vision-Language Models

Ahmadreza Jeddi, Minh Ngoc Le, Amirhossein Kazerouni, Hakki Can Karaimer+7

AI Center-Toronto, Samsung Electronics · University of Toronto · Vector Institute+1

Test-Time Adaptation of VLMs Efficient VLM Inference Test-Time Scaling for VLMs Test-Time Scaling VLM Reasoning Visually Grounded Reasoning+1

Open paper AskPDF arXiv

Modern Vision-Language Models (VLMs) benefit from chain-of-thought prompting and test-time scaling, but these gains often come with prohibitive inference cost due to large visual contexts and long decoding chains. We view this cost through two coupled axes: Visual Context Scaling (VCS), which controls how much visual evidence is passed to the language model, and Visual Reasoning Scaling (VRS), which controls how much inference-time reasoning search is performed. Existing methods typically optimize one axis at a time, leaving the joint allocation of compute across these axes underexplored. We introduce Adaptive Visual Inference Scaling (AVIS), a lightweight policy that adapts both VCS and VRS per query. AVIS realizes VCS through Key Diversity Visual (KDV) pruning, a training-free

O(N)

key-based rule for removing redundant visual tokens before prefilling, and realizes VRS through adaptive self-consistency, using a learned difficulty predictor to select the number of reasoning rollouts. AVIS is deployment-friendly and compatible with shared-prefill inference, where all rollouts reuse a single prefilling pass and KV cache. Across diverse image and video reasoning benchmarks, AVIS improves the accuracy--compute trade-off relative to VCS-only and VRS-only baselines, and remains effective on top of RL post-trained VLMs while keeping compute and latency low.

Abstract

Explore similar work

AVIS: Adaptive Test-Time Scaling for Vision-Language Models

Taking Shortcuts for Categorical VQA Using Super Neurons

MANAR: Memory-augmented Attention with Navigational Abstract Conceptual Representation