TorR: Towards Brain-Inspired Task-Oriented Reasoning via Cache-Oriented Algorithm-Architecture Co-design

Hyunwoo Oh; SungHeon Jeong; Suyeon Jang; Hanning Chen; Sanggeon Yun; Tamoghno Das; Mohsen Imani

cs.ARMar 24, 2026

TorR: Towards Brain-Inspired Task-Oriented Reasoning via Cache-Oriented Algorithm-Architecture Co-design

Authors: Hyunwoo Oh, SungHeon Jeong, Suyeon Jang, Hanning Chen, Sanggeon Yun, Tamoghno Das, Mohsen Imani

Organizations: Department of Computer Science, University of California, Irvine Irvine, CA, USA

Abstract

Task-oriented object detection (TOOD) atop CLIP offers open-vocabulary, prompt-driven semantics, yet dense per-window computation and heavy memory traffic hinder real-time, power-limited edge deployment. We present TorR, a brain-inspired algorithm--architecture co-design that replaces CLIP-style dense alignment with a hyperdimensional (HDC) associative reasoner and turns temporal coherence into reuse. On the algorithm side, TorR reformulates alignment as HDC similarity and graph composition, introducing partial-similarity reuse via (i) query caching with per-class score accumulation, (ii) exact $δ$ -updates when only a small set of hypervector bits change, and (iii) similarity/load-gated bypass under high system load. On the architecture side, TorR instantiates a lane-scalable, bit-sliced item memory with bank/precision gating and a lightweight controller that schedules bypass/ $δ$ /full paths to meet RT-30/RT-60 targets as object counts vary. Synthesized in a TSMC 28,nm process and exercised with a cycle-accurate simulator, TorR sustains real-time throughput with millijoule-scale energy per window ( $\approx$ 50,mJ at 60,FPS; $\approx$ 113,mJ at 30,FPS) and low latency jitter, while delivering competitive [email protected] across five task prompts (mean 44.27%) within a bounded margin to strong VLM baselines, but at orders-of-magnitude lower energy. The design exposes deployment-time configurability (effective dimension $D'$ , thresholds, precision) to trade accuracy, latency, and energy for edge budgets.

Figures & tables

Figure 1. From CLIP/ViT to TorR. Top: dense token aligner with fixed per-frame cost. Bottom: an event-driven encoder paired with an HDC associative aligner and a lightweight reasoner. Query caching turns temporal coherence into reuse so TorR updates on change .

Figure 2. Bottleneck shift from CLIP/ViT to TorR. Left: TaskCLIP is dominated by the ViT backbone. Right: after ViT

\rightarrow

event encoder, cost shifts to HDC associative search + graph reasoning , which are memory-bound.

Figure 3. TorR overview. Images are used only to transfer CLIP semantics to events. At run time, partial-similarity reuse ( $\delta$ -updates, caches) and FPS/QoS control align cost with scene dynamics.

Figure 4. Cache-gated HDC reasoner with $\delta$ -alignment. A query cache (depth $K$ ) supplies the nearest prior query $\mathbf{q}^{(t-1)}$ . Partial similarity $\rho=\cos(\mathbf{q}^{(t)},\mathbf{q}^{(t-1)})=1-\tfrac{2|\Delta|}{D^{\prime}}$ selects $\delta$ -update (update only flipped indices $\Delta$ ) or full . Under high load, if $\rho\!\geq\!\tau_{\mathrm{byp}}\!\wedge\!H(N,q)$ , cached scores/outputs are reused. Otherwise the aligner computes $s_{j}=\cos(\mathbf{q},\mathbf{h}_{j})$ and the reasoner (for fixed task) applies cached weights $\tilde{w}_{j}=\cos(\mathbf{g}_{P},\mathbf{h}_{j})$ with $\mathbf{g}_{P}=\mathbf{t}\odot r_{\ell_{1}}\odot\cdots\odot r_{\ell_{k}}$ , producing $\hat{s}_{j}=s_{j}\tilde{w}_{j}$ . The FPS/QoS controller gates $D^{\prime}$ (bank gating).

Figure 5. Similarity-gated top-level accelerator. PSU computes inter-query similarity

\rho

. The FPS/QoS controller selects bypass/

\delta

/full, gates

D^{\prime}

(bank enables) and precision, and programs the associative aligner. In

\delta

-mode the aligner uses a

\Delta

-index FIFO for sparse reads from the banked item memory

M\times D

; scores are accumulated, top-

k

pooled, optionally reasoned (HDC), cached, and returned via host/DMA. Solid arrows = data, dashed = control.

Figure 6. Five example tasks for TOOD evaluation. Detected objects are highlighted in blue.

Block / Component	Area (mm 2 )	Power (mW)
Logic (synthesized)
Associative Aligner	4.488	3,522.56
Lightweight Reasoner	0.642	504.32
Partial-Update Unit	0.280	220.16
Score Buffer (top- $k$ )	0.140	110.08
Sorter	0.140	110.08

Table 1. Hardware specifications (TSMC 28 nm, 1 GHz). Logic areas in mm 2 , powers in mW.

Mode	Global Min	Task (Min)	Global Max	Task (Max)
RT-60	6.8 ms	have breakfast	13.8 ms	sports
RT-30	12.9 ms	have breakfast	23.6 ms	sports

Table 2. Min/max end-to-end per-frame latency across the five tasks (1 GHz).

	RT-60 (60 FPS target)						RT-30 (30 FPS target)
Task	Median	p95	Jitter	Headroom	Power	Energy	Median	p95	Jitter	Headroom	Power	Energy
pour wine	9.4	11.3	1.9	5.37	3.20	53	17.2	19.9	2.7	13.43	3.50	116
sports	9.8	11.9	2.1	4.77	3.22	54	17.8	20.6	2.8	12.73	3.52	117
cooking	8.7	10.6	1.9	6.07	3.12	51	16.5	18.8	2.3	14.53	3.40	113
have breakfast	7.9	9.4	1.5	7.27	3.05	50	15.1	17.3	2.2	16.03	3.32	110
take a rest	8.1	9.7	1.6	6.97	3.06	50	15.4	17.6	2.2	15.73	3.33	110

Table 3. Accelerator runtime per task at RT targets. Latency in ms, power in W, energy in mJ per frame. Jitter = p95

-

median; Headroom = budget

-

p95 (16.67 ms for RT-60; 33.33 ms for RT-30).

Method	Assumptions	FPS (RTX 4090)	Energy/Frame (RTX 4090)
TOIST (DETR)	standard COCO input	15–25	30–18 J
iTaskCLIP (ViT-B/16)	120–200 crops/frame	5–12	90–38 J
iTaskCLIP (ViT-L/14)	120–200 crops/frame	2–6	225–75 J
Ours (RT-60)	single-pass similarity	60.3	50 mJ
Ours (RT-30)	single-pass similarity	30.1	113 mJ

Table 4. Throughput/energy (single RTX 4090 vs. our accelerator) for representative task-oriented pipelines. GPU energy/frame is 450 W divided by FPS.

Method	pour wine	sports	cooking	have breakfast	take a rest
GGNN	40.7	43.6	37.6	39.1	40.5
TOIST	52.9	52.8	43.1	48.1	46.7
iTaskCLIP	63.51	65.54	56.08	44.39	44.76
iTaskCLIP*	63.36	62.02	56.14	42.59	45.62
Ours (SW)	54.62	52.07	46.40	34.07	34.17

Table 5. [email protected] on the five evaluation tasks.

Efficient Object Detection

Efficient object detection reduces the computation, latency, memory, and energy needed to find and locate objects in images and video.

See all 95 papers

Efficiently Representing Algorithms With Chain-of-Thought Transformers
Jun 18, 2026Yanhong Li, Anej Svete, Ashish Sabharwal +1Transformer CoT Reasoning
Allen Institute for AI · ETH Zürich
What Survives When You Compress a Recursive Reasoner for the Edge?
Jun 25, 2026Pearse Jim, Steven Kolawole, Opegbemi Matthias Busoye +2Model Compression Efficient Neural Network Inference
ML Collective · Carnegie Mellon University
RiO-DETR: DETR for Real-time Oriented Object Detection
Mar 10, 2026Zhangchi Hu, Yifan Zhao, Yansong Peng +8Object Detection Detection Transformer
University of Science and Technology of China · Tsinghua University · Huawei Technologies Co., Ltd. +1

Jun 18, 2026J/K move · Enter open · S save

Efficiently Representing Algorithms With Chain-of-Thought Transformers

Yanhong Li, Anej Svete, Ashish Sabharwal, William Merrill

Allen Institute for AI · ETH Zürich

Transformer CoT Reasoning Neural Algorithmic Reasoning Efficient Language Model Reasoning Algorithmic Reasoning

Open paper AskPDF arXiv

The increasing popularity of reasoning models -- language models that output a series of reasoning or thought tokens before producing an answer -- is justified, in part, by theoretical results showing that chain-of-thought (CoT) transformers can simulate Turing machines, and thus perform arbitrary computation. However, the Turing machine, while suitable for complexity-theoretic analysis, is not convenient, intuitive, or efficient for discussing algorithms. Algorithms are typically designed and analyzed at a higher level of abstraction, captured by the Word RAM model with random-access memory and unit-cost operations on

\bigO(\log n)

-bit words. As a result, Word RAM algorithms can be substantially more efficient than their Turing machine counterparts, raising the question: Can CoT transformers efficiently simulate Word RAM algorithms? For instance, can they sort

n

items in

\bigO(n \log n)

steps or run Dijkstra's algorithm in

\bigO(E + V \log V)

steps? We answer affirmatively, up to poly-logarithmic overhead. We first establish this for finite-precision transformers with poly-logarithmic width and rightmost unique hard attention, then strengthen the result to two more practical settings with finite width and log-precision: continuous CoT, where reasoning takes the form of vectors rather than tokens, and a hybrid architecture in which transformer layers sit atop a recurrent (linear RNN) layer. In all three cases, we find that CoT can efficiently simulate any Word RAM algorithm with only a poly-logarithmic overhead in

n

. This overhead reduces to log-square when the Word RAM has a ``flat'' instruction set, and only logarithmic for multiplication-free flat instructions -- in stark contrast to known CoT simulations of Turing machines, which require quadratic overhead over Word RAM.

Abstract

Efficient Object Detection

Explore similar work

Efficiently Representing Algorithms With Chain-of-Thought Transformers

What Survives When You Compress a Recursive Reasoner for the Edge?

RiO-DETR: DETR for Real-time Oriented Object Detection