Neuro Inspired Robotics: What Robot Learning Can Borrow from the Sensorimotor Brain

Ruihan Zhang; Davy Deng

cs.RONov 2, 2026

Neuro Inspired Robotics: What Robot Learning Can Borrow from the Sensorimotor Brain

Authors: Ruihan Zhang, Davy Deng

Organizations: MIT

Abstract

Robot policies still fail at everyday manipulation: an unfamiliar task, an object never held before, a scene that changes mid-motion, a skill that should improve with use. They fail despite pretraining on over a million trajectories, pooled from dozens of robots at great expense, while a child acquires all of these abilities within a few years of learning from a single body. This survey argues that the gap is one of design, and proposes designs for the next generation of robot models, drawn from how the nervous system solves the same problems. From the body we take compliance: the policy commands stiffness alongside position, so the controller yields where the task allows and presses where it must, which greatly lowers the demand for precise position output. From the visual system, where a fast pathway for action runs in parallel with the slow pathway for recognition, we take a dedicated fast vision system for motor behavior: it precomputes the quantities the policy needs and drives a reactive policy at camera rate. From the cerebellum we take a predictor beside the policy that forecasts the consequence of each command and recalibrates from its own error. From motor cortex we take a single model that composes action units from primitives, verifies each at a sensory boundary, practices the weakest unit with a hypothesis in mind, and reasons each attempt closer to the goal. Proven primitives are then consolidated from code into weights, so they run fluently and respond quickly the next time. Together these describe a robot that acts on its own initiative, the basis of deployment and recursive self-improvement, and we offer each proposal as a hypothesis to be tested.

Figures & tables

Figure 1: A child has three abilities that current robot policies lack. Top: a boy opens an unfamiliar drawer on the first try, while a policy finds no training handle to copy. Middle: a girl catches a flying toy, while a robot is still reasoning about shape and coordinates when the toy lands. Bottom: the boy’s piano playing improves with days of practice, while a robot never trained on the piano cannot learn it or improve over time, because its learning stopped the day it was deployed.

Figure 2: The slot walls supply the rod’s trajectory. The hand pushes along the slot and yields across it, so the walls shape the path and the controller has far less of the curve to compute.

Figure 3: Opposing muscles set torque and stiffness separately. The difference of flexor and extensor pull sets the net torque, and their sum sets the stiffness at the same pose.

Figure 4: Contact supplies the surface detail. A yielding wrist and a soft towel follow the bumps, so the controller computes fewer corrections.

Figure 5: Touch alone finds the handle and opens the door. With eyes closed the hand feels its way to the handle, turns it and pushes the door open; in the dark a camera has nothing to offer.

Sensor type	How it works	Spatial res.	Speed	Strengths	Weaknesses
Piezoresistive [ 85 , 169 ]	Pressure squeezes a conductive polymer and lowers its resistance	mm-scale	tens of Hz	Cheap, thin, flexible; scales to large arrays	Hysteresis; reading drifts with load history; normal force only
Capacitive [ 85 , 434 ]	Pressure narrows the gap between two electrodes (touchscreen principle)	mm-scale	$\sim$ 100 Hz	Sensitive to light contact; mature technology	Picks up electrical interference; mostly normal force
Piezoelectric [ 165 , 85 ]	A polymer film emits a voltage when strained	mm-scale	kHz	Captures vibration, texture, and the onset of slip	Responds only to change ; blind to steady pressure
Barometric [ 373 ]	Air-pressure chips cast inside rubber, one chip per taxel	mm-scale	$>$ 100 Hz	Robust, inexpensive, factory-calibrated	Coarse; normal force only
Magnetic [ 37 , 379 ]	Magnetized elastomer over magnetometer chips; deformation shifts the field they read	mm-scale	hundreds of Hz	Three-axis force including shear; thin; replaceable skin	Hysteresis; signal drifts as the elastomer ages, so each new skin needs recalibration
Multimodal fluid-filled fingertip [ 111 , 409 ]	Fluid-filled fingertip with impedance electrodes, a pressure transducer and a thermistor	tens of sensing points per fingertip	$\sim$ 100 Hz, with a kHz vibration channel	Pressure, vibration, and heat flow together, the closest to a human fingertip	Expensive; signals need learned decoding

Table 1: No single sensor family combines fine resolution, kilohertz speed, and shear. Resolution and speed are order-of-magnitude values typical of each family, and the cited works are representative examples. A taxel (“tactile pixel”) is one sensing point on the skin.

Figure 6: Slowly adapting (SA) afferents report steady pressure and rapidly adapting (RA) ones report change, so the four classes split what touch carries. The traces are schematic responses to a ramp-and-hold skin indentation: SA afferents sustain firing during the hold, RA afferents respond transiently during pressing and release, and SA2 afferents also fire at a low rate before contact, driven by resting skin stretch. RA afferents can also fire during ongoing motion or vibration. Spike counts and timing are illustrative rather than measured; responses depend on the stimulus and afferent [ 194 , 2 ] .

Receptor	Where it sits	Skin patch it hears	Steady or change?	Everyday role
SA1 (Merkel ending)	Just under the surface; densest at the fingertip	Tiny (2–4 mm), sharply bounded	Keeps firing under steady pressure	Reading fine shape and texture; edges and curvature, the Braille channel
RA1 (Meissner corpuscle)	Just under the surface, in the hairless skin of the hand	Small (3–5 mm)	Fires only when contact changes	Catching micro-slips and low-frequency flutter, the trigger for tightening your grip
SA2 (Ruffini ending)	Deeper in the skin; sparse	Large, blurry borders	Keeps firing	Sensing skin stretch, finger posture, and which way a force is pulling
RA2 (Pacinian corpuscle)	Deepest of all; sparse, roughly 350 per finger	Very large, a whole finger or the hand	Fires only at sharp changes	Feeling high-frequency vibration through a held tool, pen on paper

Table 2: Four afferent classes split touch by receptive-field size and adaptation [ 186 , 389 , 194 ] . The pairing of SA2 afferents with Ruffini endings is disputed, since the human fingertip holds few such endings [ 303 ] . Figure 6 uses the same codes.

Figure 7: Vision and touch build a single object model, and each sense fills in what the other cannot. Vision covers the visible surfaces, while touch supplies weight, texture, slipperiness, and compliance, and explores the occluded back.

Figure 8: Three neurons carry a fingertip signal to primary somatosensory cortex (S1), a chain short enough to leave time for a tactile correction within about 75 ms. The first-order neuron ascends on the same side through the spinal dorsal column to the cuneate nucleus of the medulla; the second-order neuron crosses there and ascends to the ventral posterolateral (VPL) thalamus; the third-order neuron projects to S1 on the side opposite the stimulated hand. Each line ends in a dot at its synapse, and the legend colors mark the three cell bodies. Other branches and circuits are omitted [ 197 ] .

Figure 9: Descending signals from cortex turn down predicted or irrelevant tactile channels and turn up the task-relevant one, so the fingertip signal arrives at high contrast.

Figure 10: When an object’s properties cannot be seen, the robot should probe before it commits. Lifting, tilting, and pressing within safe bounds each test a hypothesis the camera cannot settle.

Figure 11: The same ball is read twice, by the two visual streams in the brain. The ventral stream resolves its pattern and identity; the dorsal stream resolves its trajectory and guides the catch.

Figure 12: Vision for action and vision for recognition separate after V1, and a collicular route bypasses cortex entirely. The geniculate route through V1 divides into a dorsal stream through posterior parietal cortex to premotor and motor areas (vision for action) and a ventral stream through V2 and V4 into inferotemporal cortex (vision for recognition); a separate collicular route drives fast orienting of eyes and head without cortex.

Figure 13: In the prevailing model, the dorsal stream is a bank of parallel channels. The figure expands the dorsal branch of Figure 12 and draws the motion areas MT and MST, which that figure folds into the parietal box, as a separate stage. Each visual area estimates one quantity the body needs (where to look, what is moving, what is within reach, where to reach, how to turn and shape the hand, and what a tool in hand reaches) and sends it to a partner in premotor, motor, or oculomotor areas; the italic line in each visual box names the reference frame the channel works in ( Section 4.2.4 ). Each label names an area’s dominant contribution rather than its only one [ 331 ] .

Figure 14: A visually guided action must re-express the retinal report in the frame its effector needs. The same dumpling has different coordinates in each frame along the chain: eye, head, body, hand, tool, and object.

Figure 15: A second viewpoint reveals what the head camera cannot see. The mug body partly hides its far-side handle from the head camera. Moving the wrist camera of the free hand to a side viewpoint reveals more of the handle before grasping. Teal arrows mark viewing directions; the curved gray arrow marks the wrist motion.

Figure 16: Having tossed the ball herself, she knows from the throw where and when it will land. A copy of the tossing command is enough to forecast the ball’s arrival, so the right hand is already in place while the ball is still in the air. The faint circles show predicted positions of the same ball.

Figure 17: Joint World Action Models let the prediction loss train the action head, while cascaded ones only pass predicted frames forward. In the cascaded design a video generator synthesizes predicted frames and a separate inverse-dynamics module reads actions off them. In the joint design a shared backbone trained under one generative objective feeds a state head that generates future frames, which some variants skip at test time, and an action head that emits the executed chunk.

Figure 18: The cerebellar circuit is three synapses deep, wide in the middle, and local along its length. The highlighted paths trace one local circuit: parallel fibers run only a few millimeters [ 315 ] , and about forty Purkinje cells converge on one nuclear neuron [ 309 ] , so coupling between body parts is local by construction. The inferior olive (amber) returns the prediction error over one climbing fiber per Purkinje cell, which adjusts the parallel-fiber synapses that were active when the faulty prediction was made. Counts are rough human figures [ 12 , 285 , 202 ] .

Figure 19: The same nuclear prediction leaves by two routes. Ascending (left), it reaches cortex through the ventrolateral thalamus. Descending (right), it joins the command in flight at the brainstem and spinal cord. Both arrive within tens of milliseconds of the command, before sensory feedback could alter it through cortex.

Figure 20: One model composes familiar primitives into a new solution. Reach, grasp, orient, and tool use, none of them learned for this task, combine to retrieve an out-of-reach key, and the same model turns the plan into movement.

Figure 21: Actions attach to a vision-language backbone in one of three ways. (a) The first generation bins each action dimension and decodes actions as text tokens, one at a time. (b) The current generation keeps the backbone and attaches a small action expert that attends to its features and denoises a noisy chunk into continuous actions by diffusion or flow matching; proprioception enters the expert, as drawn, in

\pi_{0}

and GR00T N1, and the backbone in SmolVLA [ 42 , 40 , 354 ] . (c) An omni model has no head. Images, text, and actions are tokens in one sequence, so the same network predicts the next action and the next image. Purple: transformer with pretrained weights. Teal: images, text, proprioception, and the action expert. Gray: discrete action tokens and the noise the expert denoises.

Figure 22: A robot learns mostly before it leaves the factory and afterwards executes the learned policy; a human begins with a genome rather than pretrained weights and does most of its learning after birth, through play, practice, and observation.

Figure 23: Unlocking a door is four action units in a fixed order, reach, orient, insert, and rotate, and each ends at a sensory event: the tip reaches the slot, the key aligns, it stops at the end, the lock turns.

Figure 24: Motor cortex is M1, premotor cortex, and SMA, reciprocally connected, with three cortical inputs: prefrontal goals enter through SMA and premotor cortex, the parietal state enters through premotor cortex and, from S1 and area 5, M1 directly, and ACC reaches SMA and M1 through the cingulate motor areas [ 364 ] . M1 carries most of the corticospinal tract, and SMA, premotor, and cingulate areas each add a direct projection that bypasses it [ 100 ] . Motor cortex also drives the brainstem pathways and runs loops through the basal ganglia and cerebellum [ 230 ] ; reflex afferents return to the cord and touch and proprioception return to parietal cortex. Only the principal pathways are drawn.

Figure 25: One action unit runs as a loop; the arrows show the flow of control rather than anatomical pathways. Purple marks the four stages, run clockwise, and teal what each stage reads. Solid teal arrows have direct support: the cerebellar forecast amends the unit in flight [ 31 , 252 ] and sustains and times the cortical preparation of the next [ 122 , 60 ] . Dashed arrows are hypotheses: whether the forecast also times the basal ganglia release is untested, and pre-SMA, which launches units and switches between them [ 205 , 178 ] , is proposed here to do so on the ACC verdict. Assigning one area to each stage is a simplification; each area takes part in more than one.

Figure 26: Consolidation advances a primitive from a script over many functions to an end-to-end policy. (a) At first the model writes a perception pipeline that computes the slot’s pose and the key’s geometry in the hand before calling the orient template, which runs by Cartesian control. (b) Next the model outputs both values directly from the images and supplies them to the same template, so the explicit measuring code is no longer needed. (c) In the end the template lives in the orient experts of the model’s mixture-of-experts trunk: the action head of the same model reads their latent, no number is passed, and the action stream reacts at the control rate.

Figure 27: The practice agent finds the error-prone part across attempts, corrects it during inference, and hands the corrected record to consolidation. (a) Over seven episodes of opening a lock, insert passes rarely, and the blame falls on orient’s angle, which is solved with an arbitrary offset after several model calls and fails most first attempts; it is unfamiliar and error prone and is practiced first. (b) During practice the direction to move after each attempt comes from the graded goal test, the forecast against the outcome, or the model’s reasoning; the interval halves until the goal test fires, and the angle that passed is stored. (c) Consolidation then improves the coding strategy that finds the keypoints, trains the model to output the values directly, and trains the orient experts of the trunk, so the orient primitive becomes familiar.

Stage	Current robot stack	Nervous system	Proposal
Action ( Section 2 )	A position target goes into fixed servo gains, and the same gains serve every contact.	The command is effort per muscle, and opposing pairs set torque and stiffness separately.	Emit position and stiffness together. Realize the stiffness beneath the policy by impedance or admittance, and let touch set where to yield and how hard to press.
Touch ( Section 3 )	Sensors cover only parts of the hand, and the signal differs per sensor family.	Four afferent classes cover the whole skin at high density and precision, measure both normal and shear force, and a grip reflex corrects slips.	Treat touch as a first-class input, and use it for compliance control.
Vision ( Section 4 )	One recognition encoder reads one stale frame with no velocity, and encoding is slow.	A ventral stream handles recognition, and a separate dorsal stream carries fast metric signals for guiding the hand.	Build a dedicated streaming vision encoder for control.
Predictor ( Section 5 )	A video world model or WAM takes hundreds of milliseconds per step and adapts poorly at test time.	The cerebellum forecasts within tens of milliseconds, recalibrated with a few trials by its own error.	Train a self-labeling predictor beside the policy. Consult it as a tool.
Policy and learning ( Section 6 )	A VLA copies demonstrations, a slow planner sits over a fast executor, code as policy is slow and blind between calls, and weights are frozen at deployment.	The goal is split into action units checked at their boundaries. Primitives live in motor cortex and combine to solve new tasks, and practice is scored by dopamine and consolidated offline into new motor skills.	One model composes parameterized primitives in code, verifies each at its boundary, and runs them through a reactive head. A practice agent rehearses the weakest unit, and a consolidation agent graduates each skill from code into weights.

Table 3: Discrepancies between the current robot stack and the nervous system, stage by stage, and the proposal each chapter derives from the gap.

Robot Learning

Robot learning develops methods for robots to acquire skills and adapt from demonstrations, interaction, data, and physical models.

See all 331 papers

Sep 30, 2026

A Biophysically Detailed C. elegans Circuit as a Task-Agnostic Dynamical Core for Visually Robust Robot Manipulation

Robot policies are usually trained for one task, one body and one visual environment, and generalize poorly beyond these conditions. Whether a nervous system can instead supply the sensorimotor computation through its evolved wiring and biophysics remains unresolved. Here we embed a biophysically detailed Caenorhabditis elegans sensorimotor circuit - 136 multicompartment neurons with realistic morphologies and electrophysiological characteristics - as the dynamical core of a visuomotor policy. Only thin task-specific adapters are trained; the core's synaptic weights stay fixed while its membrane voltages evolve freely. Across different MetaWorld tasks the core matches or exceeds diffusion-policy, action-chunking-transformer and neural-circuit-policy baselines, and degrades less under visual perturbations. Replacing the core with generic network models such as MLP, LSTM, transformer or reservoir networks removes the advantage. Furthermore, on a real robotic arm the core withstands diverse visual perturbations that collapse the baselines. Our results suggest that visual robustness can be inherited from biophysically detailed circuit dynamics rather than learned by task-specific controllers.

Linrui Qian, Jiajia Zhang, Gan He +7

CogLeap.AI Space Intelligence (Wuxi) Technology Co., Ltd., Beijing 100080, China · School of Mathematics and Computational Science, Xiangtan University, Xiangtan 411105, China · Institute for Brain and Intelligence, Fudan University, Shanghai 200433, China. +1

Robotic Control Robot Policy Learning Visuomotor Policy Learning Robotic Manipulation

Ask PDF

Mar 26, 2026

Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories

Scaling robot learning to long-horizon tasks remains a formidable challenge. While end-to-end policies often lack the structural priors needed for effective long-term reasoning, traditional neuro-symbolic methods rely heavily on hand-crafted symbolic priors. We introduce ENAP (Emergent Neural Automaton Policy), a neuro-symbolic imitation learning framework in which a bi-level policy adaptively emerges from visuomotor demonstrations. Specifically, we employ adaptive clustering and an extension of the L* algorithm to learn the task structure from demonstrations as a Mealy state machine, which serves as an interpretable high-level planner capturing latent task modes. This discrete structure then guides a low-level reactive residual network to learn precise continuous control via behavior cloning (BC). By explicitly modeling discrete transitions and continuous residuals, ENAP achieves sample efficiency and interpretability without task-specific labels. Experiments on complex manipulation and long-horizon tasks show that ENAP outperforms state-of-the-art end-to-end vision-language-action (VLA) policies by up to 27% in low-data regimes. ENAP can also distill task structure from the latent space of a pre-trained VLA, surpassing it on CALVIN benchmark with 39% fewer parameters (Fig. 1).

Yiyuan Pan, Xusheng Luo, Hanjiang Hu +2

Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA

Automata Learning Visuomotor Policy Learning Neurosymbolic Learning Long-Horizon Robotic Manipulation Robot Imitation Learning

Ask PDF

Aug 21, 2026

Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning

Behaviour Cloning (BC) has driven remarkable progress in robot manipulation, yet it is fundamentally limited by its inability to self-improve: a policy that fails cannot learn from that failure without additional human demonstrations. Reinforcement Learning fine-tuning offers a path to self-improvement but has proven difficult to scale to the multi-billion-parameter models underpinning modern robot policies. We propose Q-Planning, which equips a large visuomotor BC policy with a small off-policy Q-function. Because a Q-function estimates value rather than imitates actions, it can be trained on the same successful demonstrations as the BC policy and later absorb both successful and failed deployment rollouts, an asymmetry BC does not have. We exploit this asymmetry to enable value-guided action selection at inference (a single-step Q-weighted average over BC draws) and online self-improvement that fine-tunes only the Q-function, leaving the BC weights untouched. On LIBERO and bimanual RoboTwin, ten iterations of self-improvement lift every benchmark score we tested (LIBERO-10 93% to 99%, RoboTwin 83.8% to 91.4%) and shorten successful episodes on the near-ceiling suites (LIBERO-Object, LIBERO-Goal). On two contact-rich bimanual real-robot tasks, the same loop (BC frozen, no human intervention) improves purely from its own deployment rollouts: stack-cups 40% to 90% and insert-wallet 25% to 80% in five iterations, whereas SFT on successful rollouts alone stalls at 55% and 30%. Under an identical online budget Q-Planning is the only method, among Best-of-N, filtered SFT, IBRL, DSRL, and DAWR, that improves stably from failures without training an auxiliary actor.

Varun Giridhar, Anant Khandelwal, Jeremy A. Collins +2

Georgia Institute of Technology

Robot Policy Adaptation Robotic RL Robotic Manipulation Robot Imitation Learning Off-Policy RL

Ask PDF

Abstract

Robot Learning

Explore similar work

A Biophysically Detailed C. elegans Circuit as a Task-Agnostic Dynamical Core for Visually Robust Robot Manipulation

Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories

Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning