Emergence of Physical Intelligence via Controllable Information Production

Tristan Shah; Stas Tiomkin

cs.AIJan 30, 2026

Emergence of Physical Intelligence via Controllable Information Production

Authors: Tristan Shah, Stas Tiomkin

Organizations: Department of Computer Science, Texas Tech University

Abstract

Intrinsic Motivation (IM) aims to train agents without external rewards, enabling useful behavior to emerge from the agent's interaction with its environment alone. However, the dominant IM approaches rely on information-theoretic quantities with designer-chosen variables, introducing bias and lacking a principled connection to dynamics or optimal control (OC). We introduce Controllable Information Production (CIP), a novel IM method explicitly grounded in dynamical systems and OC. CIP measures the rate at which an agent produces information, capturing controllable complexity without external knowledge or bias. CIP unifies IM and OC into a single framework, formalizing physical intelligence as the control of information production. It further reveals connections between the structure of the value function and Kolmogorov-Sinai entropy. CIP consistently outperforms prior IM methods on standard benchmarks in robot learning and solves tasks they fail on, including humanoid self-righting. These results support an organizing principle: that physical intelligence emerges from driving systems toward the edge of controllable chaos.

Figures & tables

Figure 1: A humanoid-like agent discovers self-righting and stable upright behavior using Controllable Information Production (CIP) as its sole intrinsic motivation signal.

Panel of Figure 2 — Figure 2: Overview of the CIP derivation. The optimal policy is decomposed into intrinsic (feedback) and extrinsic (drift) components (Lemma 3.1 ). Analysis of the value Hessian then separates open- and closed-loop information production rates (Lemma 3.3 ). The open-loop component defines CIP (Definition 3.5 ), which upper-bounds the closed-loop information production (Theorem 3.6 ).

Figure 3: KSE estimation on the Lorenz system in two standard chaotic parameter regimes. (A)

\sigma=10,\;\rho=28,\;b=8/3

, with reference value

h_{\mbox{ks}}(g)\approx 0.906

( Sprott, 1997 ) . (B)

\sigma=16,\;\rho=45.92,\;b=4

, with reference value

h_{\mbox{ks}}(g)\approx 1.498

( Wolf et al., 1985 ) . Results are shown for the continuous-time analogue of the recursion in Equation ( 10 ).

Figure 4: Mean height $\pm$ std of the extremity over time in each environment. For the pendulum environments the extremity is defined as the tip, for the gibbon the height of the head is considered as the extremity. The heights are normalized between $0.0$ and $1.0$ where the latter indicates the fully upright position. All runs start from the hanging position and are averaged over $10$ random seeds, except empowerment which runs deterministically. The skill-based IM (DADS, DIAYN, SMM) are evaluated using the skill with the highest average height. We use the default parameters for the baselines as suggested in their official implementations ( Laskin et al., 2021 ; Sharma et al., 2019 ) .

Figure 5: CIP (nats/s) over time during MPC runs.

	True dynamics (planning)			Learned from interaction (RL)
System	CIP (ours)	Empowerment	APT (MPC)	DIAYN	DADS	SMM	ICM
Cart Pole	$\mathbf{0.9996}$	$\mathbf{0.9996}$	$0.5488$	$0.0221$	$0.0684$	$0.6171$	$0.0005$
Double Pendulum	$\mathbf{0.9913}$	$0.5433$	$0.4503$	$0.2339$	$0.0107$	$0.0112$	$0.0016$
Triple Pendulum	$\mathbf{0.9931}$	$0.3430$	$0.4622$	$0.4865$	$0.0608$	$0.2513$	$0.0069$
Gibbon	$\mathbf{0.9319}$	$0.3988$	$0.2394$	$0.4649$	$0.1430$	$0.4458$	$0.5122$

Table 1: Mean extremity height over the final 10% of the episode, averaged across 10 random seeds. Values are min-max normalized so that

0.0

corresponds to the initial hanging configuration and

1.0

corresponds to the fully upright pose.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: ( left ) A “ball in box” environment where the agent controls the acceleration of a ball inside of a box with dissipative walls. ( center ) The occupancy heatmap of the ball under a CIP maximizing policy. ( right ) The occupancy heatmap of the ball under randomized actions.

Method	$N=10$	$N=25$	$N=50$	$N=75$	$N=100$
Ours	$1.98\pm 0.01$	$6.27\pm 0.02$	$12.80\pm 0.04$	$19.26\pm 0.02$	$25.70\pm 0.03$
QR	$1.98\pm 0.01$	$6.26\pm 0.02$	$12.80\pm 0.04$	$19.25\pm 0.02$	$25.69\pm 0.03$

Appendix

Table 2:

\Lambda^{+}

on Lorenz-96 for increasing state dimension

N

, estimated with our recursion and with QR decomposition (mean

\pm

std over 4 random seeds).

Task	$\eta$	Horizon	Shots	Iterations	Elite Fraction	Smoothing	$\rho$
Cart Pole	$0.0$	$400$	$512$	$1$	$0.1$	$0.1$	$0.9$
Double Pendulum	$0.0$	$512$	$1024$	$10$	$0.1$	$0.1$	$0.9$
Triple Pendulum	$2.5$	$128$	$2048$	$10$	$0.1$	$0.1$	$0.9$
Gibbon	$9.0$	$512$	$1024$	$1$	$0.2$	$0.1$	$0.9$

Appendix

Table 3: Hyperparameters of the iCEM planner [ Pinneri et al., 2021 ] used to optimize our CIP objective across all tasks. Horizon, shots, iterations, and elite fraction follow the notation of Pinneri et al. [2021] ;

\rho

is the autocorrelation coefficient of the AR(1) colored-noise process used for action sampling, smoothing refers to the momentum coefficient on the mean of the sampling distribution, and

\eta

is the control cost penalty defined in Algorithm 1 .

Robotic RL

Robotic reinforcement learning uses trial-and-error learning to help robots control, navigate, and manipulate objects.

See all 299 papers

Bellman Meets Lyapunov: Unsupervised Reinforcement Learning via Mastering Chaos
Oct 1, 2026Tristan Shah, Wooyoung Chung, Volodomyr Makarenko +2Reinforcement Learning Intrinsic Motivation
Texas Tech University · Purdue University
A Mathematical Theory of Agency and Intelligence
Feb 26, 2026Wael Hafez, Chenan Wei, Rodrigo Pena +2AI Agent Monitoring
Stiles-Nicholson Brain Institute, Jupiter, FL, USA
InternW0: A Foundational Physical World Model for Efficient Real-World Interactions
Sep 23, 2026Jisong Cai, Yao Mu, Ganlin Yang +23World Models for Robotics Robotics
Physical Intelligence Team, Shanghai AI Laboratory

Oct 1, 2026J/K move · Enter open · S save

Bellman Meets Lyapunov: Unsupervised Reinforcement Learning via Mastering Chaos

Tristan Shah, Wooyoung Chung, Volodomyr Makarenko, Juan Wachs+1

Texas Tech University · Purdue University

Reinforcement Learning Intrinsic Motivation Unsupervised RL

Open paper AskPDF arXiv

Reinforcement learning (RL) is a powerful paradigm for training agents, yet its success rests on domain expertise of human engineers who design informative reward signals for every new task. Unsupervised RL aims to reduce this engineering with intrinsic motivation (IM): reward signals that emerge from the agent environment interaction itself. Existing IM objectives, however, involve the selection of information variables, which re-introduces domain expertise the field has sought to eliminate. We introduce Forward CIP (F-CIP), an RL-native formulation of the Controllable Information Production (CIP) objective, which is defined by the system's dynamics alone and requires no such selection. We prove that F-CIP is compatible with RL and demonstrate its effectiveness with existing algorithms. Training agents with F-CIP results in unsupervised discovery of primitive behaviors such as balancing and maintaining controllability, which are essential for more complex robot behaviors. Paired with a simple forward-velocity reward, our method produces coordinated gaits such as hopping and running which otherwise require reward engineering to learn.

Abstract

Robotic RL

Explore similar work

Bellman Meets Lyapunov: Unsupervised Reinforcement Learning via Mastering Chaos

A Mathematical Theory of Agency and Intelligence

InternW0: A Foundational Physical World Model for Efficient Real-World Interactions