On physical implementability of reinforcement-learning-driven reasoning

Eiji Konishi

cond-mat.dis-nnOct 9, 2026

On physical implementability of reinforcement-learning-driven reasoning

Authors: Eiji Konishi

Organizations: Graduate School of Human and Environmental Studies, Kyoto University, Kyoto 606-8501, Japan

Abstract

In the machine learning community, the reinforcement-learning-driven reasoning process has been intensively studied since around 2025. In this article, we present a negative result on the hardware implementation of this reinforcement-learning-driven reasoning process in a physical neural network model in the presence of a reward external potential energy for a given reward. In this hardware implementation, the existence of a chain of thought in dynamics contradicts the Lyapunov stability of the system.

Physical Neural Networks

See all 10 papers

BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning
Mar 4, 2026Tarjei Paule Hage, Markus J. BuehlerStructural Engineering RL for Language Model Reasoning
Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Civil and Environmental Engineering, Department of Mechanical Engineering, Schwarzman College of Computing, Massachusetts Institute of Technology, Cambridge, MA, USA
StaRPO: Stability-Augmented Reinforcement Policy Optimization
Apr 10, 2026Jinghan Zhang, Fengran Mo, Tharindu Cyril Weerasooriya +5RL for Language Model Reasoning Reasoning Consistency in Language Models
Clemson University · Université de Montréal · Accenture (United States) +2
Lyapunov Exponent as Physics-Informed Dense Reward: RL Discovery of Stabilization Beyond the Kapitza Pendulum
Jul 15, 2026Slava AndrejevRL Control Reinforcement Learning

Mar 4, 2026J/K move · Enter open · S save

BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning

Tarjei Paule Hage, Markus J. Buehler

Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Civil and Environmental Engineering, Department of Mechanical Engineering, Schwarzman College of Computing, Massachusetts Institute of Technology, Cambridge, MA, USA

Structural Engineering RL for Language Model Reasoning Scientific Reasoning in Language Models Reinforcement Learning with Verifiable Rewards Reinforcement Learning Language Model Robustness

Open paper AskPDF arXiv

Can reinforcement learning with hard, verifiable rewards teach a compact language model to reason about physics, or does it primarily learn to pattern-match toward correct answers? We study this question by training a 1.5B-parameter reasoning model on beam statics, a classic engineering problem, using parameter-efficient RLVR with binary correctness rewards from symbolic solvers, without teacher-generated reasoning traces. The best BeamPERL checkpoint achieves a 66.7% improvement in Pass@1 over the base model. However, the learned competence is anisotropic: the model generalizes compositionally (more loads) but fails under topological shifts (moved supports) that require the same equilibrium equations. Intermediate checkpoints yield the strongest reasoning, while continued optimization degrades robustness while maintaining reward. These findings reveal a key limitation of outcome-level alignment: reinforcement learning with exact physics rewards induces procedural solution templates rather than internalization of governing equations. The precision of the reward signal - even when analytically exact - does not by itself guarantee transferable physical reasoning. Our results suggest that verifiable rewards may need to be paired with structured reasoning scaffolding to move beyond template matching toward robust scientific reasoning.

Abstract

Physical Neural Networks

Explore similar work

BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning

StaRPO: Stability-Augmented Reinforcement Policy Optimization

Lyapunov Exponent as Physics-Informed Dense Reward: RL Discovery of Stabilization Beyond the Kapitza Pendulum