Organizations: School of Physics, Peking University, Beijing 100871, China · Center for High Energy Physics, Peking University, Beijing 100871, China · Beijing Computational Science Research Center, Beijing 100193, China
Large language model (LLM)-driven agents are emerging as a powerful new paradigm for solving complex problems. Despite the empirical success of these practices, a theoretical framework to understand and unify their macroscopic dynamics remains lacking. This paper proposes a method based on the least action principle to estimate the underlying generative directionality of LLMs embedded within agents. By experimentally measuring the transition probabilities between LLM-generated states, we report evidence consistent with a tendency toward detailed balance in LLM-generated transitions, indicating that LLM generation may not be achieved by generally learning rule sets and strategies, but rather by implicitly learning a class of underlying potential functions that may transcend different LLM architectures and prompt templates. To our knowledge, this provides an effective macroscopic description of LLM generative dynamics that does not depend on specific model details. This work is an attempt to establish a macroscopic dynamics theory of complex AI systems, aiming to elevate the study of AI agents from a collection of engineering practices to a science built on effective measurements that are predictable and quantifiable.
Figures & tables
Figure 1: A schematic of a formalization framework for studying the directionality of LLM generation, illustrating the state space and possible transitions. LLMs are embedded within an agent, which transitions from state f to state g with probability T(g←f)=P(g∣f) . The underlying potential function VT quantifies the agent’s global ordering of each state, satisfying the detailed balance condition at equilibrium.
Figure 2: In the conditioned word generation task, the Claude Sonnet 4.5 model exhibits directionality. Transition process of the Claude Sonnet 4.5 model in the prompt word state space, ordered by the potential function VT . Transitions tend to move toward states with lower potentials. States with βV(f)≫log(20000)∼10 are those where the equilibrium condition cannot be strictly satisfied.
Figure 3: In the task of conditioned word generation without a reasoning chain, verification of detailed balance through closed paths in the state transition graph of the GPT-5 Nano model. Each point represents a triplet, with all different triplets found in the experimental data. The error bars directly arise from sampling errors. χ2/ndf is the chi-square per degree of freedom. Note that both axes are in natural logarithm ( ln ), not log10 .
states f
Potential
param1 * tanh(param2 * x + param3) + param4
5.70
param1 - (param2 / (x + param3))
0.88
param1 * x / (1 + param2 * log(x + 1))
-0.57
param1 * tanh(param2 * x) + param3
-1.57
param2 + param1 * (1 - exp(-x))
-3.30
Table 1: Examples of some states of the agent Treal and their potentials. The independent variable log_v_k_nu in the fitting task is abbreviated as x .
Figure 4: In the task of symbolic fitting with a long reasoning chain, test of the detailed balance condition for the agent Treal . The error estimation includes root mean square statistical errors, and the error of the potential function is estimated through bootstrapping. r is the Pearson correlation coefficient, and χ2/ndf is the chi-squared per degree of freedom. Measurement includes state pairs with at least one measured transition between the two states. Points with ∣VT(f)−VT(g)∣>log50288 are excluded. One-tenth of the data points are displayed for clarity.
Figure 5: Illustration of the ability of the potential function discovered using IdeaSearch [ 18 ] to predict the directionality of state transitions. Each point represents a transition pair from state f to state g . The figure shows a subgraph composed of 70 states selected from the database, displaying transitions with high transition kernel Treal(g←f)>0.05 . Red and green lines represent transitions with increasing and decreasing potential functions, respectively. The horizontal axis represents the mean square error of the expression corresponding to the state in the symbolic fitting task, while the vertical axis represents the potential function.
Agent
Transition Samples
Unique States
Unique Transitions
States with Samples >1
IdeaSearchFitter
50228
7484
21697
2551
Conditioned Word Generation (GPT-5 nano)
19968
645
9473
620
Table S1: Database statistics for the two agents, including transition sampling counts, unique state counts, unique transition counts, and the number of states with sampling times greater than 1.
Figure S1: Comparison of Correct Rate=1−∑fN0(f)∑fN(failed←f) for different LLMs in the conditioned word generation task. The Qwen series models are Qwen2.5-Instruct-3B, Qwen2.5-Instruct-7B, Qwen2.5-Instruct-14B, Qwen2.5-Instruct-32B, Qwen2.5-Instruct-72B and Qwen3-Max. The Claude series models are Claude Sonnet 3.7, Claude Sonnet 4, and Claude Sonnet 4.5. The Gemini series models are Gemini 2.0 Flash and Gemini 2.5 Flash. The GPT series models are GPT-4.1 nano and GPT-5 nano.
Figure S2: Test of the detailed balance condition for the Conditioned Word Generation Agent using GPT-5 nano model. The error estimation includes root mean square statistical errors, and the error of the potential function is estimated using the bootstrap method. r is the Pearson correlation coefficient, and χ2/ndf is the chi-squared per degree of freedom. Measurement includes state pairs with at least one measured transition between the two states. The agent’s underlying potential function is consistent with the detailed balance condition, with systematic deviations from detailed balance observed at higher potential function values. Points with ∣VT(f)−VT(g)∣>log20000 are excluded. One-fifth of the data points are displayed for clarity.
Figure S3: Architecture of the IdeaSearchFitter agent (reproduced from Ref. [ 45 ] ). Input data and variable descriptions are expanded into a structured prompt once at initialization, after which the system iterates: an ansatz agent (LLM) generates candidate expressions informed by the best-performing entries in a database, an evaluator scores them, and the results are stored in an island-based database for selection and mutation.
Parameter
Value
Parallel LLM calls per generation ( samplers_num )
3
Best examples shown in prompt ( examples_num )
1
Candidates per call ( generate_num )
3
Sampling temperature
1000
Number of LLM generators
10
Number of islands
8
Table S2: Key hyperparameters of the IdeaSearchFitter agent used in this work, taken from the GitHub repository.
Figure S4: Test of detailed balance through closed paths in the state transition graph of the complex agent Treal . Each point represents a triplet, with a total of 620 different triplets found in the experimental data, where each transition was detected at least twice. This indicates that within the error range, the sums of the logarithms of the forward and backward transition kernels are roughly equal, consistent with the detailed balance condition. To clearly display the figure, only 1/5 of the data points are shown.
Figure S5: Test of the detailed balance condition for both agents using K(x)=log(1+e−x) , with all settings the same as in the main text. r is the Pearson correlation coefficient, and χ2/ndf is the chi-squared per degree of freedom. (a) Results for the IdeaSearchFitter agent. (b) Results for the Conditioned Word Generation Agent.
Figure S6: For pairs in the IdeaSearchFitter and Conditioned Word Generation Agent where the transition f←g was measured but the transition g←f was not, we compare the differences in their potential functions and the logarithm of the transition kernel ratios. The figure shows points corresponding to 500 such pairs. The black dashed line represents the 90th percentile line. It is mostly above the red dashed line representing detailed balance, indicating that the inequality Eq. ( S8 ) is basically satisfied, with those points at larger ∣βVT(f)−βVT(g)∣ possibly arising from systematic overestimation of the potential function differences. (a) Results for the IdeaSearchFitter agent with a total of 18,935 such pairs. (b) Results for the Conditioned Word Generation Agent with a total of 8,805 such pairs. From this figure, it can be directly seen that most of such transition pairs come from cases where N(f←g)=1 .
Figure S7: Results of the detailed balance test for the random agent Trand . r is the Pearson correlation coefficient, and χ2/ndf is the chi-squared per degree of freedom. (a) Examination of the detailed balance condition by comparing the differences in potential functions and the logarithm of transition kernel ratios. (b) Examination of the detailed balance condition through closed paths in the state transition graph. It can be seen that the state transitions of this agent do not satisfy the detailed balance condition.
Task
States
Trans
Npairs
r
rfloor
rceil
Semantic agents (Qwen2.5-72B):
5-Letter-S
206
4232
240
0.58
0.17
0.88
Digit Sum = 15
41
196
15
0.79
0.34
1.00
Sum ÷ 10
69
242
7
0.84
0.52
0.99
Reference models from main text:
GPT-5 nano word gen.
645
9473
329
0.77
0.15
0.90
Table S3: Multi-task test of detailed balance, with matched floor and ceiling controls. r is the measured Pearson correlation between ln(T(g←f)/T(f←g)) and V(f)−V(g) for bidirectional pairs with both counts >1 . rfloor and rceil are matched controls processed through the identical pipeline, quoted as medians over repeated randomizations; their construction and interpretation are described in the text.
Figure S8: Two overlaid pairs of distributions of the consistency measure r for the tasks of Table S3 ; the construction of the controls is described in the text. Left: the matched floor control rfloor (gray) overlaid with the measured random-agent controls (blue). Right: the matched ceiling control rceil (gray) overlaid with the measured semantic agents (red). Each box spans the min–max range across tasks within its group; the black bar marks the cross-task mean of the measured r in each group.
Figure S9: Sensitivity of the detailed-balance test to the minimum sampling threshold N0min for IdeaSearchFitter. N0(f) is the total number of outgoing transitions from state f , and Npairs is the number of state pairs with bidirectional transitions passing the threshold. Left: Pearson r between ln[T(g←f)/T(f←g)] and β(V(f)−V(g)) remains high ( r≃0.82 – 0.92 ) across all thresholds, with only a modest decline even when the dataset is restricted to the ∼600 best-sampled pairs. Right: χ2/ndf increases from ∼0.85 at N0min=1 , crossing unity near N0min=10 and reaching ∼1.2 at N0min=50 ; the rise reflects a slight mismatch between the globally fitted potential and the high-threshold subset, rather than a systematic violation of detailed balance.
Figure S10: Comparison of PageRank potential VPR=−lnπ(f) and variational potential Vvar for the GPT-5 nano word task with damping factor α=0.99 . Light blue points are per-state values; dark red points are binned means with 1σ error bars. The dashed line has slope 1; the solid line is a linear fit (slope =1.30 ). The binned Pearson correlation is r=0.97 .
From \ To
ATT.
TUR.
PER.
PRO.
BUZ.
DIS.
failed
ATT.
0
0
31
27
0
4
3480
TUR.
3521
0
0
0
0
0
74
PER.
3500
0
0
0
0
0
0
PRO.
3401
0
0
0
0
0
0
BUZ.
2055
1254
0
0
0
0
218
DIS.
2435
0
0
0
0
0
0
Table S4: Transition kernel T(g←f) for the Claude Sonnet 4.5 model, where ATT., TUR., PER., PRO., BUZ., and DIS. represent the states ATTITUDE, TURKEY, PERSONAL, PROBLEM, BUZZY, and DISCIPLINE respectively. Each row represents the number of transitions starting from state f , and each column represents the number of transitions to state g . The reason for “failed” is that some transitions are rejected because they are words but the sum of letters is not 100, or the generated word is still the prompt word, especially for transitions starting from the ATTITUDE state.
Figure S11: Transition process of the Gemini 2.5 Flash model in the prompt word state space, sorted by the potential function VT . Transitions tend to move towards states with lower potential functions. States with βV(f)≫log(20000)∼10 are those where the equilibrium condition cannot be strictly satisfied. Thick lines represent high-frequency transitions, while thin lines represent low-frequency transitions. Each horizontal line represents a state, arranged in order of increasing potential function.
Metric
Claude Sonnet 4.5
Gemini 2.5 Flash
Chains converging to ATTITUDE
89/100 (89%)
99/100 (99%)
Mean steps to first ATTITUDE
2.3
1.0
Median steps to first ATTITUDE
2
1
ATTITUDE fraction of valid outputs
68.8%
54.1%
Second most common word
TURKEY (16.6%)
DISCIPLINE (45.9%)
Overall validity rate
15.7%
75.3%
Table S5: Chain experiment: 100 independent Markov chains from diverse starting words, each running 10 steps. Starting words include common English words, rare words, and nonsensical strings satisfying the letter-sum-equals-100 constraint.
Figure S12: (a) Distribution of potential functions for all states sampled at least twice in the IdeaSearchFitter agent. It exhibits a localized structure and can be fitted with a Gaussian distribution N(μ=−0.56,σ=4.37) . (b) Distribution of potential functions for all states sampled at least twice in the Conditioned Word Generation Agent. It exhibits a localized structure and can be fitted with a Gaussian distribution N(μ=−0.93,σ=2.30) .
Agent
σ
Expected min. action
Actual min. action
IdeaSearchFitter
4.38
0.129
0.150
Conditioned Word Generation(GPT-5 nano)
2.30
0.245
0.195
Table S6: Comparison of expected minimum action size through the potential function and actual minimized action for two agents.
Figure S13: Numerical comparison of both sides of Eq. ( S22 ). The comparison is performed for M=10 , with different colors representing different values of T(g←f) . The three subplots represent the cases of n=5,7,9 , respectively.
Figure S14: Resolution contrast between the microscopic architectural energy and the macroscopic state-level potential. (a) In an energy-based model such as EBT, Eθ(x,y^) is a per-token energy; inference proceeds by gradient-descent energy minimization, and the full sequence is still generated autoregressively. (b) Our potential V(f) is a macroscopic free energy defined on the semantically closed states f (blue) and g (red), obtained by coarse-graining over all token sequences that realize a given state. The intermediate chain-of-thought reasoning tokens (gray) carry no V .
Large language models (LLMs) can generate high-level diverse phenomena without explicitly programmed rules. This capability has led to their adoption within different agent-based models (ABMs) and social simulations. Recent studies investigate their ability to generate different phenomena of interest, for example, human behavior on social media platforms or alien behavior in game-theoretic scenarios. However, capability, prediction, and explanation are different--drawing from the philosophy of science and mechanisms literature, explanation requires showing, to some degree, how a phenomenon is produced by related organized entities and activities. For modelers, describing the characteristics of an experiment or whether a simulation provides progress in capability (or explanation), can be difficult without being grounded in potentially distant research areas. We integrate recent work on LLM-ABMs with contemporary philosophy of science literature and use it to operationalize a definition of 'plausibility' in a four-level scale. Our scale separates the evaluation of a model's generative sufficiency (ability to reproduce a phenomenon) from its mechanistic plausibility (how the phenomenon could be produced), and clarifies the distinct roles of different models, such as predictive and explanatory ones. We introduce this as the Mechanism Plausibility Scale.
Complexity and interpretability rarely coincide: systems rich enough for complex behaviours to emerge are usually too opaque to question, while transparent ones are too simple for anything complex to emerge. A single large language model (LLM) is a static artefact, hardly exhibiting any of the emergent properties we associate with life. This changes through interaction: populations of LLMs display emergent dynamics absent from isolated models. Furthermore, LLMs can be endowed with persistent memory, tools and shared skills, and the capacity to initiate actions unprompted, i.e., turning LLMs agentic. In this paper, we argue that such collectives of agents can serve as a computational substrate for Artificial Life (ALife) research. Critically, since the agents communicate in natural language, their collective behaviour can be directly interrogated by examining textual traces and asking the agents themselves. We outline the notion of interpretability in language-model research and extend it for collectives of agents. Lastly, we survey recent examples of agentic LLM collectives that already instantiate the idea of agentic substrates, from controlled experiments to deployments in the wild.
Elias Najarro, Ane Espeseth, Eleni Nisioti +2
IT University of Copenhagen, Denmark · University of Oslo, Norway · Sakana AI, Japan +1
Large Language Models (LLMs) have revolutionized AI research and enabled exciting agent systems. To build a complex LLM agent system, most existing research relies on insights from other domains or heuristics to manually build the agent system. However, this approach often requires heavy hand-engineering and fails to fully optimize for the downstream task of interest. Inspired by the tremendous success of deep learning, we propose to construct LLM agent systems in a modular manner, similar to building a deep neural network. Our key insight is to make analogies between LLM building blocks, such as retrievals, memories, and prompting strategies, and the successful deep learning modules, such as MLPs, attention, and recurrent modules. We further design forward inference and feedback mechanisms for LLMs, where prompts in LLMs are considered as the weights in deep models, and the prompt optimization from feedback is analogous to the back-propagation algorithm. We additionally leverage a search algorithm to search for the best configuration of LLM agent systems, similar to the neural architecture search (NAS) in deep learning research. Comprehensive experimental results demonstrate that the proposed deep learning recipe for LLM agent systems is highly effective, in particular: (1) Organizing LLM modules into deep-learning-style architectures yields noticeable performance gain; (2) Automatic prompt optimization, equivalent to backpropagation, is efficient in incorporating feedback from the task of interest and achieves at least 5% performance improvement; (3) NAS equivalent algorithm works well for further optimizing the LLM agent system architecture with 11% performance gain compared with randomly designed architectures. Overall, our research demonstrates the exciting opportunity of transferring the success of deep learning to building LLM agent systems.