Active inference casts perception, learning and planning as variational inference, where the Expected Free Energy (EFE) defines the planning objective. Recent work showed that EFE minimization can be written as Variational Free Energy (VFE) minimization on a generative model augmented with epistemic priors, and that the resulting entropy-corrected objective is minimized by sum-product message passing (SP-MP) on modified factor functions. These formulations treat model uncertainty as inference over a finite set of known models. We extend them to planning during learning, where beliefs over the observation and transition parameters θA, θB are Dirichlet priors. Closed-form Dirichlet updates are usually obtained by a structured mean-field factorization between parameters and states, but under exactly that factorization the information-gain corrections vanish. We resolve this by giving every factor its own parameter prior, frozen during a rollout. Conjugacy then makes each factor belief a product of a marginal over the categorical variables and a one-step Bayes posterior of the parameters. This reintroduces the datum-dependent parameter posteriors that mean-field factorization destroys. We show that evaluating the epistemic priors under the resulting Bethe joint recovers the information gains. Stationarity of this single closed-form objective yields SP-MP whose factor functions are augmented by the information-gain weight. Experiments on two grid POMDPs, in which only a planner that learns the model finds the safe route, show that the method plans to efficiently learn θA and θB. It reaches the goal on every episode, whereas SP-MP planners with the same inference update plateau or take the risky route.
Figures & tables
Figure 1: Frozen-parameter factor graph (Section 4.3 ). Squares: factors; edges: variables; = : equality nodes; filled squares: clamped data; p^ : preferences. Left: the real slice carries the shared θA,θB and turns p(xprior),uprev,y0 into the planning prior p(x0) . Right: over the horizon each factor f∗ (factor functions ( 17 )–( 18 )) owns a copy θA(t),θB(t) with its own degree-two prior node (open, gray) frozen at Dir(αA) resp. Dir(αB) ; only θc is shared, through equality nodes.
Figure 2: Top (a–c): learning θA on fog-walls ( B known, H=8 , 5 seeds); bottom (d–f): learning θB on the unmapped detour (identity A , H=7 , T=8 , 3 seeds). Curves are trailing means with a ±1 std band over seeds; the glyph legend gives the cell and route vocabulary of (a) and (d).
fog-walls, learn θA
unmapped detour, learn θB
Planner
goals/100
last 20
err A
known
frozen
learned
detour
SP-EFE
95.8±0.7
20.0
0.026
300
300†
300
300
SP-Pref-Plan
81.4±4.8
16.2
0.037
300
146
146±7
0
SP-Pref
47.4±3.1
9.6
0.049
146
300†
158±8
25
Random
1.0±0.9
0.2
0.038
1
1
1.3±0.5
0
SP-EFE−dyn
94.4±2.0
20.0
0.026
300
0
298
298
Table 1: Results and ablation. Fog-walls, learning θA (100 episodes, 5 seeds): goals, goals in the last 20, final matrix error. Unmapped detour, learning θB (300 episodes, 3 seeds): goals with B known, frozen at the prior, or learned, and learned- B goals via the detour. Lower block: ablations of ( 16 ). † Entropy-following: with frozen flat doors, the most uncertain column and the detour coincide.
Planning under uncertainty requires agents to balance goal achievement with information gathering. Active inference addresses this through the Expected Free Energy (EFE), a cost function that unifies instrumental and epistemic objectives. However, existing EFE-based methods typically employ specialized optimization procedures that are difficult to extend or analyze. In this paper, we show that EFE-based planning can be formulated as Variational Free Energy minimization on a generative model augmented with epistemic priors. Our main result demonstrates that minimizing a Variational Free Energy functional with appropriately chosen priors yields a decomposition into expected plan costs (the EFE) plus a complexity term. This formulation reinforces theoretical consistency with the Free Energy Principle by casting planning as the same inferential process that governs perception and learning. We validate our approach on three environments of increasing complexity: a deterministic T-maze, a stochastic Reactivity Maze, and a partially observable MiniGrid DoorKey-8x8 environment. The experiments demonstrate that the epistemic priors induce information-seeking behavior, that the variational formulation yields policy-based inference outperforming plan-based methods under stochastic transitions, and that temporal factorization enables scalability to environments where existing tabular active inference methods cannot operate.
Wouter W. L. Nuijten, Thijs van de Laar, Bert de Vries
Eindhoven University of Technology, Eindhoven, the Netherlands · Lazy Dynamics B.V., the Netherlands
Active inference casts decision-making as inference, with the Expected Free Energy (EFE) unifying goal-directed and information-seeking behavior. Recent work showed that EFE minimization can be written as Variational Free Energy (VFE) minimization on a generative model augmented with epistemic priors. We prove that the VFE of the augmented model can be rewritten as the VFE of the predictive model plus explicit entropy-correction terms, making the EFE contribution transparent. We then show that proper EFE-based planning requires combining these epistemic corrections with a planning correction that turns marginal inference into policy optimization, yielding a full variational characterization of EFE-based planning. This clarifies which corrections are needed for cross-entropy planning and for full EFE-based planning. The same entropy-corrected formulation leads to a detailed message-passing scheme for EFE-based planning together with simpler ablations. Experiments on three grid-world environments show that full EFE-based planning outperforms ablations that omit either the planning correction or the epistemic corrections.
Wouter W. L. Nuijten, Mykola Lukashchuk, Thijs van de Laar +1
Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands · Lazy Dynamics, Utrecht, the Netherlands
Sophisticated Inference is a variant of active inference often associated with recursive belief modeling and tree search. We argue that its central computational role is simpler: within a planning horizon, it makes active inference closed-loop by allowing future actions to depend on future states and observations. This closed-loop structure can be represented in the epistemic-prior variational free energy framework. Epistemic priors supply the active-inference objective, while a joint posterior over future states and actions supplies the state-contingent control structure. We evaluate this decomposition in the Reactivity Maze, a stochastic benchmark designed to separate epistemic incentive from inner-horizon closed-loop control. The comparison includes three variational objectives with the same state-action posterior family, an action-state factorized active inference objective, Sophisticated Inference, and standard Expected Free Energy planning. The results show that neither ingredient is sufficient on its own. Methods without an epistemic component do not seek information, while methods that prevent future actions from depending on future states cannot turn information into reliable goal-reaching. By contrast, both Sophisticated Inference and full-joint epistemic-prior active inference solve the environment by combining epistemic drive with closed-loop inference. These results show that the advantage associated with Sophisticated Inference need not be specific to tree search itself. It arises from the closed-loop form of active inference, and this form can be represented in epistemic-prior variational inference when the posterior keeps future actions dependent on future states.
Wouter W. L. Nuijten, Bert de Vries
Eindhoven University of Technology, 5612 AP Eindhoven, the Netherlands · Lazy Dynamics, Utrecht, the Netherlands