Active inference selects actions by minimising an expected free energy functional over predicted futures. However, adding an expectation over yet-unobserved outcomes means the free energy functional no longer has a Kullback-Leibler structure, which hinders message passing treatments of inference procedures. We propose an alternative formulation based on a Bethe free energy functional, fully supporting inference by message passing. The epistemic drive is maintained by imposing an information constraint, next to normalisation, marginalisation and form constraints, insisting that the mutual information between future observations, states and parameters given actions must be at least as large as the entropy of the goal prior. For a specific value of the corresponding Karush-Kuhn-Tucker multiplier, the stationary point of this constrained Bethe Lagrangian recovers the expected free energy solution. We show that, as the information demand is varied, the solved multiplier moves through its inactive, interior, and saturated regimes. In the inactive regime the agent's epistemic drive switches off entirely, while in the saturated regime it is maximal. We compare the performance of the constrained Bethe agent on three tasks against EFE and Q-MDP.
Figures & tables
Figure 1: Forney-style factor graph of the planning model in ( 1 ) over the horizon τ . Each transition factor ( fx ) maps the previous state xk−1 , the control uk and the transition parameters B to xk . Each emission factor ( fy ) maps xk and the emission parameters A to the predicted observation yk , whose edge terminates in the goal-prior factor f∗=p∗(yk) . Equality nodes ( = ) share B and A across time. Forward predictive messages (1, 2) and backward messages (3, 4) realise the forward–backward sweep of Section 3 .
Reach rate
Cue-visit rate
⟨ time-to-target ⟩
⟨ return ⟩
Q-MDP
0.82±.03
0.79±.03
2.68±.05
+1.22±.06
EFE
1.00
1.00
2.00
+1.92±.03
CBFE
1.00
1.00
2.00
+1.92±.03
Table 1: Canonical T-maze over 200 trials ( T=K=3 , ρ=0.98 ; ± one standard error over trials). The constrained Bethe and standard-EFE agents behave identically. Q-MDP, blind to future information gain, reaches the cue without exploiting it and gambles on an arm in the remaining trials.
Figure 2: A representative cue-grid trial. Left: the constrained Bethe agent detours to the off-path cue (gold star) to resolve the reward condition, then proceeds to the rewarding goal (green). Right: Q-MDP ignores the cue and commits to a goal directly, here the non-rewarding one (red). Walls are grey, the open circle is the start, and paths shade from light (early) to dark (late).
Reach rate
Cue-visit rate
⟨ time-to-target ⟩
⟨ return ⟩
CBFE
0.84±.03
1.00
12.49±0.27
+0.83±.03
EFE
0.45±.04
0.00
6.00±0.00
−0.10±.07
EFE-LA
1.00±.01
1.00
12.32±0.18
+1.00±.01
Q-MDP
0.37±.03
0.01±.01
13.08±0.34
+0.02±.06
Table 2: Information-gathering grid over 200 trials ( K=6 , Tmax=25 ; ± one standard error). Reach rate is the fraction of trials reaching the rewarding goal. time-to-target is over the trials that reach a goal, which favours agents with lower reach rates. The unconditional return column is unaffected.
Figure 3: The information floor selects the dual’s regime (cue grid). (a) Solved multiplier γ against the per-horizon floor β for a representative planning state: inactive at zero, a unique interior root crossing γ=1 , and saturation at γmax . (b) Cue-visit and rewarding-goal rates as the reward-modality floor KH[p∗] is varied, rising from a blind gamble to maximal information seeking at the inactive–binding boundary of (a).
Figure 4: Salience–novelty allocation under learning. (a) The gate-probe rate falls and the cue-visit rate rises across trials (top), tracked by novelty and salience (bottom). (b) The reward rate recovers to 1 as the agent stops gambling; EFE+novelty planner shifts earlier, the salience-only agent is always correct, and Q-MDP is at chance.
Planning under uncertainty requires agents to balance goal achievement with information gathering. Active inference addresses this through the Expected Free Energy (EFE), a cost function that unifies instrumental and epistemic objectives. However, existing EFE-based methods typically employ specialized optimization procedures that are difficult to extend or analyze. In this paper, we show that EFE-based planning can be formulated as Variational Free Energy minimization on a generative model augmented with epistemic priors. Our main result demonstrates that minimizing a Variational Free Energy functional with appropriately chosen priors yields a decomposition into expected plan costs (the EFE) plus a complexity term. This formulation reinforces theoretical consistency with the Free Energy Principle by casting planning as the same inferential process that governs perception and learning. We validate our approach on three environments of increasing complexity: a deterministic T-maze, a stochastic Reactivity Maze, and a partially observable MiniGrid DoorKey-8x8 environment. The experiments demonstrate that the epistemic priors induce information-seeking behavior, that the variational formulation yields policy-based inference outperforming plan-based methods under stochastic transitions, and that temporal factorization enables scalability to environments where existing tabular active inference methods cannot operate.
Wouter W. L. Nuijten, Thijs van de Laar, Bert de Vries
Eindhoven University of Technology, Eindhoven, the Netherlands · Lazy Dynamics B.V., the Netherlands
Active inference casts decision-making as inference, with the Expected Free Energy (EFE) unifying goal-directed and information-seeking behavior. Recent work showed that EFE minimization can be written as Variational Free Energy (VFE) minimization on a generative model augmented with epistemic priors. We prove that the VFE of the augmented model can be rewritten as the VFE of the predictive model plus explicit entropy-correction terms, making the EFE contribution transparent. We then show that proper EFE-based planning requires combining these epistemic corrections with a planning correction that turns marginal inference into policy optimization, yielding a full variational characterization of EFE-based planning. This clarifies which corrections are needed for cross-entropy planning and for full EFE-based planning. The same entropy-corrected formulation leads to a detailed message-passing scheme for EFE-based planning together with simpler ablations. Experiments on three grid-world environments show that full EFE-based planning outperforms ablations that omit either the planning correction or the epistemic corrections.
Wouter W. L. Nuijten, Mykola Lukashchuk, Thijs van de Laar +1
Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands · Lazy Dynamics, Utrecht, the Netherlands
Sophisticated Inference is a variant of active inference often associated with recursive belief modeling and tree search. We argue that its central computational role is simpler: within a planning horizon, it makes active inference closed-loop by allowing future actions to depend on future states and observations. This closed-loop structure can be represented in the epistemic-prior variational free energy framework. Epistemic priors supply the active-inference objective, while a joint posterior over future states and actions supplies the state-contingent control structure. We evaluate this decomposition in the Reactivity Maze, a stochastic benchmark designed to separate epistemic incentive from inner-horizon closed-loop control. The comparison includes three variational objectives with the same state-action posterior family, an action-state factorized active inference objective, Sophisticated Inference, and standard Expected Free Energy planning. The results show that neither ingredient is sufficient on its own. Methods without an epistemic component do not seek information, while methods that prevent future actions from depending on future states cannot turn information into reliable goal-reaching. By contrast, both Sophisticated Inference and full-joint epistemic-prior active inference solve the environment by combining epistemic drive with closed-loop inference. These results show that the advantage associated with Sophisticated Inference need not be specific to tree search itself. It arises from the closed-loop form of active inference, and this form can be represented in epistemic-prior variational inference when the posterior keeps future actions dependent on future states.
Wouter W. L. Nuijten, Bert de Vries
Eindhoven University of Technology, 5612 AP Eindhoven, the Netherlands · Lazy Dynamics, Utrecht, the Netherlands