Stable Deep Reinforcement Learning via Isotropic Gaussian Representations

Ali Saheb Pasand; Johan Obando-Ceron; Aaron Courville; Pouya Bashivan; Pablo Samuel Castro

Method	Train Accuracy( $\uparrow$ )	SIGReg Loss( $\downarrow$ )	Feature Rank ( $\uparrow$ )	Dormant Neurons%
Adam	33.0	49.1	149.0	18.5
Adam+SIGReg	50.0	7.6	262.9	0.4
RAdam	29.8	42.3	148.8	12.6
RAdam+SIGReg	42.8	8.4	251.8	0.6
Kron	62.6	28.5	167.4	7.9
Kron+SIGReg	72.7	7.2	252.2	0.9

Appendix

Table 1: Non-stationary CIFAR-10. Area Under the Curve (AUC) for different methods across evaluation metrics. SIGReg loss minimization improve results across all metrics and optimizers.

Figure 8: Effect of non-stationarity and SIGReg loss minimization on representation stability and accelerating recovery in non-stationary CIFAR-10 experiment. Non-stationarity induced by label shuffling causes a sharp drop in accuracy, increase in SIGReg loss, collapse of feature rank, and higher neuron dormancy. It can be observed that Kronecker-factored optimizer implicitly lowers the SIGReg loss compared to first-order methods. Explicitly minimizing SIGReg further improves all optimizers by accelerating recovery, preserving feature rank, and reducing neuron dormancy.

	PQN		PPO
Method	% of Games Improved ( $\uparrow$ )	Avg. Improvement ( $\uparrow$ )	% of Games ( $\uparrow$ )	Avg. Improvement ( $\uparrow$ )
Gaussian (Real and Imaginary parts)	90%	305%	70%	7.9%
Laplacian	90%	234%	60%	3.2%
Logistic	90%	209%	50%	-0.8%
Real Part Only	100%	151%	60%	2.4%
Imaginary Part Only	70%	56%	50%	1.9%

Appendix

Table 2: Ablation study on Atari-10. Percentage of games improved over baseline and average AUC improvement across all games. Top: different isotropic target distributions (Laplacian and Logistic). Middle: minimizing different SIGReg loss components. Bottom: whitening the covariance matrix without controlling tails. Gaussian with real and imaginary components performs best, while enforcing isotropy alone can improve some environments. In PQN, minimizing only the real part improves more games than the baseline Isotropic Gaussian, though with significantly smaller average improvement.

	PQN			PPO
Method	% of Games Improved ( $\uparrow$ )	Avg. Improvement ( $\uparrow$ )	Avg. SPS ( $\uparrow$ )	% of Games Improved ( $\uparrow$ )	Avg. Improvement ( $\uparrow$ )	Avg. SPS ( $\uparrow$ )
Kron	98.2%	1404%	1.6k	45.6%	-2.6%	1.8k
Baseline + SIGReg	89.5%	889%	9.3k	68.4%	25.5%	2.7k
Baseline + MultiSkip + SIGReg	92.9%	1695%	11.4k	87.7%	88.6%	3.2k

Appendix

Table 3: PQN and PPO Performance Across Full Atari Suite. We report the percentage of games in which each method improves over the baseline, the average percentage of AUC improvement across all games, and the average steps per second (Avg. SPS). In terms of average AUC improvement, the baseline augmented with multi-skip residual connections and SIGReg loss minimization outperforms all other methods, while also achieving higher SPS and avoiding the memory overhead associated with computing gradient curvature. In terms of the percentage of games improved, Kronecker-factored optimization achieves the best result by a small margin in PQN, and it is outperformed by both two other cases in PPO. Overall, these results indicate that the performance gains provided by Kron do not justify the additional memory and computational cost.

Figure 9: Atari-10, PQN, Kron Optimizer. The change of reward, SIGReg loss, feature rank, and percentage of dormant neurons. In almost all games, Kron leads to implicit minimization of SIGReg loss, higher rank, and lower dormancy.

Figure 10: Atari-10, PQN, Multi-skip Residual Architecture. The change of reward, SIGReg loss, feature rank, and percentage of dormant neurons. In almost all games, using Multi-skip Residual Architecture leads to implicit minimization of SIGReg loss, higher rank, and lower dormancy.

Figure 11: Atari-10, PPO, Kron Optimizer. The change of reward, SIGReg loss, feature rank, and percentage of dormant neurons. In almost all games, Kron leads to implicit minimization of SIGReg loss, higher rank, and lower dormancy.

Figure 12: Atari-10, PPO, Multi-skip Residual Architecture. The change of reward, SIGReg loss, feature rank, and percentage of dormant neurons. In almost all games, using Multi-skip Residual Architecture leads to implicit minimization of SIGReg loss, higher rank, and lower dormancy.

Figure 13: Atari-10, PQN with Explicit SIGReg Loss Minimization. The change of reward, SIGReg loss, feature rank, and percentage of dormant neurons through time. The orange line is the baseline without SIGReg loss minimization, and the other lines are SIGReg loss minimization with different strengths (Larger

\lambda

means stronger regularization).

Figure 14: IQM Human-Normalized Results for PQN (top) and PPO (bottom) Across the Atari-10 Benchmark Games. Explicit SIGReg loss minimization encourages isotropic Gaussian embeddings, reducing the performance gap between first-order optimizers (RAdam) and second-order Kronecker-factored optimization. Adding multi-skip residual connections further improves performance.

Figure 15: PQN. Reward over time for individual games with and without SIGReg loss minimization.

Figure 16: PPO. Reward over time for individual games with and without SIGReg loss minimization.

Figure 17: Isaac Gym continuous control. Learning curves for PPO and PPO + SIGReg on four representative locomotion tasks. Encouraging isotropic Gaussian representations improves learning stability and asymptotic performance across all environments. Curves show mean episode returns over five independent runs, with shaded regions indicating variability across runs.

Hyperparameter	Value / Description
Learning rate	2.5e-4
Anneal lr	False (no learning rate annealing)
Num envs	128 (parallel environments)
Num steps	32 (steps per rollout per environment)
Gamma	0.99 (discount factor)
Num minibatches	32

Appendix

Table 4: PQN Hyperparameters

Hyperparameter	Value / Description
Learning rate	2.5e-4
Num envs	8
Num steps	128 (steps per rollout per environment)
Anneal lr	True (learning rate annealing enabled)
Gamma	0.99 (discount factor)
Gae lambda	0.95 (GAE parameter)

Appendix

Table 5: PPO Hyperparameters

Hyperparameter	Value / Description
Total timesteps	30,000,000
Learning rate	0.0026
Num envs	4096 (parallel environments)
Num steps	16 (steps per rollout)
Anneal lr	False (disable learning rate annealing)
Gamma	0.99 (discount factor)

Appendix

Table 6: PPO Hyperparameters for IsaacGym

Hyperparameter	Value
Batch size	256
Epochs	100
Learning rate	0.00025

Appendix

Table 7: Image Classification Hyperparameters (CIFAR-10)

Hyperparameter	Value
Number of Random Projections	16
Number of Frequency Samples	8
Maximum Frequency	5.0
Regularization Factor (larger $\xrightarrow{}$ stronger)	Best value between [1.0 and 0.2] (see Table 9 )

Appendix

Table 8: SIGReg Loss Hyperparameters.

Game	PQN	PPO	Game	PQN	PPO
Alien	1.0	1.0	Kangaroo	0.2	0.2
Amidar	1.0	0.2	Krull	1.0	0.2
Assault	1.0	1.0	KungFuMaster	1.0	0.2
Asterix	0.2	0.2	MontezumaRevenge	0.2	1.0
Asteroids	1.0	1.0	MsPacman	1.0	0.2
Atlantis	1.0	1.0	NameThisGame	0.2	1.0

Appendix

Table 9: Best regularization factor (

\lambda

) selected via AUC for PQN and PPO algorithms across Atari games (Larger means stronger regularization)

Figure 18: Tracking error norm

\|e_{t}\|_{2}=\|w_{t}-w^{*}\|_{2}

over all steps for isotropic Gaussian distribution , across three random seeds (columns). The learner tracks a sequence of three target weights,

w^{*}_{1}\overset{t=100}{\longrightarrow}w^{*}_{2}\overset{t=200}{\longrightarrow}w^{*}_{3}

, each drawn uniformly at random from the unit sphere. The corresponding target angles

(\theta_{1},\theta_{2},\theta_{3})

are reported in each column title. Green dots and dotted vertical lines mark the two task-switch points at

t\in\{100,200\}

. Dashed red vertical lines indicate steps at which

\|e_{t+1}\|_{2}>\|e_{t}\|_{2}

, highlighting transient divergence episodes. Each switch induces an abrupt spike in the tracking error norm due to the change in

w^{*}

, followed by reconvergence, whose rate and steady-state level depend on the geometry of the sample distribution.

Figure 19: Tracking error norm

\|e_{t}\|_{2}=\|w_{t}-w^{*}\|_{2}

over all steps for anisotropic Gaussian distribution with condition number

\kappa=10

, across three random seeds (columns). The learner tracks a sequence of three target weights,

w^{*}_{1}\overset{t=100}{\longrightarrow}w^{*}_{2}\overset{t=200}{\longrightarrow}w^{*}_{3}

, each drawn uniformly at random from the unit sphere. The corresponding target angles

(\theta_{1},\theta_{2},\theta_{3})

are reported in each column title. Green dots and dotted vertical lines mark the two task-switch points at

t\in\{100,200\}

. Dashed red vertical lines indicate steps at which

\|e_{t+1}\|_{2}>\|e_{t}\|_{2}

, highlighting transient divergence episodes. Each switch induces an abrupt spike in the tracking error norm due to the change in

w^{*}

, followed by reconvergence, whose rate and steady-state level depend on the geometry of the sample distribution.

Figure 20: Tracking error norm

\|e_{t}\|_{2}=\|w_{t}-w^{*}\|_{2}

over all steps for anisotropic Gaussian distribution with condition number

\kappa=100

, across three random seeds (columns). The learner tracks a sequence of three target weights,

w^{*}_{1}\overset{t=100}{\longrightarrow}w^{*}_{2}\overset{t=200}{\longrightarrow}w^{*}_{3}

, each drawn uniformly at random from the unit sphere. The corresponding target angles

(\theta_{1},\theta_{2},\theta_{3})

are reported in each column title. Green dots and dotted vertical lines mark the two task-switch points at

t\in\{100,200\}

. Dashed red vertical lines indicate steps at which

\|e_{t+1}\|_{2}>\|e_{t}\|_{2}

, highlighting transient divergence episodes. Each switch induces an abrupt spike in the tracking error norm due to the change in

w^{*}

, followed by reconvergence, whose rate and steady-state level depend on the geometry of the sample distribution.

Figure 21: Tracking error norm

\|e_{t}\|_{2}=\|w_{t}-w^{*}\|_{2}

over all steps for isotropic Laplace distribution , across three random seeds (columns). The learner tracks a sequence of three target weights,

w^{*}_{1}\overset{t=100}{\longrightarrow}w^{*}_{2}\overset{t=200}{\longrightarrow}w^{*}_{3}

, each drawn uniformly at random from the unit sphere. The corresponding target angles

(\theta_{1},\theta_{2},\theta_{3})

are reported in each column title. Green dots and dotted vertical lines mark the two task-switch points at

t\in\{100,200\}

. Dashed red vertical lines indicate steps at which

\|e_{t+1}\|_{2}>\|e_{t}\|_{2}

, highlighting transient divergence episodes. Each switch induces an abrupt spike in the tracking error norm due to the change in

w^{*}

, followed by reconvergence, whose rate and steady-state level depend on the geometry of the sample distribution.

Figure 22: Tracking error norm

\|e_{t}\|_{2}=\|w_{t}-w^{*}\|_{2}

over all steps for anisotropic Laplace distribution with condition number

\kappa=10

, across three random seeds (columns). The learner tracks a sequence of three target weights,

w^{*}_{1}\overset{t=100}{\longrightarrow}w^{*}_{2}\overset{t=200}{\longrightarrow}w^{*}_{3}

, each drawn uniformly at random from the unit sphere. The corresponding target angles

(\theta_{1},\theta_{2},\theta_{3})

are reported in each column title. Green dots and dotted vertical lines mark the two task-switch points at

t\in\{100,200\}

. Dashed red vertical lines indicate steps at which

\|e_{t+1}\|_{2}>\|e_{t}\|_{2}

, highlighting transient divergence episodes. Each switch induces an abrupt spike in the tracking error norm due to the change in

w^{*}

, followed by reconvergence, whose rate and steady-state level depend on the geometry of the sample distribution.

Figure 23: Tracking error norm

\|e_{t}\|_{2}=\|w_{t}-w^{*}\|_{2}

over all steps for anisotropic Laplace distribution with condition number

\kappa=100

, across three random seeds (columns). The learner tracks a sequence of three target weights,

w^{*}_{1}\overset{t=100}{\longrightarrow}w^{*}_{2}\overset{t=200}{\longrightarrow}w^{*}_{3}

, each drawn uniformly at random from the unit sphere. The corresponding target angles

(\theta_{1},\theta_{2},\theta_{3})

are reported in each column title. Green dots and dotted vertical lines mark the two task-switch points at

t\in\{100,200\}

. Dashed red vertical lines indicate steps at which

\|e_{t+1}\|_{2}>\|e_{t}\|_{2}

, highlighting transient divergence episodes. Each switch induces an abrupt spike in the tracking error norm due to the change in

w^{*}

, followed by reconvergence, whose rate and steady-state level depend on the geometry of the sample distribution.

Figure 24: Tracking error vectors

e_{t}=w_{t}-w^{*}

in the 2D weight space for isotropic Gaussian distribution . The learner is trained sequentially on three tasks defined by target weights

w^{*}_{1},w^{*}_{2},w^{*}_{3}

, each drawn uniformly at random from the unit circle, with task switches at steps

t=100

and

t=200

. Each panel corresponds to one phase of training (rows) and one random seed (columns), and the title indicates the active target

w^{*}

for that phase. Within each panel, black arrows originate at the origin and point to

e_{t}

, illustrating the instantaneous direction and magnitude of the tracking error. The green dot marks

e_{1}

, the first error vector immediately after each task switch. Red arrows connect consecutive error vectors when the error norm increases, while blue arrows indicate decreases in the error norm. Axes are shown on a symmetric logarithmic scale to simultaneously visualize transient spikes following task switches and steady-state residual errors.

Figure 25: Tracking error vectors

e_{t}=w_{t}-w^{*}

in the 2D weight space for anisotropic Gaussian distribution with condition number

\kappa=10

. The learner is trained sequentially on three tasks defined by target weights

w^{*}_{1},w^{*}_{2},w^{*}_{3}

, each drawn uniformly at random from the unit circle, with task switches at steps

t=100

and

t=200

. Each panel corresponds to one phase of training (rows) and one random seed (columns), and the title indicates the active target

w^{*}

for that phase. Within each panel, black arrows originate at the origin and point to

e_{t}

, illustrating the instantaneous direction and magnitude of the tracking error. The green dot marks

e_{1}

, the first error vector immediately after each task switch. Red arrows connect consecutive error vectors when the error norm increases, while blue arrows indicate decreases in the error norm. Axes are shown on a symmetric logarithmic scale to simultaneously visualize transient spikes following task switches and steady-state residual errors.

Figure 26: Tracking error vectors

e_{t}=w_{t}-w^{*}

in the 2D weight space for anisotropic Gaussian distribution with condition number

\kappa=100

. The learner is trained sequentially on three tasks defined by target weights

w^{*}_{1},w^{*}_{2},w^{*}_{3}

, each drawn uniformly at random from the unit circle, with task switches at steps

t=100

and

t=200

. Each panel corresponds to one phase of training (rows) and one random seed (columns), and the title indicates the active target

w^{*}

for that phase. Within each panel, black arrows originate at the origin and point to

e_{t}

, illustrating the instantaneous direction and magnitude of the tracking error. The green dot marks

e_{1}

, the first error vector immediately after each task switch. Red arrows connect consecutive error vectors when the error norm increases, while blue arrows indicate decreases in the error norm. Axes are shown on a symmetric logarithmic scale to simultaneously visualize transient spikes following task switches and steady-state residual errors.

Figure 27: Tracking error vectors

e_{t}=w_{t}-w^{*}

in the 2D weight space for isotropic Laplace distribution . The learner is trained sequentially on three tasks defined by target weights

w^{*}_{1},w^{*}_{2},w^{*}_{3}

, each drawn uniformly at random from the unit circle, with task switches at steps

t=100

and

t=200

. Each panel corresponds to one phase of training (rows) and one random seed (columns), and the title indicates the active target

w^{*}

for that phase. Within each panel, black arrows originate at the origin and point to

e_{t}

, illustrating the instantaneous direction and magnitude of the tracking error. The green dot marks

e_{1}

, the first error vector immediately after each task switch. Red arrows connect consecutive error vectors when the error norm increases, while blue arrows indicate decreases in the error norm. Axes are shown on a symmetric logarithmic scale to simultaneously visualize transient spikes following task switches and steady-state residual errors.

Figure 28: Tracking error vectors

e_{t}=w_{t}-w^{*}

in the 2D weight space for anisotropic Laplace distribution with condition number

\kappa=10

. The learner is trained sequentially on three tasks defined by target weights

w^{*}_{1},w^{*}_{2},w^{*}_{3}

, each drawn uniformly at random from the unit circle, with task switches at steps

t=100

and

t=200

. Each panel corresponds to one phase of training (rows) and one random seed (columns), and the title indicates the active target

w^{*}

for that phase. Within each panel, black arrows originate at the origin and point to

e_{t}

, illustrating the instantaneous direction and magnitude of the tracking error. The green dot marks

e_{1}

, the first error vector immediately after each task switch. Red arrows connect consecutive error vectors when the error norm increases, while blue arrows indicate decreases in the error norm. Axes are shown on a symmetric logarithmic scale to simultaneously visualize transient spikes following task switches and steady-state residual errors.

Figure 29: Tracking error vectors

e_{t}=w_{t}-w^{*}

in the 2D weight space for anisotropic Laplace distribution with condition number

\kappa=100

. The learner is trained sequentially on three tasks defined by target weights

w^{*}_{1},w^{*}_{2},w^{*}_{3}

, each drawn uniformly at random from the unit circle, with task switches at steps

t=100

and

t=200

. Each panel corresponds to one phase of training (rows) and one random seed (columns), and the title indicates the active target

w^{*}

for that phase. Within each panel, black arrows originate at the origin and point to

e_{t}

, illustrating the instantaneous direction and magnitude of the tracking error. The green dot marks

e_{1}

, the first error vector immediately after each task switch. Red arrows connect consecutive error vectors when the error norm increases, while blue arrows indicate decreases in the error norm. Axes are shown on a symmetric logarithmic scale to simultaneously visualize transient spikes following task switches and steady-state residual errors.

Figure 30: Tracking error dynamics under isotropic Gaussian and anisotropic Laplace feature distributions. The left column shows the evolution of the tracking error vector

e_{t}=w_{t}-w^{*}

after the task switch, while the right column shows the corresponding error norm

\|e_{t}\|_{2}

over time. The task switches at

t=T_{\mathrm{switch}}

from

w_{1}^{*}=(0,\pm 1)

to

w_{2}^{*}=(\cdot,\cdot)

(shown in the legend), with two scenarios overlaid in each panel. Arrow plots (left). Each arrow from the origin represents

e_{t}

at a given step; purple and black arrows correspond to the two configurations. Colored arrows between successive tips indicate temporal evolution: red for an increase in

\|e_{t}\|_{2}

, blue for a decrease. The first post-switch error

e_{1}

is highlighted by a color-coded dot. Axes use a symmetric log scale to capture small and large deviations. Norm plots (right). Solid curves show

\|e_{t}\|_{2}

for each scenario; vertical dashed red lines mark norm increases and dotted lines indicate the task switch, with markers highlighting the error at switching. Initialization.

w_{1}^{*}=(0,\pm 1)

aligns with the principal direction of the feature distribution (largest eigenvalue of covariance), which in the anisotropic case is the dominant eigenvector of

\Sigma

. Anisotropic distribution. For anisotropic Laplace, features are sampled independently with coordinate scales to match a diagonal covariance matrix with eigenvalues

(\frac{2}{1+\kappa},\frac{2\kappa}{1+\kappa})

(trace 2, condition number

\kappa

), ensuring both distributions have the same total variance. Observation. While isotropic Gaussian features show near-monotone error contraction, anisotropic Laplace features lead to frequent increases and non-monotone trajectories that plateau before reaching zero, highlighting the impact of anisotropy and non-Gaussianity on tracking stability.

Figure 31: Tracking error dynamics under different input distributions. Evolution of the tracking error

\|e_{t}\|_{2}=\|w_{t}-w^{*}\|_{2}

over time for Gaussian and Laplace input distributions with varying condition numbers (

\kappa

), shown across three random seeds (columns). The top row corresponds to

d=8

and the bottom row to

d=32

. The learner tracks a sequence of target weights,

w^{*}_{1}\xrightarrow{t=100}w^{*}_{2}\xrightarrow{t=200}w^{*}_{3}

, each sampled uniformly from the unit sphere. Each target switch induces a sharp increase in error, followed by reconvergence, whose rate and stability depend on the geometry of the input distribution. Across both dimensions and seeds, isotropic Gaussian inputs yield smooth, monotonic, and rapid convergence to zero error, whereas isotropic Laplace inputs also converge but exhibit pronounced transient spikes. Increasing the condition number (i.e., reduced isotropy) leads to slower convergence and, in some cases, failure to fully converge.

Figure 32: Existence of divergent directions under anisotropic Gaussian distributions. We test the hypothesis that when the tracking error aligns with the eigenvector corresponding to the smallest eigenvalue of the input covariance in the reverse direction (

e_{t}=-v_{d}

), the dynamics fail to converge to zero steady-state error. Specifically, for Gaussian inputs, we initialize the task such that the error vector is aligned with

-v_{d}

, where

v_{d}

is the eigenvector associated with the smallest eigenvalue, and subsequently update the target weights so that

e_{t}=-\beta v_{d}

at each step. Results show that larger condition numbers (

\kappa

) lead to increased divergence, whereas the isotropic case (

\kappa=1

) remains convergent. Moreover, increasing

\beta

amplifies divergence; however, for moderate conditioning (e.g.,

\kappa=5

), smaller values of

\beta

(e.g.,

\beta=2.4

) do not induce divergence.

Method	% of Games Improved ( $\uparrow$ )	Avg. Improvement ( $\uparrow$ )
Gaussian (Real and Imaginary parts)	90%	305%
Laplacian	90%	234%
Logistic	90%	209%
Real Part Only	100%	151%
Imaginary Part Only	70%	56%
Covariance Whitening (VICReg)	90%	144%

Appendix

Table 10: Comparison with alternative regularization methods. We report the percentage of games improved over the baseline and the average improvement across Atari-10 using the PQN algorithm. Top: different isotropic target distributions. Middle: minimizing different components. Bottom: comparison with alternative regularization methods. Among the baselines, embedding norm regularization performs the worst. Simplicial embedding improves over L2 normalization, but a substantial gap remains compared to isotropic Gaussian embeddings. Weight orthogonalization improves a similar number of games as isotropic Gaussian, but yields lower average improvement; moreover, it requires computing the Gram matrix of all weights, which is computationally expensive. VICReg, which promotes isotropy without enforcing Gaussianity, improves a similar number of games but does not match the average improvement of isotropic Gaussian embeddings.

PQN					PPO
Game	$\lambda$	Start	Mid	End	Game	$\lambda$	Start	Mid	End
	0.0	5.02	1.26	1.29		0.0	20.96	4.01	3.45
Amidar	0.2	5.02	10.63	10.45	Amidar	0.2	20.96	11.93	14.08
	1.0	5.02	15.66	17.93		1.0	20.96	15.19	25.75
	0.0	4.32	1.46	1.39		0.0	19.60	2.85	3.24
Qbert	0.2	4.32	8.00	7.52	Qbert	0.2	19.60	12.78	15.20

Appendix

Table 11: Entropy of eigenvalues. Comparison of eigenvalue entropy at the start, middle, and end of training for PQN (left) and PPO (right) across different games and

\lambda

values. Amidar and Qbert are selected as representative cases where SIGReg improves performance in both PQN and PPO. Freeway, Pitfall, Atlantis, and Venture are cases where adding the SIGReg loss degrades performance relative to the baseline. Comparing the same games and baseline settings, PPO consistently exhibits higher final entropy, indicating less severe representation collapse. For a fixed game, increasing

\lambda

leads to higher entropy, as expected, since the SIGReg loss promotes equalization of eigenvalues.

PQN			PPO
Game	$\lambda$	Avg. Condition Number Ratio	Game	$\lambda$	Avg. Condition Number Ratio
Amidar	0.2	$8.8\times 10^{3}$	Amidar	0.2	$8.3\times 10^{1}$
	1.0	$2.1\times 10^{4}$		1.0	$1.8\times 10^{2}$
Qbert	0.2	$1.2\times 10^{1}$	Qbert	0.2	$1.3\times 10^{2}$
	1.0	$6.7\times 10^{3}$		1.0	$4.3\times 10^{2}$
Freeway	0.2	$8.5\times 10^{5}$	Atlantis	0.2	$3.9$

Appendix

Table 12: Condition number ratio averaged over training. We report the time-averaged ratio of condition numbers between the baseline and SIGReg embeddings,

\frac{\kappa_{\text{baseline}}}{\kappa_{\text{SIGReg}}}

. According to our theoretical analysis, a lower condition number is desirable, as it indicates more uniform convergence rates across directions and more controlled drift term. As shown,

\lambda=1

yields a substantially lower condition number than the baseline, consistent with the effect of SIGReg in spreading eigenvalues and promoting isotropy.

Stable Deep Reinforcement Learning via Isotropic Gaussian Representations

Abstract

Figures & tables

Appendix

RL Stability

Weak-SIGReg: Covariance Regularization for Stable Deep Learning

GIFT: Global stabilisation via Intrinsic Fine Tuning

Local Guidance, Global Impact: Gaussian-Reshaped Trust Region Unlocks Behavior Transitions

Abstract

RL Stability

Explore similar work

Weak-SIGReg: Covariance Regularization for Stable Deep Learning

GIFT: Global stabilisation via Intrinsic Fine Tuning

Local Guidance, Global Impact: Gaussian-Reshaped Trust Region Unlocks Behavior Transitions