JEPA as a Transferable Optimizer: Learning and Understanding Black-Box Optimization from Reward-Agnostic Pretraining
Authors: Mayukh Debnath, Manan Roy Choudhury
Organizations: Robotics and Autonomous Systems (AI), Arizona State University, Tempe, USA · Department of Artificial Intelligence, Arizona State University, Tempe, USA
Black-box optimization evaluates candidates through simulations, rollouts, or experiments, often without gradients. Learned optimizers are usually judged by whether they outperform existing methods. We ask a complementary question: what information does a learned optimizer use from its search history? We introduce JEPA-PS (JEPA Policy Search), based on a Joint-Embedding Predictive Architecture. It is pretrained on random rollouts using parameter vectors and scalar returns. Its latent-prediction objective predicts the next point in an optimization trajectory rather than its reward or gradient, allowing us to examine what was tried and how well it performed. At test time, JEPA-PS reads a short evaluation history and proposes a candidate in one forward pass, without task descriptions, reward shaping, or test-time gradients. We evaluate it on 15 control environments against 12 optimizers, including evolutionary methods, Bayesian optimization, CMA-ES, and an LLM optimizer. JEPA-PS is competitive with tuned classical baselines and outperforms the LLM optimizer by more than 2x on Swimmer at 71x lower per-step cost. It also transfers zero-shot to environments excluded from pretraining. Null-model comparisons and decoder-training ablations indicate that its latent representation carries information beyond simple history statistics; every trained decoder variant performs worse than the default frozen representation. Our main finding is that the history JEPA-PS uses depends on the task. On smooth landscapes it uses the full trajectory, whereas on rugged landscapes the best-seen point carries nearly all the signal. This memory requirement has a weak positive correlation with fitness-distance correlation (FDC; r=0.112, n=7), while an independent reward-ordering sensitivity probe does not corroborate the relationship (r=-0.024, n=7). Code: https://github.com/mdebnath22/jepa-props.
Figures & tables
Figure 1: Overview of the JEPA optimizer.
Method
Mean ± std
CEM Baseline
500.00±0.00
JEPA (ours)
365.79±182.71
Swarm-JEPA (ours)
223.73±114.07
ProPS-Redundant (Qwen2.5:7B)
170.53±74.25
ProPS (Qwen2.5:7B)
103.01±43.47
Random Search
60.52±14.49
Table 1: Independent five-seed CartPole comparison at a matched budget. Scores report mean best reward ± standard deviation.
Variant
Split
Mean ± std
Parse-failure rate
nominal
TRAIN
499.44±1.68
0.387
gravity_x2
TRAIN
500.00±0.00
0.569
force_x0.5
TRAIN
480.92±51.04
0.045
gravity_x1.5+length_x1.3
HELD-OUT
474.06±57.59
0.000
masspole_x2
HELD-OUT
487.12±38.24
0.000
masscart_x0.5+force_x1.5
HELD-OUT
500.00±0.00
0.000
Table 2: Restart-wrapped LLM optimization on training and held-out CartPole physics variants: ten seeds, budget 400, and restart after 40 non-improving evaluations. Scores report mean best reward ± standard deviation. Parse-failure rates are fractions of calls; failed calls use random-noise fallback.
Intervention
Mean best
Δ
Control (full history)
410.9
—
Best-only (single point)
319.2
−22.3%
Remove best-seen point †
285.1
−30.6%
Permute parameter dims.
329.7
−19.8%
Shuffle reward values
341.1
−17.0%
Reverse temporal order
354.7
−13.6%
Table 3: Inference-time history interventions. Panel A uses CartPole with five seeds and 200 iterations; † denotes three seeds. Panel B uses the reduced-budget protocol in Appendix J . The panels use different protocols, so their reward-permutation effects are not directly interchangeable.
Environment
FDC
HD
OrdSens
CartPole
0.661
0.218
0.370
InvPendulum
0.548
0.050
0.000
Swimmer
0.482
0.044
0.068
Hopper
0.439
−0.006
0.000
Reacher
0.401
0.074
0.003
Walker2d
0.326
0.000
0.000
Table 4: Fitness–distance correlation ( FDC ), history dependence ( HD ), and reward-ordering sensitivity ( OrdSens ). The reported Pearson correlation between HD and FDC is 0.738 across seven environments, with a bootstrap 95% confidence interval excluding zero. The original OrdSens correlation uses five environments, excluding InvertedPendulum and Hopper. The all-seven OrdSens calculation includes their displayed zero entries. The all-seven OrdSens and Spearman correlations are computed from the rounded entries shown.
Figure 2: Landscape structure and measured history benefits. (A) History dependence versus fitness–distance correlation across seven environments. (B) Reward-ordering sensitivity versus FDC. The tested monotone reward transformations have little effect, while reward permutation has an environment-dependent effect. (C) Memory-condition performance normalized to the full-history baseline.
Appendix figures & tables32 assets
Supplementary material from the paper’s appendix.
Appendix
Environment
ARS
CEM
CMA-ES
PES
Pong
3.00±0.00
2.28±0.90
2.28±0.89
3.00±0.00
InvertedDoublePendulum
1949.3±1750.8
189.4±64.4
250.1±92.4
217.8±18.3
MountainCar (discrete)
−168.0±34.1
−199.5±1.4
−200.0±0.0
−176.3±22.0
Navigation
3414.0±68.8
2746.6±1028.5
2522.8±1306.3
3111.3±310.5
FrozenLake
0.74±0.13
0.56±0.20
0.68±0.13
0.74±0.09
Nim †
0.44±0.20
0.28±0.47
0.28±0.47
0.44±0.20
Appendix
Table 5: Classical/policy-search baselines on the eight extended environments (mean best reward ± std, n=10 seeds, timestep-matched budget; see text for T and population size per environment). CliffWalking’s zero-variance result is a diagnosed landscape property, not an error (see text).
Corpus size N
Collect
Per-epoch
50-epoch total
Per-rollout
1,000
3.5 s
1.2 s
1.1 min
3.5 ms
5,000
18.6 s
4.5 s
4.1 min
3.7 ms
10,000
36.6 s
8.4 s
7.6 min
3.7 ms
20,000
74.6 s
17.4 s
15.7 min
3.7 ms
Appendix
Table 6: Pretraining cost by corpus size, CartPole, single CPU core.
Method
CartPole
Swimmer
Hopper
InvPend.
MtnCar(C)
Random Search
55.5±16.9
120.5±38.2
19.2±3.6
88.5±27.6
24.9±19.1
CEM
360.9±95.3
250.8±59.0
620.2±201.7
436.4
−5.0
PSO
322.5±96.5
263.4±40.0
463.3±240.8
296.8
2.3
IPOP-CMA-ES
500.0±0.0
352.2±3.3
659.9±361.4
890.0±220.0
−0.0±0.0
ARS
500.0±0.0
352.1±2.5
789.1±203.2
50.9±8.5
−0.0±0.0
LLM optimizer (OPRO)
186.0±218.0
140.3±93.3
1026.7±0.8
356.0±455.4
−0.1±0.0
Appendix
Table 7: Mean best reward (mean ± std, 5 seeds) at T=400 evaluations. Bold : best in column among rows shown. The full grid (17 methods, BallLanding) is Table 8 ; stronger modern baselines are in Table 13 (e.g. TuRBO reaches 1047.4 on Hopper); the eight extended environments are in Table 5 .
Method
CartPole
Swimmer
Hopper
InvPend.
MtnCar(C)
Random Search
55.5±16.9
120.5±38.2
19.2±3.6
88.5±27.6
24.9±19.1
Tabu Search
145.2±98.4
220.1±52.3
246.0±122.2
272.3±364.8
17.8±35.6
(μ+λ) -ES
180.5±112.3
215.7±48.9
141.7±53.3
406.4±327.3
−0.0±0.0
OpenAI-ES
36.7±28.1
198.5±55.3
141.5±112.0
55.7±42.0
−20.1±39.9
BayesOpt
274.0±154.7
182.3±72.1
57.0±42.3
93.6
−13.6
PSO
322.5±96.5
263.4±40.0
463.3±240.8
296.8
2.3
Appendix
Table 8: Full baseline grid: mean best reward (mean ± std, 5 seeds), T=400 total evaluations. Bold : best per environment. Underlined : best JEPA-family result. “—”: not run.
Method
Time/step
Relative
CEM update (numpy)
0.024 ms
109-fold faster
CMA-ES update (numpy)
0.053 ms
49-fold faster
JEPA propose (CPU)
2.607 ms
—
LLM 7B (Qwen2.5)
∼200 ms
77-fold slower
LLM 14B
∼550 ms
211-fold slower
Appendix
Table 9: Per-step wall-clock time (1,000 repetitions, CPU, excluding the environment rollout).
Figure 3: Complete performance comparison across all optimizers (mean best reward ± std, 5 seeds); JEPA-family methods are outlined. Latent-CEM (Appendix P ) adds a further gain over raw-space CEM on Swimmer.
Configuration
S succ.
U succ.
B succ.
S-vs-U
B-vs-S
Baseline (no wrapper)
0.38
0.16
0.44
PASS ( +0.22 , 3/3)
FAIL ( +0.06 , 0/3)
Longer warm-up (32)
0.90
0.33
0.89
PASS ( +0.56 , 3/3)
FAIL ( +0.01 , 0/3)
Restart on stagnation (40)
0.94
0.72
0.98
PASS ( +0.22 , 2/3)
FAIL ( +0.03 , 0/3)
Multi-start ( K=8 )
0.99
0.80
0.98
PASS ( +0.19 , 3/3)
FAIL ( −0.01 , 0/3)
Combined (restart+anneal+warm-up)
0.99
0.83
1.00
FAIL (ceiling eff., +0.16 , 1/3)
FAIL ( +0.01 , 0/3)
Appendix
Table 10: Validation-stage pooled success rate (reward ≥490 ) across three variants, n=30 seeds each. S: Part 1 pretrained checkpoint. U: untrained-encoder/predictor control, identical decoder. B: broader-dynamics-curriculum checkpoint (Part 7). PASS/FAIL is the pre-registered attribution rule (pooled gap ≥0.10 and paired significance on ≥2/3 variants).
Variant
S baseline
S restart40
U baseline
U restart40
Gravity × 0.5
420.9±153.8 (0.74)
500.0±0.0 (1.00)
263.7±166.5 (0.19)
492.0±23.2 (0.87)
Length × 2
316.5±160.2 (0.27)
483.5±38.5 (0.81)
155.9±100.5 (0.03)
420.5±94.4 (0.50)
EXTREME-A
178.9±141.0 (0.07)
414.7±114.6 (0.56)
101.6±86.0 (0.02)
330.6±130.7 (0.28)
EXTREME-B
430.5±155.7 (0.81)
500.0±0.0 (1.00)
308.4±176.4 (0.30)
500.0±0.0 (1.00)
Pooled
336.7 (0.47)
474.5 (0.84)
207.4 (0.14)
435.8 (0.66)
Appendix
Table 11: Test-set results, n=100 seeds per variant. Baseline and restart-on-stagnation (40), for both the pretrained checkpoint (S) and the untrained control (U). “succ.” is the fraction of seeds reaching reward ≥490 . The from-scratch ARS/CEM reference (Part 4) reaches 1.00 success on every seed on gravity × 0.5 and length × 2; it was not separately re-run on the two extreme variants.
Variant
K
JEPA-S
JEPA-U
CMA-ES-cold
CMA-ES-warm
gravity ×1.5 +length ×1.3
10
190.3±159.7
139.9±135.7
187.0±161.4
481.5±50.4
25
212.6±161.7
140.5±135.3
313.3±138.0
500.0±0.0
50
236.3±173.3
140.6±135.2
466.4±71.3
500.0±0.0
100
245.2±173.9
141.1±134.8
500.0±0.0
500.0±0.0
250
260.1±177.0
142.7±133.8
500.0±0.0
500.0±0.0
masspole ×2
10
227.5±176.7
171.8±151.4
288.0±178.3
500.0±0.0
Appendix
Table 12: JEPA-S vs. CMA-ES-cold and CMA-ES-warm on four held-out CartPole physics variants, never seen during pretraining ( n=20 seeds). CMA-ES-warm’s mean is seeded from a one-time 200-evaluation solve on the nominal variant, amortized across all four tasks. JEPA-S beats both controls in 0 of 20 cells.
Method
CartPole
Swimmer
Hopper
MtnCar(C)
Swarm-JEPA ( K=8 , n=10 )
416.1
293.1
1039.7
−5.1
Swarm-JEPA (single agent, n=10 )
410.9±178.2
90.6±54.1
136.0±67.4
102.3
IPOP-CMA-ES
500.0±0.0
352.2±3.3
659.9±361.4
−0.0±0.0
BIPOP-CMA-ES
500.0±0.0
353.2±1.7
999.4±31.8
33.0±46.8
TuRBO
500.0±0.0
358.0±3.3
1047.4±29.5
33.0±46.7
TPE (Optuna)
500.0±0.0
293.5±120.8
1035.5±13.4
−0.1±0.1
Appendix
Table 13: Strong-baseline grid ( n=10 seeds unless noted, T=400 , runner rollout). All modern methods outperform Swarm-JEPA on Swimmer.
Figure 4: Per-step cost and model footprint. (A) Wall-clock time per proposal on a log scale. A JEPA forward pass is 77 times faster than a 7B-parameter LLM call, and 49–109 times slower than a classical covariance or elite-mean update. (B) Model size against multi-billion-parameter language models.
K
FDC
Classification
0
0.389
smooth (additive)
2
0.236
mildly rugged
4
0.113
intermediate
8
0.036
rugged
12
0.028
maximally rugged
Appendix
Table 14: FDC on NK landscapes across ruggedness levels K (network size 16, 5 instances per K ).
Env
FDC
Full
Best-only
HD
CartPole
0.661
410.87
321.43
0.218
Swimmer
0.482
263.43
251.73
0.044
MountainCar(C)
0.153
−28.41
−28.51
0.004
Appendix
Table 15: History dependence by memory condition (mean best reward, 5 seeds) and correlations with landscape descriptors (bootstrap 95% CI, 1,000 resamples).
N
FDC mean
FDC std
95% CI
CI excl. 0?
50
0.504
0.135
[0.242,0.659]
yes
100
0.431
0.093
[0.266,0.562]
yes
200
0.381
0.084
[0.274,0.552]
yes
300
0.363
0.082
[0.260,0.533]
yes
500
0.334
0.093
[0.245,0.519]
yes
1000
0.299
0.084
[0.204,0.461]
yes
Appendix
Table 16: FDC estimator robustness on CartPole across sample sizes (10 repetitions per N , 95% bootstrap CI), and the partial-correlation summary (right).
Environment
Domain
FDC↑
ACL ↑
Smooth ↓
Basin ↑
HD
OrdSens
NR
IC
DM
CartPole
balance
0.661
8
9.84
0.6
0.218
0.370
0.580
2.257
1.327
InvPendulum
balance
0.548
4
14.40
0.0
0.050
0.000
0.692
1.517
0.864
Swimmer
locomotion
0.482
3
12.84
0.37
0.044
0.068
0.424
2.770
1.220
Hopper
locomotion
0.439
5
32.57
0.0
−0.006
0.000
0.982
0.182
2.695
CliffWalking
discrete
0.424
2
1.44
0.0
—
—
—
—
—
Reacher
control
0.401
5
13.76
0.0
0.074
0.003
0.220
2.941
1.808
Appendix
Table 17: Landscape descriptors for the 15 originally-profiled environments, sorted by FDC . † : basin values of 1.0 are threshold artifacts. OrdSens is — for environments without a JEPA checkpoint. NR, IC, and DM are computed from random walks. The eight extended environments of Appendix B have not yet been profiled on these descriptors. Figure 5 visualizes this ranking.
Figure 5: Landscape ruggedness across the 15 originally-profiled environments (green: smoother; red: more rugged). CartPole is the most exploitable landscape; Maze, Navigation, and MountainCar the least.
Condition
Mean best
Δ
Full history
410.87
—
Remove best
285.10
−30.6%
Remove early half
300.90
−26.8%
Best-only
321.43
−21.8%
Recent-8
309.23
−24.7%
Random-8
320.94
−21.9%
Appendix
Table 18: History, order, and reward-transformation ablations on CartPole (and Swimmer/MountainCar where noted). The full trajectory outperforms every truncated or corrupted variant; the optimizer is sensitive to coarse temporal direction and reward rank but not to fine-grained order or reward magnitude.
Table 20: Isolating the two reward channels at test time (CartPole, n=5 seeds, T=400 ; top), and the target-return percentile sweep at a small-scale (2k/50ep) protocol and the main protocol (bottom). Corrupting r∗ has no measurable effect at any percentile from 50 to 99, at either budget; corrupting the reward values inside the history does.
r∗ condition
CartPole ( FDC=0.661 )
Swimmer ( FDC=0.482 )
Normal (80th/99th pct.)
410.9
272.1
Zero / worst / 20th pct.
408.5 – 412.7
268.4 – 276.8
Δ vs. normal
≈0%
≈0%
Appendix
Table 21: Effect of r∗ conditioning ( n=5 seeds, T=400 ). Disabling r∗ has no measurable effect on final reward on either landscape (top); yet varying r∗ measurably shifts the proposal distribution, as quantified by maximum mean discrepancy (MMD, bottom).
Decoder variant
Reward
Δ vs. frozen
Frozen random (default)
410.9
—
Reconstruction fine-tuned
293.7
−28.5%
End-to-end joint retraining
269.0
−34.5%
Appendix
Table 22: Effect of decoder training on downstream reward (CartPole, n=5 seeds), and mean gradient norm reaching each component during a training step.
Method (main protocol)
CartPole
Swimmer
Null N(μh,σh)
302.1
151.9
JEPA-PS
410.9
293.1
Appendix
Table 23: JEPA-PS vs. a null statistics-only proposer under two protocols. Top: main protocol ( n=10 seeds, T=400 ). Bottom: matched-protocol head-to-head with shared warm-starts and proposal-noise seed ( n=10 , Swimmer only), which reverses the direction of the comparison.
K
Mode
Iters/island
Total evals
Mean best
Std
1
either
400
400
410.9
178.2
3
fixed-total
133
400
421.6
156.9
3
fixed-per-island
400
1200
421.6
156.9
8
fixed-total
50
400
416.1
28.7
8
fixed-per-island
400
3200
500.0
0.0
Appendix
Table 24: Swarm-JEPA budget accounting and hyperparameter sensitivity (CartPole). Top: fixed-total vs. fixed-per-island budget accounting ( n=5 seeds). Bottom: sensitivity to migration frequency and exploration noise ( K=3 , n=3 seeds, 400 iterations).
Figure 6: Population-wrapper ablations. (A) Swarm-term ablation on CartPole and Swimmer. (B) Island-count sweep on Swimmer: the mean rises and variance falls, then widens, with diminishing returns. (C) Exploration-noise sweep, with σn=0.3 best on both tasks. (D) Candidate-selection rule and Latent-CEM comparison on CartPole.
Delay d (steps)
0
5
10
20
50
Mean best
410.9
324.1
452.9
435.2
487.8
Appendix
Table 25: Full CartPole robustness grid: evaluation-budget scaling, Gaussian reward noise, pretraining data efficiency, sparse feedback, and delayed feedback ( n=5 seeds unless noted).
Table 32
Figure 7: Policy visualization on Swimmer and Hopper. Weight matrices (left) evolve from unstructured initialization to action-specific structure; rendered frames (right) show behavior across an episode. A random Swimmer policy flails without net forward motion while an optimized policy produces coordinated undulation; a random Hopper policy collapses immediately while an optimized policy sustains hopping.
Figure 8: LLM optimizer comparison. (A) CartPole across three seeds: hints help on the hard seed and not on the easy one. (B) MountainCar: all prompt variants remain near the random floor. (C) Swimmer: the LLM optimizer trails the population wrapper. (D) History-length sweep on MountainCar.
Method
CartPole
Swimmer
CEM (raw parameter space)
360.86±95.3
215.00±42.1
Latent-CEM (JEPA latent)
393.71±45.2
304.48±38.5
Appendix
Table 30: Latent-space CEM vs. raw-space CEM (mean best reward, 3 seeds).
Figure 9: Secondary mechanistic observations. (A) Cosine similarity between JEPA proposals and classical update directions, all near zero. (B) Exploration radius contracting over a run. (C) History-length robustness. (D) Leave-one-out transfer.
Function
FDC
JEPA
JEPA std
JEPA (BBOB-pretrained)
CEM mean
CEM std
Sphere (f1)
0.861
−2.12
0.91
−1.19
−1.32
1.01
Ellipsoid (f2)
0.250
−119190
31871
−60630
−3415
2340
Rosenbrock (f8)
0.649
−524.3
112.1
−159.1
−152.2
110.2
Rastrigin (f15)
0.215
−53.0
10.7
−53.0
−16.2
5.7
Schwefel (f20)
0.109
−10.2
5.3
−1959.4
2057
821
Appendix
Table 32: JEPA optimizer vs. CEM on BBOB synthetic functions ( d=8 , n=5 seeds, 400 evaluations; higher is better as scored), with FDC measured directly on each function ( 300 random samples) and results for a checkpoint pretrained directly on BBOB rollouts (all 5 functions mixed, no control-task data).