Goal-conditioned planning with image observations and goals requires predictions that distinguish visual states and action consequences. Spiking neural networks offer sparse computation, but single-step spikes can obscure motion cues and merge actions with different physical outcomes. We propose the Spiking Latent World Model (SpikWM), an end-to-end joint-embedding predictive architecture (JEPA) with one simulation step per image and one neural update per latent position. Spiking Regional Encoding retains spatial cues through regional pooling. Bi-Spike Action Encoding combines positive and negative binary responses, while Causal Membrane Prediction carries subthreshold evidence across the latent history with learned group-wise time constants. The encoder and predictor are trained jointly on reward-free image--action trajectories using next-embedding prediction and SIGReg. With a common planner, SpikWM achieves mean success rates of 88.67%, 97.33%, 99.33%, and 86.67% on Push-T, Two-Room, Reacher, and OGBench-Cube, respectively, outperforming two spiking baselines. Push-T ablations support each module, and controlled replays examine responses to appearance and physical state changes. Under a matched planning budget, SpikWM reduces estimated arithmetic energy by 75.6% relative to LeWM. Code will be released at https://github.com/ShawnYE618/SpikWM.
Figures & tables
Figure 1: Lost motion and action distinctions on Push-T. LeWM (ANN) and Spiking-LeWM (SNN). (a) Motion direction readout errors on 305 shared moving windows. (b) Replayed endpoints and predicted goal costs. Dashed outlines mark the goal. (c) Distinct predictor conditions for one pool of action blocks.
Figure 2: SpikWM architecture and planning. Spiking Regional Encoding (SRE) combines spatial spike processing, regional pooling and an output projection. Bi-Spike Action Encoding (BAE) uses learned group thresholds and combines two binary streams through separate maps. Causal Membrane Prediction (CMP) includes conditional causal blocks, final normalization and an output projection. Within each block, LayerNorm precedes action-conditioned scaling and shifting. Shared SRE branches learn from next-latent prediction and SIGReg. The cross-entropy method replans using terminal goal distance.
Success (%) ↑
Method
Spiking
Updates
Features
Params (M)
Push-T
Two-Room
Reacher
OGBench-Cube
PLDM † [ 27 ]
×
1
Learned
18.03
71.8±6.1
93.0±2.0
69.5±5.8
51.2±1.6
DINO-WM † [ 43 ]
×
1
DINOv2
41.75
65.2±6.6
99.6±0.9
67.2±1.8
74.4±2.6
Sub-JEPA [ 42 ]
×
1
Learned
18.03
89.00±5.33
95.00±2.76
84.00±4.00
76.33±5.99
Fast-LeWM [ 15 ]
×
1
Learned
17.91
91.7±4.1
95.4±3.2
84.0±5.9
71.7±8.4
Σ -LeWM [ 46 ]
×
1
Learned
18.03
94.5±3.0
92.4±4.4
86.0±4.8
82.6±5.0
Table 1: Goal-conditioned planning success. SNN results are mean ± sample SD across three evaluation seeds. † denotes Anchor-WM’s five-seed results [ 32 ] . ‡ marks our Spiking-WM adaptation. Updates count ANN passes or SNN steps per latent position. Parameters cover the deployed Push-T modules. Bold marks the highest mean within each group.
Figure 3: Open-loop forecasts on four tasks. Frame pairs show steps 15/20 (Push-T), 30/35 (Two-Room), 20/35 (OGBench-Cube), and 25/30 (Reacher). We render state readouts of the latent forecasts under shared actions [ 20 ] .
Figure 4: Component ablations and prediction diversity. (a) Push-T success. (b) Distinct Cube forecasts under 300 shared action blocks. Bars show mean ± sample SD across three evaluation seeds (a) and 12 starts (b).
Model
Success (%) ↑
FLOPs (G)
SOPs (G)
Energy (J) ↓
Time (s) ↓
LeWM
90.40
2747.75
0.00
6.320
1.53
Spiking-LeWM
53.33
1771.70
65.82
4.134
2.21
Spiking-WM
37.33
2608.01
53.97
6.047
3.82
SpikWM
88.67
600.15
178.38
1.541
1.97
Table 2: Push-T success, planning time & arithmetic energy. Success means follow Tab. 1 . Times are means on one RTX 4090 D. Counts use one common initial-plan request.
Figure 5: Perturbation responses in Two-Room. At step 50, we recolor or displace the agent. We divide next-embedding MSE by each model’s pre-intervention mean. Bands show 95% pointwise bootstrap intervals over 16 paired starts.
Linear
MLP
Model
MSE ↓
r↑
MSE ↓
r↑
Gripper position
LeWM
0.855±1.019
0.633
0.742±0.805
0.515
Spiking-LeWM
1.007±1.625
0.604
0.557±0.720
0.666
SpikWM
0.309±0.395
0.847
0.394±0.544
0.789
Cube position
Table 3: Physical probing on OGBench-Cube. Normalized MSE (SD in subscripts) and mean Pearson r .