Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring

Zhenxin Li; Nadine Chang; Xinglong Sun; Jingde Chen; Wenhao Yao; Zi Wang; Maying Shen; Yu-Gang Jiang

cs.ROSep 7, 2026

Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring

Authors: Zhenxin Li, Nadine Chang, Xinglong Sun, Jingde Chen, Wenhao Yao, Zi Wang, Maying Shen, Yu-Gang Jiang,

Organizations: Fudan University · NVIDIA

Abstract

Behavior policies are often formulated as continuous generative models, whose iterative denoising processes are expressive but difficult to interpret and prone to producing implausible actions. We propose the Large Discrete Policy (LDiP), a fully discrete behavior modeling framework that selects actions from a large vocabulary of physically plausible candidates. Rather than perturbing actions, LDiP improves expressivity through stochastic iterative scoring: it progressively re-scores and prunes candidates with score-space stochasticity, enabling fine-grained ranking and exploration among plausible actions while preserving an explicit decision process. Across end-to-end planning, closed-loop driving, robotic manipulation, and vision-language-action settings, LDiP consistently outperforms strong discrete and continuous baselines in autonomous driving, and exceeds or matches continuous generative policies in robotic manipulation. These results show that discrete policies, when equipped with effective scoring mechanisms, offer an expressive, plausible, and interpretable alternative for behavior modeling. Project website: https://zhenxinli.net/LargeDiscretePolicy/.

Figures & tables

Figure 1: Paradigms for Behavior Modeling Policies. (a) Iterative denoising policies generate actions through stochastic iterative refinement, offering strong expressivity but remaining an implicit and less interpretable generation process. (b) One-shot discrete policies explicitly select from a fixed action vocabulary, but are limited by single-pass scoring. (c) Our approach combines the strengths of both paradigms by performing stochastic iterative scoring over a discrete action vocabulary, enabling expressive, plausible, and interpretable behavior generation.

Figure 2: The Model Architecture of LDiP. LDiP first tokenizes a fixed action vocabulary into action tokens, which attend to visual observations and optional language instructions in a Transformer decoder. The model then scores all candidates with an MLP, stochastically prunes the vocabulary, and iteratively re-scores the remaining candidates until a single action is selected.

Figure 3: Vision-Language-Action (VLA) Architecture Variants.

Backbone	Method	Behavior Modeling	NC $\uparrow$	DAC $\uparrow$	DDC $\uparrow$	TLC $\uparrow$	EP $\uparrow$	TTC $\uparrow$	LK $\uparrow$	HC $\uparrow$	EC $\uparrow$	EPDMS $\uparrow$
-	Human Agent	-	100	100	99.8	100	87.4	100	100	98.1	90.1	90.3
-	Ego Status MLP	Regression	93.1	77.9	92.7	99.6	86.0	91.5	89.4	98.3	85.4	64.0
ResNet34	Transfuser ( Chitta et al., 2022 )	Regression	96.9	89.9	97.8	99.7	87.1	95.4	92.7	98.3	87.2	76.7
	HydraMDP++ ( Li et al., 2025a )	Discrete	97.2	97.5	99.4	99.6	83.1	96.5	94.4	98.2	70.9	81.4
	DriveSuprim ( Yao et al., 2025 )	Discrete	97.5	96.5	99.4	99.6	88.4	96.6	95.5	98.3	77.0	83.1
	ZTRS ( Li et al., 2025f )	Discrete	97.9	98.0	99.7	99.8	80.1	96.9	94.9	98.1	79.0	83.2

Table 1: Performance on the Navtest v2 Benchmark, End-to-end Planning Models.

Method	VLM	Behavior Modeling	NC $\uparrow$	DAC $\uparrow$	TTC $\uparrow$	C $\uparrow$	EP $\uparrow$	PDMS $\uparrow$
Human	–	–	100	100	100	99.9	87.5	94.8
AutoVLA Zhou et al. (2025)	Qwen2.5-VL-3B Bai et al. (2025b)	Token Generation	98.4	95.6	98.0	99.9	81.9	89.1
ReCogDrive Li et al. (2025d)	InternVL3-8B Zhu et al. (2025)	Diffusion	98.2	97.8	95.2	99.8	83.5	89.6
DriveVLA-W0 Li et al. (2025c)	Emu3-8B Wang et al. (2024)	Discrete	98.7	99.1	95.3	99.3	83.3	90.2
AdaThinkDrive Luo et al. (2025)	InternVL3-8B Zhu et al. (2025)	Token Generation	98.4	97.8	95.2	100	84.4	90.3
SpanVLA Zhou et al. (2026)	Qwen2.5-VL-3B Bai et al. (2025b)	Flow Matching	99.1	97.1	95.2	100	86.3	90.3

Table 2: Performance on the Navtest v1 Benchmark, Vision-Language-Action Models.

Method	Easy		Medium		Hard		Extreme		Overall
Method	RC $\uparrow$	HD-Score $\uparrow$	RC $\uparrow$	HD-Score $\uparrow$	RC $\uparrow$	HD-Score $\uparrow$	RC $\uparrow$	HD-Score $\uparrow$	RC $\uparrow$	HD-Score $\uparrow$
VAD* ( Jiang et al., 2023 )	38.7	24.3	27.0	9.9	25.5	10.4	23.0	8.2	27.9	12.3
UniAD* ( Hu et al., 2023 )	58.6	48.7	41.2	29.5	40.4	27.3	26.0	14.3	40.6	28.9
LTF ( Chitta et al., 2022 )	68.4	52.8	40.7	24.6	36.9	19.8	25.5	8.1	41.4	24.8
Diffusion Policy ( Chi et al., 2025 )	72.4	60.8	30.5	15.2	27.4	13.4	28.0	14.9	36.9	23.1
ZTRS ( Li et al., 2025f )	74.8	66.9	46.1	37.9	31.6	21.1	20.0	9.8	42.0	32.9

Table 3: Zero-shot Performance on the HUGSIM Benchmark. UniAD* and VAD* are trained on nuScenes Caesar et al. (2020) , which partially overlap with the evaluation scenarios. Other methods are evaluated with zero-shot transfer.

\dagger

The controller is tuned for closed-loop driving.

		IBC Florence et al. (2021)	LSTM-GMM Mandlekar et al. (2022)	Diffusion Policy Chi et al. (2025)	LDiP (Ours)
Task	DoF	Continuous			Discrete
Lift	6+1	0.73	0.96	1.00	0.99
Can	6+1	0.01	0.88	0.98	0.94
ToolHang	6+1	0.00	0.49	0.47	0.52
PushT	2	0.64	0.54	0.66	0.76

Table 4: Performance on Visual-based Robotic Manipulation . Following Diffusion Policy Chi et al. (2025) , we report averaged success rates over the last 10 checkpoints. We compare against the Transformer-based Diffusion Policy, as both methods adopt a Transformer Decoder for behavior modeling.

Model Variants	NC $\uparrow$	DAC $\uparrow$	DDC $\uparrow$	TLC $\uparrow$	EP $\uparrow$	TTC $\uparrow$	LK $\uparrow$	HC $\uparrow$	EC $\uparrow$	EPDMS $\uparrow$
Hydra-MDP- $\mathcal{V}_{16384}$ Li et al. (2024)	98.5	98.7	98.9	99.9	88.5	98.2	97.0	98.3	80.5	86.2
+ Iterative Scoring	97.9	98.3	99.5	99.8	90.6	97.3	98.1	98.1	77.1	86.7
+ Improved Tokenizer & Cross-Attention Decoder	98.4	98.7	99.7	99.8	89.6	97.9	97.4	98.3	78.6	87.1
+ Stochastic Scoring	98.3	98.9	99.7	99.8	91.0	97.9	97.5	98.1	79.1	87.8
Hydra-MDP- $\mathcal{V}_{16384}$ Li et al. (2024)	98.5	98.7	98.9	99.9	88.5	98.2	97.0	98.3	80.5	86.2
+ Noise Injection on Top-8 Actions Yao et al. (2026)	98.6	98.7	99.7	99.9	88.7	98.2	97.0	98.3	80.2	87.2

Table 5: Ablation Study on Key Components.

Table 9

Figure 4: Visualizations on Navtest. The figure shows the selected trajectory , the human trajectory , and the remaining vocabulary at the intermediate and final stages.

Figure 5: Ablation Study on the Width and Depth of LDiP.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Additional Visualizations on Navtest. The figure shows the selected trajectory , the human trajectory , and the remaining vocabulary at the intermediate and final stages.

Figure 7: Additional Visualizations on Robotic Manipulation. The figure shows a full episode of the ToolHang task. We show the key frames with top-8 action chunks from the last scoring stage and the selected action chunk.

Method	Backbone	Params.	Vocab. Size	Iteration Steps	FPS	EPDMS
Diffusion Policy* [ 9 ]	V2-99	115.0M	-	100	4.1	79.1
Hydra-MDP- $\mathcal{V}_{16384}$ [ 31 ]	V2-99	83.0M	16384	1	13.5	86.2
GTRS-Dense [ 34 ]	V2-99	83.0M	8192	1	18.5	84.0
ZTRS [ 33 ]	V2-99	83.4M	8192	1	18.4	85.3
LDiP	V2-99	82.8M	16384	3	12.1	87.8

Appendix

Table 8: Comparisons on Efficiency. We compare the model complexity and runtime efficiency of different policies. The FPS metric is measured on one single NVIDIA A100 GPU with the PyTorch FP32 backend. * The Diffusion Policy utilizes extra networks to encode the Bird’s-Eye-View condition tokens following LTF [ 10 ] , while the DDPM Scheduler [ 17 ] is used for iterative denoising.

Policy Learning

Policy learning develops decision rules that choose actions or interventions to maximize outcomes from data, feedback, or interaction.

See all 488 papers

When Do Discrete Actions Make Good Continuous Controllers?
Oct 14, 2026Zeji Yi, Soham Dasgupta, Xinyang Zhang +5Robot Policy Learning VLMs for Robotics
Carnegie Mellon University · Amazon FAR (Frontier AI & Robotics)
Discrete-WAM: Unified Discrete Vision-Action Token Editing for World-Policy Learning
Jun 4, 2026Ziyang Yao, Haochen Liu, Yuncheng Jiang +10Hierarchical Planning World Model Learning
Xiaomi EV
SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Driving
Mar 31, 2026Wenchao Sun, Xuewu Lin, Keyu Chen +4Autonomous Driving Planning End-to-End Autonomous Driving
Tsinghua University, Beijing, China · Horizon Continental Technology, Beijing, China · Horizon, Beijing, China

Oct 14, 2026J/K move · Enter open · S save

When Do Discrete Actions Make Good Continuous Controllers?

Zeji Yi, Soham Dasgupta, Xinyang Zhang, Koushil Sreenath+4

Carnegie Mellon University · Amazon FAR (Frontier AI & Robotics)

Robot Policy Learning VLMs for Robotics Discrete Flow Matching

Open paper AskPDF arXiv

Though modern AI models like LLMs predict discrete tokens, today's robotic foundation models represent actions as continuous vectors. In this paper, we challenge this common consensus by asking if continuous action parametrizations are superior to their discrete counterparts. We conduct an extensive comparison of single-step regression and multi-step flow variants of both continuous and discrete policies in both from-scratch single-task and multi-task VLA training, in both simulation and on real hardware. In all regimes, we find that discrete action prediction, even the most naive, uniform-bin quantization, can match or even outperform continuous-action policies in task success rate. Moreover, we find that the challenge to discrete action policies is not action precision, but rather factorization error introduced by product-distribution predictions across time and action dimensions. We show that, when paired with the correct interventions (iterative computation through discrete flows, or distribution sharpening through classifier-free guidance), discrete flow policies can turn this weakness into a strength, enabling more responsive action changes during a predicted action chunk. Lastly, we show that attempts to explicitly inject geometric information into discrete policies, e.g. through corruption processes that emulate continuous flow matching, do not yield consistent improvements. Taken together, our findings question existing assumptions that robotic action prediction requires distinct design decisions compared to other AI domains, and motivate the possibility of discrete action prediction as a compelling alternative to today's continuous action parametrizations. Project page: https://iscoyizj.github.io/discrete-action-website/

Abstract

Policy Learning

Explore similar work

When Do Discrete Actions Make Good Continuous Controllers?

Discrete-WAM: Unified Discrete Vision-Action Token Editing for World-Policy Learning

SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Driving