When Do Discrete Actions Make Good Continuous Controllers?

Zeji Yi; Soham Dasgupta; Xinyang Zhang; Koushil Sreenath; Guannan Qu; Xi Chen; Max Simchowitz; Guanya Shi

cs.ROOct 14, 2026

When Do Discrete Actions Make Good Continuous Controllers?

Authors: Zeji Yi, Soham Dasgupta, Xinyang Zhang, Koushil Sreenath, Guannan Qu, Xi Chen, Max Simchowitz, Guanya Shi

Organizations: Carnegie Mellon University · Amazon FAR (Frontier AI & Robotics)

Abstract

Though modern AI models like LLMs predict discrete tokens, today's robotic foundation models represent actions as continuous vectors. In this paper, we challenge this common consensus by asking if continuous action parametrizations are superior to their discrete counterparts. We conduct an extensive comparison of single-step regression and multi-step flow variants of both continuous and discrete policies in both from-scratch single-task and multi-task VLA training, in both simulation and on real hardware. In all regimes, we find that discrete action prediction, even the most naive, uniform-bin quantization, can match or even outperform continuous-action policies in task success rate. Moreover, we find that the challenge to discrete action policies is not action precision, but rather factorization error introduced by product-distribution predictions across time and action dimensions. We show that, when paired with the correct interventions (iterative computation through discrete flows, or distribution sharpening through classifier-free guidance), discrete flow policies can turn this weakness into a strength, enabling more responsive action changes during a predicted action chunk. Lastly, we show that attempts to explicitly inject geometric information into discrete policies, e.g. through corruption processes that emulate continuous flow matching, do not yield consistent improvements. Taken together, our findings question existing assumptions that robotic action prediction requires distinct design decisions compared to other AI domains, and motivate the possibility of discrete action prediction as a compelling alternative to today's continuous action parametrizations. Project page: https://iscoyizj.github.io/discrete-action-website/

Figures & tables

Figure 2: Temporal variation and precision on held-out chunks. (a) Jerk-ratio violin plots; Continuous underestimates expert temporal variation. (b) EEF error, grouped by ground-truth motion.“Quantization” highlights the discrete heads’ static-error floor. (c) Mean RoboCasa EEF error by horizon; DF has the lowest error in the first steps of the chunk. More details are in Section E.1 .

Figure 3: State responsiveness on held-out RoboCasa demonstrations: each head’s median change in its mean predicted chunk, divided by the expert’s median chunk change, for frame pairs four steps apart (1.00 matches the expert). Other lags are in Tab. 10 .

Figure 4: Iterative refinement reduces token ambiguity. (a,b) At CFG 0, DF at NFE

1,2,5,10,20

show separated-mode prevalence (a) ambiguity decreasing, action diversity remains

\exp(\mathcal{H}(p))

(b); the orange dashed curve is DR. (c) Median central-90% width

W_{90}

of action proposal and its ground truth coverage, run from CFG 0 to 3. (d) DROID t-SNE; the star marks the GT; other sources are in Fig. 12 . ; additional diagnostics and protocols are in Sections F.1 and E.2 .

Figure 5: (a) Temporal mode switching within an action chunk as relative mode probabilities vary across the prediction horizon; (b) Action chunks predicted by CR, CF, and DF from the same starting position. (c) Coordinate-wise factorization within a timestep: independently combining coordinates from two valid modes can produce two additional, incompatible actions.

CFG $0\to 2$	AR	DR (ref.)	DF (ref.)
EEF (mm) $\downarrow$	18.4 $\to$ 24.4	16.4 $\to$ 16.3	14.1 $\to$ 18.2
Success (%) $\uparrow$	42.8 $\to$ 48.3	27.1 $\to$ 49.2	52.3 $\to$ 53.9

Table 1: EEF error and RoboCasa success

Figure 6: Discrete heads learn local bin order. Centered output embeddings of DR and DF trained on real bimanual robot data, over the 222 bins seen at least 50 times in validation. (a) Cosine similarity between bins. (b) Median cosine at each bin separation $k$ ; gray band: chance under shuffled bin labels. (c) Bins whose most similar bin is a direct neighbor. (d) First two principal components, colored by bin index; $\rho$ : Spearman correlation of PC1 with bin index. Details are in Section G.3 .

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Frames (M)	Weight
RoboCasa pretrain atomic	1.38	0.052
RoboCasa pretrain composite	21.89	0.206
RoboCasa pretrain MimicGen	116.25	0.202
RoboCasa target atomic	2.14	0.065
RoboCasa target composite-seen	5.17	0.100
RoboCasa target composite-unseen	5.99	0.108

Appendix

Table 2: Training mixture for the VLA-scale policies. Frames are counted after filtering. Weights are proportional to the square root of the frame count, with the MimicGen count capped at DROID’s.

Head	Output	Training loss	Generation
CR	real-valued chunk	squared error on the chunk	one pass
CF	real-valued chunk	velocity or clean-target MSE	$N$ integration steps from noise
DR	one token per coordinate	token cross-entropy (hard or distance-smoothed)	one pass, argmax on every token
DF	one token per coordinate	clean-token cross-entropy on corrupted chunks	$N$ updates, conditioned on the current chunk

Appendix

Table 3: Four heads: representation crossed with inference computation. Losses and decoders are described at the head-family level; per-experiment details are in Appendices B and H .

	CR, CF	DR, DF
Blocks	8 encoder + 8 decoder	12
Width / heads / MLP width	256 / 4 / 1024	256 / 4 / 1024
Action tokens	one per time step (10)	one per time step and coordinate (100)
Action input	MLP on the 10-d action	learned embedding of $B$ bins
Action output	linear to 10 values	logits over $B$ bins
Observation MLP width	256	512

Appendix

Table 4: Networks for the single-task RoboMimic policies. Both families are DiT-style transformers conditioned through adaptive layer norm (adaLN) on the observation embedding and the flow time. Parameter counts include the observation encoder and are given for single-arm tasks.

Shared by all four heads
	CR	CF	DR	DF
Backbone	PaliGemma-3B from the $\pi_{0}$ weights, fully fine-tuned
Condition tokens	64, from a 2-block resampler (width 1536)
Blocks / width / heads / MLP	8 / 1536 / 12 / 6144
Action chunk	$H=24$ steps, $M=7$ coordinates, one sequence element per coordinate
Shared within each representation

Appendix

Table 5: Action heads and training settings for the VLA-scale policies. Parameter counts cover the action head and resampler; the backbone adds 2.92B parameters.

Figure 7: EEF position error on RoboMimic training samples. Each violin shows one run’s RMS EEF position error (mm) over its last 10 evaluations; dots are individual evaluations and the vertical line marks the median. The spread reflects late-training variation of the metric. DF denotes the mixture path and DF metric the metric-induced path ( Section G.2 ); “quantile” marks equal-frequency bins, otherwise bins are uniform.

Head	Delta	Relative	Absolute
DF	$51.6\pm 2.6$	$\mathbf{53.9\pm 1.2}$	$47.0\pm 1.3$
CF	$50.2\pm 3.5$	$\mathbf{52.7\pm 3.4}$	$39.0\pm 2.1$
DR	$\mathbf{50.2\pm 2.9}$	$49.2\pm 1.4$	$46.8\pm 1.9$
CR	$37.3\pm 1.8$	$\mathbf{38.8\pm 3.3}$	$17.4\pm 3.1$

Appendix

Table 6: Action-space comparison on RoboCasa. Success rate (%) of the best of the last five checkpoints, as mean

\pm

standard deviation over five seeds. Bold marks the highest success rate in each row.

Figure 8: Delta, relative, and absolute actions. (a)–(c) One planned chunk in the robot base frame. Each arrow is the translation

\mathbf{a}^{\mathrm{tr}}_{h}

that the parameterization encodes for one command, drawn from its reference pose in Eq. D.1 . Rotations use the same reference poses. (d) Translation magnitude encoded along a 24-step chunk of an illustrative motion, on a logarithmic axis. The trajectories are illustrative, and (a)–(c) are not to scale.

Head	Joint space	EEF space
DF	$54.3\pm 1.5$	$53.9\pm 1.2$
CF	$51.9\pm 2.8$	$52.7\pm 3.4$
DR	$49.8\pm 1.9$	$49.2\pm 1.4$
CR	$38.1\pm 2.6$	$38.8\pm 3.3$

Appendix

Table 7: Relative actions in joint space and end-effector space on RoboCasa. Success rate (%), mean

\pm

standard deviation over five evaluation seeds.

Head	CFG / NFE	$\pi_{0.5}$ fine-tuned	$\pi_{0}$ frozen
CR	0 / –	$42.4\pm 1.2$	$10.2\pm 0.6$
CF	2 / 20	$58.2\pm 2.4$	$13.4\pm 0.6$
DR	2 / –	$55.8\pm 3.1$	$10.2\pm 1.0$
DF	2 / 5	$60.2\pm 1.9$	$11.2\pm 0.8$

Appendix

Table 8: RoboCasa success rate (%) with other backbones, mean

\pm

standard deviation over five seeds.

Figure 9: Per-source view of Figure 2 . Top: jerk-ratio violins for each of the five displayed sources. Bottom: mean EEF error by horizon (left axis) and chunk-error violins (right axis).

Figure 10: EEF error against ground truth as a function of the guidance weight

\lambda

. Mean XYZ end-effector L2 error (mm) over the 24-step chunk, for DR (argmax decoding) and DF (NFE 10, mean over 32 draws). Shaded bands show

\pm 1

standard error over observations.

\lambda=0

is unguided ( Section C.5 ). (a) All 8 validation sources. (b) The 7 sources other than LIBERO. (c) LIBERO only.

Head	$\sigma$	$F_{\pi}$	$T_{\pi}-F_{\pi}$	$R_{\pi}$
DF	1.217	1.245	0.233 / 0.490 / 1.071	0.19 / 0.39 / 0.86
CF	0.764	0.591	0.036 / 0.102 / 0.403	0.06 / 0.17 / 0.68
DR	0.182	0.141	0.301 / 0.529 / 0.927	2.13 / 3.75 / 6.57
CR	0.000	0.000	0.147 / 0.275 / 0.568	undefined

Appendix

Table 9: Proposal spread and response to one-step simulator disturbances.

\sigma

is mean nominal RMS spread over the full bank. The independent 32-vs-32 transport statistics use the existing 24-snapshot subsample per head and every other perturbation. Triples correspond to

\epsilon=0.1,0.3,1.0

. Relative excess is computed from the displayed rounded transport means. CR’s deterministic law has zero floor, so its relative excess is undefined. Spread and transport use different subsets.

Lag	$\\|\Delta s\\|$	Expert $\Delta$ chunk	DR (floor)	CR	DF (floor)	CF (floor)
1	0.018	0.66	0.71 (0.05)	0.35	0.45 (0.38)	0.36 (0.21)
2	0.036	1.13	1.16 (0.05)	0.62	0.77 (0.37)	0.64 (0.22)
4	0.073	1.88	1.77 (0.05)	1.09	1.32 (0.38)	1.18 (0.21)
8	0.149	3.00	2.71 (0.05)	1.93	2.34 (0.38)	2.25 (0.22)
16	0.292	4.32	4.01 (0.05)	3.13	3.55 (0.38)	3.28 (0.22)

Appendix

Table 10: Action changes on held-out expert trajectories.

\|\Delta s\|

is the state displacement and expert

\Delta

chunk is the demonstrated chunk change. Head columns report raw proposal-mean shifts; parentheses show same-observation sampling floors for the stochastic heads. All values are medians over pairs in normalized units; no floor subtraction or response-ratio normalization is applied.

RoboCasa
Source	CR	CF	DR	DF
DROID	21.60	26.60	25.09	19.67
LIBERO-plus	9.88	7.23	2.53	2.90
pretrain composite	16.44	17.34	15.97	14.84
MimicGen	16.18	16.95	15.76	12.55
target atomic	17.34	18.97	19.55	15.62

Appendix

Table 11: EEF target-prediction error (mm,

\downarrow

) on slow validation near 100k steps, from single runs.

Panel of Figure 11 — Figure 11: Ambiguity thresholds and width by horizon. (a) Percentage of chunks with at least $k$ ambiguous tokens at CFG 0, on the same observations and draws as Fig. 4 (a). The log axis stops at 0.1%. (b) Median central-90% width $W_{90}$ at each horizon step for CF (teal), DR (orange), and DF (gold). Dashed lines with open markers are CFG 0; solid lines with filled markers are CFG 2.

Figure 12: Additional t-SNE views of sampled action chunks. LIBERO, RoboCasa atomic, and RoboCasa composite supplement the DROID example in Figure 4 . Colors and marker shapes identify CF, DR, and DF; faded and solid marks denote CFG 0 and CFG 2, respectively. Stars denote reference actions. The original embeddings are preserved; these projections provide a qualitative view of concentration under guidance.

Figure 13: Refinement sharpens conditionals while retaining across-draw variation. Mean within-draw translation-token entropy (a) and across-draw translation-command diversity (b) use the same 4,000 validation observations and 16 draws per observation as Figure 4 . Gold curves show DF; orange horizontal lines are DR references. Dashed and solid curves denote CFG 0 and CFG 2. From NFE 5 to 20, entropy decreases while action diversity persists. Diversity is the RMS pairwise Euclidean translation-command distance across draws, averaged over horizon positions and observations, in mm.

Figure 14: RoboCasa success rate under varying NFE and guidance. Each head uses one fixed checkpoint. (a,b) NFE sweep for DF and CF at guidance weight

\lambda=0

and

\lambda=2

. (c) Guidance sweep with DF and CF at NFE 10; DR has no NFE axis. The CR line is an unguided one-pass reference. Lines show the mean and bands the sample SD over three evaluation seeds.

Figure 15: a Robocasas example showing that in the same chunk, robot arm has two mode one is move consistenly, the second is first move then hold; (b We show that this independent token sampling could cause factorization error, leads to swithcing between motions on different timestep, the same as what we have seen in push-T Fig. 5

Tokenizer	Head	CFG	NFE	SR (%)
Frequency bins	DR	0	–	$49.8\pm 2.9$
	DF	2	20	$46.0\pm 1.6$
+ metric path	DF	2	20	$51.8\pm 1.6$
nFSQ	DR	2	–	$41.6\pm 3.2$
	DF	2	5	$52.4\pm 1.4$
RVQ	DR	0	–	$14.4\pm 1.3$

Appendix

Table 12: RoboCasa success by tokenizer and head. Success rate (%) as mean

\pm

standard deviation over five evaluation seeds, at the best operating point for each tokenizer and head.

Figure 16: Conditional corruption distributions at matched expected bin error. Colors mark path progress

s=\alpha(t)

; the target is bin 10. Mixture corruption keeps a uniform background, while metric corruption concentrates around the target.

Figure 17: Output-weight diagnostics: DR (top), DF (bottom). Columns show centered cosine, lag-dependent similarity, PCA, and row norms. The banded matrices and decreasing local similarity support ordinal organization in both heads.

Figure 18: Forward-pass input-offset probes. The plot also shows a counterfactual softcap-15 DR trace; our interpretation uses only the canonical softcap-0 results.

Figure 19: Precision depends on target structure. Each target pairs original predictions (seed 0) with held-out MSE for all three seeds; marks indicate means and the dashed line indicates the hard-bin codec floor. Prediction axes use the local target range; all errors remain included. CF/DF curves show one draw, while stochastic MSE averages eight per-draw errors. Both fine-term amplitude and shape change between targets.

Robot Policy Learning

Robot policy learning develops controllers that map sensory observations, task goals, and interaction history to actions for embodied robots.

See all 372 papers

Discrete Forcing: Infusing Discrete Guidance into Continuous Denoising for Few-Step Action Experts
Sep 30, 2026Jingbo Wang, Wenxuan Song, Wenhao Yu +6Flow Matching Efficient VLA Model Inference
The Hong Kong University of Science and Technology (Guangzhou) · South China University of Technology · University of Science and Technology of China +3
SA-VLA: State-aware tokenizer for improving Vision-Language-Action Models' performance
Jun 29, 2026Tengyue Jiang, Chunpu Xu, Jiayue Kang +1Robotic Manipulation Vision-Language-Action Models
2East China University of Science and Technology · 3Hong Kong Polytechnic University · 4Xi’an University of Electronic Science and Technology +1
HiFlow: Tokenization-Free Scale-Wise Autoregressive Policy Learning via Flow Matching
Mar 28, 2026Daichi Yashima, Koki Seno, Shuhei Kurita +2Visuomotor Policy Learning Flow-Based Policy Learning
Keio University · National Institute of Informatics Research and Development Center for Large Language Models · National Institute of Informatics

Sep 30, 2026J/K move · Enter open · S save

Discrete Forcing: Infusing Discrete Guidance into Continuous Denoising for Few-Step Action Experts

Jingbo Wang, Wenxuan Song, Wenhao Yu, Han Zhao+5

The Hong Kong University of Science and Technology (Guangzhou) · South China University of Technology · University of Science and Technology of China+3

Flow Matching Efficient VLA Model Inference Robotic Manipulation Vision-Language-Action Models Hybrid Action Space RL

Open paper AskPDF arXiv

Efficient action generation in vision-language-action (VLA) models requires capturing both coarse action structure and fine-grained details. Discrete action tokens provide compact structural representations but sacrifice precision, while continuous action tokens offer high precision but often require multiple denoising steps. We introduce Discrete Forcing, a flow-matching framework that combines these representations through an explicit coarse-to-fine generation process. It first predicts discrete action tokens to establish a coarse action structure, then uses them to guide continuous action refinement. The discrete and continuous components share a common diffusion transformer backbone with specialized branches, maintaining a parameter count comparable to a conventional single-branch model while requiring only one forward pass per branch. Extensive evaluations across multiple benchmarks demonstrate improved performance and faster inference over a parameter-matched continuous action expert, with consistent performance gains as model capacity increases. Real-world experiments further demonstrate improvements on high-precision and dynamic manipulation tasks.

Abstract

Robot Policy Learning

Explore similar work

Discrete Forcing: Infusing Discrete Guidance into Continuous Denoising for Few-Step Action Experts

SA-VLA: State-aware tokenizer for improving Vision-Language-Action Models' performance

HiFlow: Tokenization-Free Scale-Wise Autoregressive Policy Learning via Flow Matching