Per-Domain Generalizing Policies: On Learning Efficient and Robust Q-Value Functions (Extended Version with Technical Appendix)

Nicola J. Müller; Moritz Oster; Isabel Valera; Jörg Hoffmann; Timo P. Gros

cs.AIMar 18, 2026

Per-Domain Generalizing Policies: On Learning Efficient and Robust Q-Value Functions (Extended Version with Technical Appendix)

Authors: Nicola J. Müller, Moritz Oster, Isabel Valera, Jörg Hoffmann, Timo P. Gros

Organizations: German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany · Saarland University, Saarland Informatics Campus, Saarbrücken, Germany · Center for European Research in Trusted Artificial Intelligence (CERTAIN)

Abstract

Learning per-domain generalizing policies is a key challenge in learning for planning. Standard approaches learn state-value functions represented as graph neural networks using supervised learning on optimal plans generated by a teacher planner. In this work, we advocate for learning Q-value functions instead. Such policies are drastically cheaper to evaluate for a given state, as they need to process only the current state rather than every successor. Surprisingly, vanilla supervised learning of Q-values performs poorly as it does not learn to distinguish between the actions taken and those not taken by the teacher. We address this by using regularization terms that enforce this distinction, resulting in Q-value policies that consistently outperform state-value policies across a range of 10 domains and are competitive with the planner LAMA-first.

Figures & tables

Figure 1: (Left) Total runtime of forward passes in hours per instance size on random states. (Right) Average coverage and confidence interval per instance size for trained policies.

	R-GNN		OE		OAE
Domain	$V$	$Q$	$V$	$Q$	$V$	$Q$
Blocksworld	0.6	0.5	1.8	2.0	0.7	0.4
Childsnack	7.2	0.9	21.7	4.3	28.6	0.7
Ferry	0.8	0.5	2.5	2.0	1.6	0.4
Floortile	1.9	1.0	7.9	4.0	4.0	0.6
Gripper	1.0	0.6	3.1	2.2	1.9	0.5

Table 1: Total runtime in hours for randomly initialized state and Q-value policies on sets of random states.

	$Q$						$\Omega^{\text{Exp.}}$						$\Omega^{\text{Heu.}}$
	R-GNN		OE		OAE		R-GNN		OE		OAE		R-GNN		OE		OAE
	Err	Diff	Err	Diff	Err	Diff	Err	Diff	Err	Diff	Err	Diff	Err	Diff	Err	Diff	Err	Diff
Average	0.54	0.71	1.37	0.96	2.15	0.84	0.70	4.69	0.70	7.66	0.91	6.66	0.73	97.55	1.25	46.25	0.83	152.52

Table 2: Prediction errors for teacher actions (

\mathrm{Err}

), and Q-value differences between teacher and non-teacher actions (

\mathrm{Diff}

	R-GNN								OE								OAE
	$V$		$Q$		$\Omega^{\text{Exp.}}$		$\Omega^{\text{Heu.}}$		$V$		$Q$		$\Omega^{\text{Exp.}}$		$\Omega^{\text{Heu.}}$		$V$		$Q$		$\Omega^{\text{Exp.}}$		$\Omega^{\text{Heu.}}$
Domain	Scale	SCov	Scale	SCov	Scale	SCov	Scale	SCov	Scale	SCov	Scale	SCov	Scale	SCov	Scale	SCov	Scale	SCov	Scale	SCov	Scale	SCov	Scale	SCov
Blocksworld	52	44.5	5	1.6	54	47.8	55	48.2	14	9.2	5	1.2	48	40.5	50	42.4	37	27.5	4	1.3	52	44.9	52	44.6
Childsnack	71	43.4	16	4.2	100	93.0	100	93.0	26	12.0	9	0.2	82	43.1	98	46.9	31	15.9	36	14.9	100	65.8	83	41.5
Ferry	100	84.5	30	12.9	100	85.2	100	84.5	33	19.8	21	10.9	100	80.2	100	80.1	45	26.4	7	2.1	100	79.5	100	80.1
Floortile	8	0	8	0	60	44.5	62	45.6	8	0	8	0	50	28.8	56	43.2	8	0	8	0	59	42.0	62	45.9

Table 3: Scale and SCov scores of the scaling behavior evaluation. Bold scores are the best for each architecture, and highlighted scores are the best overall.

	$V$		$\Omega^{\text{Exp.}}$		$\Omega^{\text{Heu.}}$		LAMA-first
	R-GNN
Domain	Cov	Len	Cov	Len	Cov	Len	Cov	Len
Blocksworld	47.8	90.0	78.9	233.1	78.9	232.1	56.7	288.7
Childsnack	5.6	41.2	51.1	41.4	54.4	41.5	35.2	41.8
Ferry	54.4	73.7	67.8	119.6	67.8	119.6	67.8	150.3
Floortile	0.0	—	36.7	93.5	37.8	109.3	16.9	51.0

Table 4: Coverage in percent (Cov) and average plan length (Len) on IPC’23 test sets. Bold scores are the best coverages for each domain.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Domain	Training Set Sizes
Blocksworld	$[2-16]$
Childsnack	$[8-38]$
Ferry	$[3-25]$
Floortile	$[7-18]$
Rovers	$[10-26]$
Satellite	$[5-31]$

Appendix

Table 5: Number of objects in training instances.

Generalization in RL

Reinforcement learning generalization studies how agents transfer learned behavior to unseen tasks, environments, dynamics, and observations.

See all 89 papers

GADAR: Generalizing Across Domains via Action Ranking for Classical Planning
Oct 26, 2026Rajesh Mangannavar, Pranay Dugar, Prasad TadepalliCross-Domain Generalization Symbolic Planning
Oregon State University, Corvallis, OR 97330, USA
Learning to Search and Searching to Learn for Generalization in Planning
May 25, 2026Michael Aichmüller, Yannik Hesse, Hector GeffnerReinforcement Learning Generalization in RL
Department of Machine Learning and Reasoning, RWTH Aachen University, Aachen, Germany.
Generative Value Classifier
Oct 15, 2026Alexander Swerdlow, Perry Dong, Chelsea FinnReinforcement Learning Q-Learning
Stanford University

Oct 26, 2026J/K move · Enter open · S save

GADAR: Generalizing Across Domains via Action Ranking for Classical Planning

Rajesh Mangannavar, Pranay Dugar, Prasad Tadepalli

Oregon State University, Corvallis, OR 97330, USA

Cross-Domain Generalization Symbolic Planning Zero-Shot Policy Transfer

Open paper AskPDF arXiv

Learned policies for classical planning act without search, but each is tied to the domain it was trained on: network modules per action schema, parameters per predicate, or vocabulary-sized encodings mean a trained policy cannot even be executed on a new domain's predicates and action schemas. We present GADAR (Generalizing Across Domains via Action Ranking), a policy that takes the domain description as input, π(s, G, D), and constructs grounded actions in domains never seen in training, without search. Symbols are described by the roles they play in the domain rather than by their names, in three layers that each extend the previous one: (i) a lifted domain layer embeds D in every state graph through structural signatures; (ii) a binding layer ties each ground action's applicability to the lifted preconditions it instantiates; and (iii) a chain layer states which schema produces what another needs and how far each schema stands from the goal. The same network therefore runs on every domain, seen or unseen. We evaluate zero-shot generalization leave-one-domain-out over eight benchmark domains. A single GADAR policy trained on seven domains solves 58.3% of their test instances, one point below the 59.3% of dedicated single-domain GADAR models, so one policy serves many domains at no cost in coverage; a multi-task control that keeps symbol identity in its weights, while strong on some of the domains it trains on, transfers to none of the held-out ones. Zero-shot, GADAR executes on every held-out domain and exceeds random choice under the same executor on the large test instances of all eight: it solves 22.6% of them on average, against 3.3% for random choice and 0.6% for the same representation without the chain layer, and reaches 68% on held-out Visitall. Zero-shot coverage nonetheless remains well below in-domain coverage on most domains, and we analyze where it is weakest.

Abstract

Generalization in RL

Explore similar work

GADAR: Generalizing Across Domains via Action Ranking for Classical Planning

Learning to Search and Searching to Learn for Generalization in Planning

Generative Value Classifier