A Fully Differentiable Neuro-Soft-Symbolic Framework for Perceptual Task Planning

Hongyan Wei; Wael AbdAlmageed

cs.AISep 18, 2026

A Fully Differentiable Neuro-Soft-Symbolic Framework for Perceptual Task Planning

Authors: Hongyan Wei, Wael AbdAlmageed

Organizations: Clemson University

Abstract

Perceptual planning tasks require two key capabilities: accurately perceiving uncertain scenes and planning valid action sequences following logical rules. Conventional methods convert perception into discrete symbolic facts and then plan, discarding perceptual uncertainty and severing task-level feedback to perception. We introduce a generic, fully differentiable neuro-soft-symbolic framework that connects visual perception and task planning within a single computational graph. The framework maintains a continuous soft symbolic state, lifts domain rules into a differentiable soft- $T_P$ transition operator, and optimizes action logits over a short planning horizon. Gradients from the planning objective can also update the perception parameters, allowing task-relevant perceptual representations to be refined during planning. On Blocksworld, our method solves 40/40 LatPlan-40 tasks and 596/600 PlanBench-600 tasks, compared with 33/40 for LatPlan and 587/600 for the reasoning-model baseline, while requiring substantially less computation and time. In the perceptual-uncertainty ablation, our method improves the success rate from 59% with frozen perception to 83%. We further conduct task-and-motion simulations on Blocksworld scenes, providing an execution-level validation of the compatibility between decoded task plans and downstream robotic motion execution.

Figures & tables

Fig. 1: Comparison of current planning methods.

Fig. 2: Overview of the proposed fully differentiable neuro-soft-symbolic planning framework.

Logic rule / constraint	ASP syntax	Soft- $T_{P}$
$\operatorname{clear}(X)$ : $X$ is clear if and only if no other block currently rests on $X$ .	clear(X) :- block(X), not blocked(X). blocked(X) :- on(Y,X), Y!=X.	$c_{t}(x)=\prod_{z\neq x}(1-\mathbf{F}_{t,z,x})$ (Eq. 4 ) — product-t-norm lift.
$\mathrm{move}(x\!\to\!y)$ is allowed only when $x$ and $y$ are available.	legal(X,Y) :- clear(X), clear(Y), X!=Y.	$\tilde{g}_{t}(x,y)=M_{t}(x,y)c_{t}(x)\bar{c}_{t}(y)e_{t}(x,y)$ (Eq. 5 ).
At most one block moves at a given timestep.	:- moves(X,T), moves(Z,T), X!=Z.	$\rho_{t}(x)=\sum_{y}\tilde{g}_{t}(x,y)$ , $q_{t}(x)=\prod_{x^{\prime}\neq x}(1-\rho_{t}(x^{\prime}))$ , and $g_{t}(x,y)=\tilde{g}_{t}(x,y)q_{t}(x)$ (Eqs. 6 – 7 ).
A block supports at most one other block; the table has unlimited capacity.	:- on(X,Y), on(Z,Y), X!=Z, block(Y).	$\mathcal{T}_{P}^{\mathrm{targ}}(\mathbf{F})_{z,y}=\mathbf{F}_{z,y}\prod_{z^{\prime}\neq z}(1-\mathbf{F}_{z^{\prime},y})$ (Eq. 9 ).
A block remains on its current support unless a move commits. STAY creates no new relation.	on(X,Y,T+1) :- on(X,Y,T), not moved(X,T).	$\mathbf{F}_{t+1,x,y}=(1-m_{t}(x))\mathbf{F}_{t,x,y}+g_{t}(x,y)$ , where $m_{t}(x)=\sum_{y}g_{t}(x,y)$ (Eq. 8 ).

TABLE I: Correspondence between Blocksworld logic rules and the differentiable soft-

T_{P}

lift used in our implementation.

Dataset	Method	Valid (SR) $\uparrow$	Avg. time $\downarrow$	Cost / 100 (USD) $\downarrow$	Optimal $\uparrow$
Blocksworld, LatPlan-40	LatPlan ( $\mathrm{AMA}^{+}_{4}$ ) [ 11 ]	33/40 (82.50%)	–	–	–
Blocksworld, LatPlan-40	Ours	40/40 (100.00%) +17.50%	3.53 s $\pm$ 0.01 s	$0.31	31/40 (77.50%)
Blocksworld, PlanBench-600	Claude 3.5 Sonnet (LLM) [ 21 ]	329/600 (54.83%)	–	$0.44	–
	LLaMA 3.1 405B (LLM) [ 21 ]	376/600 (62.67%)	–	–	–
	o1-mini (LRM) [ 21 ]	340/600 56.67%	10.84 s	$3.69	–
	o1-preview (LRM) [ 21 ]	587/600 (97.83%)	40.43 s	$42.12	–

TABLE II: Main planning results. All reported Ours results on Blocksworld use 3 independent planning seeds. Red shows the improvement of our method over the corresponding baseline.

Blocks	Method	Instances	Valid plans $\uparrow$	Optimal $\uparrow$	Inexecutable plans $\downarrow$	Non-goal-reaching plans $\downarrow$
All	o1-mini (LRM) [ 21 ]	600	340 (56.67%)	–	39.50%	3.83%
All	o1-preview (LRM) [ 21 ]	600	587 (97.83%)	–	12 (2.00%)	1 (0.17%)
Total	Ours	600	596 (99.33%)	350 (58.78% $\pm$ 0.08%)	0 (0.00%)	4 (0.67%)
3-block	Ours	100	100 (100.00%)	84 (84.00%)	0 (0.00%)	0 (0.00%)
4-block	Ours	445	445 (100.00%)	242 (54.38%)	0 (0.00%)	0 (0.00%)
5-block	Ours	55	51 (92.73%)	24 (47.06%)	0 (0.00%)	4 (7.27%)

TABLE III: PlanBench Blocksworld error analysis. The PlanBench baseline results are reported overall; our method is additionally broken down by number of blocks. Optimal is reported as a percentage of valid plans.

	Original PlanBench-600		Clear subset (100)		Blurred subset (100)
Perception Variant	Valid (SR) $\uparrow$	Time $\downarrow$	Valid (SR) $\uparrow$	Time $\downarrow$	Valid (SR) $\uparrow$	Time $\downarrow$
Ground-truth symbolic state (PDDL)	596/600	7.02 s	–	–	–	–
Frozen perception backbone	596/600	7.63 s	100/100	7.98 s	59/100	10.13 s
Planning-optimized perception backbone	596/600	9.96 s	100/100	10.22 s	83/100 +24%	13.34 s

TABLE IV: Ablation Study for Differentiable Perception-Planning

Fig. 3: Distribution of PlanBench-600 instances by optimal step count.

$T$	Valid (SR) $\uparrow$	Optimal $\uparrow$	Non-goal-reaching $\downarrow$	Avg. time $\downarrow$
1	108/150 (72.0%)	93/150 (62.0%)	42 (28.0%)	9.57 s
3 (ours)	150/150 (100.0%)	115/150 (76.7%)	0 (0.0%)	7.77 s
5	128/150 (85.3%)	114/150 (76.0%)	22 (14.7%)	13.48 s

TABLE V: Ablation study on short-horizon window-size sensitivity.

Fig. 4: Case study of the proposed perception-to-planning procedure on a 4-block PlanBench instance.

Fig. 5: Integrated task-and-motion simulation on four-block and five-block Blocksworld instances.

Dataset	Method	Valid (SR) $\uparrow$	Inexecutable $\downarrow$	Non-goal-reaching $\downarrow$	Avg. time $\downarrow$
Logistics	o1-preview (LRM)	188/200 (94.00%)	12 (6.00%)	0 (0.00%)	84.07 s
Logistics	Ours	196/200 (98.00%) +4.00%	0 (0.00%)	4 (2.00%)	36.28 s -47.79 s
Sokoban	o1-preview (LRM)	7/55 (12.73%)	37 (67.27%)	11 (20.00%)	147.98 s
Sokoban	Ours	33/55 (60.00%) +47.27%	0 (0.00%)	22 (40.00%)	53.02 s -94.96 s

TABLE VI: Cross-domain comparison on Logistics and Sokoban.

Neuro-Symbolic Planning

See all 20 papers

Bridging Learned Visual Perception and Symbolic Belief-Space Planning
Sep 15, 2026Guy Azran, Michael Navat, Sarah KerenBelief-Space Planning Symbolic Planning
Taub Faculty of Computer Science, Technion - Israel Institute of Technology · Faculty of Mathematics, Technion - Israel Institute of Technology
Write Once, Plan Many: Building Reusable Visual Planning Systems from One Scene
Oct 13, 2026Ruixiao Yang, Yilun Hao, Mingxin Yu +2Program Synthesis Model-Based Planning
MIT · MIT-IBM Watson AI Lab
Neuro-Symbolic Learning for Long-Horizon Task Planning Under Complex Logical Constraints
Jun 5, 2026Qiwei Du, Zitong Zhan, Shaoshu Su +7Symbolic Planning Neuro-Symbolic Planning
Spatial AI & Robotics (SAIR) Lab, University at Buffalo, NY 14260 · Robotics Institute, Carnegie Mellon University, PA 15213

Sep 15, 2026J/K move · Enter open · S save

Bridging Learned Visual Perception and Symbolic Belief-Space Planning

Guy Azran, Michael Navat, Sarah Keren

Taub Faculty of Computer Science, Technion - Israel Institute of Technology · Faculty of Mathematics, Technion - Israel Institute of Technology

Belief-Space Planning Symbolic Planning VLMs for Robotics

Open paper AskPDF arXiv

In partially observable settings, agents must act without full knowledge of the world state and rely on uncertain state-estimation pipelines. Obtaining grounded and verifiable symbolic plans under such uncertainty remains a key challenge. Recent work has integrated Vision-Language Models (VLMs) to bridge perception and symbolic reasoning, following two main paradigms. The first, VLM-as-planner, maps images directly to action sequences, and the second, VLM-as-grounder, grounds observations into symbolic predicates used as the initial state by off-the-shelf planners. Both approaches ignore uncertainty in the planning process, compromising robustness. We introduce a third paradigm, VLM-as-probabilistic-grounder, a novel approach that captures the uncertainty of VLM predicate groundings as a probability distribution over symbolic states. This enables planning in belief space and producing robust plans under uncertainty. Experiments in simulated household robot settings show improved robustness and task success over deterministic grounding, underscoring how our approach leverages foundation models for reliable planning under uncertainty.

Abstract

Neuro-Symbolic Planning

Explore similar work

Bridging Learned Visual Perception and Symbolic Belief-Space Planning

Write Once, Plan Many: Building Reusable Visual Planning Systems from One Scene

Neuro-Symbolic Learning for Long-Horizon Task Planning Under Complex Logical Constraints