Neurosymbolic learning can use symbolic rules to provide supervision for latent concepts from weak labels, but it commonly assumes that the entities referenced by these rules are already specified. Object-centric models decompose images into slot-like representations; however, such slots are not necessarily aligned with the predicates required for symbolic reasoning. We investigate object-centric neurosymbolic learning under distant supervision, where the object-level arguments of a logic program are learned directly from images using only global task labels. We introduce DeepPbjectLog, a probabilistic neurosymbolic model that integrates a slot-based perceptual encoder with a probabilistic logic layer. The encoder predicts objectness and class probabilities for candidate object representations, while the logic layer marginalizes over latent objectness and class assignments to compute the likelihood of the observed label. This formulation provides a differentiable task-level learning signal for object-centric perception without requiring per-object labels, masks, bounding boxes, or heuristic set matching. Evaluations across diverse visual reasoning tasks demonstrate that DeepObjectLog achieves superior out-of-distribution generalization to compositional, object-count, and rule shifts compared to neural object-centric and standard neurosymbolic baselines.
Figures & tables
Figure 1: Overview of the proposed model, DeepObjectLog , on a COCOLogic example. A slot encoder extracts N slots, and each slot gets an objectness probability and a class distribution. A logic program marginalizes over all objectness and class assignments to compute the probability of each task label, so the label alone supervises objectness, classes and slots, with no per-object labels, masks, boxes or set matching. Objectness also gates the slot decoder, so a slot that is not an object (s5) is used by neither the logic nor the reconstruction.
Figure 2: Probabilistic graphical model of DeepObjectLog
MM-A
Task Acc.
Concept Acc.
Extrapolation
Test
OOD
Test
OOD
4 digits
5 digits
CNN
79.90 ±0.17
3.43 ±0.15
-
-
22.16 ±0.90
13.16 ±0.96
SA
98.90 ±0.34
13.30 ±3.14
-
-
36.23 ±4.44
9.76 ±6.11
MESH
98.86 ±0.47
18.26 ±1.33
-
-
37.50 ±1.15
12.00 ±0.51
Tf
95.20 ±0.42
52.28 ±9.11
-
-
50.09 ±8.12
18.42 ±9.85
CoSA
93.00 ±1.92
36.2 ±15.10
-
-
52.20 ±14.01
25.06 ±12.82
Table 1: Task Accuracy (overall image) and Concept Accuracy (object-level decomposition) in MM-A. Results are on a test set of in-distribution compositions ( Test ) and on out-of-distribution compositions ( OOD ) . We also report extrapolation performance on images with four or five digits OOD counts. Dashes indicate cases where a model cannot provide results due to architectural limitations.
Test
OOD class
In-distribution class
OOD class
PokerRules
Extrapolation: 5 cards
CNN
81.96 ±1.10
-
22.03 ±5.27
-
SA
99.30 ±1.21
-
35.86 ±8.72
-
MESH
99.93 ±0.11
-
37.80 ±4.91
-
Tf
97.41 ±0.15
-
15.20 ±7.41
-
CoSA
95.46 ±3.90
-
44.26 ±17.16
-
Table 2: Task Accuracy on PokerRules for both in-distribution test classes and OOD classes. A dash indicates cases where a model cannot provide results due to architectural limitations.
Table 3: Example of predicates and background knowledge for MultiMNIST. This is a simplification of the version employed in the experiments, wherein a maximum of two digits are considered. It is important to note the role of isobj_tmp (objectness) in this process, as it facilitates the selection of elements to be added, thereby determining the final result, and those to be excluded.
Figure 5: Example images with their corresponding classes from the COCOLogic dataset.
Model
OOD Compositions
Extrap. (4 digits)
Extrap. (5 digits)
Llava-1.5-7b-hf
27.43
2.73
1.87
Qwen2.5-VL-7B-Instruct
59.93
28.67
14.37
Ours
90.00
69.73
44.06
Appendix
Table 4: Accuracy results of our model and two Vision-Language Models on the MultiMNIST-addition task.
Model
Test (0,1,or 3 digits)
Interpolation (2 digits)
CNN
76.50 ±0.70
24.90 ±0.14
SA
98.40 ±0.70
10.05 ±3.74
MESH
99.05 ±0.07
14.60 ±5.09
CoSA
87.23 ±5.85
22.53 ±20.16
NeSy
94.35 ±0.49
33.25 ±7.42
Ours
89.80 ±6.34
62.30 ±19.44
Appendix
Table 5: Accuracy results on the MultiMNIST-addition interpolation setting.
Extrapolation
OOD class extrapolation
4 digits
5 digits
4 digits
5 digits
CNN
22.16 ±0.90
13.16 ±0.96
-
-
SA
36.23 ±4.44
9.76 ±6.11
-
-
MESH
37.50 ±1.15
12.00 ±0.51
-
-
CoSA
52.20 ±14.01
25.06 ±12.82
-
-
Ours
69.73 ±10.74
44.06 ±3.85
70.50 ±9.46
46.66 ±5.14
Appendix
Table 6: Accuracy results on the MultiMNIST-addition extrapolation settings. The right columns show the case where both the class and the number of digits are out of distribution.
Task accuracy
Concept accuracy
Objects number accuracy
Test
OOD class
Test
OOD class
Test
OOD class
NeSy
80.23 ±2.11
0.46 ±0.05
18.63 ±1.37
0.13 ±0.05
80.20 ±5.82
76.53 ±24.88
Ours
97.90 ±1.17
72.23 ±16.72
32.00 ±5.88
29.26 ±13.95
81.60 ±2.78
84.13 ±6.60
Appendix
Table 7: Concept accuracy results for PokerRules
CLEVR-A
Test
Extrapolation: 7 objects
OOD Class
MESH
96.97 ±0.35
0.50 ±0.30
-
CoSA
88.79 ±3.95
3.44 ±1.69
-
Ours
93.12 ±0.56
59.81 ±12.45
28.57 ±3.71
Appendix
Table 8: Results for CLEVR-A dataset
Task Accuracy
Concept Accuracy
Test
OOD comps
Test
OOD comps
(1) Original SA
76.91 ±3.25
71.70 ±3.53
30.70 ±5.70
5.71 ±0.14
(2) One classifier only
93.49 ±0.70
64.06 ±19.09
15.05 ±9.19
20.49 ±14.14
(3) No class conditioning
92.51 ±0.28
81.84 ±3.46
66.77 ±14.96
41.16 ±12.10
(4) No reconstruction
95.75 ±1.76
89.55 ±2.19
64.64 ±2.33
52.20 ±11.59
Ours
94.26 ±2.00
90.00 ±3.01
85.16 ±2.45
65.46 ±5.70
Appendix
Table 9: Ablation results on the MultiMNIST-Addition task. Specifically, we evaluate performance under the following modifications: (1) replacing the object-centric encoder MESH with the standard Slot Attention mechanism; (2) using a single classifier that treats non-objectness as an additional class (common approach on object-centric learning), instead of our two-head design with separate objectness and class classifiers; (3) removing the conditioning of the class classifier on the objectness score; (4) excluding the final decoding phase and corresponding reconstruction loss during training. Regarding this last point, a recent work [ 22 ] has shown that incorporating perceptual cues through reconstruction can help align perception with symbolic reasoning, discouraging reliance on irrelevant features and preventing to fall in reasoning shortcuts.
Methods (↓),
MM-A
PokerRules
CLEVR-A
Batch size (→)
64
64
32
CNN
24.60
6.97
N.A.
SA
14.06
3.72
N.A.
MESH
11.06
3.12
1.98
CoSA
6.76
3.65
2.15
NeSy
12.43
3.23
N.A.
Appendix
Table 10: Training speed (iterations per second, it/s) of the evaluated models across three experimental settings.
Methods (↓),
MM-A
PokerRules
CLEVR-A
CNN
∼ 0h 31min
∼ 1h 14min
N.A.
SA
∼ 0h 55min
∼ 2h 20min
N.A.
MESH
∼ 1h 10min
∼ 2h 47min
∼ 6h 11min
CoSA
∼ 1h 55min
∼ 2h 22min
∼ 5h 03min
NeSy
∼ 4h 11min
∼ 10h 46min
N.A.
Ours
∼ 6h 47min
∼ 16h 33min
∼ 9h 20min
Appendix
Table 11: Total training time per epoch (hours:minutes) for the evaluated models across the three experimental settings.
Methods (↓),
Supervision (%) (→)
0
30
60
100
DINOSAUR
∼ 5h 27min
∼ 6h 51min
∼ 11h 03min
∼ 15h 40min
Ours
∼ 6h 44min
∼ 8h 00min
∼ 14h 33min
∼ 19h 47min
Appendix
Table 12: Total training time of the evaluated models in the COCOLogic setting.
Methods (↓),
Num. of objects (→)
1
2
3
4
5
6
10
20
MESH
24.04
16.61
12.27
9.88
6.70
5.68
3.37
1.04
Ours
16.46
13.24
7.69
4.06
2.74
1.70
TO
TO
Ours w A-NeSI
20.54
18.00
14.34
12.67
10.78
9.21
4.34
1.19
Appendix
Table 13: Number of inference iterations per second for the MM-A task (batch size 64) as the number of objects increases. We write timeout (TO) when the number of iterations per second drops below 1.
MM-A
PokerRules
CLEVR-A
MESH
0.998 ±0.011
1.156 ±0.050
2.057 ±0.037
Ours
0.733 ±0.170
0.498 ±0.072
1.287 ±0.188
Appendix
Table 14: Count Mean Absolute Error for the number of active slots in the three datasets. The comparison involves our method and the baseline MESH. The lower the better.
Figure 6: Attention weights of the Slot Attention mechanism for the MultiMNIST dataset.
Figure 7: Attention weights of the Slot Attention mechanism for the PokerRules dataset. As the images are larger than those in MultiMNIST, it is harder to see where the attention weights are highest. However, the model is exhibiting the same behaviour as in previous experiments.
Figure 8: Attention weights of the Slot Attention mechanism trained with our method for the CLEVR dataset. As the images are larger than those in MultiMNIST, it is harder to see where the attention weights are highest. However, the model is exhibiting the same behaviour as in previous experiments. For each slot, the associated objectness score is shown, with slots above 0.5 marked as yes.
Figure 9: Alpha masks produced by the Slot Attention mechanism trained with our method on the PokerRules dataset. For each slot, the associated objectness score is shown, with slots above 0.5 marked as yes .
Neurosymbolic systems promise to combine deep neural network's (DNN) processing of raw sensor inputs with few-shot performance of symbolic artificial intelligence. Two-stage approaches explicitly decouple DNN based perception from subsequent rule based reasoning. This avoids optimization and interpretability issues of end to end differentiable approaches, but requires costly labels for the perception output. This paper introduces an efficient weak supervision scheme for the perception stage to ground its output symbols for logical induction in object-centric reasoning tasks. It combines a slot-based architecture for object-centricity with a Variational Autoencoder (VAE) for self-supervision, competing with concept guidance on latent dimensions for human interpretable grounding. The resulting predictions are translated into symbolic background knowledge for reasoning frameworks, such as Inductive Logic Programming (ILP), Decision Trees, and Bayesian Networks. Our extensive empirical evaluation on synthetic and real world datasets shows that our approach can discover complex, abstract rules for object centric reasoning whilst reducing supervision to as little as 1% of labels, and being robust even under substantial domain shift. Notably, at 1% supervision it even outperforms state of the art foundation model baselines in domain generalization
Sparsh Tiwari, Bettina Finzel, Gesina Schwalbe
University of Lübeck, Germany · University of Bamberg, Germany · University of Ulm, Germany
Two-stage neuro-symbolic architectures provide an elegant paradigm for visual problem solving by cleanly separating connectionist perception of predefined symbols from possibly later defined relational reasoning thereon. However, anchoring high-level predicates into visual frames typically necessitates annotations that are expensive to acquire. In this work, we introduce the Dynamic Orthogonal Concept Bottleneck (D-OCB), an object-centric slot- VAE framework designed to extract human-aligned symbolic predicates under extremely weak supervision. D-OCB eliminates the arduous manual tuning of loss-balancing coef- ficients by dynamically learning optimal hyperparameter allocations during training. To infuse prior knowledge on independence of concept categories, in addition to standard re- construction self-supervision we penalize correlation across concept subspaces. Crucially, to combat the instability of very low supervision regimes, D-OCB incorporates a dynamic di- mensionality allocation mechanism; this adaptive formulation allows well-represented con- cepts to yield latent dimensions to underperforming concepts that are lagging behind, effectively preventing representation collapse and significantly improving overall concept accuracy. Through an extensive empirical evaluation, we demonstrate that our framework achieves high concept alignment and downstream visual reasoning accuracy using minimal label budgets, matching or outperforming end-to-end paradigms.
Sparsh Tiwari, Gesina Schwalbe, Bettina Finzel
University of Lübeck, Germany · Ulm University, Germany · University of Bamberg, Germany
Neuro-symbolic models are usually trained with supervision only on final labels, leaving the intermediate concepts unobserved. Since many concept assignments are consistent with a given label, training can predict labels correctly while recovering the wrong concepts, a failure known as a reasoning shortcut. Prototypical networks reduce shortcuts by anchoring each concept to a few labeled examples, but existing methods still couple perception and reasoning through a hand-crafted, task-specific differentiable loss that must be redesigned for every task. We introduce Soft-PNet, which removes this loss: it reframes concept grounding as a Metropolis walk over a precomputed cache of feasible symbolic solutions, guided by a prototype distribution built from a single labeled anchor per concept, and trains against one KL objective between the prototype-weighted cache and the network's concept predictions. The objective is identical across tasks and remains applicable when the solution space cannot be enumerated. On MNIST-EvenOdd, Visual Sudoku, and Kand-Logic under scarce supervision, Soft-PNet matches loss-engineered prototypical networks at the concept and label levels and recovers concepts that soft-grounding baselines miss, with no loss engineering and lower training time.