Neurosymbolic learning can use symbolic rules to provide supervision for latent concepts from weak labels, but it commonly assumes that the entities referenced by these rules are already specified. Object-centric models decompose images into slot-like representations; however, such slots are not necessarily aligned with the predicates required for symbolic reasoning. We investigate object-centric neurosymbolic learning under distant supervision, where the object-level arguments of a logic program are learned directly from images using only global task labels. We introduce DeepPbjectLog, a probabilistic neurosymbolic model that integrates a slot-based perceptual encoder with a probabilistic logic layer. The encoder predicts objectness and class probabilities for candidate object representations, while the logic layer marginalizes over latent objectness and class assignments to compute the likelihood of the observed label. This formulation provides a differentiable task-level learning signal for object-centric perception without requiring per-object labels, masks, bounding boxes, or heuristic set matching. Evaluations across diverse visual reasoning tasks demonstrate that DeepObjectLog achieves superior out-of-distribution generalization to compositional, object-count, and rule shifts compared to neural object-centric and standard neurosymbolic baselines.
Figures & tables
Figure 1: Overview of the proposed model, DeepObjectLog , on a COCOLogic example. A slot encoder extracts N slots, and each slot gets an objectness probability and a class distribution. A logic program marginalizes over all objectness and class assignments to compute the probability of each task label, so the label alone supervises objectness, classes and slots, with no per-object labels, masks, boxes or set matching. Objectness also gates the slot decoder, so a slot that is not an object (s5) is used by neither the logic nor the reconstruction.
Figure 2: Probabilistic graphical model of DeepObjectLog
MM-A
Task Acc.
Concept Acc.
Extrapolation
Test
OOD
Test
OOD
4 digits
5 digits
CNN
79.90 ±0.17
3.43 ±0.15
-
-
22.16 ±0.90
13.16 ±0.96
SA
98.90 ±0.34
13.30 ±3.14
-
-
36.23 ±4.44
9.76 ±6.11
MESH
98.86 ±0.47
18.26 ±1.33
-
-
37.50 ±1.15
12.00 ±0.51
Tf
95.20 ±0.42
52.28 ±9.11
-
-
50.09 ±8.12
18.42 ±9.85
CoSA
93.00 ±1.92
36.2 ±15.10
-
-
52.20 ±14.01
25.06 ±12.82
Table 1: Task Accuracy (overall image) and Concept Accuracy (object-level decomposition) in MM-A. Results are on a test set of in-distribution compositions ( Test ) and on out-of-distribution compositions ( OOD ) . We also report extrapolation performance on images with four or five digits OOD counts. Dashes indicate cases where a model cannot provide results due to architectural limitations.
Test
OOD class
In-distribution class
OOD class
PokerRules
Extrapolation: 5 cards
CNN
81.96 ±1.10
-
22.03 ±5.27
-
SA
99.30 ±1.21
-
35.86 ±8.72
-
MESH
99.93 ±0.11
-
37.80 ±4.91
-
Tf
97.41 ±0.15
-
15.20 ±7.41
-
CoSA
95.46 ±3.90
-
44.26 ±17.16
-
Table 2: Task Accuracy on PokerRules for both in-distribution test classes and OOD classes. A dash indicates cases where a model cannot provide results due to architectural limitations.
Table 3: Example of predicates and background knowledge for MultiMNIST. This is a simplification of the version employed in the experiments, wherein a maximum of two digits are considered. It is important to note the role of isobj_tmp (objectness) in this process, as it facilitates the selection of elements to be added, thereby determining the final result, and those to be excluded.
Figure 5: Example images with their corresponding classes from the COCOLogic dataset.
Model
OOD Compositions
Extrap. (4 digits)
Extrap. (5 digits)
Llava-1.5-7b-hf
27.43
2.73
1.87
Qwen2.5-VL-7B-Instruct
59.93
28.67
14.37
Ours
90.00
69.73
44.06
Appendix
Table 4: Accuracy results of our model and two Vision-Language Models on the MultiMNIST-addition task.
Model
Test (0,1,or 3 digits)
Interpolation (2 digits)
CNN
76.50 ±0.70
24.90 ±0.14
SA
98.40 ±0.70
10.05 ±3.74
MESH
99.05 ±0.07
14.60 ±5.09
CoSA
87.23 ±5.85
22.53 ±20.16
NeSy
94.35 ±0.49
33.25 ±7.42
Ours
89.80 ±6.34
62.30 ±19.44
Appendix
Table 5: Accuracy results on the MultiMNIST-addition interpolation setting.
Extrapolation
OOD class extrapolation
4 digits
5 digits
4 digits
5 digits
CNN
22.16 ±0.90
13.16 ±0.96
-
-
SA
36.23 ±4.44
9.76 ±6.11
-
-
MESH
37.50 ±1.15
12.00 ±0.51
-
-
CoSA
52.20 ±14.01
25.06 ±12.82
-
-
Ours
69.73 ±10.74
44.06 ±3.85
70.50 ±9.46
46.66 ±5.14
Appendix
Table 6: Accuracy results on the MultiMNIST-addition extrapolation settings. The right columns show the case where both the class and the number of digits are out of distribution.
Task accuracy
Concept accuracy
Objects number accuracy
Test
OOD class
Test
OOD class
Test
OOD class
NeSy
80.23 ±2.11
0.46 ±0.05
18.63 ±1.37
0.13 ±0.05
80.20 ±5.82
76.53 ±24.88
Ours
97.90 ±1.17
72.23 ±16.72
32.00 ±5.88
29.26 ±13.95
81.60 ±2.78
84.13 ±6.60
Appendix
Table 7: Concept accuracy results for PokerRules
CLEVR-A
Test
Extrapolation: 7 objects
OOD Class
MESH
96.97 ±0.35
0.50 ±0.30
-
CoSA
88.79 ±3.95
3.44 ±1.69
-
Ours
93.12 ±0.56
59.81 ±12.45
28.57 ±3.71
Appendix
Table 8: Results for CLEVR-A dataset
Task Accuracy
Concept Accuracy
Test
OOD comps
Test
OOD comps
(1) Original SA
76.91 ±3.25
71.70 ±3.53
30.70 ±5.70
5.71 ±0.14
(2) One classifier only
93.49 ±0.70
64.06 ±19.09
15.05 ±9.19
20.49 ±14.14
(3) No class conditioning
92.51 ±0.28
81.84 ±3.46
66.77 ±14.96
41.16 ±12.10
(4) No reconstruction
95.75 ±1.76
89.55 ±2.19
64.64 ±2.33
52.20 ±11.59
Ours
94.26 ±2.00
90.00 ±3.01
85.16 ±2.45
65.46 ±5.70
Appendix
Table 9: Ablation results on the MultiMNIST-Addition task. Specifically, we evaluate performance under the following modifications: (1) replacing the object-centric encoder MESH with the standard Slot Attention mechanism; (2) using a single classifier that treats non-objectness as an additional class (common approach on object-centric learning), instead of our two-head design with separate objectness and class classifiers; (3) removing the conditioning of the class classifier on the objectness score; (4) excluding the final decoding phase and corresponding reconstruction loss during training. Regarding this last point, a recent work [ 22 ] has shown that incorporating perceptual cues through reconstruction can help align perception with symbolic reasoning, discouraging reliance on irrelevant features and preventing to fall in reasoning shortcuts.
Methods (↓),
MM-A
PokerRules
CLEVR-A
Batch size (→)
64
64
32
CNN
24.60
6.97
N.A.
SA
14.06
3.72
N.A.
MESH
11.06
3.12
1.98
CoSA
6.76
3.65
2.15
NeSy
12.43
3.23
N.A.
Appendix
Table 10: Training speed (iterations per second, it/s) of the evaluated models across three experimental settings.
Methods (↓),
MM-A
PokerRules
CLEVR-A
CNN
∼ 0h 31min
∼ 1h 14min
N.A.
SA
∼ 0h 55min
∼ 2h 20min
N.A.
MESH
∼ 1h 10min
∼ 2h 47min
∼ 6h 11min
CoSA
∼ 1h 55min
∼ 2h 22min
∼ 5h 03min
NeSy
∼ 4h 11min
∼ 10h 46min
N.A.
Ours
∼ 6h 47min
∼ 16h 33min
∼ 9h 20min
Appendix
Table 11: Total training time per epoch (hours:minutes) for the evaluated models across the three experimental settings.
Methods (↓),
Supervision (%) (→)
0
30
60
100
DINOSAUR
∼ 5h 27min
∼ 6h 51min
∼ 11h 03min
∼ 15h 40min
Ours
∼ 6h 44min
∼ 8h 00min
∼ 14h 33min
∼ 19h 47min
Appendix
Table 12: Total training time of the evaluated models in the COCOLogic setting.
Methods (↓),
Num. of objects (→)
1
2
3
4
5
6
10
20
MESH
24.04
16.61
12.27
9.88
6.70
5.68
3.37
1.04
Ours
16.46
13.24
7.69
4.06
2.74
1.70
TO
TO
Ours w A-NeSI
20.54
18.00
14.34
12.67
10.78
9.21
4.34
1.19
Appendix
Table 13: Number of inference iterations per second for the MM-A task (batch size 64) as the number of objects increases. We write timeout (TO) when the number of iterations per second drops below 1.
MM-A
PokerRules
CLEVR-A
MESH
0.998 ±0.011
1.156 ±0.050
2.057 ±0.037
Ours
0.733 ±0.170
0.498 ±0.072
1.287 ±0.188
Appendix
Table 14: Count Mean Absolute Error for the number of active slots in the three datasets. The comparison involves our method and the baseline MESH. The lower the better.
Figure 6: Attention weights of the Slot Attention mechanism for the MultiMNIST dataset.
Figure 7: Attention weights of the Slot Attention mechanism for the PokerRules dataset. As the images are larger than those in MultiMNIST, it is harder to see where the attention weights are highest. However, the model is exhibiting the same behaviour as in previous experiments.
Figure 8: Attention weights of the Slot Attention mechanism trained with our method for the CLEVR dataset. As the images are larger than those in MultiMNIST, it is harder to see where the attention weights are highest. However, the model is exhibiting the same behaviour as in previous experiments. For each slot, the associated objectness score is shown, with slots above 0.5 marked as yes.
Figure 9: Alpha masks produced by the Slot Attention mechanism trained with our method on the PokerRules dataset. For each slot, the associated objectness score is shown, with slots above 0.5 marked as yes .