cs.LGNov 4, 2026

Structured Machine Theory of Mind from Agent Trajectories

Authors: Nigel F. Doering, Rahath Malladi, Arshia Sangwan, M Tanjid Hasan Tonmoy, Jacob Doan, David Danks, Tauhidur Rahman

Organizations: Halıcıoğlu School of Data Science and Computing, University of California San Diego · New York University · Department of Philosophy and School of Data Science, University of Virginia

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Predictive models of human behavior trained on large-scale trajectory data optimize for statistical accuracy without representing the mental states that causally generate behavior. Such models support prediction but not principled intervention: they cannot answer how an agent's behavior would change if its beliefs or preferences were different. We introduce Structured Machine Theory of Mind (SMToM), a framework that attributes explicit, independently supervised belief and desire representations from observed trajectories within a Belief-Desire-Intention (BDI) causal structure. The central architectural element is a goal head that consumes only the predicted mental-state channels and a current-trajectory embedding; counterfactual intervention on beliefs and desires is a direct operation. We instantiate SMToM on a controlled pedestrian navigation domain where ground-truth mental states are known by construction, enabling evaluation of attribution accuracy, local channel interventions, and trajectory-level counterfactual diagnostics. The resulting model, BDIBottleneck, improves over trajectory-only and unsupervised-context neural baselines. It is also competitive with a matched auxiliary-supervised context baseline while providing an editable mental-state interface that the unbottlenecked baseline lacks. Counterfactual experiments on desires confirm that substituting an agent's inferred preferences with a different activity type coherently shifts predicted destinations toward relevant locations. Counterfactual experiments on beliefs confirm that marking a location as unavailable in the agent's belief state reduces its predicted probability as a destination, though in a complex way. These results demonstrate, in a controlled navigation setting, that explicit BDI-structured supervision is a viable foundation for interpretable and counterfactual analysis of longitudinal trajectory data.

Figures & tables

Figure 1: BDIBottleneck architecture. Three trajectory streams are encoded separately: random same-agent context for desire, recent same-agent context for belief comparison, and the current trajectory prefix for goal inference. The belief head uses [zd∥zr][z_{d}\|z_{r}] , while the goal head uses the bottleneck representation [π^∥b^∥zτ][\hat{\pi}\|\hat{b}\|z_{\tau}] . Desire and belief channels can be intervened on directly for counterfactual analysis before goal prediction.
Figure 2: Top-1 goal inference including the matched auxiliary-supervised AuxContext baseline.
 Test (belief heldout)Test (new agent, belief heldout)
Model25%50%75%100%25%50%75%100%
VisitFrequency13.3313.3313.3313.3314.1714.1714.1714.17
BToM (Informed) [approximate upper bound]24.7933.9648.3394.3829.7939.7958.3394.17
GoalPredictor12.5025.8340.4242.5010.3527.5041.0443.82
ContextGoalPredictor17.3626.3238.7541.2515.3528.2637.5741.53
AuxContext16.2528.4740.2843.6117.5028.8240.9044.86
Table 1: Top-1 accuracy (%) at 25/50/75/100% path reveal for belief heldout splits. AuxContext is the unbottlenecked context model trained with the same auxiliary desire/belief losses as BDIBottleneck. Underline : BToM (Informed), approximate upper bound on achievable accuracy. Bold : best result per column among learned models.
SplitPrototypesnnMean Δ\Delta95% CIPerm. ppSign test ppPairs sig.
TestModel-derived4800.151[0.145, 0.157]≤10−4{\leq}10^{-4}7.1×10−1367.1\times 10^{-136}29/30
TestHandcrafted4800.344[0.335, 0.354]≤10−4{\leq}10^{-4}3.2×10−1453.2\times 10^{-145}30/30
Test (new agent)Model-derived4800.135[0.129, 0.140]≤10−4{\leq}10^{-4}5.9×10−1385.9\times 10^{-138}27/30
Test (new agent)Handcrafted4800.333[0.324, 0.341]≤10−4{\leq}10^{-4}3.2×10−1453.2\times 10^{-145}30/30
Table 2: Desire counterfactual significance at 50% path reveal. For each episode the mean off-diagonal Δ\Delta is computed as the average change in goal probability mass toward the transplanted category across all five non-matching prototype substitutions. Permutation pp : one-sided random-sign test (H 0 : mean == 0); sign test pp : one-sided exact test (H 0 : median == 0); CI: 95% bootstrap interval. Pairs sig.: off-diagonal (source, target) pairs with mean Δ>0\Delta>0 significant at α=0.05\alpha=0.05 after Holm–Bonferroni correction (30 pairs total).
Figure 3: Desire counterfactual at 50% path reveal (2×2 panel). Top row: held-in test split; bottom row: held-out new-agent split. Left column: model-derived prototypes; right column: handcrafted category-only prototypes. Rows denote source desire category and columns denote transplanted category.
Figure 4: Belief counterfactual at 50% path reveal (rank ≤3\leq 3 , 2×2 panel). Top row: held-in test episodes; bottom row: held-out new-agent episodes. Left column: mean alternative gain. Right column: mean goal drop. Error bars: ±1\pm 1 SE.
SplitOutcomennMean95% CIPermutation ppSign test pp
TestΔalt\Delta_{\text{alt}}2060.0061[0.0007, 0.0121]0.01940.1181
TestΔgoal\Delta_{\text{goal}}206-0.0147[-0.0224, -0.0074]1.0×10−41.0\times 10^{-4}2.93×10−72.93\times 10^{-7}
Test (new agent)Δalt\Delta_{\text{alt}}2370.0055[0.0004, 0.0110]0.02380.0969
Test (new agent)Δgoal\Delta_{\text{goal}}237-0.0240[-0.0299, -0.0183]5.0×10−55.0\times 10^{-5}1.98×10−181.98\times 10^{-18}
Table 3: Counterfactual belief significance summary at 50% path (rank ≤3\leq 3 ). CI denotes 95% bootstrap confidence interval for the mean.
RegimeSplitTypennTop-1 (%)Top-5 (%)
densetestbelief28806.4 [4.7, 8.1]34.7 [29.3, 40.6]
densetestdesire288018.1 [15.4, 20.9]43.3 [40.1, 46.4]
densetest_new_agentbelief28807.7 [6.2, 9.2]36.8 [32.5, 41.0]
densetest_new_agentdesire288021.5 [18.6, 24.4]47.2 [44.2, 50.1]
sparsetestbelief24008.9 [7.4, 10.4]37.2 [33.7, 41.2]
sparsetestdesire24009.0 [7.5, 10.6]33.4 [30.5, 36.4]
Table 4: Model-planned versus simulator-resampled counterfactual agreement at 50% path reveal. Top-1 is the percentage of cases where the model-selected counterfactual goal equals the simulator-resampled goal; Top-5 is the percentage where the simulator goal appears in the model’s edited top-5. Values are means with 95% bootstrap confidence intervals clustered by seed-agent. Top-5 agreement exceeds a within-group shuffled-goal baseline in every condition (Holm-adjusted permutation p≤0.0016p\leq 0.0016 ).
Appendix figures & tables33 assets

Supplementary material from the paper’s appendix.

Appendix

SplitConfigsAgentsTotal episodes
trainB0B_{0} – B7B_{7} (8)152040
valB0B_{0} – B7B_{7} (8)15480
testB0B_{0} – B7B_{7} (8)15480
test_new_agentB0B_{0} – B7B_{7} (8)15480
test_bhBH0 , BH1 (2)15480
test_new_agent_bhBH0 , BH1 (2)15480
Appendix
Table 5: Dataset split sizes used for training and evaluation.
SettingGoalPredictorContext GoalPredictorBDIBottleneck
Epochs300300300
Batch size643232
OptimizerAdamWAdamWAdamW
Base learning rate3×10−43\times 10^{-4}3×10−43\times 10^{-4}3×10−43\times 10^{-4}
Weight decay10−410^{-4}10−410^{-4}10−410^{-4}
LR scheduleCosine annealing; ηmin⁡=η0/10\eta_{\min}=\eta_{0}/10Warmup 0, then cosine; ηmin⁡=η0/10\eta_{\min}=\eta_{0}/10Warmup 0, then cosine; ηmin⁡=η0/10\eta_{\min}=\eta_{0}/10
Appendix
Table 6: Per-model training specification used in reported experiments.
SettingAuxContext
Epochs, seeds, context300 epochs; seeds 0, 5, 10; K=10K=10 , Krecent=5K_{\text{recent}}=5
ArchitectureSame three transformer encoders as ContextGoalPredictor and BDIBottleneck ; dmodel=64d_{\text{model}}=64 , layers=3, heads=4, dim_ff=256, dropout=0.1
Goal headMLPaux([zd∥zr∥zτ])\mathrm{MLP}_{\mathrm{aux}}([z_{d}\|z_{r}\|z_{\tau}]) ; no belief/desire bottleneck in the goal path
Auxiliary headsDesire head on zdz_{d} and belief head on [zd∥zr][z_{d}\|z_{r}] , matching BDIBottleneck ’s targets
LossesGoal cross-entropy, desire KL, and weighted belief BCE with λg=1.0\lambda_{g}=1.0 , λd=1.0\lambda_{d}=1.0 , λb=5.0\lambda_{b}=5.0 and false-belief weight 5.0
Optimizer and selectionAdamW, learning rate 3×10−43\times 10^{-4} , weight decay 10−410^{-4} , cosine schedule, best checkpoint selected by validation goal top-1
Appendix
Table 7: Training specification for the matched auxiliary-supervised context baseline. Settings are intentionally aligned with BDIBottleneck ; the only architectural difference is the goal-head input.
Figure 5: BDIBottleneck validation diagnostics across 300 training epochs. Left: overall belief accuracy (all 24 entries); inflated by the majority open class. Center: false-belief F1 on believed-closed entries only; peaks at 40.4% early and converges to 33.3%; an all-open predictor achieves 0% F1, so the converged value reflects genuine false-belief discrimination. Right: desire KL divergence against ground-truth preference distribution; drops from 0.69 to 0.22, reflecting rapid and stable desire learning. Best checkpoint is selected by validation goal top-1.
SettingBDIBottleneck-NoDesireBDIBottleneck-NoBelief
Epochs300300
Context sizesKrecent=5K_{\text{recent}}=5 ; desire context KK not usedK=10K=10 ; recent context KrecentK_{\text{recent}} not used
Goal lossλg⋅\lambda_{g}\cdot cross-entropy ( λg=1.0\lambda_{g}=1.0 )λg⋅\lambda_{g}\cdot cross-entropy ( λg=1.0\lambda_{g}=1.0 )
Desire lossNot usedλd⋅\lambda_{d}\cdot KL divergence ( λd=1.0\lambda_{d}=1.0 )
Belief lossλb⋅\lambda_{b}\cdot weighted BCE ( λb=5.0\lambda_{b}=5.0 , false-belief weight =5.0=5.0 )Not used
Checkpoint criterionBest val goal top-1Best val goal top-1
Appendix
Table 8: Training settings specific to the two ablation variants. Unlisted hyperparameters match BDIBottleneck exactly (Table 6 ).
Figure 6: Top-1 goal accuracy vs. path fraction for BDIBottleneck (full), BDIBottleneck-NoDesire , and BDIBottleneck-NoBelief . Left: held-in test split. Right: held-out new-agent split. NoBelief closely tracks the full model at all fractions; NoDesire lags substantially, demonstrating that desire context is the primary driver of goal disambiguation. Shaded bands: ±1\pm 1 SD across 3 training seeds.
CategoryPOI IDs (within-category order)
attend_classPOI-01, POI-02, POI-03, POI-04
get_foodPOI-05, POI-06, POI-07, POI-08
studyPOI-09, POI-10, POI-11, POI-12
socializePOI-13, POI-14, POI-15, POI-16
exercisePOI-17, POI-18, POI-19, POI-20
rest_homePOI-21, POI-22, POI-23, POI-24
Appendix
Table 9: POI indexing used in this appendix.
ConfigSplitFalse-belief POIs (believed closed)
B0trainnone
B1trainPOI-09, POI-10, POI-11
B2trainPOI-05, POI-06, POI-07
B3trainPOI-17, POI-18, POI-19
B4trainPOI-13, POI-14, POI-15
B5trainPOI-09, POI-10, POI-05, POI-06
Appendix
Table 10: Belief configurations (POI IDs as in Table 9 ).
Characterclassfoodstudysocialexercisehome
studious_stem3.01.05.00.30.40.8
social_athlete0.52.00.44.54.00.6
foodie_socializer0.85.00.83.50.50.4
homebody_studier1.00.62.50.20.26.0
well_rounded2.02.02.02.01.51.5
Appendix
Table 11: Category-level Dirichlet parameters by character (category order: attend_class, get_food, study, socialize, exercise, rest_home).
CharacterNode alphas by category
studious_stemclass (1.5, 1.0, 3.0, 0.5); food (0.5, 2.5, 3.0, 0.5); study (1.5, 3.5, 1.0, 3.0); social (1.5, 1.0, 0.5, 1.5); exercise (1.0, 2.0, 0.5, 1.5); home (1.0, 3.0, 1.0, 1.5)
social_athleteclass (2.0, 1.0, 1.0, 0.5); food (3.5, 0.5, 0.5, 3.0); study (2.0, 0.5, 2.5, 0.5); social (2.5, 1.0, 3.5, 1.5); exercise (4.0, 1.0, 1.5, 3.5); home (3.0, 1.0, 0.5, 2.0)
foodie_socializerclass (1.0, 1.5, 1.0, 1.5); food (3.0, 2.0, 1.0, 4.0); study (2.5, 0.5, 2.0, 0.5); social (1.5, 4.0, 1.0, 4.5); exercise (1.0, 1.0, 0.5, 2.5); home (2.0, 1.0, 2.0, 1.0)
homebody_studierclass (1.0, 1.5, 1.0, 0.5); food (0.5, 4.0, 1.0, 0.5); study (3.5, 1.0, 0.5, 2.0); social (1.0, 2.0, 0.5, 1.0); exercise (0.5, 1.5, 0.5, 2.0); home (1.0, 1.0, 4.0, 2.5)
well_roundedclass (2.0, 2.0, 2.0, 1.5); food (2.0, 2.0, 2.0, 2.0); study (3.0, 1.5, 2.0, 1.5); social (2.0, 2.0, 1.5, 2.0); exercise (2.5, 2.0, 1.5, 2.0); home (2.0, 2.0, 2.0, 1.5)
Appendix
Table 12: Within-category Dirichlet parameters by character. Each tuple is a 4-vector aligned to the category’s POI order in Table 9 .
RegimeConfigurationsClosed-entry rateClosed F1Closed PR-AUC
Sparse testB0B_{0} – B7B_{7}0.1350.3900.300
Sparse new agentB0B_{0} – B7B_{7}similar sparse rate0.3960.309
Dense testB0B_{0} – B7B_{7} , BM1–BM4, BD1–BD40.298 overall0.5240.462
Dense new agentsame dense family0.298 overall0.5210.453
Appendix
Table 13: Sparse versus dense false-belief regimes. Closed-entry F1 and PR-AUC evaluate the minority believed-closed class, not the majority open class.
RegimeConfigModelTop-1Closed F1
Sparsedefault weightedAuxContext0.253±0.0130.253\pm 0.0130.399±0.0060.399\pm 0.006
Sparsedefault weightedBDIBottleneck0.196±0.0360.196\pm 0.0360.392±0.0060.392\pm 0.006
Sparsebalanced focalAuxContext0.248±0.0080.248\pm 0.0080.389±0.0040.389\pm 0.004
Sparsebalanced focalBDIBottleneck0.195±0.0520.195\pm 0.0520.387±0.0080.387\pm 0.008
Densedefault weightedAuxContext0.302±0.0090.302\pm 0.0090.524±0.0020.524\pm 0.002
Densedefault weightedBDIBottleneck0.305±0.0640.305\pm 0.0640.523±0.0030.523\pm 0.003
Appendix
Table 14: Same-budget robustness summary. Values are mean ±\pm standard deviation across seeds on the test split. The sweep is diagnostic rather than a tuned winner selection.
Figure 7: Percentage of distractor episodes where the model ranks the true goal above the distractor POI, by belief configuration. B0 is excluded (no false beliefs). The dashed line marks chance (50%). Upper panel: held-in test agents. Lower panel: new agents.
Figure 8: Belief-head sensitivity by training evidence on the test split. Bars show mean Δ\Delta per evidence bin, where Δ=P^(C∣bj=0)−P^(C∣bj=1)\Delta=\hat{P}(C\mid b_{j}{=}0)-\hat{P}(C\mid b_{j}{=}1) . Error bars denote ±1\pm 1 standard error across (agent, POI) pairs, and labels show the number of pairs in each bin.
SplitOutcomeMean95% CIPermutation ppSign test pp
TestΔalt\Delta_{\text{alt}}0.0054[-0.0009, 0.0125]0.07840.1509
TestΔgoal\Delta_{\text{goal}}-0.0122[-0.0259, 0.0013]0.05210.0592
Test (new agent)Δalt\Delta_{\text{alt}}0.0050[-0.0040, 0.0141]0.16340.5000
Test (new agent)Δgoal\Delta_{\text{goal}}-0.0233[-0.0338, -0.0130]0.00060.0037
Appendix
Table 15: Agent-clustered sensitivity analysis for counterfactual belief effects at 50% path (rank ≤3\leq 3 ). For each agent, episode-level deltas are averaged to one value per outcome ( nagents=15n_{\text{agents}}=15 per split), then one-sided tests are applied to agent-level means.
 TestTest (new agent)
Path fractionΔalt\Delta_{\text{alt}}Δgoal\Delta_{\text{goal}}Δalt\Delta_{\text{alt}}Δgoal\Delta_{\text{goal}}
25%+0.0120+0.0120−0.0200-0.0200+0.0013+0.0013−0.0229-0.0229
50%+0.0061+0.0061−0.0147-0.0147+0.0055+0.0055−0.0240-0.0240
75%+0.0040+0.0040−0.0100-0.0100+0.0017+0.0017−0.0264-0.0264
100%+0.0072+0.0072−0.0171-0.0171+0.0007+0.0007−0.0242-0.0242
Appendix
Table 16: Mean Δalt\Delta_{\text{alt}} and Δgoal\Delta_{\text{goal}} for belief counterfactual interventions at each path fraction (rank ≤3\leq 3 , n=206n=206 test / n=237n=237 new-agent episodes).
Figure 9: Belief counterfactual at 25% path reveal (rank ≤3\leq 3 , 2 ×\times 2 panel). Top row: held-in test split; bottom row: held-out new-agent split. Left column: mean alternative gain. Right column: mean goal drop. Error bars: ±1\pm 1 SE. Compare to the 50% results in Figure 4 : goal-drop magnitude is larger here on held-in agents, consistent with weaker trajectory evidence at earlier reveals.
RegimeSplitTypennTop-1 (%)Top-5 (%)p(gsim)p(g_{\mathrm{sim}})
densetestbelief96014.7 [11.2, 18.4]57.0 [50.0, 63.6]0.0902 [0.0812, 0.0989]
densetestdesire96014.0 [11.1, 17.1]50.9 [45.4, 56.8]0.0874 [0.0823, 0.0931]
densetest_new_agentbelief96011.6 [9.1, 14.3]50.9 [45.9, 55.7]0.0807 [0.0756, 0.0862]
densetest_new_agentdesire96014.6 [12.6, 16.7]55.8 [51.6, 60.2]0.0946 [0.0884, 0.1016]
sparsetestbelief48017.5 [13.3, 21.9]58.8 [52.9, 64.6]0.0859 [0.0785, 0.0935]
sparsetestdesire4809.6 [6.5, 12.9]45.6 [39.6, 52.5]0.0729 [0.0675, 0.0789]
Appendix
Table 17: Simulator-resampled full-trajectory counterfactual prediction at 50% path reveal. Values are means with 95% bootstrap confidence intervals clustered by agent. Top-1/top-5 measure agreement with the simulator-sampled counterfactual goal; p(gsim)p(g_{\mathrm{sim}}) is the mean probability assigned to that goal.
RegimeSplitTypeChannelΔorig\Delta_{\mathrm{orig}}Δsel\Delta_{\mathrm{sel}}Δsim\Delta_{\mathrm{sim}}Δsim−cf\Delta_{\mathrm{sim-cf}}
densetestbeliefpredicted-0.0112 [-0.0137, -0.0088]0.0855 [0.0765, 0.0949]-0.0870 [-0.0968, -0.0774]-0.1725 [-0.1815, -0.1641]
densetestdesirepredicted-0.0705 [-0.0836, -0.0582]0.1220 [0.1042, 0.1391]-0.0552 [-0.0719, -0.0399]-0.1772 [-0.1888, -0.1652]
densetest_new_agentbeliefpredicted-0.0095 [-0.0115, -0.0075]0.0976 [0.0883, 0.1074]-0.0653 [-0.0736, -0.0576]-0.1629 [-0.1719, -0.1543]
densetest_new_agentdesirepredicted-0.0532 [-0.0641, -0.0427]0.1467 [0.1275, 0.1647]-0.0276 [-0.0420, -0.0137]-0.1743 [-0.1880, -0.1602]
sparsetestbeliefpredicted0.0016 [0.0009, 0.0022]0.0453 [0.0430, 0.0476]-0.0234 [-0.0274, -0.0196]-0.0687 [-0.0737, -0.0642]
sparsetestdesirepredicted-0.0182 [-0.0223, -0.0148]0.0450 [0.0386, 0.0512]-0.0265 [-0.0333, -0.0202]-0.0715 [-0.0781, -0.0653]
Appendix
Table 18: Model-planned trajectory counterfactual probability deltas at 50% path reveal. Values are means with 95% bootstrap confidence intervals clustered by seed-agent. Δorig=pcf(gorig)−p(gorig)\Delta_{\mathrm{orig}}=p^{\mathrm{cf}}(g_{\mathrm{orig}})-p(g_{\mathrm{orig}}) ; Δsel=pcf(gcf)−p(gorig)\Delta_{\mathrm{sel}}=p^{\mathrm{cf}}(g_{\mathrm{cf}})-p(g_{\mathrm{orig}}) ; Δsim=pcf(gsim)−p(gorig)\Delta_{\mathrm{sim}}=p^{\mathrm{cf}}(g_{\mathrm{sim}})-p(g_{\mathrm{orig}}) ; and Δsim−cf=pcf(gsim)−pcf(gcf)\Delta_{\mathrm{sim-cf}}=p^{\mathrm{cf}}(g_{\mathrm{sim}})-p^{\mathrm{cf}}(g_{\mathrm{cf}}) , where gcfg_{\mathrm{cf}} is the model-selected counterfactual goal and gsimg_{\mathrm{sim}} is the simulator-resampled counterfactual goal.
RegimeSplitTypepadj(Δorig<0)p_{\mathrm{adj}}(\Delta_{\mathrm{orig}}<0)padj(Δsel>0)p_{\mathrm{adj}}(\Delta_{\mathrm{sel}}>0)padj(Δsim≠0)p_{\mathrm{adj}}(\Delta_{\mathrm{sim}}\neq 0)padj(Δsim−cf≠0)p_{\mathrm{adj}}(\Delta_{\mathrm{sim-cf}}\neq 0)
densetestbelief0.00320.00320.00320.0032
densetestdesire0.00320.00320.00320.0032
densetest_new_agentbelief0.00320.00320.00320.0032
densetest_new_agentdesire0.00320.00320.00320.0032
sparsetestbelief1.00000.00320.00320.0032
sparsetestdesire0.00320.00320.00320.0032
Appendix
Table 19: Cluster random-sign tests for the model-planned probability deltas in Table 18 . Values are Holm-adjusted pp -values across the displayed 50% predicted-channel tests. One-sided alternatives are used for the original-goal decrease and model-selected-goal increase; two-sided tests are used for simulator-goal deltas.
PathTestTest (new agent)
frac. (%)GoalCtxAuxBDIGoalCtxAuxBDI
106.1112.2212.8513.895.3511.3210.6311.74
2010.9718.4016.6720.499.5814.8613.4719.10
3016.4621.3921.3224.1016.6720.2818.9623.54
4023.5427.5727.2231.7423.1224.6524.3833.26
5027.9931.5331.5336.3226.4628.7527.5036.11
Appendix
Table 20: Top-1 goal inference accuracy (%) at 10% path-fraction increments. Aux is the matched auxiliary-supervised context baseline.
SplitComparison10%20%30%40%50%
testBDI vs. GP< 0.001< 0.0010.282< 0.0010.001
BDI vs. CTX1.0001.0001.0001.0001.000
test_new_agentBDI vs. GP0.001< 0.0010.276< 0.001< 0.001
BDI vs. CTX1.0001.0000.9191.0001.000
Appendix
Table 21: McNemar’s exact test (BDIBottleneck vs. baselines) at 10% path-fraction increments. Holm–Bonferroni adjusted pp -values across 40 tests. Bold = significant at α=0.05\alpha=0.05 . Top panel: 10–50%; bottom panel: 60–100%.
SplitComparisonFracnnn01n_{01}n10n_{10}ppp∗p^{*}
testBDI vs. GoalPredictor10%4801551< 0.001< 0.001
20%4801969< 0.001< 0.001
30%48044700.0190.282
40%4803990< 0.001< 0.001
50%4804086< 0.0010.001
60%4803697< 0.001< 0.001
Appendix
Table 22: Full McNemar’s exact test results at 10% path-fraction increments. n01n_{01} : BDI correct & baseline wrong; n10n_{10} : baseline correct & BDI wrong; pp : raw two-sided pp -value; p∗p^{*} : Holm–Bonferroni adjusted (40 tests total). Bold = significant at α=0.05\alpha=0.05 .
Figure 10: Top-5 goal accuracy vs. path fraction on the held-in test split for the three learned models.
Figure 11: Top-5 goal accuracy vs. path fraction on the held-out new-agent split. The generalization gap between test and new-agent performance narrows relative to the top-1 setting, consistent with the relaxed identification requirement.
Figure 12: Full four-fraction desire counterfactual delta matrices, model-derived prototypes, held-in test split. Panels correspond to 25%, 50%, 75%, and 100% path reveal (left to right). Rows are source desire categories; columns are transplanted categories; each cell is the mean change in category-level goal probability relative to the no-swap baseline.
Figure 13: Full four-fraction desire counterfactual delta matrices, model-derived prototypes, held-out new-agent split. The off-diagonal structure generalizes across held-out agents, indicating that the desire channel’s influence on goal prediction is not agent-specific.
Figure 14: Full four-fraction desire counterfactual delta matrices, handcrafted category-only prototypes, held-in test split. Handcrafted prototypes place uniform mass over the four POIs in the target category and zero elsewhere, producing a sharper desire signal than model-derived prototypes and correspondingly larger delta magnitudes.
Figure 15: Full four-fraction desire counterfactual delta matrices, handcrafted category-only prototypes, held-out new-agent split. The stronger category signal of handcrafted prototypes produces consistently larger shifts than model-derived prototypes across both splits and all fractions.
Figure 16: Model-derived prototype distributions on the held-in test split. Each panel shows the 24-POI mean desire vector for one target category, with POIs grouped and color-coded by category. Non-trivial mass outside the target category reflects the smoothness of individual desire estimates.
Figure 17: Model-derived prototype distributions on the held-out new-agent split. The within-category concentration is comparable to the test split, indicating that the desire encoder learns category-consistent representations that generalize to unseen agents.
Figure 18: Handcrafted category-only prototype distributions on the held-in test split. Each prototype assigns uniform mass over the four POIs in the target category and zero mass elsewhere, providing a sharp, idealized desire signal for comparison with model-derived prototypes.
Figure 19: Handcrafted category-only prototype distributions on the held-out new-agent split. Identical in construction to Figure 18 ; shown separately to confirm that prototype construction is split-invariant by design.
More in this topic

Theory of Mind

Theory of Mind studies how people and AI infer others’ beliefs, goals, and knowledge to predict behavior and coordinate interaction.

See all 95 papers

Explore similar work

CardsList

  1. Teleological Inference in Structural Causal Models via Intentional Interventions

    Mar 19, 2026Dario Compagno, Fabio Massimo ZennaroStructural Causal ModelsCausal Intervention

  2. DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories

    Apr 22, 2026Neemesh Yadav, Palakorn Achananuparp, Jing Jiang +1LLM EvaluationTheory of Mind

    Singapore Management University · Australian National University

  3. The Theory of Mind Utility: Formal Specification of a Mentalizing Mechanism

    Jun 10, 2026Nikolos Gurney, Stacy MarsellaComputational Cognitive ModelingTheory of Mind

    Institute for Creative Technologies, University of Southern California California, United States of America · Khoury College of Computer Sciences, Northeastern University Massachusetts, United States of America