Pragmatic Curiosity: A Unified Framework for Hybrid Learning and Optimization via Active Inference
Organizations: Department of Aeronautics and Astronautics Massachusetts Institute of Technology · School of Industrial and Systems Engineering Georgia Institute of Technology
Abstract
Many engineering and scientific workflows rely on expensive black-box evaluations, requiring sequential decisions that must both improve task performance and reduce uncertainty. Bayesian optimization (BO) and Bayesian experimental design (BED) provide powerful but largely separate treatments of goal-directed optimization and information-seeking experimentation, leaving limited guidance for hybrid settings in which learning and optimization are intrinsically coupled. We propose Pragmatic Curiosity (PraC), a unified framework for hybrid learning and optimization via active inference. PraC evaluates candidate queries by trading information gain about a task-relevant latent symbol against an expected regret-based potential over outcomes. This formulation foregrounds three operational design choices: which latent quantity should be clarified, how task value is encoded as regret, and how strongly information gain should be exchanged against pragmatic regret. We instantiate PraC across three regimes of increasing complexity: decision-oriented monitoring with fixed global symbols and known downstream losses, targeted active search with induced local symbols and evolving coverage goals, and composite Bayesian optimization with hierarchical regret learning under unknown preferences. Across these regimes, PraC reduces downstream decision risk, improves coverage of critical outcome regions, and jointly learns predictive and preference structures without relying on task-specific staging rules.
Figures & tables
| Acquisition Strategy | Acquisition Function | Pragmatic Curiosity | |
|---|---|---|---|
| Expected Information Gain ( Chaloner and Verdinelli, 1995 ) | - | ||
| Entropy Search ( Hennig and Christian J. Schuler, 2012 ) | - | ||
| Max-value Entropy Search ( Wang and Jegelka, 2017 ) | - | ||
| Joint Entropy Search ( Hvarfner et al., 2022 ) | - | ||
| Bayesian Algorithm Execution ( Neiswanger et al., 2021 ) | - |
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Center | Prior weight | Prior scale | Consequence weight | Consequence scale |
|---|---|---|---|---|
| 3.2 | 13 | 0.4 | 18 | |
| 0.9 | 16 | 4.0 | 16 | |
| 1.7 | 14 | 0.8 | 16 |
| Source | Location | True | ||||||
|---|---|---|---|---|---|---|---|---|
| 1 | 12 | 30 | 0.05 | 10 | no | |||
| 2 | 12 | 30 | 10 | no | ||||
| 3 | 12 | 36 | 0.05 | 0.10 | 12 | no | ||
| 4 | 12 | 34 | 0.10 | 12 | no | |||
| 5 | 4 | 55 | 0.05 | 18 | yes | |||
| 6 | 4 | 55 | 18 | yes |
| Domain | Hypotheses | Actions | Candidates | Budget |
|---|---|---|---|---|
| Response localization | 1,600 | 25 | 400 | 8 |
| Consequence-Weighted dispatch | 1,600 | 25 | 400 | 8 |
| Leak Repair Prioritization | 63 | 15 | 400 | 8 |
| Method | Bayes-risk AUC | Final Bayes risk | Response loss | Response success | MAP error |
|---|---|---|---|---|---|
| PraC dynamic beta | |||||
| PraC fixed beta | |||||
| IDS | |||||
| Greedy | |||||
| EIG | |||||
| Random |
| Method | Bayes-risk AUC | Final Bayes risk | Response loss | Response success | MAP error |
|---|---|---|---|---|---|
| PraC dynamic beta | |||||
| PraC fixed beta | |||||
| IDS | |||||
| Greedy | |||||
| EIG | |||||
| Random |
| Method | Bayes-risk AUC | Final Bayes risk | Missed risk | Top-2 recall | Weighted recall |
|---|---|---|---|---|---|
| PraC dynamic beta | |||||
| PraC fixed beta | |||||
| IDS | |||||
| Greedy | |||||
| EIG | |||||
| Random |
| Application | Baseline | AUC | 95% paired -interval | Wins/ties | ||
|---|---|---|---|---|---|---|
| Response localization | PraC fixed beta | 18/0 | ||||
| Response localization | IDS | 18/0 | ||||
| Response localization | Greedy | 19/0 | ||||
| Response localization | EIG | 18/0 | ||||
| Response localization | Random | 20/0 | ||||
| Risk-aware dispatch | PraC fixed beta | 19/0 |
| Dynamic PraC | Fixed PraC | ||||||
|---|---|---|---|---|---|---|---|
| Task | Within 5% | Within 5% | |||||
| Response localization | 4/9 (44%) | 1/5 (20%) | |||||
| Risk-aware dispatch | 4/9 (44%) | 1/5 (20%) | |||||
| Leak repair | 4/9 (44%) | 1/5 (20%) | |||||
| Across tasks | 12/27 (44%) | none | 3/15 (20%) | ||||
| Domain | Target oracle | Pool | Targets | Target families | Hidden families | Initialization | Budget |
|---|---|---|---|---|---|---|---|
| BACE scaffold hopping | 1,040 | 132 | 32 | 31 | 1 target + 4 non-targets in anchor scaffold | 50 | |
| 3DSC-95K prototype hopping | K | 672 | 30 | 6 | 5 | 1 target + 4 non-targets in anchor prototype | 50 |
| MVTec cable mechanism coverage | Membership in four actionable types | 150 | 44 | 4 | 2 | 2 known-type targets + 2 good + 2 hard negatives | 30 |
| Method | Primary AUC | Hidden macro recall | Hidden families | Hidden targets | Anchor recall |
|---|---|---|---|---|---|
| PraC | |||||
| IDS | |||||
| Greedy | |||||
| EIG | |||||
| Random |
| Method | Primary AUC | Hidden macro recall | Hidden families | Hidden targets | Anchor recall |
|---|---|---|---|---|---|
| PraC | |||||
| IDS | |||||
| Greedy | |||||
| EIG | |||||
| Random |
| Method | Targeted AUC | Hidden-type AUC | Final targeted macro | Actionable hits | Hidden types covered |
|---|---|---|---|---|---|
| PraC | |||||
| IDS | |||||
| Greedy | |||||
| EIG | |||||
| Random |
| Domain | Baseline | AUC | 95% paired -interval | Wins/ties | ||
|---|---|---|---|---|---|---|
| BACE | Greedy | 19/0 | ||||
| BACE | IDS | 10/0 | ||||
| BACE | EIG | 19/0 | ||||
| BACE | Random | 19/0 | ||||
| 3DSC-95K | Greedy | 10/1 | ||||
| 3DSC-95K | IDS | 13/0 |
| Outcome | Operational definition in the experiment | Motivation |
|---|---|---|
| Voltage fairness | Normalized network-wide voltage-uniformity score, constructed monotonically from across-bus voltage dispersion; values nearer one indicate more uniform delivery. | Complements OPF’s explicit bus-voltage feasibility constraints by distinguishing feasible solutions with different spatial uniformity. |
| Total cost | DER capital cost plus reactive-power-support cost. | Frozen coefficients are DER_INSTALL_COST and Q_COST_FACTOR ; additive cost is consistent with the pandapower OPF formulation. |
| Priority service | Weighted service score at designated priority buses. | Frozen priority-bus indices are and the priority multiplier is . This criterion represents an explicit policy preference, not an electrical feasibility constraint. |
| Resilience | Fraction of simulator perturbations for which all bus voltages remain inside the safe operating interval. | A scenario-based reliability surrogate: it evaluates the design under load / network perturbations rather than only at the nominal OPF point ( Watson et al., 2015 ) . |
| Application | Seeds | Acquisition optimizer | ||||||
|---|---|---|---|---|---|---|---|---|
| Vehicle safety | 5 | 3 | 5 | 1 | 2 | 80 | 0–19 | 10 restarts; 100 raw samples |
| Penicillin production | 7 | 3 | 5 | 1 | 2 | 100 | 0–19 | 10 restarts; 100 raw samples |
| Energy allocation | 10 | 4 | 10 | 10 | 2 | 95 | 0–19 | 10 restarts; 100 raw samples |
| Domain | Method | Final | Anytime AUC | |
|---|---|---|---|---|
| Vehicle Safety | Random | 20 | -2.783\text{\times}{10}^{04}$\,\pm\,$143 | -2.829\text{\times}{10}^{04}$\,\pm\,$129 |
| EIG-only | 20 | -2.766\text{\times}{10}^{04}$\,\pm\,$97.2 | -2.817\text{\times}{10}^{04}$\,\pm\,$109 | |
| PraC w/o IG | 20 | -2.565\text{\times}{10}^{04}$\,\pm\,$0.852 | -2.714\text{\times}{10}^{04}$\,\pm\,$96.7 | |
| PraC w/o outcome IG | 20 | -2.577\text{\times}{10}^{04}$\,\pm\,$83.4 | -2.718\text{\times}{10}^{04}$\,\pm\,$147 | |
| PraC full | 20 | -2.574\text{\times}{10}^{04}$\,\pm\,$68.5 | -2.715\text{\times}{10}^{04}$\,\pm\,$134 | |
| Penicillin Production | Random | 20 | -3.35\text{\times}{10}^{05}$\,\pm\,$2.93\text{\times}{10}^{03} | -3.409\text{\times}{10}^{05}$\,\pm\,$1.57\text{\times}{10}^{03} |
| Method | Final best | Best AUC | Final selected |
|---|---|---|---|
| BOPE-I | 6.9$\,\pm\,$0.1 | 6.634$\,\pm\,$0.142 | 5.1$\,\pm\,$0.748 |
| BOPE-II | 6.3$\,\pm\,$0.2 | 6.037$\,\pm\,$0.213 | 4.6$\,\pm\,$0.292 |
| BOPE-III | 6$\,\pm\,$0.447 | 5.543$\,\pm\,$0.465 | 4.3$\,\pm\,$0.122 |
| BOPE-IV | 5.8$\,\pm\,$0.339 | 5.595$\,\pm\,$0.355 | 4.8$\,\pm\,$0.406 |
| PraC | 7$\,\pm\,$0 | 6.248$\,\pm\,$0.216 | 5.4$\,\pm\,$0.992 |
| Random | 5.1$\,\pm\,$0.187 | 4.6$\,\pm\,$0.0965 | 2.8$\,\pm\,$0.374 |
| Application | Comparator | AUC [95% CI] | ||
|---|---|---|---|---|
| Vehicle Safety | Random | 20 | 838.81441 | |
| EIG-only | 20 | 725.71305 | ||
| PraC w/o IG | 20 | -349.6323.7 | ||
| PraC w/o outcome IG | 20 | -376.8419.9 | ||
| Penicillin Production | Random | 20 | -39537359 | |
| EIG-only | 20 | -28257717 |
Bayesian Optimization
Bayesian optimization uses probabilistic models and sequential experiments to find strong solutions when evaluating a black-box objective is costly.
See all 276 papers