Decision-Making under Uncertainty

Follow this topic Ask about this topic

Decision-making under uncertainty studies how agents should choose actions when outcomes, states, preferences, or even the governing model are imperfectly known. Common approaches include Bayesian beliefs and partially observable Markov decision processes for sequential learning, robust optimization for misspecified distributions, risk-sensitive criteria such as CVaR, and conformal methods that calibrate when to act, gather information, or abstain. These methods are tested in settings such as autonomous driving and robotics, clinical diagnosis, energy and supply-chain scheduling, finance, and human–AI interaction. A central tension is that better-calibrated or more conservative uncertainty estimates do not automatically improve decisions: methods must balance safety and tail risk against utility, computational cost, and the value of additional information.

Latest papers

203 papers24 papers in the last four weeks, up 26% on the four weeks before. 0.1% of all new papers.

Mar 10, 2026

Sim2Act: Robust Simulation-to-Decision Learning via Adversarial Calibration and Group-Relative Perturbation

Simulation-to-decision learning enables safe policy training in digital environments without risking real-world deployment, and has become essential in mission-critical domains such as supply chains and industrial systems. However, simulators learned from noisy or biased real-world data often exhibit prediction errors in decision-critical regions, leading to unstable action ranking and unreliable policies. Existing approaches either focus on improving average simulation fidelity or adopt conservative regularization, which may cause policy collapse by discarding high-risk high-reward actions. We propose Sim2Act, a robust simulation-to-decision framework that addresses both simulator and policy robustness. First, we introduce an adversarial calibration mechanism that re-weights simulation errors in decision-critical state-action pairs to align surrogate fidelity with downstream decision impact. Second, we develop a group-relative perturbation strategy that stabilizes policy learning under simulator uncertainty without enforcing overly pessimistic constraints. Extensive experiments on multiple supply chain benchmarks demonstrate improved simulation robustness and more stable decision performance under structured and unstructured perturbations.

Hongyu Cao, Jinghan Zhang, Kunpeng Liu +5

Arizona State University · Clemson University · University of Kansas +3

Robust RL Decision-Making under Uncertainty

Mar 9, 2026

Efficient Policy Learning with Hybrid Evaluation-Based Genetic Programming for Uncertain Agile Earth Observation Satellite Scheduling

The Uncertain Agile Earth Observation Satellite Scheduling Problem (UAEOSSP) is a novel combinatorial optimization problem and a practical engineering challenge that aligns with the current demands of space technology development. It incorporates uncertainties in profit, resource consumption, and visibility, which may render pre-planned schedules suboptimal or even infeasible. Genetic Programming Hyper-Heuristic (GPHH) shows promise for evolving interpretable scheduling policies; however, their simulation-based evaluation incurs high computational costs. Moreover, the design of the constructive method, denoted as Online Scheduling Algorithm (OSA), directly affects fitness assessment, resulting in evaluation-dependent local optima within the policy space. To address these issues, this paper proposes a Hybrid Evaluation-based Genetic Programming (HE-GP) for effectively solving UAEOSSP. A Hybrid Evaluation (HE) mechanism is integrated into the policy-driven OSA, combining exact and approximate filtering modes: exact mode ensures evaluation accuracy through elaborately designed constraint verification modules, while approximate mode reduces computational overhead via simplified logic. HE-GP dynamically switches between evaluation models based on real-time evolutionary state information. Experiments on 16 simulated instance sets demonstrate that HE-GP significantly outperforms handcrafted heuristics and single-evaluation based GPHH, achieving substantial reductions in computational cost while maintaining excellent scheduling performance across diverse scenarios. Specifically, the average training time of HE-GP was reduced by 17.77% compared to GP employing exclusively exact evaluation, while the optimal policy generated by HE-GP achieved the highest average ranks across all scenarios.

Junhua Xue, Yuning Chen, Mingyan Shao +3

Genetic Programming Automated Heuristic Design Decision-Making under Uncertainty Remote Sensing

Mar 9, 2026

Sequential Service Region Design with Capacity-Constrained Investment and Spillover Effect

Service region design determines the geographic coverage of service networks, shaping long-term operational performance. Capital and operational constraints preclude simultaneous large-scale deployment, requiring expansion to proceed sequentially. The resulting challenge is to determine when and where to invest under demand uncertainty, balancing intertemporal trade-offs between early and delayed investment and accounting for network effects whereby each deployment reshapes future demand through inter-regional connectivity. This study addresses a sequential service region design (SSRD) problem incorporating two practical yet underexplored factors: a

k

-region constraint that limits the number of regions investable per period and a stochastic spillover effect linking investment decisions to demand evolution. The resulting problem requires sequencing regional portfolios under uncertainty, leading to a combinatorial explosion in feasible investment sequences. To address this challenge, we propose a solution framework that integrates real options analysis (ROA) with a Transformer-based Proximal Policy Optimization (TPPO) algorithm. ROA evaluates the intertemporal option value of investment sequences, while TPPO learns sequential policies that directly generate high option-value sequences without exhaustive enumeration. Numerical experiments on realistic multi-region settings demonstrate that TPPO converges faster than benchmark DRL methods and consistently identifies sequences with superior option value. Case studies and sensitivity analyses further confirm robustness and provide insights on investment concurrency, regional prioritization, and the increasing benefits of adaptive expansion via our approach under stronger spillovers and dynamic market conditions.

Tingting Chen, Feng Chu, Jiantong Zhang

School of Economics and Management, Tongji University, Shanghai, 200092, China · Laboratoire IBISC, Univ-Evry, Université Paris-Saclay, Evry, 91000, France

Sequential Decision Making Decision-Making under Uncertainty

Mar 4, 2026

Risk-Aware Rulebooks for Multi-Objective Trajectory Evaluation under Uncertainty

We present a risk-aware formalism for evaluating system trajectories in the presence of uncertain interactions between the system and its environment. The proposed formalism supports reasoning under uncertainty and systematically handles complex relationships among requirements and objectives, including hierarchical priorities and non-comparability. Rather than treating the environment as exogenous noise, we explicitly model how each system trajectory influences the environment and evaluate trajectories under the resulting distribution of environment responses. We prove that the formalism induces a preorder on the set of system trajectories, ensuring consistency and preventing cyclic preferences. Finally, we illustrate the approach with an autonomous driving example that demonstrates how the formalism enhances explainability by clarifying the rationale behind trajectory selection.

Tichakorn Wongpiromsarn

Department of Computer Science, Iowa State University, Ames, IA 50010

Risk-Aware Path Planning Decision-Making under Uncertainty

Mar 3, 2026

What Capable Agents Must Know: Selection Theorems for Robust Decision-Making under Uncertainty

As artificial agents become increasingly capable, what internal structure is necessary for an agent to act competently under uncertainty? Classical results show that optimal control can be implemented using belief states or world models, but not that such representations are required. We prove quantitative "selection theorems" showing that strong task performance (low average-case regret) forces world models, belief-like memory and -- under task mixtures -- persistent regime-tracking variables resembling functional primitives of emotion, along with informational modularity under block-structured tasks. Our results cover stochastic policies, partial observability, and evaluation under task distributions, without assuming optimality, determinism, or access to an explicit model. Technically, we reduce predictive modeling to binary "betting" decisions and show that regret bounds limit probability mass on suboptimal bets, enforcing the predictive distinctions needed to separate high-margin outcomes. In fully observed settings, this yields approximate recovery of the interventional transition kernel; under partial observability, it implies necessity of predictive state and belief-like memory, addressing an open question in prior world-model recovery work.

Aran Nayebi

Machine Learning Department and Neuroscience & Robotics Institutes, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA

World Models Decision-Making under Uncertainty Predictive State Representations

Feb 24, 2026

When Alpha Breaks: Two-Level Uncertainty for Safe Deployment of Cross-Sectional Stock Rankers

Cross-sectional ranking models are often deployed as if point predictions were sufficient: the model outputs scores and the portfolio follows the induced ordering. Under non-stationarity, rankers can fail during regime shifts. In the AI Stock Forecaster, a LightGBM ranker performs well overall at a 20-day horizon, yet the 2024 holdout coincides with an AI thematic rally and sector rotation that breaks the signal at longer horizons and weakens 20d. This motivates treating deployment as two decisions: (i) whether the strategy should trade at all, and (ii) how to control risk within active trades. We adapt Direct Epistemic Uncertainty Prediction (DEUP) to ranking by predicting rank displacement and defining an epistemic uncertainty signal ehat relative to a point-in-time (PIT-safe) baseline. Empirically, ehat is structurally coupled with signal strength (median correlation between ehat and absolute score is about 0.6 across 1,865 dates), so inverse-uncertainty sizing de-levers the strongest signals and degrades performance. To address this, we propose a two-level deployment policy: a strategy-level regime-trust gate G(t) that decides whether to trade (AUROC around 0.72 overall and 0.75 in FINAL) and a position-level epistemic tail-risk cap that reduces exposure only for the most uncertain predictions. The operational policy, trade only when G(t) is at least 0.2, apply volatility sizing on active dates, and cap the top epistemic tail, improves risk-adjusted performance in the 20d policy comparison and indicates DEUP adds value mainly as a tail-risk guard rather than a continuous sizing denominator.

Ursina Sanderink

Quantitative Finance Decision-Making under Uncertainty Epistemic Uncertainty Learning to Rank

Feb 24, 2026

A Dynamic Survey of Soft Set Theory and Its Extensions

Soft set theory provides a direct framework for parameterized decision modeling by assigning to each attribute (parameter) a subset of a given universe, thereby representing uncertainty in a structured way [1, 2]. Over the past decades, the theory has expanded into numerous variants-including hypersoft sets, superhypersoft sets, TreeSoft sets, bipolar soft sets, and dynamic soft sets-and has been connected to diverse areas such as topology and matroid theory. In this book, we present a survey-style overview of soft sets and their major extensions, highlighting core definitions, representative constructions, and key directions of current development.

Takaaki Fujita, Florentin Smarandache

Decision-Making under Uncertainty

Feb 24, 2026

POMDPPlanners: Open-Source Package for POMDP Planning

We present POMDPPlanners, an open-source Python package for empirical evaluation of Partially Observable Markov Decision Process (POMDP) planning algorithms. The package integrates state-of-the-art planning algorithms, a suite of benchmark environments with safety-critical variants, automated hyperparameter optimization via Optuna, persistent caching with failure recovery, and configurable parallel simulation -- reducing the overhead of extensive simulation studies. POMDPPlanners is designed to enable scalable, reproducible research on decision-making under uncertainty, with particular emphasis on risk-sensitive settings where standard toolkits fall short.

Yaacov Pariente, Vadim Indelman

Faculty of Mathematics, Technion - Israel Institute of Technology, Haifa 3200003, Israel · Stephen B. Klein Faculty of Aerospace Engineering, Faculty of Data and Decision Sciences, Technion - Israel Institute of Technology, Haifa 3200003, Israel

Partially Observable Markov Decision Processes Decision-Making under Uncertainty Belief-Space Planning

Feb 23, 2026

Wasserstein Distributionally Robust Online Learning

We study online Wasserstein distributionally robust optimization (DRO), where the ambiguity set is a Wasserstein ball centered at the empirical distribution and evolves with each observation. We formulate each round as a zero-sum game and develop an online distributional best-response algorithm that requires only one projected subgradient step per iteration. Combining regret analysis with measure concentration, we establish explicit, instance-dependent, and high-probability convergence rates to the optimal offline DRO value.

Guixian Chen, Salar Fattahi, Soroosh Shafiee

University of Michigan · Cornell University

Distributionally Robust Optimization Decision-Making under Uncertainty Wasserstein Distributionally Robust Optimization Stochastic Optimization

Feb 18, 2026

Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents

LLM agents are deployed in environments where they must interact to acquire information. In these scenarios, the agent must reason about inherent cost-uncertainty tradeoffs in how to act, such as when to stop exploring and commit to an answer. For instance, on a programming task, an agent might run the code it generates, or it might generate tests for that code snippet; the cost of writing and running a test is nonzero, but typically lower than the cost of running buggy code. In this work, we show that we can induce LLM agents to explicitly reason about balancing these cost-uncertainty tradeoffs, then act more optimally in their environments. We formalize multiple tasks, including retrieval-augmented QA and a file reading coding task, as sequential decision-making problems under uncertainty. Each problem has latent environment state that impacts the agent's performance. We introduce a framework called Calibrate-Then-Act (CTA), where we pass the agent an inferred prior about this environment state to enable it to act more optimally. This information qualitatively changes agent behavior, and adds environment sensitivity to the agent which is not learned via standard RL training. Our results on a synthetic task, QA, and file reading show that making cost-benefit tradeoffs explicit with CTA helps agents discover more optimal decision-making strategies.

Wenxuan Ding, Nicholas Tomlin, Greg Durrett

New York University

LLM Agents Decision-Making under Uncertainty LLM Agent Planning

Feb 16, 2026

AI Arms and Influence: Frontier Models Exhibit Sophisticated Reasoning in Simulated Nuclear Crises

Today's leading AI models engage in sophisticated behaviour when placed in strategic competition. They spontaneously attempt deception, signaling intentions they do not intend to follow; they demonstrate rich theory of mind, reasoning about adversary beliefs and anticipating their actions; and they exhibit credible metacognitive self-awareness, assessing their own strategic abilities before deciding how to act. Here we present findings from a crisis simulation in which three frontier large language models (GPT-5.2, Claude Sonnet 4, Gemini 3 Flash) play opposing leaders in a nuclear crisis. Our simulation has direct application for national security professionals, but also, via its insights into AI reasoning under uncertainty, has applications far beyond international crisis decision-making. Our findings both validate and challenge central tenets of strategic theory. We find support for Schelling's ideas about commitment, Kahn's escalation framework, and Jervis's work on misperception, inter alia. Yet we also find that the nuclear taboo is no impediment to nuclear escalation by our models; that strategic nuclear attack, while rare, does occur; that threats more often provoke counter-escalation than compliance; that high mutual credibility accelerated rather than deterred conflict; and that no model ever chose accommodation or withdrawal even when under acute pressure, only reduced levels of violence. We argue that AI simulation represents a powerful tool for strategic analysis, but only if properly calibrated against known patterns of human reasoning. Understanding how frontier models do and do not imitate human strategic logic is essential preparation for a world in which AI increasingly shapes strategic outcomes.

Kenneth Payne

King’s College London

Strategic Reasoning Risks in Language Models Large Language Model-Based Social Simulation Decision-Making under Uncertainty Deception in Language Models

Feb 16, 2026

Multimodal Belief-Space Covariance Steering with Active Probing and Influence for Interactive Driving

Autonomous driving in complex traffic requires reasoning under uncertainty. Common approaches rely on prediction-based planning or risk-aware control, but these are typically treated in isolation, limiting their ability to capture the coupled nature of action and inference in interactive settings. This gap becomes especially critical in uncertain scenarios, where simply reacting to predictions can lead to unsafe maneuvers or overly conservative behavior. Our central insight is that safe interaction requires not only estimating human behavior but also shaping it when ambiguity poses risks. To this end, we introduce a hierarchical belief model that structures human behavior across coarse discrete intents and fine motion modes, updated via Bayesian inference for interpretable multi-resolution reasoning. On top of this, we develop an active probing strategy that identifies when multimodal ambiguity in human predictions may compromise safety and plans disambiguating actions that both reveal intent and gently steer human decisions toward safer outcomes. Finally, a runtime risk-evaluation layer based on Conditional Value-at-Risk (CVaR) ensures that all probing actions remain within human risk tolerance during influence. Our simulations in lane-merging and unsignaled intersection scenarios demonstrate that our approach achieves higher success rates and shorter completion times compared to existing methods. These results highlight the benefit of coupling belief inference, probing, and risk monitoring, yielding a principled and interpretable framework for planning under uncertainty.

Devodita Chakravarty, John Dolan, Yiwei Lyu

Department of Mechanical Engineering, Indian Institute of Technology Kharagpur, India · Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA · Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA

Risk-Sensitive Planning Autonomous Driving Planning Decision-Making under Uncertainty Belief-Space Planning Active Perception

Jan 30, 2026

Uncertainty-aware Causal Decision Making via Effect Bound Decomposition

Causal inference from observational data can provide strong evidence for finding the best action in a decision-making scenario without having to perform expensive randomized trials. The causal effect of an action is often not pointwise identifiable even with infinite data due to unobserved confounding factors. Furthermore, having only finitely many samples adds another layer of uncertainty to causal effect estimation. Several existing methods can be used to obtain upper and lower bounds to the causal effect, ranging from symbolic methods to the more recent neural network-based approaches, which implicitly incorporate both sources of uncertainty. However, these methods do not inform whether collecting more samples may or may not help identify the best action from observational data, leaving experts in the dark about their data collection strategies. We address this problem with a novel framework that can distinguish the range of causal effect values that might be eliminated by collecting more samples from the range of values that, with high probability, cannot be eliminated with more observational samples. We show that this partitioning can be obtained by solving max-min and min-max optimization problems. We leverage neural causal models to approximately recover this decomposition in practice. We demonstrate via experiments on synthetic and real-world datasets that our algorithm can determine when collecting more samples will not help determine the best action. Our framework can help practitioners decide when to resort to non-observational studies or seek to measure some of the unmeasured confounders for optimal decision-making.

Md Musfiqur Rahman, Ziwei Jiang, Hilaf Hasson +1

Electrical and Computer Engineering, Purdue University · Computer Science, Johns Hopkins University · Cohesity

Causal Effect Estimation Parameter Identifiability Partial Identification Causal Identifiability Decision-Making under Uncertainty Causal Inference with Unobserved Confounding

Nov 30, 2025

Bayesian dynamic scheduling of multipurpose batch processes under incomplete look-ahead information

Multipurpose batch processes become increasingly popular in manufacturing industries since they adapt to low-volume, high-value products and shifting demands. These processes often operate in a dynamic environment, which faces disturbances such as processing delays and demand changes. To minimise long-term cost and system nervousness (i.e., disruptive changes to schedules), schedulers must design rescheduling strategies to address such disturbances effectively. Existing methods often assume complete look-ahead information over the scheduling horizon. This assumption contrasts with realistic situations where schedulers can only access incomplete look-ahead information. Sticking with existing methods may lead to suboptimal long-term costs and high-level system nervousness. In this work we propose a Bayesian dynamic scheduling method. Our method relies on learning a Bayesian Network from the probability distribution of disturbances. Specifically, the Bayesian Network represents how likely each operation will be impacted by disturbances. During the online execution, when new disturbances become observed, this method updates the posterior distribution and therefore guides the rescheduling strategy. We compare our method with the existing periodic rescheduling strategy (which generates new schedules from scratch at fixed intervals) on four benchmark problems. Computational results show that our method achieves statistically better long-term costs and system nervousness. In the theoretical aspect, we prove that if disturbances are mutually independent, the impact-quantifying variables inherently satisfy the independence assumptions required by Bayesian Networks. As an implication, practitioners can extend the method to other scheduling problems (such as job shop scheduling and continuous processes), given that they define the problem-specific dependencies between operations.

Taicheng Zheng, Dan Li, Jie Li

Centre for Process Integration, the Department of Chemical Engineering, The University of Manchester, Oxford Road, Manchester, M13 9SS, United Kingdom

Decision-Making under Uncertainty

Oct 13, 2025

People use fast and flat simulation to reason about new games

Games have long been a microcosm for studying planning and reasoning in both natural and artificial intelligence (AI), often focusing on expert-level or even super-human play. But real life also pushes human intelligence along a different frontier, requiring people to flexibly navigate decision-making problems that they have never thought about before. Here, we use novice gameplay to study how people reason about new problem settings. Through a series of large-scale behavioral studies with over 1000 participants and 121 two-player strategic board games (almost all novel to our participants), we show that people are systematic and adaptively rational in how they play a game for the first time, or evaluate a game (e.g., how fair or how fun it is likely to be) before they have played it even once. We explain these capacities via a computational cognitive model that we call the 'Intuitive Gamer', a model based on mechanisms of fast and flat (depth-limited) goal-directed probabilistic simulation. Our work offers new insights into how people rapidly evaluate, act, and make suggestions when encountering novel problems, and could inform the design of more flexible and human-like AI systems that can determine not just how to solve new tasks, but whether a task is worth thinking about at all.

Katherine M. Collins, Cedegao E. Zhang, Lionel Wong +6

Massachusetts Institute of Technology, Cambridge, MA, United States · Princeton University, Princeton, NJ, United States · University of Cambridge, Cambridge, United Kingdom +3

Computational Cognitive Modeling Human Behavior Simulation Bounded Rationality Decision-Making under Uncertainty

Aug 28, 2025

Meta-learning ecological priors from large language models explains human learning and decision making

Human cognition is profoundly shaped by the environments in which it unfolds. Yet, it remains an open question whether learning and decision making can be explained as a principled adaptation to the statistical structure of real-world tasks. We introduce ecologically rational analysis, a computational framework that unifies the normative foundations of rational analysis with ecological grounding. Leveraging large language models to generate ecologically valid cognitive tasks at scale, and using meta-learning to derive rational models optimized for these environments, we develop a new class of learning algorithms: Ecologically Rational Meta-learned Inference (ERMI). ERMI internalizes the statistical regularities of naturalistic problem spaces and adapts flexibly to novel situations, without requiring hand-crafted heuristics or explicit parameter updates. We show that ERMI captures human behavior across 15 experiments spanning function learning, category learning, and decision making, outperforming several established cognitive models in trial-by-trial prediction. Our results suggest that much of human cognition may reflect adaptive alignment to the ecological structure of the problems we encounter in everyday life.

Akshay K. Jagadish, Mirko Thalmann, Julian Coda-Forno +2

Institute for Human-Centered AI, Helmholtz Computational Health Center, Munich, Germany · Computational Principles of Intelligence, Max Planck Institute for Biological Cybernetics, Tübingen, Germany · Eberhard Karls University of Tübingen, Tübingen, Germany +1

Computational Cognitive Modeling Meta-Learning Decision-Making under Uncertainty

Aug 24, 2025

The price of uncertainty for social consensus

How hard is it to achieve consensus in a social network under uncertainty? In this paper we model this problem as a social graph of agents where each vertex is initially colored red or blue. The goal of the agents is to achieve consensus, which is when the colors of all agents align. Agents attempt to do this locally through steps in which an agent changes their color to the color of the majority of their neighbors. In real life, agents may not know exactly how many of their neighbors are red or blue, which introduces uncertainty into this process. Modeling uncertainty as perturbations of relative magnitude

1+\varepsilon

to these color neighbor counts, we show that even small values of

\varepsilon

greatly hinder the ability to achieve consensus in a social network. We prove theoretically tight upper and lower bounds on the price of uncertainty, a metric defined in previous work by Balcan et al. to quantify the effect of uncertainty in network games.

Yunzhe Bai, Alec Sun

Opinion Dynamics Multi-Agent Consensus Decision-Making under Uncertainty

Aug 22, 2025

Sequential Cohort Selection under Uncertainty

We study the problem of fair cohort selection under uncertainty, motivated by university admissions where applicant outcomes are only partially observed. We consider both a one-shot setting, where a fixed policy is applied to a population, and a sequential setting, where policies are updated over time using data from previous admission years. We propose a policy optimization framework that combines probabilistic modeling of outcomes with policy gradient methods, supporting both logistic and neural network policies. In the sequential setting, the approach jointly updates the policy and the underlying models to adapt to evolving applicant populations. Experiments on a simulator grounded in real admission data show that adaptive policies substantially outperform static baselines in term of expected utility, especially under higher admission costs. Neural policies consistently achieve higher utility and adapt more effectively than simpler models, while maintaining favorable fairness properties over time. Our results demonstrate the importance of adaptivity and model expressiveness for decision-making under uncertainty.

Hortence Yiepnou, Christos Dimitrakakis

University of Neuch atel

Sequential Decision Making Policy Optimization Group Fairness Decision-Making under Uncertainty

Aug 12, 2025

Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty

Real-world decision-making often involves uncertainty expressed in linguistic rather than numerical terms, and Prospect Theory (PT) provides a classic framework for modeling human behavior under such uncertainty. Although recent studies have developed frameworks to estimate PT parameters for Large Language Models (LLMs), few have examined whether PT itself adequately describes LLM decision-making behavior. To address these gaps, we develop a streamlined workflow grounded in a classic behavioral economics experimental paradigm. First, we estimate PT parameters and evaluate how well the resulting model captures LLM decision-making behavior. We then derive probability mappings for epistemic markers in the same context and inject them into prompts to examine the stability of PT parameters under linguistic uncertainty. Our findings suggest that PT does not consistently provide a reliable account of LLM decision-making across models, and that its application to LLMs is likely sensitive to epistemic uncertainty. The findings caution against the deployment of PT-based frameworks in real-world applications where epistemic ambiguity is prevalent, giving valuable insights in behaviour interpretation and future alignment direction for LLM decision-making.

Rui Wang, Qihan Lin, Jiayu Liu +7

Hong Kong University of Science and Technology · Huazhong University of Science and Technology · University of Illinois Urbana-Champaign

Prospect Theory LLM Decision-Making Decision-Making under Uncertainty Epistemic Uncertainty

May 6, 2025

Sufficient Decision Proxies for Decision-Focused Learning

When solving optimization problems under uncertainty with contextual data, utilizing machine learning to predict the uncertain parameters' values is a popular and effective approach. Decision-focused learning (DFL) aims at learning a predictive model such that decision quality, instead of prediction accuracy, is maximized. Common practice is to predict a single scenario representing the uncertain parameters, implicitly assuming that there exists a deterministic problem approximation (proxy) that allows for optimal decision-making. The opposite has also been considered, where the underlying distribution is estimated with a parameterized distribution. However, little is known about when either choice is valid. This paper investigates for the first time problem properties that justify using a certain decision proxy. Using this, we present alternative decision proxies for DFL, with little or no compromise on the complexity of the learning task. We show the effectiveness of presented approaches in experiments on continuous and discrete problems, as well as problems with uncertainty in the objective function and in the constraints.

Noah Schutte, Grigorii Veviurko, Krzysztof Postek +1

Delft University of Technology · Independent Researcher

Stochastic Optimization Learning to Optimize Decision-Focused Learning Decision-Making under Uncertainty

May 5, 2025

Plan-Driven Adaptive Bidding for First-Price Auctions with Budget Constraints under Nonstationarity

We study budget pacing in repeated first-price auctions when an advertiser's private-value distributions change over time and the stationary competing-bid distribution is unknown. We ask how a feasible expenditure plan should enter online bid shading, learning, and hard budget control. We establish a plan-to-performance decomposition for a plan-driven projected-dual policy. The policy uses any feasible expenditure plan as a soft target, learns an unknown stationary competing-bid CDF from thresholds revealed after each auction, and enforces the campaign budget on every sample path. Against a distribution-informed expected-budget fluid benchmark, the uniform-plan reward gap is

O(\sqrt T)+O(\mathcal W_T)

, where

\mathcal W_T

measures heterogeneity in private-value distributions. With a supplied feasible plan, the global gap decomposes into a one-sided

O(\sqrt{T})

fixed-plan execution term and a plan-mismatch term bounded by

(b/2a)PlanError

. The same analysis provides guarantees for strict and relaxed period-cap comparators, exact recovery of the global benchmark under a specific allowance vector, and separate lower bounds establishing the necessity of the temporal-heterogeneity and Plan Error terms. An upstream planner can translate forecasts or managerial priorities into a feasible spending trajectory, while the online controller adapts bids using realized thresholds and expenditures. The guarantee is modular: it evaluates the final normalized or projected plan through

PlanError

. A specific forecasting model can be linked to the guarantee by establishing how its primitive estimation errors propagate to this plan-quality metric.

Yige Wang, Jiashuo Jiang

Department of Industrial Engineering & Decision Analytics, Hong Kong University of Science and Technology

Primal-Dual Optimization Automated Bidding Online Advertising Constrained Optimization Online Learning Decision-Making under Uncertainty

May 29, 2024

Active Exploration via Autoregressive Generation of Missing Data

We pose uncertainty quantification and exploration in online decision-making as a problem of training and generation from an autoregressive sequence model, an area experiencing rapid innovation. Our approach rests on viewing uncertainty as arising from missing future outcomes that could be revealed through action choices, rather than from unobservable latent parameters of the environment. This reformulation aligns naturally with modern machine learning capabilities: we can i) train generative models through next-outcome prediction rather than fit explicit priors, ii) assess uncertainty through autoregressive generation rather than sampling latent parameters from posteriors, and iii) adapt to new information by extending the sequence model's context rather than explicit posterior updating. Our main theoretical result establishes a reduction from online decision-making to offline next-outcome prediction: Bayesian regret is controlled directly by the sequence model's offline prediction loss, without requiring an explicit latent-variable posterior. Experiments, including a semi-synthetic news recommendation task, show that autoregressive generation produces calibrated epistemic uncertainty and enables effective exploration by using article text as prior information to focus exploration on resolving remaining uncertainties.

Tiffany Tianhui Cai, Hongseok Namkoong, Daniel Russo +1

Department of Statistics Columbia University New York, NY 10025, USA · Decision, Risk, and Operations Division Columbia Business School New York, NY 10025, USA · Department of Mathematics Imperial College London London SW7 2AZ, United Kingdom

RL Exploration Autoregressive Generation Decision-Making under Uncertainty Epistemic Uncertainty Uncertainty-Aware Exploration

Nov 30, 2021

Double Fuzzy Probabilistic Interval Linguistic Term Set and a Dynamic Fuzzy Decision Making Model based on Markov Process with tts Application in Multiple Criteria Group Decision Making

The probabilistic linguistic term has been proposed to deal with probability distributions in provided linguistic evaluations. However, because it has some fundamental defects, it is often difficult for decision-makers to get reasonable information of linguistic evaluations for group decision making. In addition, weight information plays a significant role in dynamic information fusion and decision making process. However, there are few research methods to determine the dynamic attribute weight with time. In this paper, I propose the concept of double fuzzy probability interval linguistic term set (DFPILTS). Firstly, fuzzy semantic integration, DFPILTS definition, its preference relationship, some basic algorithms and aggregation operators are defined. Then, a fuzzy linguistic Markov matrix with its network is developed. Then, a weight determination method based on distance measure and information entropy to reducing the inconsistency of DFPILPR and obtain collective priority vector based on group consensus is developed. Finally, an aggregation-based approach is developed, and an optimal investment case from a financial risk is used to illustrate the application of DFPILTS and decision method in multi-criteria decision making.

Zongmin Liu

Stanford University, Stanford, CA 94305 USA

Multi-Criteria Decision Analysis Fuzzy Logic Decision-Making under Uncertainty

Sep 24, 2020

Learning in a Small/Big World

Complexity and limited ability affect how we learn and make decisions under uncertainty. Using finite automata to model belief formation, this paper studies the characteristics of optimal learning behavior in small and big worlds, where the complexity of the environment is low and high, respectively, relative to the cognitive ability of the decision maker. The optimal long-run average payoff and distribution of actions are well approximated by the Bayesian benchmark in very small worlds, but the gap widens as the worlds get bigger. In big worlds, it can be optimal for the decision maker to ignore some states entirely, concentrating limited cognitive resources on a subset of states. This ``optimal ignorance'' is reminiscent of well-documented non-Bayesian learning patterns such as the use of heuristics, correlation neglect, and inattentive learning. The results establish a relationship between complexity, cognitive ability, and the scope for non-Bayesian learning behavior.

Benson Tsz Kin Leung

Hong Kong Baptist University.

Decision-Making under Uncertainty Bounded Rationality Cognitive Modeling Bayesian Decision Theory