Reward Delay

Reward delay, the time lag between an action and its associated reward, is a critical challenge in various decision-making scenarios, impacting the efficiency of learning algorithms and the optimality of resulting policies. Current research focuses on developing algorithms that effectively handle reward delays within frameworks like multi-armed bandits and reinforcement learning, often incorporating techniques to address issues such as fairness, composite rewards, and partial anonymity. These advancements are crucial for optimizing performance in diverse applications, including online advertising, network routing, and personalized incentive design, where delayed feedback is inherent. The ultimate goal is to create robust and efficient algorithms that achieve near-optimal performance despite the presence of unpredictable or variable reward delays.

Papers

October 26, 2024

Beyond Simple Sum of Delayed Rewards: Non-Markovian Reward Modeling for Reinforcement Learning
Yuting Tang, Xin-Qiang Cai, Jing-Cheng Pang, Qiyu Wu, Yao-Xiang Ding, Masashi Sugiyama
Reinforcement Learning Reward Function Partial Sum Non Markovian Reward Reward Delay Markovian Reward

August 27, 2024

Delay as Payoff in MAB
Ofir Schlisselberg, Ido Cohen, Tal Lancewicki, Yishay Mansour
Multi Armed Bandit Optimal Regret Payoff Function Account Time Delay Sub Optimality Gap Reward Delay Gradient Delay

July 22, 2024

Merit-based Fair Combinatorial Semi-Bandit with Unrestricted Feedback Delays
Ziqun Chen, Kechao Cai, Zhuoyue Chen, Jinbei Zhang, John C. S. Lui
Bandit Algorithm Fairness Constraint Delayed Feedback Combinatorial Semi Bandit Reward Delay

June 18, 2024

Reinforcement-Learning based routing for packet-optical networks with hybrid telemetry
A. L. García Navarro, Nataliia Koneva, Alfonso Sánchez-Macián, José Alberto Hernández, Óscar González de Dios, J. M. Rivas-Moscoso
Reinforcement Learning Q Learning Routing Problem Optical Network Satellite Telemetry Reward Delay

May 4, 2023

Reinforcement Learning with Delayed, Composite, and Partially Anonymous Reward
Washim Uddin Mondal, Vaneet Aggarwal
Reinforcement Learning Efficient Hybrid Near Optimal Policy Significant Delay Change Reward Feedback Reward Delay

December 2, 2022

Multi-Agent Reinforcement Learning with Reward Delays
Yuyang Zhang, Runyu Zhang, Yuantao Gu, Na Li
Multi Agent Reinforcement Learning Action Space Reward Delay

September 16, 2022

Sales Channel Optimization via Simulations Based on Observational Data with Delayed Rewards: A Case Study at LinkedIn
Diana M. Negoescu, Pasha Khosravi, Shadow Zhao, Nanyu Chen, Parvez Ahammad, Humberto Gonzalez
Multi Armed Bandit Simulation Study Observational Data Based Policy Randomized Experiment LinkedIn Dataset Decision Policy Reward Delay

September 8, 2022

Reward Delay Attacks on Deep Reinforcement Learning
Anindya Sarkar, Jiarui Feng, Yevgeniy Vorobeychik, Christopher Gill, Ning Zhang
Deep Reinforcement Learning Untargeted Attack Novel Attack Targeted Attack Reward Delay

June 2, 2022

Incrementality Bidding via Reinforcement Learning under Mixed and Delayed Rewards
Ashwinkumar Badanidiyuru, Zhe Feng, Tianxi Li, Haifeng Xu
Reinforcement Learning Automated Conversion Episodic Markov Decision Process Novel Reinforcement Learning Mixed Speech Incremental Model Reward Feedback Reward Delay

March 11, 2022

Overcoming Temptation: Incentive Design For Intertemporal Choice
Shruthi Sukumar, Adrian F. Ward, Camden Elliott-Williams, Shabnam Hakimi, Michael C. Mozer
Incentive Mechanism Design Reward Delay Self Restraint Goal Directed Decision Intertemporal Choice

December 22, 2021

Decentralized Task Offloading in Edge Computing: A Multi-User Multi-Armed Bandit Approach
Xiong Wang, Jiancheng Ye, John C. S. Lui
Edge Computing Mobile Edge Computing Task Offloading Multi User Optimal Assignment Reward Delay