Reward Tree

Reward trees are interpretable models used to represent reward functions in reinforcement learning, particularly within the context of learning from human feedback. Current research focuses on developing efficient algorithms, such as Monte Carlo Tree Search and genetic programming, to construct and refine these tree structures, often integrating them with large language models for improved performance and scalability in diverse applications like biomedical knowledge retrieval and traffic signal control. The ability to learn and utilize interpretable reward functions offers significant advantages in ensuring agent alignment, facilitating debugging, and improving the explainability of complex decision-making systems.

Papers

November 18, 2024

Technical Report: Enhancing LLM Reasoning with Reward-guided Tree Search
Jinhao Jiang, Zhipeng Chen, Yingqian Min, Jie Chen, Xiaoxue Cheng, Jiapeng Wang, Yiru Tang, Haoxiang Sun, Jia Deng, Wayne Xin Zhao, Zheng Liu, Dong Yan, Jian Xie, Zhongyuan Wang, Ji-Rong Wen
Large Language Model Reasoning Task Technical Report LLM Reasoning Reward Tree

June 17, 2024

SeRTS: Self-Rewarding Tree Search for Biomedical Retrieval-Augmented Generation
Minda Hu, Licheng Zong, Hongru Wang, Jingyan Zhou, Jingjing Li, Yichen Gao, Kam-Fai Wong, Yu Li, Irwin King
Retrieval Augmented Generation Proximal Policy Optimization Retrieval Augmented Biomedical Knowledge Reward Tree Retrieval Augmented Lm

June 6, 2024

ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, Jie Tang
Reinforcement Learning Self Training Reward Model LLM Reasoning LLM Adaptation Reward Tree

May 2, 2024

QxEAI: Quantum-like evolutionary algorithm for automated probabilistic forecasting
Kevin Xin, Lizhi Xin
Evolutionary Algorithm State of the Art Forecasting Probabilistic Forecast Reward Tree Quantum Decision Quantum Inspired

March 26, 2024

Learning Traffic Signal Control via Genetic Programming
Xiao-Cheng Liao, Yi Mei, Mengjie Zhang
Genetic Programming Traffic Signal Control Reward Tree Signal Control

February 15, 2024

Reward Generalization in RLHF: A Topological Perspective
Tianyi Qiu, Fanzhi Zeng, Jiaming Ji, Dong Yan, Kaile Wang, Jiayi Zhou, Yang Han, Josef Dai, Xuehai Pan, Yaodong Yang
Reinforcement Learning Reward Function Preference Learning Reinforcement Learning From Human Feedback Preference Representation Topological Perspective Reward Tree

December 13, 2023

Conflict Transformation and Management. From Cognitive Maps to Value Trees
Berkay H. Tosunlu, Joseph H. A. Guillaume, Alexis Tsoukiàs
Cognitive Map Decision Support Decision Problem Reward Tree Conflict Management

May 26, 2023

Learning Interpretable Models of Aircraft Handling Behaviour by Reinforcement Learning from Human Feedback
Tom Bewley, Jonathan Lawry, Arthur Richards
Reinforcement Learning Human Feedback Interpretable Model Preference Feedback Robust Flight Reward Tree

October 3, 2022

Reward Learning with Trees: Methods and Evaluation
Tom Bewley, Jonathan Lawry, Arthur Richards, Rachel Craddock, Ian Henderson
Global Evaluation NCD Method Reward Function Tree Specie Reward Learning Interpretable Tree Interpretable Reward Reward Tree

December 20, 2021

Interpretable Preference-based Reinforcement Learning with Tree-Structured Reward Functions
Tom Bewley, Freddy Lecue
Reinforcement Learning Reward Function Preference Based Reinforcement Learning Reward Tree