Reasoning Task
Reasoning tasks in large language models (LLMs) focus on improving the ability of these models to perform multi-step inferences and solve complex problems requiring logical deduction and induction. Current research emphasizes developing novel prompting techniques, such as those inspired by Bloom's taxonomy or employing dynamic reasoning trajectories, and improving model training through knowledge distillation and learning from mistakes. These advancements are significant because enhanced reasoning capabilities in LLMs have broad implications for various fields, including improving question answering systems, enhancing personalized recommendation systems, and advancing applications in education and scientific discovery.
Papers
Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
Tiberiu Musat
Understanding Chain-of-Thought in LLMs through Information Theory
Jean-Francois Ton, Muhammad Faaiz Taufiq, Yang Liu
Technical Report: Enhancing LLM Reasoning with Reward-guided Tree Search
Jinhao Jiang, Zhipeng Chen, Yingqian Min, Jie Chen, Xiaoxue Cheng, Jiapeng Wang, Yiru Tang, Haoxiang Sun, Jia Deng, Wayne Xin Zhao, Zheng Liu, Dong Yan, Jian Xie, Zhongyuan Wang, Ji-Rong Wen
PSPO*: An Effective Process-supervised Policy Optimization for Reasoning Alignment
Jiawei Li, Xinyue Liang, Yizhe Yang, Chong Feng, Yang Gao
PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks
Matthew Chang, Gunjan Chhablani, Alexander Clegg, Mikael Dallaire Cote, Ruta Desai, Michal Hlavac, Vladimir Karashchuk, Jacob Krantz, Roozbeh Mottaghi, Priyam Parashar, Siddharth Patki, Ishita Prasad, Xavier Puig, Akshara Rai, Ram Ramrakhya, Daniel Tran, Joanne Truong, John M. Turner, Eric Undersander, Tsung-Yen Yang
Thought Space Explorer: Navigating and Expanding Thought Space for Large Language Model Reasoning
Jinghan Zhang, Fengran Mo, Xiting Wang, Kunpeng Liu
Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics
Yaniv Nikankin, Anja Reusch, Aaron Mueller, Yonatan Belinkov
Flaming-hot Initiation with Regular Execution Sampling for Large Language Models
Weizhe Chen, Zhicheng Zhang, Guanlin Liu, Renjie Zheng, Wenlei Shi, Chen Dun, Zheng Wu, Xing Jin, Lin Yan
Improving Small-Scale Large Language Models Function Calling for Reasoning Tasks
Graziano A. Manduzio, Federico A. Galatolo, Mario G. C. A. Cimino, Enzo Pasquale Scilingo, Lorenzo Cominelli
SIKeD: Self-guided Iterative Knowledge Distillation for mathematical reasoning
Shivam Adarsh, Kumar Shridhar, Caglar Gulcehre, Nicholas Monath, Mrinmaya Sachan