Logical Reasoning Capability
Logical reasoning capability in large language models (LLMs) is a burgeoning research area focused on evaluating and enhancing the ability of these models to perform complex deductive, inductive, and abductive reasoning tasks. Current research emphasizes developing robust benchmarks, such as those based on logic games, puzzles, and knowledge graph question answering, to assess LLMs' performance and identify weaknesses in their reasoning processes, often employing techniques like chain-of-thought prompting and contrastive learning. These efforts are crucial for improving the reliability and trustworthiness of LLMs across diverse applications, ranging from legal and medical domains to more general-purpose problem-solving.
Papers
TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles
Qingchen Yu, Shichao Song, Ke Fang, Yunfeng Shi, Zifan Zheng, Hanyu Wang, Simin Niu, Zhiyu Li
Proceedings of the First International Workshop on Next-Generation Language Models for Knowledge Representation and Reasoning (NeLaMKRR 2024)
Ken Satoh, Ha-Thanh Nguyen, Francesca Toni, Randy Goebel, Kostas Stathis