cs.CLJul 26, 2026

Do Diagrams Help Large Language Models Reason? Evidence from Syllogistic Reasoning

Authors: Risako AndoKoji Mineshima

Organizations: Keio University, 2-15-45 Mita, Minato-ku, Tokyo 108-8345, Japan

Abstract

Diagrams are widely used to support logical reasoning, and prior studies suggest that representations such as Euler diagrams can improve human reasoning performance. Recent work has also explored their effects on large language models (LLMs). In this paper, we compare four representational conditions for syllogistic reasoning: natural language, logical notation, linear diagrams, and Euler diagrams. Using 285 problems from Ando et al. (2024), we evaluate two contemporary LLMs, Claude 3.5~Sonnet and GPT-4o-mini. Our results show that diagrammatic representations do not consistently improve performance. Although the models perform well on entailment and contradiction problems, they struggle with neutral problems and often make systematic conversion errors. Overall, the results suggest that the tested models gain limited benefit from diagrams in logical reasoning tasks.

Explore similar work

CardsList
  1. Are you Talking Logic to Me? Assessing Language Models Syllogistic Reasoning Capabilities

    Jul 22, 2026Hanna Abi Akl, Fabien Gandon, Catherine Faron +1