cs.CLAug 26, 2025

LongReasonArena: A Long Reasoning Benchmark for Large Language Models

Authors: Jiayu Ding, Shuming Ma, Lei Cui, Nanning Zheng, Furu Wei

Organizations: IAIR, Xi’an Jiaotong University · Microsoft Research

Abstract

Existing long-context benchmarks for Large Language Models (LLMs) focus on evaluating comprehension of long inputs, while overlooking the evaluation of long reasoning abilities. To address this gap, we introduce LongReasonArena, a benchmark specifically designed to assess the long reasoning capabilities of LLMs. Our tasks require models to solve problems by executing multi-step algorithms that reflect key aspects of long reasoning, such as retrieval and backtracking. By controlling the inputs, the required reasoning length can be arbitrarily scaled, reaching up to 1 million tokens of reasoning for the most challenging tasks. Extensive evaluation results demonstrate that LongReasonArena presents a significant challenge for both open-source and proprietary LLMs. For instance, even GPT-5 achieves only 27.8% accuracy on Level 3. Further analysis also reveals that the accuracy exhibits a linear decline with respect to the logarithm of the expected number of reasoning steps. Our code and data are available at https://github.com/LongReasonArena/LongReasonArena.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning

    Apr 15, 2026Sumeet Ramesh Motwani, Daniel Nichols, Charles London +17Long-Horizon LLM ReasoningCoT Reasoning

  2. Beyond Context Length: Characterising the Difficulty of Long-Context Reasoning

    Oct 15, 2026Shiwen Qin, Cameron Barker, Henry Gouk +4Long-Context Reasoning in Language ModelsLong-Context Language Model Evaluation