Scientific Reasoning in Language Models

Follow this topic Ask about this topic

The field studies whether language models can do more than recall scientific facts: interpret data, connect evidence, design tests, explain mechanisms, and generate useful hypotheses. Common approaches include chain-of-thought and structured reasoning traces, retrieval and tool use, multi-agent collaboration, and training with expert examples, simulators, or reinforcement learning with verifiable rewards. Evaluation spans scientific question answering, mathematics and physics problems, papers and tables, multimodal data, and interactive experimental workflows. Models are improving on specialized tasks, but reliable evidence use, robust execution, and mechanistic understanding remain difficult, and correct answers do not always show that a model reasoned scientifically.

Latest papers

144 papers40 papers in the last four weeks, up 233% on the four weeks before. 0.2% of all new papers.

Evaluating Relational Reasoning in LLMs with REL
Apr 14, 2026Lukas Fesser, Yasha Ektefaie, Ada Fang +2Relational Reasoning LLM Evaluation
Harvard University · Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard
Solving Physics Olympiad via Reinforcement Learning on Physics Simulators
Apr 13, 2026Mihir Prabhudesai, Aryan Satpathy, Yangmin Li +6Scientific Reasoning in Language Models Synthetic Data Generation for Language Models
Carnegie Mellon University
Towards Autonomous Mechanistic Reasoning in Virtual Cells
Apr 13, 2026Yunhui Jang, Lu Zhu, Jake Fawkes +3Scientific Reasoning in Language Models Multi-Agent Reasoning
Korea Advanced Institute of Science and Technology (KAIST) · Valence Labs · Recursion +1
PaperScope: A Multi-Modal Multi-Document Benchmark for Agentic Deep Research Across Massive Scientific Papers
Apr 13, 2026Lei Xiong, Huaying Yuan, Zheng Liu +2AI Agent Benchmarks Multimodal Model Evaluation
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · Beijing Academy of Artificial Intelligence, Beijing, China
When Verification Fails: How Compositionally Infeasible Claims Escape Rejection
Apr 13, 2026Muxin Liu, Delip Rao, Grace Kim +1Scientific Claim Verification Scientific Reasoning in Language Models
University of Pennsylvania
SciPredict: Can LLMs Predict the Outcomes of Scientific Experiments in Natural Sciences?
Apr 12, 2026Udari Madhushani Sehwag, Elaine Lau, Haniyeh Ehsani Oskouie +14AI-Assisted Scientific Research Scientific Reasoning in Language Models
Scale AI · University of California, Los Angeles · Human Frontier Collective, Scale AI +2
Aryabhata 2: Scaling Reinforcement Learning for Advanced STEM Reasoning
Apr 10, 2026Ritvik Rastogi, Vishal Singh, Tejas Chaudhari +1Scientific Reasoning in Language Models Efficient Language Model Reasoning
PhysicsWallah
Scientific Knowledge-driven Decoding Constraints Improving the Reliability of LLMs
Apr 8, 2026Maotian Ma, Zheni Zeng, Zhenghao Liu +1LLM Hallucination Mitigation Knowledge Augmentation for Language Models
Nanjing University · Tsinghua University · Northeastern University
MedConclusion: A Benchmark for Biomedical Conclusion Generation from Structured Abstracts
Apr 7, 2026Weiyue Li, Ruizhi Qian, Yi Li +5Evidence-Grounded Generation Scientific Reasoning in Language Models
Can Large Language Models Reinvent Foundational Algorithms?
Apr 7, 2026Jian Zhao, Haoren Luo, Yu Wang +3Algorithmic Reasoning Scientific Reasoning in Language Models
Xiongan AI Institute · Beijing University of Posts and Telecommunications, Beijing, China · Institute for Interdisciplinary Information Sciences, Tsinghua University +3
Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation
Apr 6, 2026Srujan P Mule, Aniketh Garikaparthi, Manasi PatwardhanScientific Idea Evaluation Large Language Model-Guided Scientific Discovery
IISER Pune · TCS Research
GENFIG1: Visual Summaries of Scholarly Work as a Challenge for Vision-Language Models
Apr 5, 2026Yaohan Guan, Pristina Wang, Najim Dehak +3T2I Generation Evaluation VLM Evaluation
Johns Hopkins University
FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning
Apr 4, 2026Zeyu Wang, Jingye Xu, Xiaogang Li +7Scientific Reasoning in Language Models Multimodal Reasoning
Alibaba Group, Beijing, China · Skylenage, Beijing, China
BioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data
Apr 3, 2026Brian Hsu, Ozan Gökdemir, Carlo Siebenschuh +7Scientific Reasoning in Language Models Scientific QA
Department of Computer Science, University of Chicago, Chicago, IL 60637 · Data Science and Learning Division, Argonne National Laboratory, Lemont, IL 60439
PolyReal: A Benchmark for Real-World Polymer Science Workflows
Apr 3, 2026Wanhao Liu, Weida Wang, Jiaqing Xie +12Multimodal Model Evaluation Multimodal Large Language Models
University of Science and Technology of China · Shanghai Artificial Intelligence Laboratory · Fudan University +5
Grading the Unspoken: Evaluating Tacit Reasoning in Quantum Field Theory and String Theory with LLMs
Apr 1, 2026Xingyang Yu, Yinghuan Zhang, Yufei Zhang +1Scientific Reasoning in Language Models LLM Evaluation
Department of physics, Virginia Tech · Independent Researcher · Department of Computer Science and Engineering, Michigan State University
CrossTrace: A Cross-Domain Dataset of Grounded Scientific Reasoning Traces for Hypothesis Generation
Mar 30, 2026Andrew Bouras, OMS-II Research FellowScientific Hypothesis Generation Scientific Reasoning in Language Models
Nova Southeastern University Dr. Kiran C. Patel College of Osteopathic Medicine Clearwater, FL 33760, USA
Beyond the Answer: Decoding the Behavior of LLMs as Scientific Reasoners
Mar 30, 2026Rohan Pandey, Eric Ye, Michael LiLLM Prompting LLM Interpretability
Carnegie Mellon University
PRBench: End-to-end Paper Reproduction in Physics Research
Mar 29, 2026Shi Qiu, Junyi Deng, Yiwei Deng +48AI Agent Benchmarks AI Coding Agents
Are LLMs Good For Quantum Software, Architecture, and System Design?
Mar 27, 2026Sourish Wawdhane, Poulami DasLLM Evaluation Scientific Reasoning in Language Models
The University of Texas at Austin
Not Search, But Scan: Benchmarking MLLMs on Scan-Oriented Academic Paper Reasoning
Mar 27, 2026Rongjin Li, Zichen Tang, Xianghe Wang +9LLM Evaluation Scientific Reasoning in Language Models
Beijing University of Posts and Telecommunications
MolQuest: A Benchmark for Agentic Evaluation of Abductive Reasoning in Chemical Structure Elucidation
Mar 26, 2026Taolin Han, Shuang Wu, Jinghang Wang +4Abductive Reasoning in Language Models Scientific Reasoning in Language Models
Alibaba Group, Hangzhou, China
Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scale
Mar 26, 2026Yicheng Zou, Dongsheng Zhu, Lin Zhu +174Multimodal Foundation Models AI for Science
ThermoQA: A Three-Tier Benchmark for Evaluating Thermodynamic Reasoning in Large Language Models
Mar 25, 2026Kemal DüzkarLLM Evaluation Scientific Reasoning in Language Models
Olivenet Kyrenia, Cyprus
Bridging the Know-Act Gap via Task-Level Autoregressive Reasoning
Mar 23, 2026Jihyun Janice Ahn, Ryo Kamoi, Berk Atil +34Scientific Reasoning in Language Models
Penn State · UIUC · Columbia University +1
Embodied Science: Closing the Discovery Loop with Agentic Embodied AI
Mar 20, 2026Xiang Zhuang, Chenyi Zhou, Kehua Feng +10AI Agents for Scientific Discovery Autonomous Experimentation
Zhejiang University, Hangzhou, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Tongji University, Shanghai, China
MoRI: Learning Motivation-Grounded Reasoning for Scientific Ideation in Large Language Models
Mar 19, 2026Chenyang Gu, Jiahao Cheng, Meicong Zhang +3Scientific Reasoning in Language Models Large Language Model-Guided Scientific Discovery
School of Economics and Management, East China Normal University
Mathematical Reasoning Enhanced LLM for Formula Derivation: A Case Study on Fiber NLI Modellin
Mar 19, 2026Yao Zhang, Yuchen Song, Xiao Luo +4Scientific Reasoning in Language Models LLM Mathematical Reasoning
State Key Laboratory of Information Photonics and Optical Communications, Beijing University of Posts and Telecommunications, Beijing 100876, China
PRISM: Demystifying Retention and Interaction in Mid-Training
Mar 17, 2026Bharat Runwal, Ashish Agrawal, Anurag Roy +1LLM Training Scientific Reasoning in Language Models
IBM Research, MIT-IBM Watson AI Lab
OMNIFLOW: A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning
Mar 16, 2026Hao Wu, Yongheng Zhang, Yuan Gao +7Scientific Reasoning in Language Models Physics-Informed ML
Tsinghua University · Tencent · Shenzhen Loop Area Institute +2

Apr 14, 2026J/K move · Enter open · S save

Evaluating Relational Reasoning in LLMs with REL

Lukas Fesser, Yasha Ektefaie, Ada Fang, Sham M. Kakade+1

Harvard University · Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard

Relational Reasoning LLM Evaluation Scientific Reasoning in Language Models

Open paper AskPDF arXiv

Relational reasoning is the ability to infer relations that jointly bind multiple entities, attributes, or variables. This ability is central to scientific reasoning, but existing evaluations of relational reasoning in large language models often focus on structured inputs such as tables, graphs, or synthetic tasks, and do not isolate the difficulty introduced by higher-arity relational binding. We study this problem through the lens of Relational Complexity (RC), which we define as the minimum number of independent entities or operands that must be simultaneously bound to apply a relation. RC provides a principled way to vary reasoning difficulty while controlling for confounders such as input size, vocabulary, and representational choices. Building on RC, we introduce REL, a generative benchmark framework spanning algebra, chemistry, and biology that varies RC within each domain. Across frontier LLMs, performance degrades consistently and monotonically as RC increases, even when the total number of entities is held fixed. This failure mode persists with increased test-time compute and in-context learning, suggesting a limitation tied to the arity of the required relational binding rather than to insufficient inference steps or lack of exposure to examples. Our results identify a regime of higher-arity reasoning in which current models struggle, and motivate re-examining benchmarks through the lens of relational complexity.