Scientific Reasoning in Language Models

Ask about this topic

Overview

The field studies whether language models can do more than recall scientific facts: interpret data, connect evidence, design tests, explain mechanisms, and generate useful hypotheses. Common approaches include chain-of-thought and structured reasoning traces, retrieval and tool use, multi-agent collaboration, and training with expert examples, simulators, or reinforcement learning with verifiable rewards. Evaluation spans scientific question answering, mathematics and physics problems, papers and tables, multimodal data, and interactive experimental workflows. Models are improving on specialized tasks, but reliable evidence use, robust execution, and mechanistic understanding remain difficult, and correct answers do not always show that a model reasoned scientifically.

Latest papers

CardsList

144 papers40 papers in the last four weeks, up 233% on the four weeks before. 0.2% of all new papers.

  1. Can LLMs Uncover Hidden Physical Laws through Free Exploration?

    Oct 27, 2026Zhen Yang, Wen Wang, Xander Xu +1AI Agents for Scientific DiscoveryLarge Language Model-Guided Scientific Discovery

    HKUST(GZ) · ZJU · Alibaba Group +1

  2. Prospector: learning a compact graph traversal policy for scientific reasoning

    Oct 27, 2026Haiqian Yang, Markus J. BuehlerLLM Reasoning with GraphsScientific Reasoning in Language Models

    Department of Civil and Environmental Engineering Department of Mechanical Engineering Schwarzman College of Computing Massachusetts Institute of Technology Cambridge, MA, USA

  3. What Outcome RL Buys: Finishing, Reliability, Rarely Reach

    Oct 26, 2026Shan YangReinforcement LearningRL for Language Model Reasoning

    Individual Researcher.

  4. The Right Fact, the Wrong Decision: SciValidityBench for Scientific Agent Memory

    Oct 26, 2026Yuexi Pan, Yongchao ChenScientific Agent BenchmarksMemory Agent Benchmarks

    Tsinghua University · University of Science and Technology of China

  5. LLMs Can Tackle High Energy Theoretical Physics, yet Gaps Remain: Part I, Quantum Field Theory

    Oct 23, 2026Tong Liu, Bohan Li, Gongrui Cheng +3Scientific Reasoning in Language ModelsLanguage Model Reasoning Evaluation

    Ludwig-Maximilians-Universität München, München, Germany · Munich Center for Machine Learning · Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan +3

  6. PHYRD: A Living Benchmark for Reproducing and Extending Theoretical Physics Research

    Oct 23, 2026Qituan Zhang, Guo Chen, Linfeng Zhang +1Scientific Reasoning in Language ModelsLanguage Model Reasoning Evaluation

    EPIC Lab, Shanghai Jiao Tong University · Fudan University · University of Electronic Science and Technology of China +1

  7. Principia: Test-Time Hypotheses to Principle Construction for Scientific Discovery

    Oct 23, 2026Dongki Kim, Sangwoo Park, Jinheon Baek +2Large Language Model-Guided Scientific DiscoveryScientific Hypothesis Generation

    KAIST

  8. Tampered ML: Evaluating LLM Judgment of Machine Learning Research

    Oct 23, 2026Soroush Ebadian, Parand A. Alamdari, Joe Benton +1LLM EvaluationScientific Reasoning in Language Models

    Anthropic Fellows Program · Anthropic

  9. From Prompts to Posteriors: Can LLM Agents Perform Bayesian Inference in Context?

    Oct 21, 2026Linus Filbry, Luckeciano C. Melo, Yarin GalBelief Updating in LLMsBayesian Inference

    OATML, University of Oxford

  10. CollabSciReM: Aligning Language Models for Scientific Conversations via Science-Aware Reward Decomposition

    Oct 20, 2026Sasi Kiran Gaddipati, Gollam Rabby, Sören AuerLLM AlignmentReward Modeling

    TIB—Leibniz Information Centre for Science and Technology, Hannover, Germany · L3S Research Center, Hannover, Germany

  11. Zephyrus-R1: Agentic Reinforcement Learning for Weather Reporting

    Oct 19, 2026Fengfei Yu, Marshall Fisher, Isadora White +13RL for VLMsReinforcement Learning with Verifiable Rewards

    University of California, San Diego

  12. Auditing Tool Traces in Agentic LLMs: Source Localization and Counterfactual Dependence Testing

    Oct 18, 2026Junhao Liu, Tianxiao Li, Yongxian Wu +4LLM AuditingAI Agent Auditing

    University of California, Irvine · NEC Labs America

  13. Dynamic System Inspired Continuous Large Language Model for AI4Science Tasks

    Oct 18, 2026Zhaoxin Fan, Zheng Zhu, Yuqing Ma +3Neural ODEsScientific Reasoning in Language Models

    Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University, Beijing, 100191, Beijing, China. · School of Artificial Intelligence, Beihang University, Beijing, 100191, Beijing, China. · GigaAI, Beijing, 100191, Beijing, China.

  14. TIRE: Top-Down Formula Instantiation with Reliable Execution for Physics Problem Synthesis

    Oct 18, 2026Xiaoyu Wu, Zhaoxin Yu, Xinglin Xiao +1Synthetic Data Generation for Language ModelsScientific Reasoning in Language Models

    ScienceOne AI · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences

  15. Axiomatic World Modeling for Physics Reasoning

    Oct 17, 2026Xinye Yang, Zhenyang Liu, Yuxuan Wang +1Physical ReasoningScientific Reasoning in Language Models

    Northwestern University, Evanston, IL · Department of Physics, University of Florida, Gainesville, FL · Computer & Information Science and Engineering, University of Florida, Gainesville, FL

  16. CME-SpectrumBench: Can LLMs Analyze Condensed Matter Spectral Data?

    Oct 16, 2026Jin Gene Wong, Anjney Midha, Joseph Mathew Tennyson +3LLM EvaluationScientific Reasoning in Language Models

    Physics Department and Stanford Laboratory for Intelligent Matter, Stanford University, Stanford, CA 94305 · Applied Physics Department, Stanford University, Stanford, CA 94305 · Electrical Engineering & Computer Sciences, University of California, Berkeley, Berkeley, CA 94720 +1

  17. Evaluating Inquiry Learning in LLM-Based Agents: Hypothesizing, Reasoning, and Exploration

    Oct 15, 2026Marta Knežević, Seyed Parsa Neshaei, Jibril Frej +2Instructional ScaffoldingScientific Reasoning in Language Models

    EPFL

  18. Do Agents Need a New Kind of Paper? SPRA: Structured Paper Representation for Agents

    Oct 15, 2026Yutong Leng, Haojiong Chen, Fukai Shang +4Knowledge RepresentationKG Construction

    Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University

  19. Learning to Reason with Selective Internal Feedback

    Oct 15, 2026Xun Lu, Shumeng Yang, Xinrui Wang +3RL for Language Model ReasoningReinforcement Learning

    State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Pengcheng Laboratory

  20. SciΠΠvot: Benchmarking Principal-Investigator Judgment at Pivotal Moments of Scientific AutoResearch Agents

    Oct 15, 2026Youran Sun, Xingyu Ren, Chugang Yi +15LLM Agent EvaluationAI Agent Benchmarks

    University of Maryland, College Park · The Chinese University of Hong Kong · Gradient Network +3

  21. Same Claim, Different Interpretations: How LLM Agents Resolve Ambiguity in Scientific Claims

    Oct 14, 2026Delip Rao, Ahmed Muharram, Chris Callison-BurchLLM Agent VerificationScientific Claim Verification

    University of Pennsylvania

  22. ChemArgus: Automated Scientific Diagnostics for Lab-Grade Multimodal Chemistry Reasoning: A Frontier Benchmark

    Oct 14, 2026Hanyao Guo, Mo Cui, Zihan Tan +21Chemical Reasoning in LLMsScientific Reasoning in Language Models

  23. LabOPBench: A Multimodal Benchmark for Evidence-Grounded Laboratory Decision Making

    Oct 14, 2026Jingyuan Li, Yujie Liu, Yijie Dai +18Evidence-Grounded ReasoningScientific Reasoning in Language Models

    Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University · Fudan University +6

  24. SHELDON: Reasoning through Dual Search over Hypothesis and Observation Spaces

    Oct 13, 2026Harshit Joshi, Priyank Shethia, Monica S. LamScientific Reasoning in Language ModelsScientific Hypothesis Generation

    Stanford University, USA

  25. Learning from fracture: Teaching language models to design through physical experience

    Oct 13, 2026Bo NiScientific Reasoning in Language ModelsLarge Language Model-Guided Optimization

    Department of Materials Science and Engineering, Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh, PA 15213, USA · Generative Mechanics Laboratory (GenMech Lab), independent research initiative

  26. Large Language Models Extend Known Research But Do Not Create New Research Directions

    Oct 12, 2026Zhihua Liu, Kurt Butler, Damian Machlanski +2Large Language Model-Guided Scientific DiscoveryScientific Hypothesis Generation

    Causality in Healthcare AI Hub (CHAI) School of Engineering, University of Edinburgh

  27. ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal Evidence

    Oct 9, 2026Yiwei Chen, Amila Weerasinghe, Siddharth Biswal +4AI Agent BenchmarksCausal Reasoning in Language Models

    UC San Diego · Amazon

  28. Masked Diffusion Language Agents for Tool-Integrated Chemical Reasoning

    Oct 9, 2026Mengdi Liu, Chenghao Jia, Hong Chang +1Chemical Reasoning in LLMsLLM Tool Use

    State Key Laboratory of AI Safety, Institute of Computing Technology, CAS, China · University of Chinese Academy of Sciences (CAS), China

  29. EvoBelief: Falsifiable World Models for Self-Improving LLM Agents

    Oct 8, 2026Janet Wang, Jiang Guo, Yi Xiang +6World ModelsBelief Updating in LLMs

    Tulane University · Amazon

  30. SAIL: Scientific Agentic Intelligence via a Science-Aware Loop

    Oct 8, 2026SAIL Model Team, Boyuan Sun, Bryan Dai +9Autonomous Research AgentsAI Agents for Scientific Discovery

    IQuest Research