Scientific QA

QA: Question Answering

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 39

All topics

Arctic Questions, Missing Answers: A Dataset and Benchmark for LLM Abstention in Arctic Science
Oct 7, 2026Benjamin Wilcox, Dawei Gao, Pradeeban Kathiravelu +3LLM Abstention LLM Evaluation
University of North Texas, Denton, TX · University of Alaska Anchorage, Anchorage, AK
Leveraging a four-quadrant approach for evaluating Redpine Science
Oct 6, 2026Filip Dorm, Leonora VesterbackaScientific QA RAG Evaluation
Redpine, Sweden.
From Papers to Mechanisms: An Evidence-Grounded Knowledge Substrate for Scientific Language Models
Oct 5, 2026Qiuhui Chen, Yibo Liu, Tao Dai +3Knowledge Augmentation for Language Models Scientific QA
School of Information Science and Engineering, East China University of Science and Technology · School of Computer Science, Shanghai Jiao Tong University · Tencent
PhysicsMate: A Curriculum-Grounded Bengali Benchmark for Secondary Physics QA with Small-Model Adaptation
Sep 30, 2026Rashid Azraf Jahin, Saadman Sajid, Khan Raiyan Ibne Reza +1Benchmark Design Scientific QA
Dept. of Electrical and Computer Engineering North South University Dhaka, Bangladesh
PathAnchor: Path-Structured Evidence for Scientific Agents
Sep 30, 2026Qiuhui Chen, Jiafan Lu, Shuaimin Tang +5Evidence-Grounded Reasoning Scientific QA
School of Information Science and Engineering, East China University of Science and Technology · College of Intelligent Robotics and Advanced Manufacturing, Fudan University · Tencent
Where Scientific Search Agents Fail: Decision-Checkpoint Auditing of Exposure and Inspection Attempts
Sep 29, 2026Hongmin Li, Wanli ZhaoAI Agent Auditing Scientific QA
School of Life Science and Technology, Institute of Science Tokyo Tokyo, Japan · Department of Computational Biology and Medical Sciences Graduate School of Frontier Sciences, The University of Tokyo Kashiwa, Japan · Department of Information Science Graduate School of Advanced Science and Engineering, Hiroshima University Higashihiroshima, Japan
Re-ranking and Late Interaction Drive Retrieval Quality: A Controlled Comparison of RAG Strategies for Scientific Question Answering
Sep 29, 2026Bhagyesh Rathi, Eshan Chawla, William B. AndreopoulosLate Interaction Retrieval Scientific QA
San Jose State University, Department of Computer Science
AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering
Sep 28, 2026Chanhee Park, Jeongho Yoon, Sungbin Han +2Multi-Hop QA Tool-Augmented Language Model Agents
Korea University · Sookmyung Women’s University
Large Knowledge Model: A Knowledge Foundation for Agentic Science at Scale
Sep 23, 2026Yuan Huang, Sihan Hu, Hongyu Gu +18KG Construction AI Agents for Scientific Discovery
DP Technology, Beijing, China · Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing, China · International Center for Quantum Materials, School of Physics, Peking University, Beijing, China +4
PageRecall: Measuring Page Selection in Literature-Grounded Question Answering
Sep 16, 2026Aaditya ChauhanDocument Information Extraction Scientific QA
Independent Researcher
Rethinking Domain Specialization for Open-Ended Scientific Reasoning in Astronomy Language Models
Sep 15, 2026Vanessa Lama, Sanjay Das, Emily Herron +5LLM Evaluation Scientific QA
Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA · The Ohio State University, Columbus, Ohio, USA · Max-Planck-Institut für Astronomie, Königstuhl 17, D-69117 Heidelberg, Germany +1
Lit3R: Retrieve-Relate-Read for Evidence-Grounded Question Answering over Scientific Literature
Sep 15, 2026Akira Ise, Kotaro Kumagai, Yuta Yamaguchi +3Scientific QA Evidence-Grounded Generation
♠Tokyo University of Science · ♣Dentsu Soken Inc. · ♡Studio Ousia
SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding
Sep 4, 2026Shenxi Wu, Yuhong Liu, Haosong Zhang +8Document Understanding LLM Evaluation
The Chinese University of Hong Kong · Shanghai Artificial Intelligence Laboratory · New York University +5
From Terminology to Diagrams: Visual-Instruction Generation for Scientific Diagram Understanding
Sep 1, 2026Raul Ortega, José Manuel Gómez-PérezVisual Question Answering Scientific QA
Language Technology Research Laboratory Expert.ai 17 Henri Dunant, 28036 Madrid, Spain
EGT-KG: Evidence-Grounded Typed KG Retrieval for Practical Scientific QA with Small Language Models
Aug 31, 2026Muran Yu, Jiechao Gao, Yuandong Pan +5Graph Retrieval-Augmented Generation Small Language Models
Department of Civil and Environmental Engineering, Stanford University · Department of Civil Engineering, University of Calgary
LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answering
Aug 7, 2026Xuye Liu, Yimu Wang, Peng Shi +7Scientific QA Evidence-Grounded Generation
University of Waterloo · 2Amazon · 3City University of Hong Kong +1
SciRet: A Compute-Aware Empirical Study of Retrieval and Reranking for Scientific RAG
Aug 4, 2026Kaysarul Anas Apurba, Md. Hasibul Hasan, Rofiqul Alam Shehab +1Cross-Encoder Reranking Scientific QA
Laurentian University · North South University
EMBL AI Librarian: Life-Sciences Knowledge Layer for AI Agents
Jul 30, 2026Luigi Sigillo, Matteo Silvestri, Francesco Tabaro +9LLM Agent Orchestration Scientific QA
1EMBL Rome, European Molecular Biology Laboratory, Monterotondo, Italy. · 3Sapienza University of Rome, Italy. · 2European Bioinformatics Institute (EMBL-EBI), European Molecular Biology Laboratory, Hinxton, UK. +3
SciDataSailor: Deep Scientific Data Exploring
Jul 29, 2026Jiyong Rao, Yicheng Qiu, Chi Zhang +2Scientific QA AI Agent Benchmarks
Shanghai Artificial Intelligence Laboratory · Tongji University
A corrective agentic hybrid RAG and an operations-grounded evaluation for a scientific facility
Jul 27, 2026Rajat Sainju, Dariusz Jarosz, Hairong Shang +5Graph Retrieval-Augmented Generation Scientific QA
Advanced Photon Source, Argonne National Laboratory, Lemont, Illinois 60439, USA
VecTree-RAG: An Agentic Retrieval-Augmented Generation Framework Combining Vector and Tree Retrieval for Efficiency and Accuracy
Jul 25, 2026Xinyan Zhong, Yuwei Shi, Yuqi Wei +3Scientific QA Agentic RAG
Department of Chemistry and Materials Science, Xi’an Jiaotong-Liverpool University, Suzhou 215123, Jiangsu, P. R. China · Suzhou Lab, Suzhou, P. R. China · College of Carbon Neutrality Future Technology, State Key Laboratory of Heavy Oil Processing, China University of Petroleum (Beijing), Beijing 102249, China +1
Querying Multimodal Scientific Papers with AI: Practices and Preferences Across Blind, Low-Vision, and Sighted Scientists
Jul 20, 2026Arnavi Chheda-Kothary, Lucy Lu Wang, Joseph Chee Chang +1Multimodal Large Language Models Scientific QA
Paul G. Allen School of Computer Science & Engineering University of Washington Seattle, WA, USA · Allen Institute for AI · The Information School University of Washington Allen Institute for AI Seattle, WA, USA +1
DS@GT ARC at LongEval: Citation Integrity and Factual Grounding in Scientific QA
Jul 15, 2026Brandon Michaels, Brendon JohnsonRetrieval-Augmented Generation Citation Verification
Georgia Institute of Technology, North Ave NW, Atlanta, GA 30332
ResearchQA: Benchmarking Citation-Grounded Question-Answering on Scientific Papers
Jul 13, 2026Saba Imran, Debanjum Singh SolankyLLM Evaluation Citation Verification
Khoj Inc.
NuclearQAv2: A Structured Benchmark for Evaluating Domain-Science Competence in Large Language Models
Jun 25, 2026Henry Shaowu Yuchi, Michal Kucer, Benjamin H. Sims +2LLM Evaluation Scientific QA
Los Alamos National Laboratory
Agentic Hybrid RAG for Evidence-Grounded Muon Collider Analysis
Jun 9, 2026Ruobing Jiang, Dawei Fu, Cheng Jiang +6Scientific QA High-Energy Physics
State Key Laboratory of Nuclear Physics and Technology, Peking University, China
MMClima: A Framework for Multimodal Climate Science Data and Evaluation
Jun 8, 2026Muhammad Umer Sheikh, Hassan Abid, Khawar Shehzad +2Climate Science Scientific QA
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates · University of Missouri, Columbia, Missouri, United States
TechRAG: Evidence-Gated Multimodal Agentic RAG for Technical Literature Reasoning
Jun 1, 2026Kanwar Bharat SinghMultimodal RAG Scientific QA
Global Tire Intelligence and Solutions (GTIS) The Goodyear Tire & Rubber Company
ProtStructQA: A Denotation Threshold in Protein Structural Reasoning
May 30, 2026Aravind Mandiga, Guoming Li, Jin Lu +3LLM Evaluation Scientific QA
University of Georgia
OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields
May 28, 2026Wanhao Liu, Jiaqing Xie, Qian Tan +10Benchmark Design Scientific QA
University of Science and Technology of China · Shanghai Artificial Intelligence Laboratory · Fudan University +4

Oct 7, 2026, cs.CLJ/K move · Enter open · S save

Arctic Questions, Missing Answers: A Dataset and Benchmark for LLM Abstention in Arctic Science

Benjamin Wilcox, Dawei Gao, Pradeeban Kathiravelu, Douglas Causey+2

University of North Texas, Denton, TX · University of Alaska Anchorage, Anchorage, AK

LLM Abstention LLM Evaluation Scientific QA

Open paper AskPDF arXiv

Large language models (LLMs) should abstain from scientific multiple-choice questions when no option is valid, but frequent abstention alone does not demonstrate sensitivity to answer availability. We introduce ArcticQA, a dataset of 194 questions derived from primary Arctic research, with automated checks of answer support and distractor contradiction against source evidence. We further develop ArcticAbstain, a paired benchmark comparing answer-present and answer-absent conditions, with the correct answer replaced by a distractor in the latter and an explicit abstention option in both. We evaluate eight models from the Gemini, Claude, and ChatGPT families at high reasoning effort, with three trials per condition, yielding 9,312 recorded responses. Answer-present abstention rates range from 0.0% to 63.0%, whereas replacing the correct answer increases abstention by 5.05 percentage points on average. These findings highlight substantial baseline differences and the need to evaluate abstention frequency and responsiveness jointly. The dataset and benchmark are available at https://github.com/BenWilcox8/arctic-qa.

Momentum

Related topics

Latest papers 39