Large Language Models (LLMs) are increasingly used as research collaborators for hypothesis refinement, methodological planning, and technical analysis across multi-turn interactions. In this setting, a useful assistant must be accurate, methodologically justified, logically consistent, and token-efficient. Current alignment approaches rarely achieve all of these together. Instruction-tuned models gain conversational quality through verbosity, and we show that science-agnostic general collaborative rewards compress responses at the expense of scientific rigor. We propose CollabSciReM, a goal-conditioned reward framework that decomposes the session-level reward into conversational quality, scientific rigor, and structural efficiency. We also introduce SciQuality, a pairwise evaluation of multi-turn research conversations grounded in 300 held-out scientific papers and scored on six dimensions: methodological soundness, justification, logical consistency, factual correctness, novelty, and responsiveness. Fine-tuned on 8.62K synthetic research conversations, CollabSciReM is preferred on SciQuality over a science-agnostic reward baseline trained on the same data for all four Llama and Qwen models, including a 65.3% win rate at matched response length. For the 7B and 8B models, CollabSciReM outperforms the science-agnostic reward baseline on the primary metric in 8 of 10 single- and multi-turn benchmark settings. In a blind study, domain experts rated CollabSciReM significantly higher on novelty than the general-reward variant and reported fewer hallucinated responses than both the general-reward variant and the base model.
Figures & tables
Figure 1: Overview of the CollabSciReM Framework. The pipeline begins with Synthetic Data Generation, represented by dotted arrows, in which a simulated researcher and a teacher model engage in scientific dialogue. These interactions are evaluated by a Critic/Judge Model to compute multi-faceted rewards, including conversational, structural, and scientific metrics. The resulting preference-labeled dataset is used to train a Policy Model via Supervised Fine-Tuning (SFT) and Offline Identity Preference Optimization (IPO). In inference, a user query triggers the Learned Policy to generate a scientifically grounded response.
Figure 2: Pairwise scientific-quality win rates of CollabSciReM against (a) CollabLLMSci and (b) the base Instruct models on 300 held-out scientific papers. Bars show 95% confidence intervals. Meth. : Methodological soundness, Just. : Justification, Logic : Logical Consistency, Fact. : Factual Correctness, Novel : Novelty, Resp. : Responsiveness and Ovrl. : Overall
(a) Multi-turn benchmarks
MT-Bench-STEM
MINT
Model
Overall ( ↑ )
Acc. ( ↑ )
Format ( ↑ )
Useful. ( ↑ )
Relev. ( ↑ )
TC ( ↓ )
SR@5 (%) ( ↑ )
TC ( ↓ )
APT ( ↑ )
Llama-3.1-8B-Instruct
6.83 ± 0.04
6.55 ± 0.05
7.92 ± 0.07
6.67 ± 0.08
8.37 ± 0.01
469.00 ± 8.00
47.11 ± 3.20
296.00 ± 14.06
159.20 ± 9.08
Llama-3.1-8B-CollabLLMSci
7.02 ± 0.19
6.69 ± 0.22
8.19 ± 0.01
6.88 ± 0.22
8.55 ± 0.02
298.00 ± 6.00
43.41 ± 3.22
123.91 ± 0.62
350.25 ± 24.14
Llama-3.1-8B-CollabSciReM
7.15 ± 0.20
6.84 ± 0.25
8.19 ± 0.04
6.94 ± 0.16
8.52 ± 0.08
310.00 ± 4.00
36.86 ± 0.89
147.60 ± 8.63
250.47 ± 15.27
Qwen-2.5-7B-Instruct
7.91 ± 0.09
7.62 ± 0.11
8.70 ± 0.07
7.80 ± 0.04
8.90 ± 0.03
555.00 ± 10.00
66.07 ± 1.03
226.34 ± 16.16
292.73 ± 16.98
Table 1: Performance of 7B and 8B models on (a) multi-turn benchmarks (MT-Bench-STEM, MINT) and (b) single-turn benchmarks (GPQA-D, MMLU-Valid, TheoremQA). Acc : accuracy (%). SR@5 : success rate within five interaction turns (%). TC : average tokens per response (per turn for MT-Bench-STEM). APT : accuracy per token, computed as Acc/TC×1000 (SR@5 in place of Acc for MINT). Results are mean ± sample standard deviation over three seeds; bold values indicate the best performance in each column within each model family.
(a) Multi-turn benchmarks
MT-Bench-STEM
MINT
Reward Model
Overall ( ↑ )
Acc. ( ↑ )
Format ( ↑ )
Useful. ( ↑ )
Relev. ( ↑ )
TC ( ↓ )
SR@5 (%) ( ↑ )
TC ( ↓ )
APT ( ↑ )
Rgen+Rstruct
6.81 ± 0.23
6.48 ± 0.23
8.03 ± 0.09
6.67 ± 0.26
8.38 ± 0.07
319 ± 3.00
40.59 ± 2.57
139.31 ± 1.87
291.45 ± 19.60
Rgen+Rsci
6.58 ± 0.26
6.33 ± 0.25
7.74 ± 0.27
6.46 ± 0.43
8.10 ± 0.26
622 ± 15.00
48.30 ± 4.45
699.95 ± 57.14
69.05 ± 4.17
Rstruct+Rsci
6.92 ± 0.15
6.67 ± 0.10
8.06 ± 0.19
6.67 ± 0.12
8.30 ± 0.17
319 ± 8.00
40.74 ± 2.72
149.81 ± 0.99
271.88 ± 16.36
CollabSciReM ∣0.7γ\textscCSciReM
7.14 ± 0.14
6.84 ± 0.08
8.27 ± 0.09
6.85 ± 0.15
8.52 ± 0.23
308 ± 6.00
41.78 ± 4.00
148.24 ± 6.28
282.92 ± 39.05
Table 2: Ablation and sensitivity analysis of reward components for CollabSciReM on Llama-3.1-8B-Instruct, fine-tuned with SFT followed by offline IPO. The upper block reports multi-turn benchmarks, MT-Bench-STEM and MINT, and the lower block reports single-turn benchmarks, GPQA-D, MMLU-Valid, and TheoremQA. Rgen , Rstruct , and Rsci denote the conversational, structural, and science-aware rewards, respectively.
Figure 3: Real-world user study results showcasing participant interactions with anonymized base Llama-3.1-8B-Instruct , general rewards (accuracy and interaction, w/o token penalty) and CollabSciReM models. Users rated (a) logical justification (1-10), (b) overall interactive experience (1-10), and (c) novelty (1-5).
Table 4: Hyperparameters for SFT and IPO training stages.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Synthetic dataset generation pipeline from a research paper to a conversational excerpt. (a) A research paper (excerpt from ( Yadav and Claeys Bouuaert, 2021 ) ) is transformed into (b) a structured representation in which research questions and methodologies are extracted. (c) These components are formalized into conversational excerpts, with user queries grounded in research questions and model responses derived from the associated methodologies.
Figure 5: Snippets from the user study evaluation interface.
Model
Strengths
Weaknesses
Llama-3.1-8B-Instruct
“Clear and answered the questions appropriately.", “Strong structure and clarity."
“Flawed reasoning.", “The model’s approach was generic and lacked innovation, …", “The model provided too many options to try, and its initial response was not thorough at all. It felt as though I had to pry additional steps out of it."
General-reward Variant
“Decent response formatting." , “Clear, concise", “Clear language flow and formatting."
“I had to point it more specifically at research methodology over answering the question.", “It got some technical information wrong..", “Limited novelty."
CollabSciReM
“It was easy to understand; well structured and I liked how it was straight to the point and just gave an overview.", “Clear, structured, and readable"
“While methodologically solid, the responses rely heavily on standard qualitative techniques.", “Too concise…"
Appendix
Table 5: Qualitative Feedback from Human Evaluators.
(a) Multi-turn benchmarks
MT-Bench-STEM
MINT
Model
Overall ( ↑ )
Acc. ( ↑ )
Format ( ↑ )
Useful. ( ↑ )
Relev. ( ↑ )
TC ( ↓ )
SR@5 (%) ( ↑ )
TC ( ↓ )
APT ( ↑ )
Llama-3.2-3B-Instruct
6.40 ± 0.38
6.02 ± 0.46
7.82 ± 0.20
6.25 ± 0.48
8.12 ± 0.29
447 ± 26
35.56 ± 1.94
265.06 ± 20.44
135.01 ± 17.10
Llama-3.2-3B-CollabLLMSci
6.48 ± 0.38
6.06 ± 0.42
7.93 ± 0.24
6.15 ± 0.25
8.24 ± 0.15
344 ± 9
31.85 ± 2.89
174.44 ± 4.47
182.94 ± 21.00
Llama-3.2-3B-CollabSciReM
6.24 ± 0.20
5.83 ± 0.24
7.82 ± 0.09
5.98 ± 0.21
8.26 ± 0.13
358 ± 16
35.41 ± 2.60
181.63 ± 3.06
195.13 ± 17.35
Qwen-2.5-3B-Instruct
7.44 ± 0.06
7.15 ± 0.02
8.54 ± 0.05
7.22 ± 0.09
8.82 ± 0.06
521 ± 23
52.15 ± 0.93
287.66 ± 1.29
181.29 ± 3.24
Appendix
Table 6: Performance of 3B models on (a) multi-turn benchmarks (MT-Bench-STEM, MINT) and (b) single-turn benchmarks (GPQA-D, MMLU-Valid, TheoremQA). Metrics and evaluation settings follow Table 1 . Results are mean ± sample standard deviation over three seeds; bold values indicate the best performance in each column within each model family.
Dimension
Definition
Methodological soundness
Assesses whether the proposed study design is valid, with appropriate controls, careful handling of confounds, sound measurement and analysis, and a feasible plan.
Justification
Assesses whether each design choice is supported by specific, checkable reasons. Citations count only if they identify a real, relevant work, and invented, vague or unverifiable references count against the response.
Logical consistency
Assesses whether the response is free of contradictions within and across turns and whether later turns build coherently on earlier ones.
Factual correctness
Assesses the technical and factual accuracy of the response. The judge lists the concrete errors it finds for each side.
Novelty
Assesses whether the response offers non-obvious, useful ideas that go beyond a textbook answer.
Responsiveness
Assesses how well the assistant addresses the researcher’s evolving requests and questions across turns.
Large Language Models (LLMs) trained on extensive scientific research are increasingly integrated as assistants for scientific discovery. However, most research papers omit the fine-grained cognitive process of examining constraints, failed alternatives, and iterative decisions required to achieve the desired goal. Such cognitive processes are vital for real-world scientists working toward specific goals under constraints. In this paper, we show that LLMs, when trained to produce such cognitive traces, perform better as scientific discovery assistants than when trained solely on scientific literature. We propose COGTRL, a trajectory-level reinforcement learning framework that trains LLMs to emulate cognitively grounded reasoning by jointly optimizing cognitive traces and the scientific steps produced in an interleaved manner. Across two 3B-parameter models and two scientific domains (AI and Materials Science), COGTRL improves method quality by an average of 7.85 points over comparable 3B model baselines and achieves competitive performance relative to 70B parameter models. Moreover, analysis by domain experts shows a preference for methods generated by COGTRL over the baselines.
Large language models (LLMs) are increasingly used to support scientific work, but it is unclear whether they uphold responsible conduct of research (RCR) norms or help undermine them. We introduce SciIntBench, an adversarial benchmark of 810 prompts across ten RCR categories and three scientific domains. Each scenario appears as an Overt Adversarial, Covert Adversarial, and Benign version, allowing us to jointly measure framing-sensitive refusal of misconduct and helpfulness on legitimate requests. We evaluate 16 commercial and open-weight LLMs from six providers (2024--2026), producing 12,960 responses. We find that scientific integrity alignment is strongly framing-sensitive: models refuse explicit misconduct far more reliably than covert violations, especially failing when misconduct is presented as a pressure-driven shortcut. Refusals vary by RCR category, with weaker boundaries around transparency, plagiarism, and fabrication.
Almene De Meran Meguimtsop, Maria Leonor Pacheco, Daniel E. Acuna
Department of Computer Science, University of Colorado Boulder, Boulder, CO 80309
Large language models (LLMs) have recently shown strong progress on scientific reasoning, yet two major bottlenecks remain. First, explicit retrieval fragments reasoning, imposing a hidden "tool tax" of extra tokens and steps. Second, multi-agent pipelines often dilute strong solutions by averaging across all candidates. We address these challenges with a unified framework that combines implicit retrieval and structured collaboration. At its foundation, a Monitor-based retrieval module operates at the token level, integrating external knowledge with minimal disruption to reasoning. On top of this substrate, Hierarchical Solution Refinement (HSR) iteratively designates each candidate as an anchor to be repaired by its peers, while Quality-Aware Iterative Reasoning (QAIR) adapts refinement to solution quality. On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy - the highest reported to date, surpassing the strongest agent baseline by 13.4 points and leading frontier LLMs by up to 18.1 points, while simultaneously reducing token usage by 53.5% and agent steps by 43.7%. Results on SuperGPQA and TRQA confirm robustness across domains. Error analysis shows that reasoning failures and knowledge gaps co-occur in over 85% of cases, while diversity analysis reveals a clear dichotomy: retrieval tasks benefit from solution variety, whereas reasoning tasks favor consensus. Together, these findings demonstrate how implicit augmentation and structured refinement overcome the inefficiencies of explicit tool use and uniform aggregation. The code is available at https://github.com/tangxiangru/Eigen-1.
Xiangru Tang, Wanghan Xu, Yujie Wang +13
Yale University · Shanghai Jiao Tong University · Fudan University +4