cs.CLMar 27, 2026
SaveCoAuthorAI: A Human in the Loop System For Scientific Book Writing
Organizations: Kexin Technology, Beijing 100012, China · Institute for Sustainable Industries and Liveable Cities, Victoria University, VIC 3011, Australia · School of Information Technology, Deakin University, Melbourne, VIC 3125, Australia
Abstract
Large language models (LLMs) are increasingly used in scientific writing but struggle with book-length tasks, often producing inconsistent structure and unreliable citations. We introduce CoAuthorAI, a human-in-the-loop writing system that combines retrieval-augmented generation, expert-designed hierarchical outlines, and automatic reference linking. The system allows experts to iteratively refine text at the sentence level, ensuring coherence and accuracy. In evaluations of 500 multi-domain literature review chapters, CoAuthorAI achieved a maximum soft-heading recall of 98%; in a human evaluation of 100 articles, the generated content reached a satisfaction rate of 82%. The book AI for Rock Dynamics generated with CoAuthorAI and Kexin Technology's LUFFA AI model has been published with Springer Nature. These results show that systematic human-AI collaboration can extend LLMs' capabilities from articles to full-length books, enabling faster and more reliable scientific publishing.
Figures & tables
Figure 1: Overview of CoAuthorAI, illustrating the frontend for expert inputs and revision, and the backend for PDF parsing, content compression, section generation, and post-processing through LLM interaction.
Figure 2: User interface and workflow of the CoAuthorAI system. The interface guides users through a seven-step human–AI collaborative pipeline: (1) selecting or creating a project, (2) viewing and managing book-level information, (3) uploading outlines and reference PDFs, (4) parsing PDF content , (5) compressing literature, (6) manual refinement of AI-generated text, and (7) performing citation tracing.
| LLMs used | S-H Recall | ROUGE-1 | ROUGE-2 | ROUGE-L |
|---|---|---|---|---|
| claude-3.5-sonnet | 0.9802 | 0.4932 | 0.1504 | 0.1467 |
| GPT-4o | 0.9721 | 0.4603 | 0.1417 | 0.1416 |
| KIMI | 0.9604 | 0.4423 | 0.1459 | 0.1301 |
| GPT-4-turbo | 0.9576 | 0.4136 | 0.1357 | 0.1351 |
| Qwen2.5-70B-Instruct | 0.9501 | 0.3903 | 0.1274 | 0.1293 |
| GPT-3.5-turbo | 0.8843 | 0.2651 | 0.0783 | 0.0983 |
Table 1: Performance Comparison of Large Language Models on S-H Recall and ROUGE Metrics
| LLMs used | Prim-eval | Sec-eval | Fir-exam | Fin-exam | Avg_score |
|---|---|---|---|---|---|
| claude-3.5-sonnet | 0.853 | 0.789 | 0.691 | A | 0.821 |
| GPT-4o | 0.788 | 0.673 | 0.621 | B | 0.731 |
| KIMI | 0.751 | 0.636 | 0.632 | B | 0.694 |
| GPT-4-turbo | 0.669 | 0.623 | 0.621 | C | 0.646 |
| Qwen2.5-70B-Instruct | 0.616 | 0.587 | 0.613 | C | 0.602 |
| GPT-3.5-turbo | 0.603 | 0.511 | 0.542 | D | 0.557 |
Table 2: Results of Human Evaluation
| Chapter | citation accuracy | correction rate |
|---|---|---|
| Chapter 2 | 73% | 16% |
| Chapter 3 | 76% | 18% |
| Chapter 4 | 82% | 21% |
| Chapter 5 | 81% | 13% |
| Chapter 6 | 79% | 11% |
| Chapter 7 | 77% | 15% |
Table 3: Citation Accuracy and Manual Correction Rate per Chapter
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Key | Values |
|---|---|
| title | A formula for a quartic integral: a survey of old proofs and some new ones |
| article_id | arXiv:0707.2118 |
| subject | ["Classical Analysis and ODEs"] |
| abstract | We discuss several existing proofs of the value of a quartic integral and present a new proof that evolved from rational Landen … |
| content | 1. Introduction The evaluation of definite integrals has attracted the scientific community, both professional and amateurs, for a long … |
| reference | [ [1] B. Berndt. Ramanujan’s Notebooks, Part I. Springer-Verlag, New York, 1985., [2] G. Boros and V. Moll. An integral hidden in Gradshteyn and Ryzhik. Jour. Comp. Applied Math., 106:361–368, 1999., … ] |
Table 4: An Example from datasets
| Prompts | Content |
|---|---|
| Compression Prompt | Write a detailed research report in English aimed at helping users quickly understand and grasp the core content and key findings of the provided paper. When writing the research report, please follow these steps: 1. Initial Understanding: First, gain a preliminary understanding of the research area or direction provided by the user to ensure clarity regarding their needs. 2. Paper Analysis: Carefully read and analyze the full text of the paper to identify all key research assumptions, princinples, formulas,methods, results, and conclusions. 3. Key Points Extraction: Extract the most important points from the paper, including but not limited to experimental design, data analysis, main findings, conclusions, and formulas. 4. Language Expression: Use precise, clear, and professional language, avoiding vague or ambiguous expressions. 5. Proofreading and Revision: After completing the draft, carefully proofread and revise to ensure there are no grammatical or spelling errors, while also ensuring the accuracy and completeness of the report. The specific requirements for the research report are as follows (!!! Each point is very important !!!): * The length of the research report should be around 4,000 words, with substantial content, avoiding hollow or meaningless sentences. * The report should cover all important points from the paper, ensuring comprehensiveness and avoiding excessive detail on any one specific point while omitting other important content. * The report should retain the most important formulas, tables, and key data from the paper so that users can fully understand the research results. Use markdown format to write these important formulas and tables in the report!!! * Remember, users are interested in the technical points and important findings in the required research paper; the research report should not include content such as authors, acknowledgments, or future directions!!! |
| Generation Prompt | You are a distinguished scholar specializing in writing comprehensive review books. Your current task is to assist users in crafting a specific section of their review book with meticulous attention to detail and scholarly rigor. Input: The user will provide you with: 1. The overall topic, outline, and title of their review book 2. The hierarchical headings for the section you’re tasked to write 3. Titles and abstracts of reference papers (in English or Chinese) to be used for this section Your focus: Write the content for the lowest-level section heading provided (e.g., if given headings up to the third level, you’ll write content for the third-level heading). Requirements: 1. Language: Write in English, regardless of the language of the references. 2. Citations: Use the format "[idx]" or "[idx_1, idx_2, …]" (e.g., [3] or [3, 49]). 3. Content: - Focus solely on the body text; exclude section headings, reference lists, or other supplementary content. - Write comprehensively, aiming for a minimum of 8000 words. - Describe all relevant details extensively. - Include important formulas and tables from the references. - Formulas can be screen on the makrdown file 4. References: Ensure each provided reference is cited at least once. 5. Style: Write as part of a larger work, avoiding introductory or concluding sentences that might be more appropriate for a standalone piece. |
Table 5: Prompts for the CoAuthorAI