CoWriter: Guiding Methodological and Synergetic Human-LLM Co-Authoring

Shmuel Amar; Benjamin Lieber; Ori Shapira; Debanjana Kar; Sebastian Maximilian Dennerlein; Yufang Hou; Ido Dagan

cs.CLOct 22, 2026

CoWriter: Guiding Methodological and Synergetic Human-LLM Co-Authoring

Authors: Shmuel Amar, Benjamin Lieber, Ori Shapira, Debanjana Kar, Sebastian Maximilian Dennerlein, Yufang Hou, Ido Dagan

Organizations: Bar-Ilan University · OriginAI · IT:U Interdisciplinary Transformation University Austria · IBM Software Innovation Lab, India ·

Abstract

While LLMs have transformed text generation, standard chat-and-canvas interfaces often reduce human authoring to quite passive supervision, causing review fatigue, over-reliance on AI, and the sidelining of the human role in long-form, source-grounded writing. Prior systems typically address isolated writing phases, leaving comprehensive end-to-end human-LLM collaboration largely unexplored. In this paper, we introduce CoWriter, a proactive agentic scheme that adapts expert human (co-)authoring methodologies from cognitive science to flexibly steer the entire writing workflow while actively encouraging the user to engage. In an exploratory user study, compared to a standard LLM baseline, authors using CoWriter better expressed their perspectives and maintained a higher sense of document ownership, supported quantitatively by $2.2\times$ more successful agent engagements. Furthermore, external reader judges rated the resulting texts higher in "human touch" qualities like storytelling and integrating unique personal experiences. Overall, our results demonstrate that proactively engaging users through established human practices is beneficial for effective LLM-supported co-authoring, providing useful methodologies, techniques and insights for future research on collaborative agents.

Figures & tables

Figure 1: The CoWriter platform (§ 3 ). The environment comprises a UI that exposes functionality to the user and the agentic system. The Leader proactively guides the iterative authoring workflow ( Table 1 ), outlined within the Leader ’s box. Monitor tracks UI events to update the Writing Log and trigger Leader with system-reminders to maintain the authoring workflow. Reviewer simulates an external reader and provides feedback.

Panel of Table 1 — Table 1: The iterative co-authoring workflow steps (§ 3.3.1 ) encoded in the Leader system prompt (adapted from the writing activities of Lowry et al., 2004 ). The per-step invite instruction specifies how the agent encourages engagement. Colors denote workflow phases ( Figure 1 ): planning , production , finalization , and ad-hoc requests .

Figure 2: Percentage of sessions in which each agent (baseline vs. CoWriter ) discussed each of the seven writing activities. Bold bars indicate

p<0.05

significance of CoWriter over the baseline.

Figure 3: Comparative preferences between the baseline and CoWriter by 14 authors (normalized counts). ∗ , ∗∗ , ∗∗∗ denote

p<0.05,0.01,0.001

, respectively.

Figure 4: Percentage of pairs preferred per system, rated by five human annotators for the four key dimensions. LLM-as-a-judge counterpart in Figure 11 .

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: The user interface for CoWriter , at the end of a session. 1 The task description. 2 A source with its title and short LLM-generated summary; clicking opens the full source text. 3 The title of the document being authored; the body is co-edited by the user and CoWriter via the Yjs-synchronized canvas. 4 An inline link referring to a span in a source document, to support the written claim. 5 Message-level buttons including a diff view of CoWriter ’s edits. 6 CoWriter ’s thought process (expandable), which shows the reasoning and sequence of actions of Leader in real time. 7 When clicked, opens the Writing Log (see an example of a full log in Figure 21 ).

Tool (signature)	Description
Sources
list_source_files() $\to$ SourceFile[]	List metadata for all available source files ( id , type, title, description, status).
get_source_file( id ) $\to$ SourceFile	Return a source file’s full content by id or title.
Document
read_document() $\to$ XML	Return the current shared document as XML ( <heading> , <paragraph> , inline tags).
update_document( xpath , xml , mode )	Insert, replace, or delete a block node using XPath; mode $\in$ { sibling-after , sibling-before , first-child , last-child , replace-node }.

Appendix

Table 2: Tools exposed to the Leader (main) agent. All tool responses are formatted to text for easier consumption by the LLM. The rules for usage of the tools are detailed in the system prompt ( Figure 28 ).

Tool (signature)	Description
get_source_file( id , start =1, end =-1 ) $\to$ lines	Read a source file (by id or title), line-numbered and sliced to the inclusive range [start, end] ; used to verify a <deeplink> ’s cited range.
message( content )	Final review, posted by Leader into chat as a quoted block (one per invocation).

Appendix

Table 3: Tools available to the Reviewer sub-agent. The Reviewer’s ReAct loop interleaves get_source_file lookups (to spot-check cited spans against the actual source lines) and a single final message that returns the structured review ( Appendix E ).

Source document	DOI / ERIC ID
Topic 1: The Influence of Family and Friends on Young Adults (Family)
Family and peer influences on adolescent psychological inflexibility: a regression mixture analysis	10.3389/fpsyg.2025.1650930
Parent-child attachment and mental health in young adolescents: a moderated mediation analysis	10.3389/fpsyg.2023.1298485
Family intimacy and adolescent peer relationships: investigating the mediating role of psychological capital and the moderating role of self-identity	10.3389/fpsyg.2023.1165830
Effects of peer relationships on parent-youth relationships and self-differentiation	10.17220/ijpes.2018.03.002
Parenting, Peer Relationships, Academic Self-efficacy, and Academic Achievement: Direct and Mediating Effects	10.3389/fpsyg.2017.02120

Appendix

Table 4: The 40 research papers used as writing sources in the user study, 10 per topic. Each title links to the paper; the DOI (or ERIC identifier, where no DOI is registered) is given for reference.

Figure 9: Pre-study survey responses of the 14 study participants.

Participant	Session 1	Session 2
P1	B $\cdot$ Family	C $\cdot$ Smartphones
P2	B $\cdot$ Family	C $\cdot$ Education
P3	B $\cdot$ Smartphones	C $\cdot$ Family
P4	B $\cdot$ Education	C $\cdot$ Family
P5	B $\cdot$ Education	C $\cdot$ WLB
P6	C $\cdot$ Education	B $\cdot$ WLB

Appendix

Table 5: Per-participant assignment for the 14 study participants (P1–P14). Each participant completed one B aseline session and one CoWriter session ( C ), each on a different topic. System order and the topic–system pairing were counterbalanced across participants. Topics are given by short code. Full titles and sources are in Table 4 .

Construct	Per-session questionnaire statement
Engagement	I felt the system made me actively engaged in writing this document.
	The system enabled me to express my perspectives during the authoring process.
	The system enabled me to use my choice of content during the authoring process.
	The system offered me useful suggestions throughout the authoring process.
	I felt I am not part of the writing process. †
Task Load (NASA-TLX)	How mentally demanding was the task? †

Appendix

Table 6: The 37 statements of the per-system questionnaire (§ 4.1 ), grouped by construct. After each writing session, participants rated every statement on a 7-point Likert scale (1–7): strongly disagree , disagree , somewhat disagree , neither agree nor disagree , somewhat agree , agree , strongly agree ; the five Task Load items use a 7-point numeric scale adapted from NASA-TLX ( Hart and Staveland, 1988 ) . Statements marked

\dagger

are reverse-coded — their scores are inverted

<8-\text{score}>

so that a higher value always denotes a more positive response.

Construct	Comparative questionnaire question
Engagement	Which system made you more engaged in the writing process?
Source Understanding	Which system enabled you to better understand the sources for the needs of the writing?
Cognitive Load	Which system was less mentally demanding to use?
System Guidance	Which system provided clearer steps for writing the document?
Document Factuality	Which system made it easier to verify cited claims against the sources?
Self-Expression	Which system’s final document feels more like your own tone and writing style?

Appendix

Table 7: The 13 questions of the comparative questionnaire, grouped by construct. After completing both writing sessions, participants answered each question on a 5-point comparative scale: System A more , System A slightly more , tied / no preference , System B slightly more , System B more , where System A and System B denote the participant’s first- and second-session systems, respectively.

Figure 10: Preference counts for the five questions with the highest average preference toward CoWriter in the human reader evaluation. LLM-as-a-judge counterpart in Figure 12 .

Figure 11: Percentage of pairs preferred per system across the four key dimensions, as rated by the three LLM judges over the 48 evaluated pairs. Human-rater counterpart in Figure 4 .

Figure 12: Preference counts for the five questions with the highest average preference toward CoWriter in the LLM-as-a-judge reader evaluation.

Aspect	Reader-evaluation question
Interestingness	Which of the two was written in a more appealing storytelling approach?
	Which of the two was written in a more friendly tone for the readers?
	Which of the two contains more unique ideas or perspectives from the author?
	Which of the two uses the personal experience of its writer that produces more unique writing?
	Which of the two was written in a more convincing manner?
	Which of the two was more enjoyable to read?

Appendix

Table 8: The 19 questions of the human reader questionnaire adapted from ( Lee et al., 2025 ) , grouped by aspect (§ 5.3 ). Annotators answered each question on a 5-point comparative scale: Document A more , Document A slightly more , Tied / No preference , Document B slightly more , Document B more .

System	$P$	$R$	$R_{\text{cited}}$	$R_{\text{uncited}}$	$F_{1}$	$CL$
Baseline	0.988 (0.030)	0.741 (0.123)	0.699 (0.111)	0.760 (0.159)	0.841 (0.091)	12754 (925)
CoWriter	0.996 (0.017)	0.743 (0.105)	0.622 (0.117)	0.802 (0.107)	0.847 (0.073)	205 (41)

Appendix

Table 9: Citation correctness of the final documents, evaluated with LongCite ( Zhang et al., 2025 ) : mean over the 14 baseline and 14 CoWriter documents, standard deviation in parentheses.

P

: citation precision;

R

: citation recall;

F_{1}=2PR/(P+R)

;

CL

: citation length (mean number of words per cited snippet; a lower

CL

marks more localized citations).

R_{\text{cited}}

is recall over statements that cite at least one snippet (judged fully / partially / not supported

=1/0.5/0

);

R_{\text{uncited}}

is recall over statements with no citation (

1

if the statement is a functional sentence requiring no citation, else

0

Figure 14: Workflow adherence per writing activity (§ 5.1 ). Left: Total chat turns per activity, split into agent and user turns. Right: Percentage of sessions discussing each activity at least once, categorized by participation (both parties, agent only, or user only). While 54% of the baseline agent turns concentrate in localized reviewing & revising (vs. 29% for CoWriter ) with low agent involvement in early stages, CoWriter distributes the conversation more uniformly and successfully drives higher user participation across most writing activities.

Figure 15: Aggregated writing activity transitions for the baseline (top) and CoWriter (bottom) across the 28 co-authoring sessions (§ 5.1 ). Arrows indicate transitions between activities, labeled with their total counts. Activity nodes display the percentage of their incoming transitions that are adjacent (i.e., from an immediately preceding or succeeding activity). While both systems are iterative, each featuring over 30% backward transitions, CoWriter transitions are predominantly to adjacent activities (74% vs. 56% for the baseline;

p=0.015

Figure 16: Agent-initiated (left) and user-initiated (right) engagement expressions by type (§ 5.1 ). Light bars show total expressions, while darker subsets denote the portion of successful engagements (i.e., followed up by the other collaborator). While user-initiated engagements are dominated by action requests and remain comparable across systems, CoWriter yields more agent-initiated engagements across all types.

Figure 17: Agent-initiated (left) and user-initiated (right) engagement expressions per writing activity (§ 5.1 ). Light bars show total expressions, while darker subsets denote the portion of successful engagements (i.e., followed up by the other collaborator). While baseline agent-initiated engagements concentrate in localized reviewing & revising (48%), CoWriter distributes engagements much more uniformly across the workflow and achieves higher success rates.

Figure 18: Per-system averaged absolute ratings of the first 19 of the 37 statements ( Table 6 ) on the 7-point Likert scale. The per-statement paired-test significance marker is shown above each pair.

Figure 19: Per-system averaged absolute ratings of the remaining 18 of the 37 statements ( Table 6 ) on the 7-point Likert scale. The per-statement paired-test significance marker is shown above each pair.

Figure 20: Per-question distributions of the 13 comparative questions ( Table 7 ).

P#	System	Moment the system helped	Moment the system got in the way
P1	Baseline	It helped concise the blog post without losing meaningful information	The way I interacted with the system is to let the system take lead in formulating everything upfront and I did the editing. Due to this nature of interaction, there was no way it conflicted with what I did.
P1	CoWriter	It reminded me the missing tasks that I am supposed to do - at the end of the review.Though i didn’t follow through, it was a very important help or reminder.	Nothing
P2	Baseline	help me to write a text within the time limits	used the title of the writing task instead of making a new one
P2	CoWriter	highlight relevant citations	by suggesting a topic ideas that were not factual but match the title, like the importance of formal education on children
P3	Baseline	Focus on the subject	Editing
P3	CoWriter	It helped me aggregate my thoughts in the right direction	No interruption

Appendix

Table 10: Free-text qualitative responses from the per-session questionnaire, one row per (participant, system) and one column per prompt. All 14 participants answered both prompts for both systems. Responses are verbatim.

Metric	Baseline	CoWriter	Sig.
Session duration (min)	67.6 (22.0)	61.6 (19.6)	ns
LLM token cost (USD)	5.57 (5.03)	5.21 (3.80)	ns
User messages count	16.6 (9.7)	22.1 (11.0)	ns
Agent messages count	16.3 (9.9)	25.0 (12.1)	**
Agent total words count	6988 (3897)	5120 (3102)	ns
User revisions count	23.6 (31.5)	14.7 (21.9)	ns

Appendix

Table 11: Per-system automatic measurements across the 14 writing sessions per system. Each cell shows average (standard deviation) across all sessions. Significance (Sig.) markers are from paired within-subject tests on the

<

proactive

-

baseline

>

differences (

p<0.01

** ,

p<0.05

* ; ns otherwise); the t-test / Wilcoxon selection rule is described in Appendix N . CoWriter -only metrics report CoWriter sessions.

Human-AI Collaboration

Human-AI collaboration studies how people and AI systems divide, coordinate, and adapt work to support decisions, creativity, and learning.

See all 440 papers

Detecting LLM-Generated Tokens in Human--LLM Coauthored Text
Jul 23, 2026Yangjun Lu, Hongyi Zhou, Fabian Spill +3AI-Generated Text Detection Human-AI Co-Creation
School of Mathematics, University of Birmingham, Birmingham, UK · School of Statistics and Data Science, Shanghai University of Finance and Economics, Shanghai, China · Department of Statistics, The London School of Economics and Political Science, London, UK
Value Over Language Model: Detecting Original Contribution in Writing
Sep 1, 2026Vibhhu Sharma, Thorsten Joachims, Sarah DeanLLM Evaluation AI-Generated Text Detection
Cornell University
How LLMs Distort Our Written Language
Mar 18, 2026Marwa Abdulhai, Isadora White, Yanming Wan +4Text Generation
UC Berkeley · UC San Diego · University of Washington +2

Jul 23, 2026J/K move · Enter open · S save

Detecting LLM-Generated Tokens in Human--LLM Coauthored Text

Yangjun Lu, Hongyi Zhou, Fabian Spill, Kai Ye+2

School of Mathematics, University of Birmingham, Birmingham, UK · School of Statistics and Data Science, Shanghai University of Finance and Economics, Shanghai, China · Department of Statistics, The London School of Economics and Political Science, London, UK

AI-Generated Text Detection Human-AI Co-Creation

Open paper AskPDF arXiv

The rise of human-AI collaborative writing has created a growing need for fine-grained detection methods that support localizing likely LLM-generated content in mixed-authorship documents. Existing methods for detecting LLM-generated text mainly focus on document-level classification and cannot identify which parts of the text are generated by LLMs. This paper introduces a new method to address this urgent need. Our method operates at the token level, the natural unit of modern language models, and builds on existing token-level detection scores. The key idea is to smooth adjacent token scores to reduce their variability, while using an adaptive Lepski-type rule to select the bandwidth according to the local authorship structure. Our method is simple to implement and does not require token-level labeled data for training. Theoretically, we characterize this trade-off and show that the proposed method achieves favorable mean square error performance in estimating the underlying signal. Empirically, we demonstrate strong performance of our method against a wide range of baselines in both synthetic datasets and a realistic dataset. We deploy a publicly accessible website that implements the methods as well.

Abstract

Human-AI Collaboration

Explore similar work

Detecting LLM-Generated Tokens in Human--LLM Coauthored Text

Value Over Language Model: Detecting Original Contribution in Writing

How LLMs Distort Our Written Language