CWF: A Collaborative Writing Framework for Personalized and Reliable Popular Science Writing

Ruibiao Fu; Di Tang; Yunlong Yang; Ran Wang; Sicheng Lu; Peixuan Wu; Xiaoyu Fan; Jiacheng Ma

cs.AISep 5, 2026

CWF: A Collaborative Writing Framework for Personalized and Reliable Popular Science Writing

Authors: Ruibiao Fu, Di Tang, Yunlong Yang, Ran Wang, Sicheng Lu, Peixuan Wu, Xiaoyu Fan, Jiacheng Ma,

Organizations: School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, China · Philosophy and Social Sciences Laboratory of Big Data and National Communication Strategy, Ministry of Education, Wuhan 430074, China · The National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China

Abstract

We introduce Personalized and Reliable Popular Science Writing, a novel task that requires adapting scientific explanations to audiences with different cognitive levels while preserving factual accuracy. However, improving personalization often introduces simplifications that increase the risk of hallucination and factual distortion. To address these challenges, we first construct a dataset of 39,134 entries and a reader-centric Personalized Science Communication Benchmark (PSCB) that jointly evaluates audience adaptation and factual accuracy. To reduce data and computational requirements while improving generalization across domains and audiences, we introduce DA-MoE, which explicitly decouples audience adaptation from domain knowledge through separate modeling. To enable robust verification and revision in evidence-scarce scenarios, a multi-agent fact-checking mechanism that augments limited evidence with role-specific agent debate and propagates confidence over a graph is proposed. Experiments on PSCB show that our approach achieves state-of-the-art performance. Our code is open-sourced at https://github.com/DPInnovationWorks/CWF.

Figures & tables

Figure 1: Comparison between Specialized Approaches (Fine-tuning and RAG-based methods) and Collaborative Writing Framework (CWF).

Figure 2: Overview of CWF, which integrates collaborative writing, personalized style modeling, and fact-checking with multi-agent discussion to enhance the quality and relevance of AI-generated science writings.

Reader Group	Mean	Std.	Median	Min	Max
Children	0.686	0.101	0.697	0.383	0.846
Teenagers	0.262	0.080	0.256	0.095	0.480
Adults	0.183	0.059	0.176	0.079	0.351

Table 1: Distribution of

\alpha

across different target reader groups (

n=100

per group).

Method	Training Time	Data Amount	Data Type	Average PSCB score
LoRA	2.1 hrs	5000	Dual-style	2.985
MoE	8.3 hrs	20000	Single-style	3.059
DA-MoE	22 min	1000	Single-style	3.451

Table 2: Comparison of training time, data requirements, data type, and average PSCB score in different methods.

Group	Method	LLM	User	Fact. Acc.
Child	Base	2.776	3.293	3.18
	LoRA	2.955	3.286	3.31
	MoE	3.185	3.391	3.47
	Qwen2.5-14B	3.606	3.535	3.67
	CWF	3.683	3.542	3.72
	CWF -R	3.330	3.480	4.05

Table 4: Overall performance across all topics for six methods and three target reader groups. LLM and User denote personalization scores; Fact. Acc. denotes article-level graph-based factual accuracy. All three metrics are reported on a 0–5 scale.

Figure 3: Comparison of consistency with human evaluation across different LLM-as-a-judge evaluation. The closer evaluation curve is to the human evaluation, the stronger its alignment with human judgments.

Method	Dynamic	Average	One-Step	User
Base	2.965	2.276	2.183	3.510
LoRA	2.984	2.602	2.576	3.275
MoE	3.059	2.672	2.462	3.449
Qwen2.5-14B	3.209	2.863	2.763	3.705
CWF	3.451	3.119	2.828	3.794
CWF -R	3.302	2.914	3.011	3.736

Table 5: Consistency between LLM & human scores.

Setting	S	P	U	SP $\uparrow$	PU $\uparrow$	Var. $\downarrow$
Full	4.209	2.720	1.327	1.489	1.393	0.482
w/o DC	4.183	2.737	1.364	1.446	1.373	0.561
w/o DB	3.947	2.886	1.618	1.061	1.268	0.876
w/o GP	4.028	2.813	1.742	1.215	1.071	0.694

Table 6: Claim-level diagnostic ablation on SciFact. S / P / U : mean graph confidence in

[0,5]

for supported / partially supported / unsupported claims. SP = S

-

P and PU = P

-

U are confidence gaps, rather than article-level scores. Var. : average confidence variance. w/o DC: without claim decomposition; w/o DB: without debate; w/o GP: without graph propagation.

Method	Sup. (#) $\uparrow$	Unsup. (#) $\downarrow$	Prec $\uparrow$	R@64 $\uparrow$	F1@64 $\uparrow$
CWF	36.93	27.01	0.578	0.577	0.577
RAG	40.35	23.52	0.632	0.630	0.631
SAFE	42.08	21.37	0.663	0.657	0.660
MAD	44.43	18.86	0.702	0.694	0.698
CWF -R	45.76	17.92	0.719	0.715	0.717

Table 7: Auxiliary factuality evaluation using SAFE. Sup./Unsup. are the average numbers of supported / not-supported atomic facts per article. Precision is

S/(S+N)

, R@64 is

\min(S/64,1)

. These three ratio-based metrics range from 0 to 1.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Judge model	MAE $\downarrow$	Spearman $\rho$ $\uparrow$	Kendall $\tau_{b}$ $\uparrow$	PairAcc $\uparrow$	CWF rank
Qwen-Plus	0.417	0.829	0.733	86.7%	1
Qwen3.6-Plus	0.382	0.943	0.867	93.3%	1
GPT-5.5	0.401	0.886	0.800	90.0%	1
DeepSeek-V4-Pro	0.361	0.914	0.822	91.1%	2
DeepSeek-V3.2	0.395	0.900	0.800	90.0%	1
MiniMax-M2.5	0.446	0.829	0.733	86.7%	1

Appendix

Table 9: Alignment of each judge model with human evaluation over the six systems.

Reader group	Kendall’s $W$ $\uparrow$	Interpretation
Child	0.902	Very high
Teen	0.811	High
Adult	0.784	Substantial
Macro average	0.832	High

Appendix

Table 10: Cross-judge concordance of system rankings.

Protocol	MAE $\downarrow$	MARD $\downarrow$	Spearman $\rho$ $\uparrow$	Kendall $\tau_{b}$ $\uparrow$	PairAcc $\uparrow$
Dynamic dimension weighting	0.417	0.667	0.829	0.733	86.7%
Average dimension weighting	0.837	0.667	0.829	0.733	86.7%
One-step LLM scoring	0.941	1.000	0.714	0.467	73.3%

Appendix

Table 11: Qwen-Plus–human agreement under three scoring protocols.

Group	$\rho$	$\tau_{b}$	Correct pairs	PairAcc
Child	0.943	0.867	14/15	93.3%
Teen	0.714	0.600	12/15	80.0%
Adult	0.714	0.600	12/15	80.0%
Macro average	0.790	0.689	38/45	84.4%

Appendix

Table 12: Qwen-Plus–human ordinal agreement by reader group.

Metric	Child	Teen	Adult	Overall
Pairwise preference accuracy	90.6%	85.8%	86.9%	87.8%
Cohen’s $\kappa$	0.807	0.716	0.739	0.754
Left/right consistency	97.1%	96.2%	96.8%	96.7%
Evaluated decisions	900	900	900	2,700

Appendix

Table 13: Direct topic-level preference agreement on held-out topics.

Rounds	S–P $\uparrow$	P–U $\uparrow$	Var. $\downarrow$
1	1.125	1.050	0.852
2	1.340	1.258	0.635
3	1.489	1.393	0.482
4	1.492	1.398	0.527

Appendix

Table 14: Effect of the maximum number of debate rounds. S–P and P–U denote the support–partial and partial–unsupported confidence gaps, respectively; Var. denotes average confidence variance.

Model	Sup.	Par.	Unsup.	S–P	P–U	Var.
GPT-5.5	4.547	2.763	1.138	1.784	1.625	0.458
deepSeek V4 pro	4.478	2.751	1.198	1.727	1.553	0.463
gpt-4o-mini	4.317	2.708	1.312	1.609	1.396	0.488
deepSeek V3.2	4.376	2.729	1.247	1.647	1.482	0.502
qwen-3.6-plus	4.347	2.726	1.231	1.621	1.495	0.478
qwen-plus	4.209	2.720	1.327	1.489	1.393	0.482

Appendix

Table 15: Robustness across base models. Sup./Par./Unsup. : mean scores under supported / partially supported / unsupported settings; S–P / P–U : support–partial and partial–unsupport gaps; Var. : mean variance.

Figure 4: Effect of Maximum Debate Rounds. The figure illustrates how confidence margins and average variance evolve as the number of debate rounds increases from 1 to 4.

Figure 5: Robustness across Different Base Models. The confidence scores for supported, partially supported, and unsupported claims are compared across three backbone models.

Type	Condition	CWF -R mechanism	Outcome or limitation
Success	Evidence-direction conflict	The Skeptic Agent identifies whether the retrieved evidence supports the opposite direction, and graph propagation incorporates the negative relation.	Surface-relevant evidence is less likely to be incorrectly treated as factual support.
Success	Claim–evidence relation mismatch	The graph models the relation between an atomic claim and its evidence instead of relying only on keyword or topic overlap.	Evidence that is relevant but does not entail the claim receives lower influence.
Success	Numerical inconsistency	Claim decomposition isolates numerical statements so that their values can be verified independently.	Local numerical errors are less likely to be hidden within an otherwise plausible paragraph.
Success	Weak agent consensus	Stance strength and evidence direction are propagated together rather than treating all agent votes equally.	Agreement based on weak or indirect evidence does not automatically produce high confidence.
Success	Over-generalized claim	Claim decomposition separates broad statements into independently verifiable factual units.	Evidence supporting only a narrow conclusion is less likely to validate a broader claim.
Failure	Missing retrievable evidence	The graph receives insufficient external support and therefore assigns low confidence or triggers a conservative revision.	A correct but unverifiable claim may be removed or weakened.

Appendix

Table 16: Applicability conditions and failure modes of CWF -R . Success conditions contain detectable conflict or uncertainty signals that can be calibrated by graph propagation; failure conditions lack a reliable corrective signal.

Setting	Prompt	Resp.	Total
SAFE	8967	6220	15187
Multi-agent Debate	85004	4535	89539
CWF(3 rounds)	50355	2968	53323

Appendix

Table 17: Token usage (prompt / response / total).

Figure 6: Token Usage across Fact-Checking Methods. Prompt, response, and total token consumption are compared for SAFE, Multi-agent Debate, and CWF.

Baseline	Child	Teen	Adult	Total
Base	30	30	30	90
LoRA	30	30	30	90
MoE	30	30	30	90
Qwen2.5-14B	30	30	30	90
CWF	30	30	30	90
CWF -R	30	30	30	90

Appendix

Table 18: Distribution of completed questionnaires across audience groups and baselines. Each participant evaluates all six baselines once.

Group	Mean Age	Gender (M/F)	Reading Frequency
Child	$10.4\pm 1.1$	15 / 15	3.2 / 5
Teen	$15.6\pm 1.3$	14 / 16	3.5 / 5
Adult	$26.8\pm 6.4$	16 / 14	3.8 / 5

Appendix

Table 19: Participant demographics. Gender is reported as male/female, and reading frequency is measured on a five-point scale.

Stage	Child	Teen	Adult	Total
Recruited participants	33	33	32	98
Questionnaires received before exclusion	197	197	191	585
Incomplete six-system set	1	1	1	3
Duplicate/identity check failure	0	1	1	2
Reading-time or invariant-response exclusion	2	1	0	3
Retained participants	30	30	30	90

Appendix

Table 20: Recruitment and response-quality flow. Exclusion counts are participant-level except for questionnaires received and retained.

Diagnostic	Result
Missing rating items	0/21,600 (0.00%)
Out-of-range rating cells	0/21,600 (0.00%)
Duplicate retained submissions	0
Median questionnaire time	824 s
5th–95th percentile of time	361–1,742 s
Responses retained	540/585 (92.3%)

Appendix

Table 21: Diagnostics for retained questionnaires.

Metric	$\alpha_{K}$	95% CI	ICC $(2,k)$	Mean $r_{wg}$	$\alpha_{C}$
Cognitive Load	0.781	[0.742, 0.816]	0.973	0.792	0.908
Personalization Alignment	0.803	[0.769, 0.835]	0.976	0.773	0.939
Reader Attitude	0.793	[0.756, 0.827]	0.975	0.759	0.915
Overall personalization	0.812	[0.779, 0.841]	0.977	0.794	0.948

Appendix

Table 22: Metric-level human-evaluation reliability.

Metric	Dimension	$\alpha_{K}$	95% CI	ICC $(2,k)$	Mean $r_{wg}$
CL	INTR: Intrinsic Fit	0.768	[0.719, 0.811]	0.971	0.787
CL	EXTR: Extraneous Burden Control	0.741	[0.689, 0.787]	0.966	0.774
CL	GERM: Germane Support	0.806	[0.765, 0.842]	0.976	0.815
PA	CONT: Content Relevance	0.779	[0.734, 0.819]	0.972	0.770
PA	KNOW: Knowledge-Level Fit	0.824	[0.786, 0.856]	0.979	0.792
PA	STYLE: Style Consistency	0.752	[0.701, 0.797]	0.968	0.755

Appendix

Table 23: Dimension-level inter-participant agreement and reliability of participant-group means.

Group	CL	PA	RA	Overall
Child	0.754	0.773	0.761	0.785
Teen	0.789	0.811	0.802	0.819
Adult	0.801	0.826	0.815	0.833

Appendix

Table 24: Metric-level ordinal Krippendorff’s

\alpha_{K}

by reader group.

Score	Meaning
0	completely disagree / completely not applicable
1	mostly disagree
2	slightly disagree
3	neutral / hard to judge
4	mostly agree
5	completely agree / perfectly applicable

Appendix

Table 25: Response scale used in the human evaluation questionnaire.

Group	Comparison	$p_{\mathrm{Holm}}$	$\|d_{z}\|$	Sig.
Child	CWF vs Qwen2.5-14B	0.0060	0.230	**
	CWF -R vs Qwen2.5-14B	0.0016	0.283	**
	CWF -R vs CWF	$<0.0001$	0.381
Teen	CWF vs Qwen2.5-14B	0.0032	0.275	**
	CWF -R vs Qwen2.5-14B	0.0160	0.220
	CWF -R vs CWF	0.0160	0.210

Appendix

Table 28: Paired significance tests for LLM personalization scores after aggregating across scientific domains.

p_{\mathrm{Holm}}

denotes the two-sided paired-randomization

p

-value after Holm correction within each reader group;

|d_{z}|

is the magnitude of the paired standardized effect. Significance codes are *

p<0.05

, **

p<0.01

, ***

p<0.001

, and ns

p\geq 0.05

. Method means are omitted because they are reported in Table 4 .

Domain	Group	Comparison	$t$ -statistic	$p$ -value	Sig.
AI	Adult	CWF -R vs CWF	2.145	0.0425
	Teens	CWF -R vs CWF	3.210	0.0035	**
	Child	CWF -R vs CWF	1.856	0.0782	$\dagger$
Biology	Adult	CWF -R vs CWF	3.942	0.0008
	Teens	CWF -R vs CWF	2.406	0.0242
	Child	CWF -R vs CWF	3.093	0.0051	**

Appendix

Table 29: Two-sided paired

t

-tests comparing article-level factual accuracy between CWF -R and CWF on matched evaluation topics. Positive

t

-statistics indicate higher factual accuracy after revision. Reported

p

-values are unadjusted. Significance codes are *

p<0.05

, **

p<0.01

, and ***

p<0.001

;

\dagger

denotes

p<0.1

but not conventional statistical significance.

Scientific Communication

Scientific communication studies how research is written, shared, translated, visualized, and understood by specialist and public audiences.

See all 50 papers

Uncertainty-Aware Web-Conditioned Scientific Fact-Checking
Apr 13, 2026Ashwin Vinod, Katrin ErkAutomated Fact-Checking Scientific Claim Verification
University of Texas at Austin · University of Massachusetts Amherst
Same Verdict, Different Readers: Personalizing Fact-Checking Explanations from Relative Preferences
Oct 27, 2026Haokun Wen, Xuemeng Song, Mong-Li Lee +1Automated Fact-Checking Personalized Language Model Alignment
National University of Singapore Singapore · Southern University of Science and Technology Shenzhen, China
ScienceJudge: Pattern Perturbation Augmentation for Sentence-Level Scientific Claim Verification
Oct 29, 2026Chuhai Cai, Xin Chen, Min Yang +1Claim Verification Scientific Claim Verification
Institute of Artificial Intelligence, Shenzhen University of Advanced Technology, Shenzhen, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

Apr 13, 2026J/K move · Enter open · S save

Uncertainty-Aware Web-Conditioned Scientific Fact-Checking

Ashwin Vinod, Katrin Erk

University of Texas at Austin · University of Massachusetts Amherst

Automated Fact-Checking Scientific Claim Verification

Open paper AskPDF arXiv

Scientific fact-checking is vital for assessing claims in specialized domains such as biomedicine and materials science, yet existing systems often hallucinate or apply inconsistent reasoning, especially when verifying technical, compositional claims against an evidence snippet under source and cost/latency constraints. We present a pipeline centered on atomic predicate-argument decomposition and calibrated, uncertainty-gated corroboration: atomic facts are aligned to local snippets via embeddings, verified by a compact evidence-grounded checker, and only facts with uncertain support trigger domain-restricted web search over authoritative sources. The system supports both binary and tri-valued classification where it predicts labels from Supported, Refuted, NEI for three-way tasks. We evaluate under two regimes, Context-Only (no web) and Context+Web (uncertainty-gated web corroboration); when retrieved evidence conflicts with the provided context, we abstain with NEI rather than overriding the context. On multiple benchmarks, our framework surpasses the strongest benchmarks. In our experiments, web corroboration was invoked for only a minority of atomic facts on average, indicating that external evidence is consulted selectively under calibrated uncertainty rather than routinely. Overall, coupling atomic granularity with calibrated, uncertainty-gated corroboration yields more interpretable and context-conditioned verification, making the approach well-suited to high-stakes, single-document settings that demand traceable rationales, predictable cost/latency, and conservative.

Abstract

Scientific Communication

Explore similar work

Uncertainty-Aware Web-Conditioned Scientific Fact-Checking

Same Verdict, Different Readers: Personalizing Fact-Checking Explanations from Relative Preferences

ScienceJudge: Pattern Perturbation Augmentation for Sentence-Level Scientific Claim Verification