CWF: A Collaborative Writing Framework for Personalized and Reliable Popular Science Writing
Authors: Ruibiao Fu, Di Tang, Yunlong Yang, Ran Wang, Sicheng Lu, Peixuan Wu, Xiaoyu Fan, Jiacheng Ma, HaoZhe Luo, Yang Xiao
Organizations: School of Journalism and Information Communication, Huazhong University of Science and Technology, Wuhan 430074, China · Philosophy and Social Sciences Laboratory of Big Data and National Communication Strategy, Ministry of Education, Wuhan 430074, China · The National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China
We introduce Personalized and Reliable Popular Science Writing, a novel task that requires adapting scientific explanations to audiences with different cognitive levels while preserving factual accuracy. However, improving personalization often introduces simplifications that increase the risk of hallucination and factual distortion. To address these challenges, we first construct a dataset of 39,134 entries and a reader-centric Personalized Science Communication Benchmark (PSCB) that jointly evaluates audience adaptation and factual accuracy. To reduce data and computational requirements while improving generalization across domains and audiences, we introduce DA-MoE, which explicitly decouples audience adaptation from domain knowledge through separate modeling. To enable robust verification and revision in evidence-scarce scenarios, a multi-agent fact-checking mechanism that augments limited evidence with role-specific agent debate and propagates confidence over a graph is proposed. Experiments on PSCB show that our approach achieves state-of-the-art performance. Our code is open-sourced at https://github.com/DPInnovationWorks/CWF.
Figures & tables
Figure 1: Comparison between Specialized Approaches (Fine-tuning and RAG-based methods) and Collaborative Writing Framework (CWF).
Figure 2: Overview of CWF, which integrates collaborative writing, personalized style modeling, and fact-checking with multi-agent discussion to enhance the quality and relevance of AI-generated science writings.
Reader Group
Mean
Std.
Median
Min
Max
Children
0.686
0.101
0.697
0.383
0.846
Teenagers
0.262
0.080
0.256
0.095
0.480
Adults
0.183
0.059
0.176
0.079
0.351
Table 1: Distribution of α across different target reader groups ( n=100 per group).
Method
Training Time
Data Amount
Data Type
Average PSCB score
LoRA
2.1 hrs
5000
Dual-style
2.985
MoE
8.3 hrs
20000
Single-style
3.059
DA-MoE
22 min
1000
Single-style
3.451
Table 2: Comparison of training time, data requirements, data type, and average PSCB score in different methods.
Group
Method
LLM
User
Fact. Acc.
Child
Base
2.776
3.293
3.18
LoRA
2.955
3.286
3.31
MoE
3.185
3.391
3.47
Qwen2.5-14B
3.606
3.535
3.67
CWF
3.683
3.542
3.72
CWF -R
3.330
3.480
4.05
Table 4: Overall performance across all topics for six methods and three target reader groups. LLM and User denote personalization scores; Fact. Acc. denotes article-level graph-based factual accuracy. All three metrics are reported on a 0–5 scale.
Figure 3: Comparison of consistency with human evaluation across different LLM-as-a-judge evaluation. The closer evaluation curve is to the human evaluation, the stronger its alignment with human judgments.
Method
Dynamic
Average
One-Step
User
Base
2.965
2.276
2.183
3.510
LoRA
2.984
2.602
2.576
3.275
MoE
3.059
2.672
2.462
3.449
Qwen2.5-14B
3.209
2.863
2.763
3.705
CWF
3.451
3.119
2.828
3.794
CWF -R
3.302
2.914
3.011
3.736
Table 5: Consistency between LLM & human scores.
Setting
S
P
U
SP ↑
PU ↑
Var. ↓
Full
4.209
2.720
1.327
1.489
1.393
0.482
w/o DC
4.183
2.737
1.364
1.446
1.373
0.561
w/o DB
3.947
2.886
1.618
1.061
1.268
0.876
w/o GP
4.028
2.813
1.742
1.215
1.071
0.694
Table 6: Claim-level diagnostic ablation on SciFact. S / P / U : mean graph confidence in [0,5] for supported / partially supported / unsupported claims. SP = S − P and PU = P − U are confidence gaps, rather than article-level scores. Var. : average confidence variance. w/o DC: without claim decomposition; w/o DB: without debate; w/o GP: without graph propagation.
Method
Sup. (#) ↑
Unsup. (#) ↓
Prec ↑
R@64 ↑
F1@64 ↑
CWF
36.93
27.01
0.578
0.577
0.577
RAG
40.35
23.52
0.632
0.630
0.631
SAFE
42.08
21.37
0.663
0.657
0.660
MAD
44.43
18.86
0.702
0.694
0.698
CWF -R
45.76
17.92
0.719
0.715
0.717
Table 7: Auxiliary factuality evaluation using SAFE. Sup./Unsup. are the average numbers of supported / not-supported atomic facts per article. Precision is S/(S+N) , R@64 is min(S/64,1) . These three ratio-based metrics range from 0 to 1.
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
Judge model
MAE ↓
Spearman ρ↑
Kendall τb↑
PairAcc ↑
CWF rank
Qwen-Plus
0.417
0.829
0.733
86.7%
1
Qwen3.6-Plus
0.382
0.943
0.867
93.3%
1
GPT-5.5
0.401
0.886
0.800
90.0%
1
DeepSeek-V4-Pro
0.361
0.914
0.822
91.1%
2
DeepSeek-V3.2
0.395
0.900
0.800
90.0%
1
MiniMax-M2.5
0.446
0.829
0.733
86.7%
1
Appendix
Table 9: Alignment of each judge model with human evaluation over the six systems.
Reader group
Kendall’s W↑
Interpretation
Child
0.902
Very high
Teen
0.811
High
Adult
0.784
Substantial
Macro average
0.832
High
Appendix
Table 10: Cross-judge concordance of system rankings.
Protocol
MAE ↓
MARD ↓
Spearman ρ↑
Kendall τb↑
PairAcc ↑
Dynamic dimension weighting
0.417
0.667
0.829
0.733
86.7%
Average dimension weighting
0.837
0.667
0.829
0.733
86.7%
One-step LLM scoring
0.941
1.000
0.714
0.467
73.3%
Appendix
Table 11: Qwen-Plus–human agreement under three scoring protocols.
Group
ρ
τb
Correct pairs
PairAcc
Child
0.943
0.867
14/15
93.3%
Teen
0.714
0.600
12/15
80.0%
Adult
0.714
0.600
12/15
80.0%
Macro average
0.790
0.689
38/45
84.4%
Appendix
Table 12: Qwen-Plus–human ordinal agreement by reader group.
Metric
Child
Teen
Adult
Overall
Pairwise preference accuracy
90.6%
85.8%
86.9%
87.8%
Cohen’s κ
0.807
0.716
0.739
0.754
Left/right consistency
97.1%
96.2%
96.8%
96.7%
Evaluated decisions
900
900
900
2,700
Appendix
Table 13: Direct topic-level preference agreement on held-out topics.
Rounds
S–P ↑
P–U ↑
Var. ↓
1
1.125
1.050
0.852
2
1.340
1.258
0.635
3
1.489
1.393
0.482
4
1.492
1.398
0.527
Appendix
Table 14: Effect of the maximum number of debate rounds. S–P and P–U denote the support–partial and partial–unsupported confidence gaps, respectively; Var. denotes average confidence variance.
Model
Sup.
Par.
Unsup.
S–P
P–U
Var.
GPT-5.5
4.547
2.763
1.138
1.784
1.625
0.458
deepSeek V4 pro
4.478
2.751
1.198
1.727
1.553
0.463
gpt-4o-mini
4.317
2.708
1.312
1.609
1.396
0.488
deepSeek V3.2
4.376
2.729
1.247
1.647
1.482
0.502
qwen-3.6-plus
4.347
2.726
1.231
1.621
1.495
0.478
qwen-plus
4.209
2.720
1.327
1.489
1.393
0.482
Appendix
Table 15: Robustness across base models. Sup./Par./Unsup. : mean scores under supported / partially supported / unsupported settings; S–P / P–U : support–partial and partial–unsupport gaps; Var. : mean variance.
Figure 4: Effect of Maximum Debate Rounds. The figure illustrates how confidence margins and average variance evolve as the number of debate rounds increases from 1 to 4.
Figure 5: Robustness across Different Base Models. The confidence scores for supported, partially supported, and unsupported claims are compared across three backbone models.
Type
Condition
CWF -R mechanism
Outcome or limitation
Success
Evidence-direction conflict
The Skeptic Agent identifies whether the retrieved evidence supports the opposite direction, and graph propagation incorporates the negative relation.
Surface-relevant evidence is less likely to be incorrectly treated as factual support.
Success
Claim–evidence relation mismatch
The graph models the relation between an atomic claim and its evidence instead of relying only on keyword or topic overlap.
Evidence that is relevant but does not entail the claim receives lower influence.
Success
Numerical inconsistency
Claim decomposition isolates numerical statements so that their values can be verified independently.
Local numerical errors are less likely to be hidden within an otherwise plausible paragraph.
Success
Weak agent consensus
Stance strength and evidence direction are propagated together rather than treating all agent votes equally.
Agreement based on weak or indirect evidence does not automatically produce high confidence.
Success
Over-generalized claim
Claim decomposition separates broad statements into independently verifiable factual units.
Evidence supporting only a narrow conclusion is less likely to validate a broader claim.
Failure
Missing retrievable evidence
The graph receives insufficient external support and therefore assigns low confidence or triggers a conservative revision.
A correct but unverifiable claim may be removed or weakened.
Appendix
Table 16: Applicability conditions and failure modes of CWF -R . Success conditions contain detectable conflict or uncertainty signals that can be calibrated by graph propagation; failure conditions lack a reliable corrective signal.
Table 23: Dimension-level inter-participant agreement and reliability of participant-group means.
Group
CL
PA
RA
Overall
Child
0.754
0.773
0.761
0.785
Teen
0.789
0.811
0.802
0.819
Adult
0.801
0.826
0.815
0.833
Appendix
Table 24: Metric-level ordinal Krippendorff’s αK by reader group.
Score
Meaning
0
completely disagree / completely not applicable
1
mostly disagree
2
slightly disagree
3
neutral / hard to judge
4
mostly agree
5
completely agree / perfectly applicable
Appendix
Table 25: Response scale used in the human evaluation questionnaire.
Group
Comparison
pHolm
∣dz∣
Sig.
Child
CWF vs Qwen2.5-14B
0.0060
0.230
**
CWF -R vs Qwen2.5-14B
0.0016
0.283
**
CWF -R vs CWF
<0.0001
0.381
Teen
CWF vs Qwen2.5-14B
0.0032
0.275
**
CWF -R vs Qwen2.5-14B
0.0160
0.220
CWF -R vs CWF
0.0160
0.210
Appendix
Table 28: Paired significance tests for LLM personalization scores after aggregating across scientific domains. pHolm denotes the two-sided paired-randomization p -value after Holm correction within each reader group; ∣dz∣ is the magnitude of the paired standardized effect. Significance codes are * p<0.05 , ** p<0.01 , *** p<0.001 , and ns p≥0.05 . Method means are omitted because they are reported in Table 4 .
Domain
Group
Comparison
t -statistic
p -value
Sig.
AI
Adult
CWF -R vs CWF
2.145
0.0425
Teens
CWF -R vs CWF
3.210
0.0035
**
Child
CWF -R vs CWF
1.856
0.0782
†
Biology
Adult
CWF -R vs CWF
3.942
0.0008
Teens
CWF -R vs CWF
2.406
0.0242
Child
CWF -R vs CWF
3.093
0.0051
**
Appendix
Table 29: Two-sided paired t -tests comparing article-level factual accuracy between CWF -R and CWF on matched evaluation topics. Positive t -statistics indicate higher factual accuracy after revision. Reported p -values are unadjusted. Significance codes are * p<0.05 , ** p<0.01 , and *** p<0.001 ; † denotes p<0.1 but not conventional statistical significance.
Scientific fact-checking is vital for assessing claims in specialized domains such as biomedicine and materials science, yet existing systems often hallucinate or apply inconsistent reasoning, especially when verifying technical, compositional claims against an evidence snippet under source and cost/latency constraints. We present a pipeline centered on atomic predicate-argument decomposition and calibrated, uncertainty-gated corroboration: atomic facts are aligned to local snippets via embeddings, verified by a compact evidence-grounded checker, and only facts with uncertain support trigger domain-restricted web search over authoritative sources. The system supports both binary and tri-valued classification where it predicts labels from Supported, Refuted, NEI for three-way tasks. We evaluate under two regimes, Context-Only (no web) and Context+Web (uncertainty-gated web corroboration); when retrieved evidence conflicts with the provided context, we abstain with NEI rather than overriding the context. On multiple benchmarks, our framework surpasses the strongest benchmarks. In our experiments, web corroboration was invoked for only a minority of atomic facts on average, indicating that external evidence is consulted selectively under calibrated uncertainty rather than routinely. Overall, coupling atomic granularity with calibrated, uncertainty-gated corroboration yields more interpretable and context-conditioned verification, making the approach well-suited to high-stakes, single-document settings that demand traceable rationales, predictable cost/latency, and conservative.
Ashwin Vinod, Katrin Erk
University of Texas at Austin · University of Massachusetts Amherst
Fact-checking explanations help Web users understand why a claim receives its verdict, yet the same explanation may not satisfy readers with different backgrounds and preferences. We formulate Faithful Personalized Fact-Checking Explanation Rewriting, which adapts an existing explanation to individual readers while preserving factual fidelity. Rather than relying on reader-written reference explanations, we learn personalized rewriting from the relative preferences of LLM-simulated readers. However, this presents two challenges: distinguishing persona-specific preferences from general explanation quality and learning beyond a limited set of predefined explanation styles. To address these challenges, we propose Personalized Explanation Rewriting via Preference Learning (PERPL), a framework consisting of two components. Simulated Preference Construction derives persona-specific supervision from simulated reader preferences through cross-persona reciprocal pairing. Progressive Preference Learning first trains a lightweight rewriter on the constructed supervision and then refines it through exploration-guided preference optimization, enabling it to discover preferred explanations beyond the predefined styles. Extensive experiments demonstrate that PERPL outperforms much larger general-purpose LLMs in personalized preference alignment while maintaining high factual fidelity.
Haokun Wen, Xuemeng Song, Mong-Li Lee +1
National University of Singapore Singapore · Southern University of Science and Technology Shenzhen, China
Sentence-level scientific claim verification determines whether each sentence in an AI-generated scientific paragraph is supported by a given multimodal evidence bundle. Fine-grained reporting errors often preserve fluent and plausible statements while introducing subtle evidence-claim deviations, making them difficult to detect under sparse and imbalanced supervision. We propose Pattern Perturbation Augmentation (PPA), which models reporting errors as label-specific faithfulness breaking transformations and induces reusable error patterns from labeled examples. PPA applies these patterns to supported claims to construct controlled hard negatives and expands their coverage through pattern-level, operation-level, and cross-domain augmentation. Based on these augmented views, we develop ScienceJudge, which integrates PPA-trained vision-language predictors and instruction-guided LLM predictors through class-weighted ensemble inference. On the local development set, ScienceJudge improves the average score from 0.8124 to 0.8692, achieving a Macro-F1 of 0.8466 and a Paragraph Exact Match score of 0.8919. Our submitted system ranks first on the official NLPCC 2026 Task 10 Track 1 leaderboard with a final score of 0.6890.
Chuhai Cai, Xin Chen, Min Yang +1
Institute of Artificial Intelligence, Shenzhen University of Advanced Technology, Shenzhen, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China