Readers Prefer Outputs of AI Trained on Copyrighted Books over Expert Human Writers
Authors: Tuhin Chakrabarty, Jane C. Ginsburg, Paramveer Dhillon
Organizations: Department of Computer Science, Stony Brook University. · Columbia Law School. · School of Information Science, University of Michigan. · MIT Initiative on the Digital Economy.
The use of copyrighted books for training AI has sparked lawsuits from authors concerned about AI generating derivative content. Yet whether these models can produce high-quality literary text emulating authors' voices remains unclear. We conducted a preregistered study comparing MFA-trained writers with three frontier models (ChatGPT, Claude, Gemini) writing up to 450-word excerpts emulating 50 award-winning authors' styles. In blind pairwise evaluations by 28 MFA-trained readers and 516 college-educated general readers, AI text from in-context prompting was strongly disfavored by MFA readers for stylistic fidelity (OR=0.16) and quality (OR=0.13), while general readers showed no fidelity preference (OR=1.06) but favored AI for quality (OR=1.82). Fine-tuning ChatGPT on authors' complete works reversed these results: MFA readers favored AI for fidelity (OR=8.16) and quality (OR=1.87), with general readers showing even stronger preference (fidelity OR=16.65; quality OR=5.42). Both groups preferred fine-tuned AI, but the writer-type X reader-type interaction remained significant (p=0.021 for fidelity; p<10^-4 for quality), indicating general readers favored AI by a wider margin. Effects are robust under cluster-robust inference and generalize across authors in heterogeneity analyses. Fine-tuned outputs were rarely flagged as AI-generated (3% vs. 97% for prompting) by leading detectors. Mediation analysis shows fine-tuning eliminates detectable AI quirks that penalize in-context outputs, altering the nexus between detectability and preference. While not accounting for effort to transform AI output into publishable prose, the median fine-tuning cost of $81 per author represents a 99.7% reduction versus typical writer compensation. Author-specific fine-tuning enables non-verbatim AI writing preferred over expert human writing, providing evidence relevant to copyright's fourth fair-use factor.
Figures & tables
Figure 1: Figure showing our study design. (1) Select a target author and prompt. (2) Generate up to 450-word candidate excerpts from MFA experts and from LLMs under two settings: in-context prompting (instructions + few-shot examples) and author-specific fine-tuning (model fine-tuned on that author’s works). (3) Readers (MFA-trained and college-educated) perform blinded, pairwise forced-choice evaluations on two outcomes: stylistic fidelity to the target author and overall writing quality. Pair order and left/right placement are randomized on every trial.
Figure 2: (A-B) Forest plots showing odds ratios (OR) and 95% confidence intervals comparing AI-generated and human-written excerpts in pairwise evaluations of stylistic fidelity (A) and writing quality (B) where values > 1 favor AI and values < 1 favor humans. MFA-trained readers show preference for human writing when prompted in an in-context setting (OR = 0.16 and 0.13) but that changes when AI is fine-tuned (OR = 8.16 and 1.87). College-educated general readers show a distinct preference pattern rather than mere noise: under in-context prompting they are at parity with humans on stylistic fidelity (OR = 1.06) but prefer AI-generated text on writing quality (OR = 1.82). (C-D) Probability of choosing AI excerpts across individual language models for stylistic fidelity (C) and writing quality (D). Error bars represent 95% confidence intervals. Dashed line indicates chance performance (50%). (E) AI detection accuracy with chosen threshold of τ =0.9 using two state-of-the-art AI detectors (Pangram and GPTZero). Human written text was never misclassified (0.00), in-context AI was detected with 97% accuracy by Pangram and 91% by GPTZero, but fine-tuned AI evaded detection 97% of the time (0.03) for Pangram and 100% of the time in case of GPTZero. (F) Relationship between AI detectability (Pangram) and preference for writing quality across detection score bins. For the in-context prompting setup, higher detection scores predict lower AI preference mainly among MFA-trained readers. After fine-tuning, that negative detectability penalty is strongly attenuated, and the model-implied slopes are flat-to-positive. Fine-tuning on an author’s complete oeuvre gets rid of AI quirks while achieving expert-level performance. n = 28 MFA-trained readers, 516 college-educated general readers; 10,920 reader-level pairwise judgments/evaluations.
Figure 3: Author-level AI preference and its association with fine-tuning corpus size (fidelity and quality) (A) For each fine-tuned author, the share of blinded pairwise trials in which the AI excerpt was preferred over the human (MFA expert) on stylistic fidelity (top) and overall quality (bottom). Points show Jeffreys-prior estimates (k+0.5)/(n+1) ; vertical bars are 95% Jeffreys intervals (Beta (21,21) ); the dotted line at 0.5 marks human–AI parity. Readers are pooled (MFA-trained and college-educated general readers). (B) AI preference rate versus the fine-tuning corpus size for that author (million tokens), shown for stylistic fidelity (top) and overall quality (bottom). Each point is a fine-tuned author; the line is an OLS fit with heteroskedasticity-robust standard errors (no CI displayed). Slopes are near zero in both panels, indicating little association between corpus size (in this range) and AI preference.
Figure 4: Fine-tuning substantially reduces stylometric signatures of AI text, improves stylistic fidelity and perceived writing quality over in-context prompting, and substantially cuts costs of producing first draft versus professional writers. (A) Mediation analysis linking stylometric features → AI-detector score → human preference. Standardized logistic coefficients with 95% CIs are shown for three features for in-context prompting (red) and fine-tuned models (green). Cliché density mediates 16.4% of the detector effect on choice for in-context prompting but only 1.3% for author fine-tuned models; all three features together mediate 25.4% vs -3.2% for in-context prompted and fine-tuned models respectively. (B) “Fine-tuning premium,” defined as Δ = P(prefer fine-tuned over human) - P(prefer in-context over human), as a function of fine-tuning corpus size. Top: stylistic fidelity; bottom: writing quality. Points are authors; colors denote improvement (green, Δ>0 ), no change (gray), or degradation (red, Δ<0 ). Median Δ : +34.2 (fidelity) and +13.4 percentage points (quality). (C) Cost to produce 100,000 words of raw text vs. publishable prose. MFA-trained writers in our study would earn 25,000fora100k−wordnovel−lengthmanuscript(red).Bycontrast,AIpipelinescangenerate100kwordsofrawtextfor25–276dependingonfine−tuningcorpussize(greenbars=fine−tuning;hatched=in−contextprompting,3). This figure reflects direct compute/API costs only, not the additional human steering, chunking, and editing required to turn raw AI text into a cohesive publishable work. Authors ordered by total AI cost.
#
Author
#
Author
#
Author
1
Alice Munro
19
J.D. Salinger
37
Philip Roth
2
Annie Ernaux (✓)
20
Jhumpa Lahiri (✓)
38
Rachel Cusk
3
Annie Proulx (✓)
21
Joan Didion
39
Roxane Gay (✓)
4
Ben Lerner (✓)
22
Jonathan Franzen (✓)
40
Sally Rooney (✓)
5
Charles Bukowski
23
Junot Díaz (✓)
41
Salman Rushdie (✓)
6
Cheryl Strayed (✓)
24
Kazuo Ishiguro (✓)
42
Shirley Jackson
Table 1: List of Authors
Figure 5: In-Context Writing Prompt used in AI Condition 1.
Figure 6: The pipeline used to fine-tune ChatGPT on an author’s entire oeuvre
Figure 7: Original author written excerpt and extracted content using an LLM
Figure 8: Total training token distribution for each author
Figure 9: Table showing how several award winning authors from US all had MFA degree
Figure 10: Table showing a few accomplished alumni with MFA degree from programs where we recruited our participants
Figure 12: Evaluation Interface for writing quality evaluation where readers just see two excerpts without any disclaimer of its source and choose their preference supported by a reason
Figure 13: Evaluation Interface for stylistic fidelity evaluation where readers just see three excerpts (original and two emulations) without any disclaimer of its source and choose their preference supported by a reason
Figure 14: Pearson Correlation Coefficient between writing quality and style judgments
Figure 15: In AI Condition 1(In-context prompt) we contrast MFA1/2/3 vs AI1/2/3. In AI Condition 2(Fine-tuning) we contrast MFA1/2/3 vs AI4
Figure 16: In AI Condition 1(In-context prompt) we contrast MFA1/2/3 vs AI1/2/3. In AI Condition 2(Fine-tuning) we contrast MFA1/2/3 vs AI4
Figure 17: In AI Condition 1(In-context prompt) we contrast MFA1/2/3 vs AI1/2/3. In AI Condition 2(Fine-tuning) we contrast MFA1/2/3 vs AI4
Figure 18: In AI Condition 1(In-context prompt) we contrast MFA1/2/3 vs AI1/2/3. In AI Condition 2(Fine-tuning) we contrast MFA1/2/3 vs AI4
Figure 19: Total overlap percentage calculated by number of words in generated output that come from authors overall corpus
Figure 20: MFA vs AI emulation of Tony Tulathimutte, read by 3 expert(MFA-trained) readers all of whom agree that MFA written excerpt is superior in terms of writing quality.
Figure 21: MFA vs AI emulation of Tony Tulathimutte, judged by 3 expert(MFA-trained) readers all of whom agree that Fine-tuned AI written excerpt is superior in terms of writing quality
Figure 22: MFA vs AI emulation of Junot Diaz, judged by 3 expert readers(MFA-trained) all of whom agree that MFA written excerpt is superior in terms of stylistic fidelity.
Figure 23: MFA vs AI emulation of Junot Diaz, judged by 3 expert readers(MFA-trained) all of whom agree that Fine-tuned AI written excerpt is superior in terms of stylistic fidelity
Figure 24: Pangram AI likelihood scores vs GPTZero AI Likelihood scores for MFA, In-context and Fine-tuned
Figure 25: Prompt to identify Clichés
Outcome
Setting
Writer type
Reader type
n
style
In_Context
Human
MFA-trained
450
style
In_Context
Human
College-educated
2850
style
In_Context
GPT-4o_baseline
MFA-trained
150
style
In_Context
GPT-4o_baseline
College-educated
950
style
In_Context
Claude_baseline
MFA-trained
150
style
In_Context
Claude_baseline
College-educated
950
Table 3: Observation counts for stylistic fidelity in the in-context prompting setting. Each row is a long-format observation entering the H1 analysis.
Outcome
Setting
Writer type
Reader type
n
style
Fine_tuned
Human
MFA-trained
270
style
Fine_tuned
Human
College-educated
1890
style
Fine_tuned
GPT-4o_finetuned
MFA-trained
270
style
Fine_tuned
GPT-4o_finetuned
College-educated
1890
Table 4: Observation counts for stylistic fidelity in the fine-tuned setting. Balanced counts (Human = GPT-4o_finetuned) reflect the one-to-one comparison design for H2.
Outcome
Setting
Writer type
Reader type
n
quality
In_Context
Human
MFA-trained
450
quality
In_Context
Human
College-educated
2850
quality
In_Context
GPT-4o_baseline
MFA-trained
150
quality
In_Context
GPT-4o_baseline
College-educated
950
quality
In_Context
Claude_baseline
MFA-trained
150
quality
In_Context
Claude_baseline
College-educated
950
Table 5: Observation counts for writing quality in the in-context prompting setting (same Human:AI baseline ratio as stylistic fidelity).
Outcome
Setting
Writer type
Reader type
n
quality
Fine_tuned
Human
MFA-trained
270
quality
Fine_tuned
Human
College-educated
1890
quality
Fine_tuned
GPT-4o_finetuned
MFA-trained
270
quality
Fine_tuned
GPT-4o_finetuned
College-educated
1890
Table 6: Observation counts for writing quality in the fine-tuned setting.
Writer category
Description
Mapping rule (from raw fields)
Human
Human-written (MFA authors)
excerpt_type = Human
GPT-4o_baseline
GPT-4o in-context prompting
AI excerpt; excerpt_model ∈ { GPT-4o }; setting = In_Context
Claude_baseline
Claude 3.5 Sonnet (in-context)
AI excerpt; excerpt_model = Claude3.5Sonnet ; setting = In_Context
Gemini_baseline
Gemini 1.5 Pro (in-context)
AI excerpt; excerpt_model = Gemini_1.5_Pro ; setting = In_Context
GPT-4o_finetuned
GPT-4o fine-tuned
AI excerpt; excerpt_model = GPT-4o_Finetuned
Table 7: Mapping of writer categories from raw data fields to analysis labels.
Term
Est. (log-odds)
SE
z
p
Style — In-context Prompting (n = 6,600)
(Intercept)
0.901
0.148
6.098
1.08e-09
writer_typeGPT-4o_baseline
− 1.655
0.402
− 4.121
3.78e-05
writer_typeClaude_baseline
− 1.390
0.276
− 5.039
4.67e-07
writer_typeGemini_baseline
− 2.510
0.426
− 5.894
3.77e-09
reader_typeCollege-educated
− 0.929
0.153
− 6.053
1.43e-09
Table 8: GLM coefficients with CR2 robust standard errors for all primary models. Negative coefficients indicate preference for the reference category (Human), while positive coefficients indicate preference for AI. The sign reversal between in-context prompting and fine-tuned models represents the core finding. In fine-tuned models, the writer-type × reader-type interaction is significant, indicating that college-educated general readers favor AI by a wider margin than MFA-trained readers.
Outcome
Hyp.
Reader
Est.
SE
p
pHolm
OR
95% CI
(log-odds)
(log-odds)
exp(Est.)
(OR scale)
style
H1
MFA-trained
− 1.852
0.314
3.65e-09
7.31e-09
0.157
[0.08, 0.29]
style
H1
College-educated
0.056
0.083
0.501
0.501
1.058
[0.90, 1.25]
style
H2
MFA-trained
2.100
0.282
1.04e-13
1.04e-13
8.163
[4.69, 14.20]
style
H2
College-educated
2.813
0.125
4.91e-112
9.81e-112
16.652
[13.03, 21.28]
quality
H1
MFA-trained
− 2.020
0.381
1.19e-07
1.19e-07
0.133
[0.06, 0.28]
Table 9: Primary contrasts testing H1 (baseline LLMs vs Human) and H2 (fine-tuned vs Human). Holm correction applied within each outcome and hypothesis across reader types. The OR column shows odds ratios with values <1 favoring humans and >1 favoring AI. Note: The p and pHolm columns report exact two-sided Wald p-values and their Holm-corrected counterparts. The main text reports Holm-corrected p-values using inequality bounds ( p<10−k ), capped at p<10−15 as a conservative floor reflecting the practical limits of the normal approximation.
Outcome
Setting
Model
Reader
p^
95% CI
style
In_Context
GPT-4o (In-Context)
MFA-trained
0.320
[0.21, 0.45]
style
In_Context
GPT-4o (In-Context)
College-educated
0.525
[0.49, 0.56]
style
In_Context
Claude (In-Context)
MFA-trained
0.380
[0.30, 0.46]
style
In_Context
Claude (In-Context)
College-educated
0.541
[0.51, 0.57]
style
In_Context
Gemini (In-Context)
MFA-trained
0.167
[0.10, 0.27]
style
In_Context
Gemini (In-Context)
College-educated
0.455
[0.42, 0.49]
Table 10: Predicted probabilities of selecting AI excerpts by model and reader type, corresponding to Figure 2C–D. Values above 0.5 indicate AI preference. After fine-tuning, both reader groups prefer AI, but college-educated general readers show higher predicted probabilities than MFA-trained readers.
Outcome
Setting
Term
Est.
SE
z
p
style
In_Context
GPT-4o_baseline:reader_typeCollege-educated
1.784
0.414
4.307
1.66e-05
style
In_Context
Claude_baseline:reader_typeCollege-educated
1.583
0.292
5.412
6.22e-08
style
In_Context
Gemini_baseline:reader_typeCollege-educated
2.357
0.438
5.384
7.27e-08
style
Fine_tuned
GPT-4o_finetuned:reader_typeCollege-educated
0.713
0.309
2.308
0.021
quality
In_Context
GPT-4o_baseline:reader_typeCollege-educated
3.197
0.484
6.601
4.08e-11
quality
In_Context
Claude_baseline:reader_typeCollege-educated
1.935
0.381
5.073
3.91e-07
Table 11: Individual interaction terms showing differential preferences between MFA-trained and college-educated general readers. Large positive coefficients indicate college-educated general readers are more favorable to AI. In fine-tuned models, interactions are significant, indicating that reader-group differences persist after fine-tuning.
Outcome
Setting
Term
df
χ2
p
style
In_Context
writer_type:reader_type
3
36.606
5.57e-08
style
Fine_tuned
writer_type:reader_type
1
5.328
0.021
quality
In_Context
writer_type:reader_type
3
46.884
3.68e-10
quality
Fine_tuned
writer_type:reader_type
1
15.978
6.41e-05
Table 12: Joint Wald tests for writer-type × reader-type interactions. Interactions are significant in both in-context prompting and fine-tuned models, indicating that college-educated general readers are more favorable to AI than MFA-trained readers in both conditions.
Outcome
Author
Rank
AI Win Rate (Jeffreys)
Human Win Rate (Jeffreys)
AI Win Rate (Raw)
quality
Cheryl Strayed
1
0.92
0.08
0.931
quality
Marilynne Robinson
2
0.91
0.09
0.917
quality
Han Kang
3
0.88
0.12
0.889
quality
Salman Rushdie
4
0.84
0.16
0.847
quality
Lydia Davis
5
0.80
0.20
0.806
quality
Zadie Smith
6
0.80
0.20
0.806
Table 13: Per-author AI preference rates (Jeffreys-smoothed and raw) ranked by performance. Values above 0.5 indicate AI preference. Each author has 72 pooled trials (9 MFA-trained, 63 college-educated general readers). Note the wide variation across authors and the divergence between style and quality rankings for some authors.
Outcome
Slope
Slope CI (Low)
Slope CI (High)
Intercept
Intercept CI (Low)
Intercept CI (High)
R2
N Authors
quality
0.0029
− 0.0173
0.0231
0.676
0.580
0.771
0.0031
30
style
0.0089
− 0.0194
0.0373
0.760
0.626
0.894
0.0153
30
Table 14: OLS regression of author-level AI win rate on fine-tuning corpus size (millions of tokens). Slopes are near zero with CIs spanning zero and very low R2 , indicating no detectable relationship between corpus size and AI preference rate.
Outcome
N
N Readers
Term
Estimate
SE
z
p
OR
OR (Low)
OR (High)
quality
10920
542
(Intercept)
0.990
0.169
5.87
4.43e-09
2.69
1.93
3.74
quality
10920
542
pangram_score
− 2.010
0.333
− 6.03
1.65e-09
0.13
0.07
0.26
quality
10920
542
setting (Fine-tuned)
− 1.025
0.175
− 5.86
4.64e-09
0.36
0.26
0.51
quality
10920
542
reader_type (College-educated)
− 1.288
0.174
− 7.40
1.36e-13
0.28
0.20
0.39
quality
10920
542
pangram_score × setting
2.904
0.876
3.32
9.13e-04
18.20
3.28
102.00
quality
10920
542
pangram_score × reader_type
2.613
0.344
7.59
3.24e-14
13.60
6.94
26.80
Table 15: Pangram GLM coefficients with CR2 robust standard errors (clustered by reader). Negative coefficients for pangram_score in in-context prompting indicate detection reduces preference; positive interactions with setting show that fine-tuning attenuates this penalty. Significant three-way interactions indicate the detectability penalty differs across reader groups.
Bin
Setting
Reader Type
Prob
Prob (Low)
Prob (High)
0.1
In-context Prompting
MFA-trained
0.674
0.621
0.722
0.5
In-context Prompting
MFA-trained
0.497
0.494
0.500
0.9
In-context Prompting
MFA-trained
0.320
0.273
0.372
0.1
Fine-tuned
MFA-trained
0.511
0.491
0.530
0.5
Fine-tuned
MFA-trained
0.581
0.435
0.714
0.9
Fine-tuned
MFA-trained
0.648
0.380
0.847
Table 16: Predicted AI preference probabilities for style at different Pangram detection scores. Fine-tuning attenuates the relationship with detection score for MFA-trained readers (with wider uncertainty at high detection). College-educated general readers show a weak relationship with detectability in the in-context condition and a positive slope after fine-tuning.
Bin
Setting
Reader Type
Prob
Prob (Low)
Prob (High)
0.1
In-context Prompting
MFA-trained
0.688
0.628
0.742
0.5
In-context Prompting
MFA-trained
0.496
0.493
0.499
0.9
In-context Prompting
MFA-trained
0.306
0.254
0.363
0.1
Fine-tuned
MFA-trained
0.514
0.490
0.537
0.5
Fine-tuned
MFA-trained
0.602
0.431
0.751
0.9
Fine-tuned
MFA-trained
0.683
0.373
0.887
Table 17: Predicted AI preference probabilities for quality . MFA-trained readers show strong detection sensitivity in in-context prompting ( ∼ 0.69 →∼ 0.31) that attenuates after fine-tuning (point estimate at 0.9 ∼ 0.68; 95% CI 0.37–0.89). College-educated general readers show a weaker in-context penalty and modest positive slopes after fine-tuning.
Metric
Pearson r
Spearman ρ
N
readability_ease
− 0.232
− 0.312
330
cliche_density
0.600
0.602
330
total_adjective_count
0.104
0.129
330
num_cliches
0.614
0.633
330
Table 18: Correlations between Pangram AI detection scores and stylometric features. These correlations are computed at the excerpt level ( N=330 unique excerpts) and are unchanged by the expanded reader sample.
N Authors
Fine-tune Min
Fine-tune Med
Fine-tune Max
Total Min
Total Med
Total Max
30
$22.25
$77.88
$272.50
$25.25
$80.88
$275.50
Table 19: Cost comparison for generating 100,000 words. Fine-tuning plus inference costs (25–276) represent about 0.1% to 1.1% of professional writer compensation ($25,000), with a median of approximately 0.3%.