LitArc-Bench: Disentangling Text-Specific Reader-Response Dynamics from Position and Lexical Affect
Organizations: Capital University of Economics and Business · Shanghai International Studies University · Tsinghua University
Abstract
When model predictions track an unfolding human affect trajectory, alignment may reflect narrative position, affective words, or text-specific reader-response dynamics. We introduce LitArc-Bench to separate these explanations: 39 Chinese-language literary excerpts segmented into seven checkpoints, with 430 questionnaires from 81 adults in two classroom groups (9,020 VAD ratings). We compare three future-blind model runs with leave-one-text-out position priors, lexical affect, and human-human references. Position alone reaches r = .470 for arousal, making raw alignment insufficient. Yet all six V/A transition advantages over a joint position-lexicon control have text-bootstrap intervals above zero. Cross-fitted models match the correct human text better than 38 mismatched texts in all nine model-dimension cells (p < .0001); a text-invariant negative control returns chance (p = 1). On human transitions at least 0.5 points against the position trend, models recover 69-76% of valence directions and 53-68% for arousal and dominance. Under crossed participant-text resampling, all nine identity intervals remain above zero; joint-control advantages are robust for valence and directional but less certain for arousal. High shape agreement still coexists with scale bias. LitArc-Bench identifies which components of literary reader response are recoverable beyond shallow explanations and why shape, calibration, and human variability require separate evaluation.
Figures & tables
| Resource | Language / unit | Primary affect signal | Same-reader sequence? | Human arc reference? | Future-blind model test? |
|---|---|---|---|---|---|
| EmoBank ( Buechel and Hahn, 2017a ) | English / sentence | Reader/writer VAD | No | No | No |
| CDVAI ( Huang et al., 2022 ) | Chinese / sentence+context | VA and irony | No | No | No |
| EmotionArcs ( Ohman et al., 2024 ) | English / novel bins | Automated text emotion | No | No | No |
| CR4-NarrEmote ( Piper and Budac, 2025 ) | English / passage | Open-vocabulary reader report | No | No | No |
| LitArc-Bench | Chinese / 7 increments | Incremental reader VAD | Yes | Yes | Yes |
| Statistic | Value |
|---|---|
| Excerpts / segments per excerpt | 39 / 7 |
| Original Chinese / translated | 9 / 30 |
| Participants | 81 |
| Reader–text questionnaires | 430 |
| Raters per text, mean [range] | 11.03 [5, 17] |
| Segment-level rows | 3,010 |
| Dimension | Typical reader [95% CI] | MAE [95% CI] | Transition [95% CI] | Raw split-half [95% CI] | Reader–text pairs |
|---|---|---|---|---|---|
| Valence | .516 [.414, .555] | .987 [.909, 1.048] | .395 [.303, .549] | .597 [.470, .660] | 427 |
| Arousal | .274 [.187, .352] | 1.169 [1.071, 1.273] | .252 [.213, .378] | .409 [.299, .477] | 426 |
| Dominance | .330 [.181, .499] | 1.286 [1.130, 1.440] | .343 [.175, .409] | .411 [.295, .589] | 427 |
| Dimension | Median text [95% CI] | Pooled | Median MAE | Transition [95% CI] | Mean bias |
|---|---|---|---|---|---|
| Valence | .725 [.460, .814] | .766 | 1.286 | .692 [.524, .814] | |
| Arousal | .607 [.462, .700] | .676 | 1.039 | .574 [.365, .718] | |
| Dominance | .618 [.376, .658] | .599 | 1.179 | .505 [.423, .656] |
| Baseline | V | A | D |
| Global mean | – | – | – |
| Position mean | .167 | .470 | .268 |
| Chinese EmoBank | .259 | .252 | – |
| Permuted position | .013 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Dim. | Spearman | Leave-one-position | Paired advantage [95% CI] | Paired transition advantage [95% CI] |
|---|---|---|---|---|
| V | .709 | .738 | .577 [.332, .688] | .858 [.610, 1.145] |
| A | .558 | .600 | .173 [.050, .378] | .267 [.111, .461] |
| D | .468 | .607 | .298 [ , .485] | .595 [.231, .724] |
| Model | Dim. | Median [95% CI] | Median CCC [95% CI] | Median MAE [95% CI] | Transition [95% CI] |
|---|---|---|---|---|---|
| Claude | V | .636 [.547, .764] | .318 [.151, .444] | 1.024 [.798, 1.163] | .661 [.533, .744] |
| A | .628 [.558, .791] | .339 [.189, .472] | .844 [.733, .939] | .587 [.398, .651] | |
| D | .490 [.332, .633] | .158 [.106, .300] | .964 [.765, 1.317] | .460 [.239, .579] | |
| DeepSeek | V | .629 [.449, .730] | .210 [.136, .340] | 1.357 [1.012, 1.494] | .611 [.427, .668] |
| A | .573 [.482, .689] | .242 [.188, .276] | 1.163 [.987, 1.345] | .617 [.420, .658] | |
| D | .416 [.310, .612] | .164 [.076, .252] | 1.369 [1.000, 1.673] | .524 [.283, .628] |
| Model | Dim. | Shape advantage [95% CI] | Positive texts | Transition advantage [95% CI] |
|---|---|---|---|---|
| GPT-5.6-sol | V | .577 [.332, .688] | 79.5% | .858 [.610, 1.145] |
| A | .173 [.050, .378] | 71.8% | .267 [.111, .461] | |
| D | .298 [ , .485] | 64.1% | .595 [.231, .724] | |
| Claude | V | .607 [.316, .749] | 76.3% | .925 [.485, 1.222] |
| A | .292 [.087, .431] | 74.4% | .414 [.058, .529] | |
| D | .148 [ , .412] | 60.5% | .459 [.334, .590] |
| Model | Dim. | Shape advantage over joint [95% CI] | Transition advantage over joint [95% CI] | Matched residual advantage [95% CI] |
|---|---|---|---|---|
| GPT-5.6-sol | V | .488 [.293, .645] | .284 [.174, .408] | .550 [.427, .643] |
| A | .153 [.023, .366] | .217 [.047, .439] | .477 [.408, .641] | |
| Claude | V | .406 [.263, .569] | .208 [.072, .349] | .452 [.358, .570] |
| A | .202 [.040, .473] | .297 [.108, .461] | .573 [.349, .682] | |
| DeepSeek | V | .373 [.230, .609] | .172 [.039, .404] | .465 [.371, .623] |
| A | .159 [ , .262] | .333 [.078, .448] | .505 [.281, .644] |
| Model | Dim. | Crossed identity advantage [95% CI] | Joint transition advantage [95% CI] |
|---|---|---|---|
| GPT-5.6-sol | V | .550 [.250, .697] | .284 [.116, .537] |
| A | .477 [.233, .649] | .217 [ , .524] | |
| D | .533 [.257, .713] | – | |
| Claude | V | .452 [.243, .636] | .208 [.041, .485] |
| A | .573 [.258, .702] | .297 [ , .580] | |
| D | .447 [.212, .636] | – |
| Control | Dim. | GPT | Claude | DeepSeek | |
|---|---|---|---|---|---|
| Position | V | 58 | .759 | .690 | .707 |
| A | 41 | .610 | .683 | .659 | |
| D | 43 | .674 | .535 | .651 | |
| Chinese EmoBank | V | 41 | .683 | .610 | .634 |
| A | 48 | .542 | .646 | .729 |
| Dim. | Raw | Partial controlling arc SD | |
|---|---|---|---|
| V | .345 | .011 [ , .322] | .948 |
| A | .520 | .324 [ , .612] | .045 |
| D | .398 | .114 [ , .428] | .491 |
Computational Literary Studies
Computational Literary Studies uses data analysis and language technologies to examine literary style, themes, narratives, influence, and change.
See all 53 papers