Beyond Detection: Ethical Foundations for Automated Dyslexic Error Attribution
Authors: Samuel Rose, Debarati Chakraborty
Organizations: School of Digital and Physical Sciences, University of Hull, Cottingham Road, Hull, HU6 7RX, United Kingdom. · Everybody Counts LTD, Montgomery Lane, Edinburgh, EH7 5JT, Scotland.
Dyslexic spelling errors exhibit systematic phonological and orthographic patterns that distinguish them from the errors produced by typically developing writers. While this observation has motivated dyslexic-specific spell-checking and assistive writing tools, prior work has focused predominantly on error correction rather than attribution, and has largely neglected the ethical risks. The risk of harmful labelling, covert screening, algorithmic bias, and institutional misuse that automated classification of learners entails requires the development of robust ethical and legal frameworks for research in this area. This paper addresses both gaps. We formulate dyslexic error attribution as a binary classification task. Given a misspelt word and its correct target form, determine whether the error pattern is characteristic of a dyslexic or non-dyslexic writer. We develop a comprehensive feature set capturing orthographic, phonological, and morphological properties of each error, and propose a twin-input neural model evaluated against traditional machine learning baselines under writer-independent conditions. The neural model achieves 93.01% accuracy and an F1-score of 94.01%, with phonetically plausible errors and vowel confusions emerging as the strongest attribution signals. We situate these technical results within an explicit ethics-first framework, analysing fairness across subgroups, the interpretability requirements of educational deployment, and the conditions, consent, transparency, human oversight, and recourse, under which a system could be responsibly used. We provide concrete guidelines for ethical deployment and an open discussion of the systems limitations and misuse potential. Our results demonstrate that dyslexic error attribution is feasible at high accuracy while underscoring that feasibility alone is insufficient for deployment in high-stakes educational contexts.
Figures & tables
Model Architecture
Accuracy
F1 Score
AUC
Random Forest
85.2%
85.8%
0.8597
Logistic Regression
84.60%
85.24%
0.8551
KNN
82.50%
83.38%
0.8312
SVC
82.56%
83.30%
0.8330
Gausian NB
48.90%
54.10%
0.4818
Neural Network
93.01%
94.01%
0.9274
Table 1: Comparative model performance for dyslexia classification from typed error patterns
Predicted label
Non dyslexic
Dyslexia
True label
Non dyslexic
71 (TN, 38.2%)
7 (FP, 3.8%)
Dyslexia
6 (FN, 3.2%)
102 (TP, 54.8%)
Table 2: Neural Network Confusion Matrix
Figure 1: Left: reliability plot showing calibration of predicted probabilities against observed positive rates across ten equal-width bins. The dashed diagonal represents perfect calibration; the shaded region indicates the gap between model confidence and empirical accuracy (ECE =0.1098 ). Right: sensitivity–specificity trade-off across classification thresholds. The Youden-optimal threshold ( τ=0.74 ) and high-sensitivity screening point ( sensitivity≥0.95 ) are marked. The conventional τ=0.50 default is shown for reference.
Metric
Value
Reference
Brier Score
0.0439
No-skill baseline ≈0.25
Expected Calibration Error (ECE)
0.1098
Perfect calibration =0.00
Youden-optimal threshold ( τ )
0.7368
Maximises sensitivity + specificity
Table 3: Calibration and robustness metrics for the neural attribution model.
Condition
Accuracy
F1
AUC-ROC
A: Text only
0.7194
0.6745
0.7109
B: Engineered features only
0.9247
0.9307
0.9050
C: No productivity features
0.8194
0.8386
0.8348
D: Full model
0.9301
0.9401
0.9274
Δ A − D
− 0.2107
− 0.2656
− 0.2165
Δ B − D
− 0.0054
− 0.0094
− 0.0224
Table 4: Ablation study results. All conditions were trained from scratch under identical settings. Δ rows show the difference from the full model (D); negative values indicate worse performance than the full model. Classification threshold τ=0.74 (Youden-optimal).
Error type
Accuracy (%)
Capitalisation
100.00*
Substitution
100.00
Spatial
100.00*
Omission
94.44
Multiple errors
94.12
Transposition
92.86
Table 5: Neural network classification accuracy by error type. ∗ Capitalisation and Spatial Errors were only made by a single dyslexic participant in the study
Subgroup
Accuracy
F1
Recall
Phonetically plausible
0.9846
0.9875
0.9753
Non-phonetic
0.9031
0.9059
0.9005
Disparity ( Δ )
0.0815
0.0816
0.0748
Table 6: Model performance by phonetic plausibility of the spelling error. Classification threshold τ=0.74 (Youden-optimal).
Co-morbidity
Subgroup
Accuracy
F1
Recall
Visual Stress
Absent
0.9732
0.9735
0.9630
Present
1.0000
1.0000
1.0000
ADD/ADHD
None
0.9769
0.9781
0.9630
Suspected
1.0000
1.0000
1.0000
Confirmed
0.8889
0.0000
0.0000
Dyspraxia/Dysgraphia
Absent
0.9755
0.9765
0.9630
Table 7: Model performance by comorbidity subgroup. Classification threshold τ=0.74 (Youden-optimal). ASD and SLCN are excluded due to zero positive cases in the validation set Subgroups with no positive cases are reported qualitatively. Perfect scores for small subgroups should be interpreted with caution.
Dyslexic learners increasingly use artificial intelligence (AI) tools to support reading, writing, organisation, and study-related tasks. However, their lived experiences with these tools remain largely underexamined. This paper proposes DysLexLens, a low-resource LLM framework, designed to analyse dyslexic learners experience with AI through online forum discussions. DysLexLens is designed as an end-to-end, evidence-traceable architecture which transforms noisy social media posts into a dictionary-driven corpora, provides knowledge-graph (KG)-based question reasoning, generates verifiable query responses, and enables response evaluation through quantitative and human-grounded assessment. DysLexLens has four key features. First, it employs a dictionary-driven filtering method to construct a more focused Reddit corpus on dyslexia and AI, filtering out noisy and weakly related posts to improve the relevance of data collected from low-resource forum contexts. Second, it integrates LLM-assisted semantic analysis with KG-based query reasoning to uncover meaningful patterns. Third, it has quantitative evaluation metrics (RAGAS and Query Robustness) to measure LLM-generated response performance. Fourth, it provides structured qualitative validation guidelines for assessing response quality, with a specific focus on hallucination and evidence alignment. We demonstrate the effectiveness of DysLexLens using dyslexia-related Reddit forum data and 30 questions. The results show its potential generalisability to other low-resource forum data contexts. DysLexLens, sample data, questions and evaluation results are available at Github to support reproducibility.
Dana Rezazadegan, Atie Kia, Phongpadid Nandavong +6
Swinburne University of Technology Melbourne, VIC, Australia
Automatic speech recognition replaces typing only when correction costs less than manual entry - a threshold determined by error types, not counts: fixing a misrecognized domain term costs far more than inserting a comma. Word error rate (WER) fails on two fronts: it collapses distinct error categories into a single scalar, and it structurally penalizes agglutinative languages where valid sandhi merges inflate scores. We introduce SCRIBE, a diagnostic framework offering categorical error decomposition into lexical, punctuation, numeral, and domain-entity rates via sandhi-tolerant alignment with domain vocabulary injection. Human validation confirms SCRIBE aligns with expert judgment where WER does not. We release SCRIBE, an LLM curation pipeline, benchmarks, and open-weight rich transcription models for Hindi, Malayalam, and Kannada.
Automatic speech recognition (ASR) systems remain brittle on dysarthric and other atypical speech. Recent audio-language models raise the possibility of improving performance by conditioning on additional clinical context at inference time, but it is unclear whether these models can make use of such information. We introduce a benchmark built on the Speech Accessibility Project (SAP) dataset that tests whether diagnosis labels, clinician-derived speech ratings, and progressively richer clinical descriptions improve transcription accuracy for dysarthric speech. Across matched comparisons on nine models, we find that current models do not meaningfully use this context: diagnosis-informed and clinically detailed prompts yield negligible improvements and often degrade word error rate. We complement the prompting analysis with context-dependent fine-tuning, showing that LoRA adaptation with a mixture of clinical prompt formats achieves a WER of 0.066, a 52% relative reduction over the frozen baseline, while preserving performance when context is unavailable. Subgroup analyses reveal significant gains for Down syndrome and mild-severity speakers. These results clarify where current models fall short and provide a testbed for measuring progress toward more inclusive ASR.
Pehuén Moure, Niclas Pokel, Bilal Bounajma +4
Institute of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland · Department of Mathematics, ETH Zurich, Switzerland · Department of Computational Linguistics, University of Zurich, Switzerland