Singlish is a creole rooted in Singapore's multilingual environment that continues to evolve alongside social and technological change. We examine diachronic stylistic change across a decade of informal digital messages and ask whether Large Language Models (LLMs) can generate temporally neutral outputs approximating the stable essence of the variety. Using lexical, pragmatic, psycholinguistic, and encoder-based features, we find that stylistic separability increases with temporal distance, driven primarily by structural features such as length and complexity. Evaluated against a null distribution baseline, most LLMs fail to achieve both authenticity and temporal neutrality simultaneously, revealing a structural trade-off: models generating realistic Singlish inherit its temporal biases, while temporally neutral models produce inauthentic outputs. These findings position temporal neutrality as a diagnostic metric for assessing sociolectal grounding in LLMs.
Figures & tables
Figure 1: Pipeline for our diachronic analysis of CoSEM and LLM-generated Singlish texts.
Figure 2: On CoSEM data, similarity score decreases as the year gap between data increases.
Figure 3: Feature Importance of each of the handcrafted features plotted against year-gap. The feature importance is measured by taking the mean (absolute) SHAP values.
Figure 4: Trend of top LIWC features across categories.
Model
Baseline
CoT
Diverse-Decoding
Self-Consistency
Mean
Std
Mean
Std
Mean
Std
Mean
Std
Mistral
0.367
0.024
0.444
0.056
0.523
0.054
0.373
0.043
Mistral-FT
0.437
0.059
0.303
0.042
0.283
0.040
0.329
0.050
Qwen
0.168
0.020
0.337
0.043
0.105
0.010
0.240
0.031
Qwen-FT
0.668
0.034
0.673
0.043
0.659
0.021
0.570
0.031
DeepSeek-R1
0.198
0.017
0.228
0.041
0.268
0.028
0.321
0.028
Table 1: Comparison of mean similarity scores and standard deviations across models and prompting schemes. Bold text indicates the highest mean similarity score for each model, denoting peak authenticity.
Figure 5: Normalized frequency/1000 words of individual (solid) and combined (dotted) Singlish particles in CoSEM over time.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Year
# Messages
2012
13,877
2013
42,792
2014
77,921
2015
210,354
2016
346,540
2017
265,285
Appendix
Table 2: Number of CoSEM messages per year used in the diachronic analysis. 2022 is excluded from all analyses due to its low volume.
Figure 6: Similarity trends are consistent across classifiers.
Feature
Slope
Spearman ρ
p-value
Summary Variables
Clout*
2.33
0.915
< 0.001
WPS*
2.38
0.842
0.002
Analytic*
1.60
0.697
0.025
Authentic*
-1.23
-0.818
0.004
Tone
-0.61
-0.309
0.385
Appendix
Table 3: Trend analysis of LIWC features over time. * are features with statistically significant trends ( p<0.05 , Spearman correlation)
Model
Method
Std
χ2 stat
p-val
SeaLLM
CoT
0.0330
8.47
0.4877
DeepSeek-R1
SC
0.0282
6.19
0.7212
Mistral
DD
0.0543
22.99
0.0062
Mistral-FT
Baseline
0.0586
26.74
0.0015
Qwen-FT
CoT
0.0432
14.54
0.1045
Qwen
CoT
0.0434
14.64
0.1013
Appendix
Table 4: Variance test results for different models and best-performing prompting methods. χ2 tests whether cross-year standard deviation exceeds σ95=0.034 (one-sided, df=9 ).
Feature
Abbreviation
Unit
Handcrafted Features
Lexico-structural attributes
No. of characters
length_char
Message
No. of words
length_word
Message
Average word length
avg_word_len
Message
No. of repeated characters (e.g., soooo )
num_repeated_char_words
Message
Appendix
Table 5: Features used in the stylistic similarity analysis.
Feature
Mean SHAP
Std SHAP
t-stat
p-value
length_char*
2.68e-01
1.21e-01
1.46e+01
3.73e-07
avg_word_len*
1.06e-01
1.85e-02
2.13e+01
4.83e-05
length_word*
7.80e-02
1.91e-02
9.75e+00
2.70e-04
num_emoticons*
4.42e-02
2.63e-02
6.51e+00
2.51e-04
num_repeated_char_words*
2.97e-02
1.78e-02
6.70e+00
2.24e-04
has_emoji*
8.49e-03
3.62e-03
6.44e+00
1.28e-03
Appendix
Table 6: Statistical analysis of SHAP feature importance. * are features with Mean absolute SHAP values significantly above 0 ( p<0.05 )
While large language model outputs are frequently analysed as a collective super variety termed "AI language," this chapter argues that this perspective coexists with distinct, model-specific linguistic signatures akin to human idiolects. We analyse two datasets of LLM-generated texts on societal topics: a 2024 corpus of six models (Improta et al. 2024) and a newly generated 2026 corpus using the same prompts featuring six contemporary models. Our findings, utilising computational descriptors and stylometric principal component analysis reveal a generational shift between the style of the 2024 and 2026 cohorts, while demonstrating that each individual model maintains a unique linguistic profile. This multi-layered interplay is illustrated by contraction frequencies, which vary from over 1,200 to over 30,000 per million words within the same cohort of models (2026). Ultimately, we conclude that treating LLM output as idiolectal in nature provides a valuable framework with potential implications for research on variation and change, LLM-generated text detection, forensic linguistics and usage-based approaches to language.
Karolina Rudnicka, Thomas Stephan Juzek
Faculty of Languages, University of Gdańsk, Poland · Department of Scientific Computing, Florida State University, United States
Large Language Models (LLMs) are increasingly used to ``professionalize'' workplace communication, often at the cost of linguistic identity. We introduce "Cultural Ghosting", the systematic erasure of linguistic markers unique to non-native English varieties during text processing. Through analysis of 22,350 LLM outputs generated from 1,490 culturally marked texts (Indian, Singaporean,& Nigerian English) processed by five models under three prompt conditions, we quantify this phenomenon using two novel metrics: Identity Erasure Rate (IER) & Semantic Preservation Score (SPS). Across all prompts, we find an overall IER of 10.26%, with model-level variation from 3.5% to 20.5% (5.9x range). Crucially, we identify a Semantic Preservation Paradox: models maintain high semantic similarity (mean SPS = 0.748) while systematically erasing cultural markers. Pragmatic markers (politeness conventions) are 1.9x more vulnerable than lexical markers (71.5% vs. 37.1% erasure). Our experiments demonstrate that explicit cultural-preservation prompts reduce erasure by 29% without sacrificing semantic quality.
Satyam Kumar Navneet, Joydeep Chandra, Yong Zhang
Independent Researcher Bihar, India · BNRIST, Dept. of CST, Tsinghua University Beijing, China
Large Language Models (LLMs) are now capable of generating highly fluent, human-like text. They enable many applications, but also raise concerns such as large scale spam, phishing, or academic misuse. While much work has focused on detecting LLM-generated text, only limited work has gone into understanding the stylistic differences between human-written and machine-generated text. In this work, we perform a large scale analysis of stylistic variation across human-written text and outputs from 11 LLMs spanning 8 different genres and 4 decoding strategies using Douglas Biber's set of lexicogrammatical and functional features. Our findings reveal insights that can guide intentional LLM usage. First, key linguistic differentiators of LLM-generated text seem robust to generation conditions (e.g., prompt settings to nudge them to generate human-like text, or availability of human-written text to continue the style); second, genre exerts a stronger influence on stylistic features than the source itself; third, chat variants of the models generally appear to be clustered together in stylistic space, and finally, model has a larger effect on the style than decoding strategy, with some exceptions. These results highlight the relative importance of model and genre over prompting and decoding strategies in shaping the stylistic behavior of machine-generated text.
Swati Rallapalli, Shannon Gallagher, Ronald Yurko +4
Software Engineering Institute, AI Division, Carnegie Mellon University · Department of Statistics & Data Science, Carnegie Mellon University