Singlish is a creole rooted in Singapore's multilingual environment that continues to evolve alongside social and technological change. We examine diachronic stylistic change across a decade of informal digital messages and ask whether Large Language Models (LLMs) can generate temporally neutral outputs approximating the stable essence of the variety. Using lexical, pragmatic, psycholinguistic, and encoder-based features, we find that stylistic separability increases with temporal distance, driven primarily by structural features such as length and complexity. Evaluated against a null distribution baseline, most LLMs fail to achieve both authenticity and temporal neutrality simultaneously, revealing a structural trade-off: models generating realistic Singlish inherit its temporal biases, while temporally neutral models produce inauthentic outputs. These findings position temporal neutrality as a diagnostic metric for assessing sociolectal grounding in LLMs.
Figures & tables
Figure 1: Pipeline for our diachronic analysis of CoSEM and LLM-generated Singlish texts.
Figure 2: On CoSEM data, similarity score decreases as the year gap between data increases.
Figure 3: Feature Importance of each of the handcrafted features plotted against year-gap. The feature importance is measured by taking the mean (absolute) SHAP values.
Figure 4: Trend of top LIWC features across categories.
Model
Baseline
CoT
Diverse-Decoding
Self-Consistency
Mean
Std
Mean
Std
Mean
Std
Mean
Std
Mistral
0.367
0.024
0.444
0.056
0.523
0.054
0.373
0.043
Mistral-FT
0.437
0.059
0.303
0.042
0.283
0.040
0.329
0.050
Qwen
0.168
0.020
0.337
0.043
0.105
0.010
0.240
0.031
Qwen-FT
0.668
0.034
0.673
0.043
0.659
0.021
0.570
0.031
DeepSeek-R1
0.198
0.017
0.228
0.041
0.268
0.028
0.321
0.028
Table 1: Comparison of mean similarity scores and standard deviations across models and prompting schemes. Bold text indicates the highest mean similarity score for each model, denoting peak authenticity.
Figure 5: Normalized frequency/1000 words of individual (solid) and combined (dotted) Singlish particles in CoSEM over time.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Year
# Messages
2012
13,877
2013
42,792
2014
77,921
2015
210,354
2016
346,540
2017
265,285
Appendix
Table 2: Number of CoSEM messages per year used in the diachronic analysis. 2022 is excluded from all analyses due to its low volume.
Figure 6: Similarity trends are consistent across classifiers.
Feature
Slope
Spearman ρ
p-value
Summary Variables
Clout*
2.33
0.915
< 0.001
WPS*
2.38
0.842
0.002
Analytic*
1.60
0.697
0.025
Authentic*
-1.23
-0.818
0.004
Tone
-0.61
-0.309
0.385
Appendix
Table 3: Trend analysis of LIWC features over time. * are features with statistically significant trends ( p<0.05 , Spearman correlation)
Model
Method
Std
χ2 stat
p-val
SeaLLM
CoT
0.0330
8.47
0.4877
DeepSeek-R1
SC
0.0282
6.19
0.7212
Mistral
DD
0.0543
22.99
0.0062
Mistral-FT
Baseline
0.0586
26.74
0.0015
Qwen-FT
CoT
0.0432
14.54
0.1045
Qwen
CoT
0.0434
14.64
0.1013
Appendix
Table 4: Variance test results for different models and best-performing prompting methods. χ2 tests whether cross-year standard deviation exceeds σ95=0.034 (one-sided, df=9 ).
Feature
Abbreviation
Unit
Handcrafted Features
Lexico-structural attributes
No. of characters
length_char
Message
No. of words
length_word
Message
Average word length
avg_word_len
Message
No. of repeated characters (e.g., soooo )
num_repeated_char_words
Message
Appendix
Table 5: Features used in the stylistic similarity analysis.
Feature
Mean SHAP
Std SHAP
t-stat
p-value
length_char*
2.68e-01
1.21e-01
1.46e+01
3.73e-07
avg_word_len*
1.06e-01
1.85e-02
2.13e+01
4.83e-05
length_word*
7.80e-02
1.91e-02
9.75e+00
2.70e-04
num_emoticons*
4.42e-02
2.63e-02
6.51e+00
2.51e-04
num_repeated_char_words*
2.97e-02
1.78e-02
6.70e+00
2.24e-04
has_emoji*
8.49e-03
3.62e-03
6.44e+00
1.28e-03
Appendix
Table 6: Statistical analysis of SHAP feature importance. * are features with Mean absolute SHAP values significantly above 0 ( p<0.05 )