Environmental, Social and Governance Sentiment Analysis on Slovene News: A Novel Dataset and Models
Authors: Paula Dodig, Boshko Koloski, Katarina Sitar Šuštar, Senja Pollak, Matthew Purver
Organizations: Eindhoven University of Technology, Eindhoven · Jožef Stefan Institute and Postgraduate School, Ljubljana · Faculty of Economics, University of Ljubljana · Queen Mary University of London
Environmental, Social, and Governance (ESG) considerations are increasingly integral to assessing corporate performance, reputation, and long-term sustainability. Yet, reliable ESG ratings remain limited for smaller companies and emerging markets. We introduce the first publicly available Slovene ESG sentiment dataset and a suite of models for automatic ESG sentiment detection. The dataset, derived from the MaCoCu Slovene news collection, combines large language model (LLM)-assisted filtering with human annotation of company-related ESG content. We evaluate the performance of monolingual (SloBERTa) and multilingual (XLM-R) models, embedding-based classifiers (TabPFN), hierarchical ensemble architectures, and large language models. Results show that LLMs achieve the strongest performance on Environmental (Gemma3-27B, F1-macro: 0.61) and Social aspects (gpt-oss 20B, F1-macro: 0.45), while fine-tuned SloBERTa is the best model on Governance classification (F1-macro: 0.54). We then show in a small case study how the best-preforming classifier (gpt-oss) can be applied to investigate ESG aspects for selected companies across a long time frame.
Figures & tables
Figure 1: Student annotation results
Split
Aspect
Irrel.
Neg.
Neut.
Pos.
Train (440)
E
288
41
39
72
S
144
48
69
179
G
193
78
86
83
Test (110)
E
77
6
12
15
S
37
15
22
36
G
53
23
19
15
Table 1: Dataset distribution.
Figure 2: Methodology pipeline
Hyperparameter
Value
Optimizer
AdamW
Learning rate
3×10−5
Weight decay
0.01
Batch size
128
Max sequence length
192 tokens
Warmup steps
100
Table 2: Hyperparameters for transformer fine-tuning.
Model
Accuracy
F1-macro
BAcc
AUPRC
Baseline
Majority
0.7000
0.2059
0.2500
0.2500
Ensemble
FinalTowerA
0.7394
0.4469
0.5010
0.5638
FinalTowerB
0.7182
0.4291
0.4476
0.4815
Fine-tuned Transformers
Table 3: Test-set results for Aspect E (mean over seeds). Primary metric is F1-macro; we also report AUPRC, Balanced Accuracy (BAcc), and Accuracy. Best results per column are in bold.
Model
Accuracy
F1-macro
BAcc
AUPRC
Baseline
Majority
0.3364
0.1259
0.2500
0.2500
Ensemble
FinalTowerA
0.4606
0.3033
0.3589
0.3515
FinalTowerB
0.4364
0.3252
0.3602
0.3557
Fine-tuned Transformers
Table 4: Test-set results for Aspect S (mean over seeds). Primary metric is F1-macro; we also report AUPRC, Balanced Accuracy (BAcc), and Accuracy. Best results per column are in bold.
Model
Accuracy
F1-macro
BAcc
AUPRC
Baseline
Majority
0.4818
0.1626
0.2500
0.2500
Ensemble
FinalTowerA
0.5152
0.3742
0.3997
0.4351
FinalTowerB
0.4939
0.3935
0.4158
0.4342
Fine-tuned Transformers
Table 5: Test-set results for Aspect G (mean over seeds). Primary metric is F1-macro; we also report AUPRC, Balanced Accuracy (BAcc), and Accuracy. Best results per column are in bold.
Company
Category
Total
Relevant
Positive
Negative
Neutral
Irrelevant
Talum
E
4072
1234
460
448
326
2838
S
2446
952
900
594
1626
G
2404
512
1080
812
1668
Sdh
E
30338
2504
668
838
998
27834
S
15758
1892
7908
5958
14580
G
27640
2082
14610
10948
2698
Table 6: ESG Sentiment Analysis Summary by Company and Category
Figure 3: Normalized sentiment scores from media mentions (2010 2025)