Analysing multilingual social media discourse remains a major challenge in natural language processing, particularly when large-scale public debates span across diverse languages. This study investigates how different approaches for cross-lingual text classification can support reliable analysis of global conversations. Using hydrogen energy as a case study, we analyse a decade-long dataset of over nine million tweets in English, Japanese, Hindi, and Korean (2013--2022) for topic discovery. The online keyword-driven data collection results in a significant amount of irrelevant content. We explore four approaches to filter relevant content: (1) translating English annotated data into target languages for building language-specific models for each target language, (2) translating unlabelled data appearing from all languages into English for creating a single model based on English annotations, (3) applying English fine-tuned multilingual transformers directly to each target language data, and (4) a hybrid strategy that combines translated annotations with multilingual training. Each approach is evaluated for its ability to filter hydrogen-related tweets from noisy keyword-based collections. Subsequently, topic modeling is performed to extract dominant themes within the relevant subsets. The results highlight key trade-offs between translation and multilingual approaches, offering actionable insights into optimising cross-lingual pipelines for large-scale social media analysis.
Figures & tables
Figure 1: Architecture of the proposed multi-step methodology, including data collection, data preprocessing, and relevance classification.
Approach
Training Data
Test Data
Model Setup
1. Language-specific
5,000 English annotated tweets and their translations (JA, KO, and HI) a
750 tweets each language taken from original language data
Separate language-specific models for each language
2. English-only
5,000 English annotated tweets
Single English model; all tweets translated to English
3. English Fine-Tuned mBERT
5,000 English annotated tweets
Single mBERT model fine-tuned on English data; zero-shot prediction on other non-English languages
4. Hybrid
5,000 English annotated tweets and their translations (JA, KO, and HI) a
Single mBERT model trained on combined multilingual data
Table 1: Training and testing setup for four relevance-classification approaches.
Approach
Language
Model
Accuracy (%)
F1-score (%)
Approach 1: Language-Specific
EN
\ttfamily BERT
97.72 ± 0.23
97.70 ± 0.23
JA
\ttfamily Japanese BERT
75.79 ± 0.49
74.88 ± 0.56
KO
\ttfamily Korean BERT
91.40 ± 0.31
91.35 ± 0.31
HI
\ttfamily Hindi BERT
78.08 ± 1.10
78.04 ± 1.10
Approach 2: English-only
EN
\ttfamily BERT
97.72 ± 0.23
97.70 ± 0.23
JA
\ttfamily BERT
79.85 ± 0.66
79.30 ± 0.69
Table 2: Test accuracy and F1-score for four cross-lingual relevance classification methods. Best results are shown in bold blue; second-best in bold.
App.
L
Total
R
I
’13
’14
’15
’16
’17
’18
’19
’20
’21
’22
#1
EN
4,769,850
47.80
52.20
3.33
4.01
5.47
5.24
5.96
6.32
4.77
14.88
24.12
25.91
JA
4,364,391
20.48
79.52
3.40
7.23
8.64
11.82
7.20
5.35
5.88
9.38
21.89
19.18
HI
27,258
35.44
64.56
0.50
1.22
1.01
2.49
2.52
5.16
3.21
7.75
26.44
49.70
KO
148,040
21.08
78.92
4.15
4.42
5.50
5.07
5.14
11.81
6.09
18.05
15.73
24.01
#2
EN
4,769,850
50.61
49.39
3.25
3.93
5.33
5.12
5.85
6.33
4.77
14.91
24.25
26.26
JA
2,459,340
16.16
83.84
3.29
6.55
8.95
10.00
6.38
5.01
7.35
10.01
22.32
20.05
Table 3: Distribution of Relevant (R) and Irrelevant (I) Tweets with Year-wise Percentage Breakdown (2013-2022) Across Four Classification Approaches
Figure 2: Topic Modelling 2013-2022 on a line graph showing the trending themes across multiple languages
Figure 3: Topic Modelling 2013-2022 on a bar graph showing the trending themes across multiple languages