cs.CLOct 23, 2026

From Promises to Accountability: Comparative AI Modeling and Visualization of Electoral Commitments in Nepali Political Texts

Authors: Prakash Poudyal, Nischal Karki, Amrit Dahal, Rajani Chulyadyo, Sameer Tamrakar, Bal Krishna Bal

Organizations: Information and Language Processing Research Lab, Kathmandu University, Kavre, Nepal · Department of Artificial Intelligence, Kathmandu University, Nepal · Department of Computer Science and Engineering, Kathmandu University, Nepal

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Election Commitment Mining refers to the analysis of election commitments made by electoral candidates or political parties during election campaigns. However, many elected candidates and political parties, especially in developing South Asian countries, do not fully deliver on the promises they make before elections. Even after the elections, commitments presented through election manifestos, posters, pamphlets, videos, and other campaign materials are often not systematically or transparently archived. As a result, these promises tend to fade away over time, making it difficult to hold the elected candidates accountable for their commitments. To address this persistent issue, this research aims to develop a system capable of structuring the commitments made by candidates during election periods. In this work, we created the first gold-standard dataset of Nepali political texts (Cohen's kappa = 0.777) and evaluated machine learning, deep learning and large language models to automatically extract and analyze election commitments. LaBSE-LR achieved the best results among machine learning models (0.897 accuracy, 0.896 macro F1-score, 0.947 AUROC), while fastText-LSTM achieved the highest performance among deep learning models (0.914 accuracy, 0.913 macro F1-score, 0.961 AUROC). Among the three evaluated large language models, Qwen3-8B performed best in the few-shot setting (0.848 macro F1-score). The outcome of this work is integrated into a mobile application, where commitments can be visualized with the support of GAI, enhancing transparency, civic engagement, and electoral accountability.

Figures & tables

Figure 1: System architecture for mining commitments from Nepali political texts
ProvincesNagarpalikasPeopleImagesVideos
Koshi88326
Madhesh56272
Bagmati13192812
Lumbini910189
Sudurpashchim22140
Total374511929
Table 1: Dataset composition by province: candidates, images, and videos collected from Facebook profiles and party websites
Figure 2: Preprocessing pipeline for pamphlet image-to-text conversion: from Google Lens and Tesseract OCR to refined data
  Annotator 1 (A1)
 LabelCNC\engbold NA1N_{A1}
Annotator 2 (A2)C1,2542231,477
 NC44909953
 \engbold NA2N_{A2}1,2981,132N=2,430N=2{,}430
Table 2: Inter-annotator agreement confusion matrix
ProvincesNo. of Candidates/PartiesNo. of Commitments (C)No. of Non-Commitments (NC)
Koshi622899
Madhesh619664
Bagmati11688899
Lumbini310830
Sudurpashchim210117
Total281,3211,109
Table 3: Distribution of commitment and non-commitment statements by province
Figure 3: Profile images of the candidates as displayed in the mobile application
Figure 4: Workflow diagram of the mobile application
Name of CandidateCommitments (in Nepali)Summarized Commitments (in Nepali)Commitments (in English)
\devanagarifont बालेन्द्र शाह [Figure 3(a) ]\devanagarifont काठमाडौं महानगरपालिकाका सामुदायिक विद्यालयहरूको पूर्वाधार, गुणस्तर र वातावरणमा सुधार गर्नु\devanagarifont सामुदायिक विद्यालयको गुणस्तर सुधारTo improve the infrastructure, quality, and environment of community schools in Kathmandu Metropolitan City and to emphasize technical education.
 \devanagarifont अन्तर्राष्ट्रिय विद्यालयहरूसँगको सहकार्य गरी जीवनशैली, संस्कृति, शिक्षा प्रणाली आदिको आदानप्रदान गर्नु\devanagarifont अन्तर्राष्ट्रिय विद्यालयसँग जीवनशैलीको आदानप्रदानInternational schools will be partnered with national schools and sisterhood relationships will be initiated to exchange culture, lifestyle, education systems, etc.
\devanagarifont सुरेश साह सोनार [Figure 3(b) ]\devanagarifont कृषि क्षेत्रको प्रवर्द्धन एवं विकासको लागि किसानलाई सहज तथा समयमा सहुलियत रकममा बिउ, बिजन र कृषि उपकरण उपलब्ध गराउने छु\devanagarifont किसानलाई सहुलियतमा बिउ र उपकरण उपलब्ध गराउनेFor the promotion and development of the agricultural sector, I will provide seeds, seedlings and agricultural equipment to farmers easily and on time at subsidized rates.
 \devanagarifont प्रत्येक सामुदायिक विद्यालयमा अंग्रेजी माध्यमबाट पठन पाठन संचालन गर्नको साथै उच्च-शिक्षा हासिल गर्न जेहेन्दार छात्र छात्राहरुलाई छात्रवृत्ति प्रदान गर्ने छु\devanagarifont अंग्रेजी माध्यममा जेहेन्दार विद्यार्थीलाई छात्रवृत्ति दिनेछI will provide scholarships to meritorious students to pursue higher education by conducting English medium teaching in every community school.
Table 4: A few example candidate commitments shown in original Nepali, summarized Nepali, and English translation
Model NameAccuracyma-Precisionma-Recallma- F1F_{1} -scoreAUROC
TF-IDF-LR0.8700.8690.8690.8690.943
LaBSE-LR\engbold 0.897\engbold 0.896\engbold 0.896\engbold 0.8960.947
TF-IDF-DT0.7350.7360.7400.7350.809
LaBSE-DT0.7470.7440.7410.7420.813
TF-IDF-RF0.8600.8580.8610.8590.931
LaBSE-RF0.8700.8710.8660.8680.939
Table 5: Performance of ML models under TF-IDF and LaBSE text representations. The best result is \engbold bolded, and the second best is underlined .
Model NameAccuracyma-Precisionma-Recallma- F1F_{1} -scoreAUROC
fastText-SRN0.8990.8980.8990.8980.951
GloVe-SRN0.8520.8500.8540.8510.924
fastText-LSTM\engbold 0.914\engbold 0.912\engbold 0.915\engbold 0.913\engbold 0.961
GloVe-LSTM0.8400.8410.8450.8390.915
fastText-GRU0.9050.9040.9040.9040.959
GloVe-GRU0.8500.8490.8530.8490.920
Table 6: Performance of DL models under fastText and GloVe embeddings, and transformer-based models. The best result is \engbold bolded, and the second best is underlined .
Model NameAccuracyma-Precisionma-Recallma- F1F_{1} -scoreAUROC
Gemma-4-E4B-it- \engital zero-shot0.8150.8190.8220.8150.822
Gemma-4-E4B-it- \engital few-shot0.8330.8380.8410.8330.841
Llama-3.1-8B-Instruct- \engital zero-shot0.6090.6870.6390.5930.639
Llama-3.1-8B-Instruct- \engital few-shot0.8420.8440.8480.8410.848
Qwen3-8B- \engital zero-shot0.8170.8310.8290.8170.829
Qwen3-8B- \engital few-shot\engbold 0.848\engbold 0.855\engbold 0.857\engbold 0.848\engbold 0.857
Table 7: Performance of LLMs under zero-shot and few-shot prompting. The best result is \engbold bolded, and the second best is underlined .
Model AModel Bbbccnnpmid-pp_{\text{mid-}p}
fastText-SRNfastText-LSTM1016260.248
fastText-GRUfastText-LSTM1215270.572
\engbold fastText-CNN\engbold fastText-LSTM\engbold 6\engbold 17\engbold 23\engbold 0.0227
\engbold XLM-R\engbold fastText-LSTM\engbold 10\engbold 24\engbold 34\engbold 0.0167
\engbold TF-IDF-LR\engbold fastText-LSTM\engbold 10\engbold 30\engbold 40\engbold 0.00145
\engbold TF-IDF-SVM\engbold fastText-LSTM\engbold 10\engbold 30\engbold 40\engbold 0.00145
Table 8: Pairwise McNemar mid- pp test results. Model B denotes the higher-performing model (higher F1F_{1} -score) in each comparison. bb = pairs where Model A is correct and Model B is wrong; cc = pairs where Model A is wrong and Model B is correct; n=b+cn=b+c . The statistically significant pairs are \engbold bolded.
ModelAccuracyma-Precisionma-Recallma- F1F_{1} -scoreAUROC
TF-IDF-LR \engital w augmentation0.8700.8690.8690.8690.943
TF-IDF-LR \engital w/o augmentation0.8480.8460.8460.8460.938
Δ\Delta TF-IDF-LR\engbold +2.20%\engbold +2.30%\engbold +2.30%\engbold +2.30%\engbold +0.50%
LaBSE-LR \engital w augmentation0.8970.8960.8960.8960.947
LaBSE-LR \engital w/o augmentation0.8970.8960.8960.8960.947
Δ\Delta LaBSE-LR0.00%0.00%0.00%0.00%0.00%
Table 9: Ablation Study 1: Impact of data augmentation across embedding families and architectures. \engital w augmentation denotes training with 155 synthetic-reviewed NC instances added to the training set, \engital w/o augmentation denotes training on original data only. Δ\Delta indicates change over the no-augmentation baseline. Meaningful improvement are \engbold bolded ( ≥1.00%\geq 1.00\% ).
ModelAccuracyma-Precisionma-Recallma- F1F_{1} -scoreAUROC
fastText-LSTM \engital (fine-tuned)0.9140.9120.9150.9130.961
fastText-LSTM \engital (pre-trained)0.8500.8480.8520.8490.922
Δ\Delta fastText-LSTM\engbold +6.40%\engbold +6.40%\engbold +6.30%\engbold +6.40%\engbold +3.90%
fastText-GRU \engital (fine-tuned)0.9050.9040.9040.9040.959
fastText-GRU \engital (pre-trained)0.8520.8500.8540.8510.934
Δ\Delta fastText-GRU\engbold +5.30%\engbold +5.40%\engbold +5.00%\engbold +5.30%\engbold +2.50%
Table 10: Ablation Study 2: Impact of fine-tuning pretrained embeddings on the electoral commitment dataset. \engital fine-tuned denotes pretrained embeddings further trained on the electoral commitment training corpus, \engital pre-trained denotes frozen pretrained embeddings used without any domain adaptation. Δ\Delta indicates change of fine-tuned over pre-trained baseline.Meaningful improvement are \engbold bolded ( ≥1.00%\geq 1.00\% ).
Figure 5: Confusion matrices for LaBSE-LR, fastText-LSTM, XLM-RoBERTa, and Qwen3-8B (few-shot).
FactorStatement (ne)Statement (en)Error Type
 \devanagarifont युवाहरुलाई सीपमूलक तालिमको व्यवस्था गरी लघु तथा घरेलु उद्योग सञ्चालनमा प्रोत्साहन गरिने छ।Youth will be encouraged to operate small and cottage industries by providing skill-based training.FP
Lexical cue\devanagarifont प्रत्येक तहमा जिम्मेवारी विने, कामको मुल्याङ्‌कन गर्ने र पदोन्नति गर्ने वैज्ञानिक पद्धतिको निर्माण गरी अवलम्वन गरिने छ।A scientific system for assigning responsibilities, evaluating work, and promoting will be developed and adopted at each level.FP
 \devanagarifont नगरभित्र कवड हल र रतुवामाई ४ गढीमाई महाराजथानमा रंगशाला निर्माणका लागि आवश्यक पहल गरिने छ।Necessary initiatives will be taken to construct a stadium in the city’s Kawad Hall and Ratuwamai 4 Gadhimai Maharajthan.FP
 \devanagarifont ईन्टरनेट सेवा जिल्ला भरInternet service throughout the districtFN
 \devanagarifont उदाहरणीय सुशासनExemplary Good GovernanceFN
Syntactic completeness\devanagarifont जस अन्तरगत ५ वर्ष भित्र १ लाख नयाँ बोट रोपनि छनUnder which 1 lakh new trees will be planted within 5 years.FN
Table 11: Representative example error cases grouped by linguistic factor. FP = false positive (NC predicted as C), FN = false negative (C predicted as NC).
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

ModelText RepresentationHyperparameters
LRTF-IDFC=10.0, class_weight=None, penalty=l2, max_features=20000, ngram_range=(1, 2)
LaBSEC=1.0, class_weight=balanced, penalty=l2
DTTF-IDFclass_weight=None, criterion=entropy, max_features=5000, ngram_range=(1, 1), max_depth=20, min_samples_split=10, min_samples_leaf=2
LaBSEclass_weight=balanced, criterion=entropy, max_depth=5, min_samples_split=2, min_samples_leaf=1
RFTF-IDFclass_weight=None, n_estimators=300, max_features=5000, max_depth=None, min_samples_split=2, min_samples_leaf=1, ngram_range=(1, 2)
LaBSEclass_weight=None, n_estimators=100, max_depth=10, min_samples_split=10, min_samples_leaf=2
Appendix
Table 12: Optimal hyperparameter settings for ML models under TF-IDF and LaBSE text representations
ModelText RepresentationModel ArchitectureHyperparameters
SRNfastTextEmbedding (max_length=64, dim=300), SRN (36 units), Sigmoid (1)Batch size=16, epochs=30, patience=3, lr=3e-4, dropout=0.3, l2_reg=0.0005
GloVeEmbedding (max_length=64, dim=300), SRN (48 units), Dense (32 units), Sigmoid (1)Batch size=16, epochs=30, patience=3, lr=3e-4, dropout=0.5, l2_reg=0.001
LSTMfastTextEmbedding (max_length=64, dim=300), LSTM (36 units), Sigmoid (1)Batch size=16, epochs=30, patience=3, lr=3e-4, dropout=0.3, l2_reg=0.0005
GloVeEmbedding (max_length=64, dim=300), LSTM (48 units), Dense (24 units), Sigmoid (1)Batch size=16, epochs=30, patience=3, lr=5e-4, dropout=0.3, l2_reg=0.0005
GRUfastTextEmbedding (max_length=64, dim=300), GRU (36 units), Sigmoid (1)Batch size=16, epochs=30, patience=3, lr=3e-4, dropout=0.3, l2_reg=0.0005
GloVeEmbedding (max_length=64, dim=300), GRU (36 units), Dense (24 units), Sigmoid (1)Batch size=16, epochs=30, patience=3, lr=5e-4, dropout=0.3, l2_reg=0.001
Appendix
Table 13: Architecture and optimal hyperparameter settings for DL models under fastText, GloVe, and transformer-based representations

Explore similar work

CardsList
Apr 21, 2026

Seeing Candidates at Scale: Multimodal LLMs for Visual Political Communication on Instagram

This paper presents a computational case study that evaluates the capabilities of specialized machine learning models and emerging multimodal large language models for Visual Political Communication (VPC) analysis. Focusing on concentrated visibility in Instagram stories and posts during the 2021 German federal election campaign, we compare the performance of traditional computer vision models (FaceNet512, RetinaFace, Google Cloud Vision) with a multimodal large language model (GPT-4o) in identifying front-runner politicians and counting individuals in images. GPT-4o outperformed the other models, achieving a macro F1-score of 0.89 for face recognition and 0.86 for person counting in stories. These findings demonstrate the potential of advanced AI systems to scale and refine visual content analysis in political communication while highlighting methodological considerations for future research.

Michael Achmann-Denkler, Mario Haim, Christian Wolff

Media Informatics Group University of Regensburg Regensburg, Germany · Department of Media and Communication Ludwig-Maximilians-Universität Munich, Germany

VLM EvaluationSocial Media AnalysisFace Recognition
AskPDF
Aug 12, 2026

Who Would You Vote For? Auditing Political Alignment in LLMs: An Italian Case-Study

As users increasingly turn to Large Language Models (LLMs) for information and advice on political matters, particularly during election periods, the political preferences expressed by these systems have become a matter of public interest. Prior research has shown that interactions with LLMs can influence users' political attitudes and choices, raising questions about how these models themselves evaluate political actors. In this paper, we investigate whether and how LLMs express preferences toward political parties and political leaders. We introduce a systematic and reproducible auditing framework in which multiple LLMs are prompted to evaluate parties and leaders across nine criteria. Rather than attempting to infer the models' "true" political beliefs, we focus on their observable behavior, examining consistency across evaluations, differences between models, refusal rates, and sensitivity to prompt formulation. We further investigate how these evaluations vary when models are instructed to adopt different personas. We demonstrate the framework through an Italian case study, providing a systematic analysis of LLM-generated political evaluations on italian parties and leaders.

Simone Mungari

Revelis s.r.l.

LLM AuditingPolitical Bias in Language ModelsLanguage Model Bias Evaluation
AskPDF
Aug 16, 2026

Large Language Models as Implicit Sociological Models: Reconstructing Voting Behaviour from Sociodemographic Profiles

Large language models (LLMs) trained on large-scale internet corpora encode extensive statistical regularities about social identities, attitudes, and political behaviour. This paper introduces and evaluates a methodological framework that leverages these latent representations to reconstruct aggregate voting behaviour from individual-level sociodemographic profiles. We operationalize LLMs as implicit sociological models by conditioning them on demographic descriptions, eliciting probabilistic turnout and party preferences, and aggregating individual outputs via a soft voting procedure. Using the 2021 Czech parliamentary election as a validation case, we demonstrate that contemporary LLMs reproduce official election outcomes with low mean absolute error, recover known political bloc structures, and align with independently established sociodemographic gradients. The contribution of this work is methodological rather than predictive: we show how LLMs can be systematically interrogated as compressed representations of social reality, offering a novel exploratory instrument for computational social science while clearly delineating its epistemic and ethical limits.

Roman Neruda, Martin Bakoš, Josef Šlerka +3

Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic. · Faculty of Arts, Charles University, Prague, Czech Republic. · distil labs, Berlin, Germany.

Computational Social ScienceLarge Language Model-Based Social Simulation
AskPDF