As Large Language Models (LLMs) become integrated into daily life, they are increasingly used for personal queries, including Sexual and Reproductive Health (SRH), allowing users to chat anonymously without fear of judgment. However, current evaluation methods primarily focus on accuracy, often for objective queries in high-resource languages, and lack criteria to assess usability and safety, especially for low-resource languages and culturally sensitive domains like SRH. This paper introduces LLM Evaluation Framework (LEAF), that conducts assessments across multiple criteria: accuracy, language, usability gaps (including relevance, adequacy, and cultural appropriateness), and safety gaps (safety, sensitivity, and confidentiality). Using the LEAF framework, we assessed 14K SRH queries in Nepali from over 9K users. Responses were manually annotated by SRH experts according to the framework. Results revealed that only 35.1% of the responses were "proper", meaning they were accurate, adequate and had no major usability or safety related gaps. Insights include differences in performance between ChatGPT versions, such as similar accuracy but varying usability and safety aspects. This evaluation highlights significant limitations of current LLMs and underscores the need for improvement. The LEAF Framework is adaptable across domains and languages, particularly where usability and safety are critical, offering a pathway to better address sensitive topics.
Figures & tables
Figure 1: Overall Methodology : Architecture of the data collection and conversation handling platform, illustrating the flow of user queries to the LLM and the subsequent evaluation of responses by SRH experts using the LEAF Framework.
Criteria
Index
Description
Variables (Labels)
\multirow 34emAccuracy
\multirow 32cmAccuracy
\multirow 38cmDescribes how accurate the response is. A response is accurate if all major facts are correct. It is partially accurate if one or more components are incorrect, and inaccurate if most information is wrong.
\tabitem Accurate
\tabitem Partially accurate
\tabitem Inaccurate
\multirow 34emLanguage
\multirow 32cmDifferent Language
\multirow 38cmRefers to instances where the response language differs from the language of the question. For example, if a question is asked in Nepali but the response is in English.
\tabitem Same language
\tabitem Different Language
Table 1: Assessment Criteria Table
Figure 2: Number of Conversations
Figure 3: Demographic and Geographic Profile. (A) Distribution of user background, highlighting the Female Community Health Volunteers (FCHVs). (B) Geographic coverage across Nepal, showing user density spanning all seven provinces.
Variables
Age
Gender
Education Level
Location (Province)
Device Used
Label
N%
Label
N%
Label
N%
Label
N%
Label
N%
0-10
0.14
Male
36.10
Literate
5.10
Bagmati
38.27
Mobile
81.91
11-20
30.36
Female
63.41
Primary
5.10
Gandaki
7.58
Tablet
0.05
21-30
35.88
Other
0.49
Secondary
15.11
Karnali
7.42
Laptop
18.04
31-40
15.09
High School
35.89
Koshi
16.46
Table 2: Demographic Profile of Participants. Bold values indicate the most common label within each variable.
Most Discussed Topics
N%
Least Discussed Topics
N%
menstruation
13.46
sexual-violence
0.003
sex
11.58
narcotics
0.02
knowledge-srhr
9.85
stigma
0.05
contraceptives
7.99
information-on-project
0.05
Table 3: Most and Least Discussed Topics of Conversations
Figure 4: Language and Accuracy Metrics. (A) The shift in script usage: users predominantly query in Romanized Nepali (Orange), while the bot responds in Devanagari (Blue). (B) Overall response accuracy, with 62.1% of responses categorized as fully accurate.
Figure 5: Usability Gap Analysis. (A) The dominance of ‘Inadequacy’ (74.0%) is clearly visible, while ‘Cultural’ and ‘Outdated’ gaps are negligible in volume (<1%), so they have been categorized as others. (B) Detailed breakdown by accuracy. The inset panel magnifies the ‘Cultural’ and ‘Outdated’ categories, revealing that while rare, these gaps are predominantly associated with inaccurate responses (Red/Yellow), unlike ‘Inadequacy’ which often occurs in accurate responses (Blue).
Safety Gap
Count
Percentage of Total Responses(%)
unsafe
98
0.74
insensitive/offensive
51
0.38
non-confidential
7
0.05
Table 4: Analysis of safety gaps across all the Responses
Figure 6: Model Comparison (GPT-3.5 vs. GPT-4). Analysis of usability gaps across 100 sampled queries. The grouped bars allow for direct comparison between GPT-3.5 (Left bar of each pair) and GPT-4 (Right bar). GPT-4 demonstrates a significant reduction in ‘Irrelevant’ responses (from 28 to 10) and ‘Inaccurate’ content (Red blocks).
Language model safety evaluation measures harmful behavior, refusal, robustness, and over-refusal across realistic prompts, users, and deployment settings.
Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts, often overlooking the linguistic diversity and culturally grounded safety risks present in other languages. To address this gap, we introduce SurakshaEval, a novel safety benchmark composed of human-written prompts spanning real-world scenarios, explicitly designed for ten major Indian languages - Assamese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Punjabi, Tamil, and Telugu, along with English. SurakshaEval includes both generic prompts common across India and region- and language-specific prompts that capture localized sociocultural sensitivities. We benchmark a broad range of state-of-the-art LLMs on SurakshaEval, establish baseline safety performance, and identify recurring failure modes, including over-refusal, missed detection of implicit bias, and insufficient contextual awareness in regionally sensitive settings. Our results show that even strong multilingual LLMs struggle to reliably meet nuanced safety requirements when operating in Indic languages, particularly in native scripts. These findings highlight the urgent need for safety evaluation frameworks that incorporate region-specific data and structured assessment protocols, enabling the development and deployment of AI systems that operate securely, ethically, and in alignment with diverse societal values. Our code and data are available at https://github.com/debobanerjee/SurakshaEval. Warning: This paper contains text that may be offensive or unsafe.
Existing evaluation methods for healthcare LLMs primarily assess factual correctness,safety, and fluency, while providing limited insight into whether generated interactions reflect culturally situated healthcare reasoning. This limitation is particularly important in maternal health, where care decisions are shaped by social and relational norms. We introduce MH-INDIC, a culturally grounded evaluation framework for maternal-health interactions in urban and semi-urban North Indian contexts that operationalises cultural behaviour through ten dimensions of maternal-health reasoning. Using a 26-item survey administered to 102 pregnant and postpartum women from urban and semi-urban North India, we evaluate ten LLMs. We distinguish population level cultural alignment from profile-level behavioural variation. Although several models approximate the human population-level distribution, all evaluated systems exhibit substantially lower variation across demographic and household profiles than the human cohort, revealing a gap between aggregate alignment and profile-conditioned sensitivity. As a downstream application of MH-INDIC, we use the strongest-aligned proprietary and open-source models to generate culturally conditioned maternal-health dialogues under zero-shot, self-conditioned, and human-grounded prompting. Human-grounded conditioning produces stronger profile alignment and dialogue quality ratings, suggesting that measured cultural profiles can improve the cultural grounding of generated interactions
Umaira Izhar, Gunjan Arora, Pushpendra Singh
Indraprastha Institute of Information Technology Delhi
As large language models (LLMs) are deployed in multilingual settings, their safety behavior in culturally diverse, low-resource languages remains poorly understood. We present the first systematic evaluation of LLM safety across 12 Indic languages, spoken by over 1.2 billion people but underrepresented in LLM training data. Using a dataset of 6,000 culturally grounded prompts spanning caste, religion, gender, health, and politics, we assess 10 leading LLMs on translated variants of the prompt. Our analysis reveals significant safety drift: cross-language agreement is just 12.8%, and SAFE rate variance exceeds 17% across languages. Some models over-refuse benign prompts in low-resource scripts, overflag politically sensitive topics, while others fail to flag unsafe generations. We quantify these failures using prompt-level entropy, category bias scores, and multilingual consistency indices. Our findings highlight critical safety generalization gaps in multilingual LLMs and show that safety alignment does not transfer evenly across languages. We release IndicSafe, the first benchmark to enable culturally informed safety evaluation for Indic deployments, and advocate for language-aware alignment strategies grounded in regional harms.