As Large Language Models (LLMs) become integrated into daily life, they are increasingly used for personal queries, including Sexual and Reproductive Health (SRH), allowing users to chat anonymously without fear of judgment. However, current evaluation methods primarily focus on accuracy, often for objective queries in high-resource languages, and lack criteria to assess usability and safety, especially for low-resource languages and culturally sensitive domains like SRH. This paper introduces LLM Evaluation Framework (LEAF), that conducts assessments across multiple criteria: accuracy, language, usability gaps (including relevance, adequacy, and cultural appropriateness), and safety gaps (safety, sensitivity, and confidentiality). Using the LEAF framework, we assessed 14K SRH queries in Nepali from over 9K users. Responses were manually annotated by SRH experts according to the framework. Results revealed that only 35.1% of the responses were "proper", meaning they were accurate, adequate and had no major usability or safety related gaps. Insights include differences in performance between ChatGPT versions, such as similar accuracy but varying usability and safety aspects. This evaluation highlights significant limitations of current LLMs and underscores the need for improvement. The LEAF Framework is adaptable across domains and languages, particularly where usability and safety are critical, offering a pathway to better address sensitive topics.
Figures & tables
Figure 1: Overall Methodology : Architecture of the data collection and conversation handling platform, illustrating the flow of user queries to the LLM and the subsequent evaluation of responses by SRH experts using the LEAF Framework.
Criteria
Index
Description
Variables (Labels)
\multirow 34emAccuracy
\multirow 32cmAccuracy
\multirow 38cmDescribes how accurate the response is. A response is accurate if all major facts are correct. It is partially accurate if one or more components are incorrect, and inaccurate if most information is wrong.
\tabitem Accurate
\tabitem Partially accurate
\tabitem Inaccurate
\multirow 34emLanguage
\multirow 32cmDifferent Language
\multirow 38cmRefers to instances where the response language differs from the language of the question. For example, if a question is asked in Nepali but the response is in English.
\tabitem Same language
\tabitem Different Language
Table 1: Assessment Criteria Table
Figure 2: Number of Conversations
Figure 3: Demographic and Geographic Profile. (A) Distribution of user background, highlighting the Female Community Health Volunteers (FCHVs). (B) Geographic coverage across Nepal, showing user density spanning all seven provinces.
Variables
Age
Gender
Education Level
Location (Province)
Device Used
Label
N%
Label
N%
Label
N%
Label
N%
Label
N%
0-10
0.14
Male
36.10
Literate
5.10
Bagmati
38.27
Mobile
81.91
11-20
30.36
Female
63.41
Primary
5.10
Gandaki
7.58
Tablet
0.05
21-30
35.88
Other
0.49
Secondary
15.11
Karnali
7.42
Laptop
18.04
31-40
15.09
High School
35.89
Koshi
16.46
Table 2: Demographic Profile of Participants. Bold values indicate the most common label within each variable.
Most Discussed Topics
N%
Least Discussed Topics
N%
menstruation
13.46
sexual-violence
0.003
sex
11.58
narcotics
0.02
knowledge-srhr
9.85
stigma
0.05
contraceptives
7.99
information-on-project
0.05
Table 3: Most and Least Discussed Topics of Conversations
Figure 4: Language and Accuracy Metrics. (A) The shift in script usage: users predominantly query in Romanized Nepali (Orange), while the bot responds in Devanagari (Blue). (B) Overall response accuracy, with 62.1% of responses categorized as fully accurate.
Figure 5: Usability Gap Analysis. (A) The dominance of ‘Inadequacy’ (74.0%) is clearly visible, while ‘Cultural’ and ‘Outdated’ gaps are negligible in volume (<1%), so they have been categorized as others. (B) Detailed breakdown by accuracy. The inset panel magnifies the ‘Cultural’ and ‘Outdated’ categories, revealing that while rare, these gaps are predominantly associated with inaccurate responses (Red/Yellow), unlike ‘Inadequacy’ which often occurs in accurate responses (Blue).
Safety Gap
Count
Percentage of Total Responses(%)
unsafe
98
0.74
insensitive/offensive
51
0.38
non-confidential
7
0.05
Table 4: Analysis of safety gaps across all the Responses
Figure 6: Model Comparison (GPT-3.5 vs. GPT-4). Analysis of usability gaps across 100 sampled queries. The grouped bars allow for direct comparison between GPT-3.5 (Left bar of each pair) and GPT-4 (Right bar). GPT-4 demonstrates a significant reduction in ‘Irrelevant’ responses (from 28 to 10) and ‘Inaccurate’ content (Red blocks).
Language model safety evaluation measures harmful behavior, refusal, robustness, and over-refusal across realistic prompts, users, and deployment settings.