Large Language Models (LLMs) consistently under perform in low-resource linguistic contexts such as Konkani. This performance deficit stems from acute training data scarcity compounded by high script diversity across Devanagari, Romi and Kannada orthographies. To address this gap, we introduce Konkani-Instruct-100k, a comprehensive synthetic instruction-tuning dataset generated through Gemini 3. We establish rigorous baseline benchmarks by evaluating leading open-weights architectures including Llama 3.1, Qwen2.5 and Gemma 3 alongside proprietary closed-source models. Our primary contribution involves the development of Konkani LLM, a series of fine-tuned models optimized for regional nuances. Furthermore, we are developing the Multi-Script Konkani Benchmark to facilitate cross-script linguistic evaluation. In machine translation, Konkani LLM delivers consistent gains over the corresponding base models and is competitive with and in several settings surpasses proprietary baselines
Figures & tables
ID
Foundational topic
1
Food
2
Family relationships
3
Numbers
4
Time
5
Body parts
6
Colors
Table 1: Foundational topics used to generate tutor-style lessons for basic Konkani competence.
Script
Train
Validation
Devanagari
26,006
4,820
Romi (Latin)
38,382
4,008
Kannada
26,845
3,917
English
1,000
1,000
Total
92,233
13,745
Table 2: Distribution of Konkani-Instruct-100k across scripts and splits.
Setting
Value (default)
Sequence length
4096
Epochs
2
Per-device batch size
Train 1 (some runs: 4), Eval 1
Gradient accumulation
32 (some runs: 8; effective batch size 32)
Optimizer
AdamW 8-bit ( adamw_8bit )
Learning rate
2×10−4
Table 3: Default fine-tuning configuration used across models unless otherwise stated.
Model
BLEU
chrF++
COMET
Proprietary & Large API Models
gemini-flash-latest
86.76
92.13
91.14
gemini-3-pro-preview
71.50
83.73
88.56
gpt-5.2
57.37
74.04
85.93
claude-sonnet-4-6
57.16
74.65
84.43
gpt-5-mini
34.87
54.76
75.53
Table 4: Automatic metrics for translation (200 samples). The “Task” column has been omitted as all entries represent translation tasks. Models are grouped by category and sorted by COMET score descending within groups.
Model
BLEU
chrF++
D → K
D → R
K → D
K → R
R → D
R → K
D → K
D → R
K → D
K → R
R → D
R → K
gemini-flash-latest
29.83
43.83
35.07
47.66
78.13
67.01
66.69
77.34
75.90
79.59
91.86
87.70
claude-sonnet-4-6
21.11
10.48
23.75
10.08
44.77
30.75
58.32
43.42
57.82
39.51
72.86
67.60
gpt-5.2
7.79
13.35
33.46
12.58
31.89
15.09
44.76
46.70
67.02
47.76
66.52
56.03
gpt-5-mini
1.95
6.57
20.41
9.13
11.74
5.90
33.89
43.54
57.81
48.53
47.84
41.16
grok-4-fast-non-reasoning
8.28
7.87
13.30
10.66
6.81
4.16
44.47
41.92
52.12
47.53
42.50
37.78
Table 5: Automatic metrics for transliteration. Directions are abbreviated as follows: D (Devanagari), K (Kannada), R (Romi). Note that COMET is excluded as it is not applicable to transliteration. All pairs evaluated on 200 samples except where marked with * (160 samples).
Base model
Simple chat
Creative writing
Romi
Devanagari
Kannada
Romi
Devanagari
Kannada
Qwen2.5 1.5B
2.3
3.1
3.9
2.1
1.3
1.9
Qwen2.5 14B
4.0
4.4
4.4
3.7
2.8
3.6
Gemma 4B
3.4
4.2
3.1
1.9
1.9
2.0
Gemma 12B
3.9
3.2
4.5
2.5
2.2
1.9
Gemma 27B
4.6
4.5
4.3
2.8
3.2
3.0
Table 6: LLM-as-a-judge scores (1–5) for Simple chat and Creative writing by script (judge: Gemini 3 Flash).
Base model
Logical reasoning
Translation
Romi
Devanagari
Kannada
Romi
Devanagari
Kannada
Qwen2.5 1.5B
2.0
1.6
2.1
2.8
2.2
2.5
Qwen2.5 14B
3.9
3.5
3.4
4.5
4.3
4.8
Gemma 4B
2.4
2.8
2.8
3.1
3.5
3.1
Gemma 12B
3.0
3.3
2.4
4.0
3.7
4.0
Gemma 27B
3.3
4.4
3.2
3.5
3.5
1.8
Table 7: LLM-as-a-judge scores (1–5) for Logical reasoning and Translation by script (judge: Gemini 3 Flash).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Response
gemma-3-4b
In the Goan Catholic dialect (often spoken by the Susegad community), the phrase is typically expressed as a question about whether one has finished their meal. 1. The Konkani Answer The most authentic way to ask this, focusing on the completion of the meal: \devanagarifont तुजें जेवण जालां? (Tujẽ jevonn jalañ?) English Translation: Is your food/lunch over? (Literally: Has your eating happened?) 2. Key Vocabulary & Grammar • \devanagarifont तुजें (Tujẽ): Your / Yours • \devanagarifont जेवण (Jevon): Lunch / Meal • \devanagarifont जालां (Jalañ): Finished / Done • \devanagarifont सांजचो (Sanjcho): Of the evening • \devanagarifont जेवंक (Jeunk): To eat 3. Alternative Version If you are asking if someone has already eaten and wants to know when they will be back from work or a meeting: • Alternative: \devanagarifont तुमी जेवल्यात? • Translation: (Have you eaten yet?) • Key words: \devanagarifont तुमी (Tumi) is the formal/plural form of ”you,” which is used in polite inquiries or when referring to a group of people. \devanagarifont जेवल्यात (Jevleat) is the plural past tense of the verb ”to eat.”
gemma-3-12b
In standard Goan Konkani (Antruz), this question is typically phrased by asking if one has eaten or if one’s stomach/belly is full. 1. The Konkani Answer • Introduction: In Goa, it is more natural to ask if someone has eaten than if they had a meal. • Konkani Text: \devanagarifont तुजें जेवण जालें? • English Translation: Did you eat your lunch? (Literally: Has your food happened?) • Show mistakes: A common mistake for learners is using the word jevop (act of eating) instead of the neuter noun jevon (the meal). Since Jevon is Neuter, the possessive pronoun must be \devanagarifont तुजें (Tujẽ) and not \devanagarifont तुजो (Masculine) or \devanagarifont तुजी (Feminine). 2. Key Vocabulary & Morphology • \devanagarifont तुजें (Tujẽ): Your / Yours • \devanagarifont जेवण (Jevon): Meal / Lunch • \devanagarifont जालें (Zalẽ): Happened / Done • \devanagarifont भुरगें (Bhurgem): Child (Used as ’Baby’) • \devanagarifont उदक (Udok): Water • \devanagarifont पियेंवप (Piyenvp): To drink / Drinking 3. Alternative Version If you are speaking to a child or in a very casual setting, you might use a shorter version focusing on whether their stomach is empty. • Alternative: \devanagarifont पोट रिकां? • Translation: (Is your stomach empty?) • Key words: \devanagarifont पोट (Pot): Stomach/Abdomen. \devanagarifont रिकें (Rikẽ): Empty. \devanagarifont रिकां (Rikã): The nasalized form used when addressing a person directly (Informal).
gemma-3-27b
In Goan culture, especially among the Catholic community, this is a very common way to greet someone or check on their well-being. 1. The Standard Way (Casual/Informal) When speaking to a friend, younger person, or an equal: \devanagarifont तुजें जेवण जालां? (Tujẽ jevonn jalañ?) Translation: Has your meal happened/been done? 2. The Formal Version When speaking to elders, teachers, or strangers: \devanagarifont तुमचें जेवण जालां? (Tumchẽ jevonn jalañ?) Translation: Has your meal been done? 3. A More Direct Inquiry If you want to ask specifically if they ”ate” rather than just asking if it was finished: \devanagarifont तूं जेवलो/जेवली? (Tuñ jeulo/jeuli?) Translation: Did you eat? (Masculine / Feminine). Key Vocabulary & Grammar Logic • Jevonn ( \devanagarifont जेवण): This refers to the act of eating or the meal itself. In Konkani, we often say ”the food has happened” (jevonn zalam) rather than just saying ”I ate.” • Tujẽ vs Tumchẽ: Use Tujẽ for friends and family. Use Tumchẽ for respect. • The Suffix (-lañ): This is the standard interrogative suffix used at the end of sentences in the Antruz dialect to turn a statement into a question.
llama-3.1-8b
In Goan culture, asking about someone’s meal is a sign of hospitality and concern. The most authentic way to ask this in Devanagari Konkani (Antruz dialect) is: \devanagarifont तुका जेवण मेळ्ळें? (Tuka jevonn mell’llem?) Translation: Did you get your food/lunch? Key Vocabulary & Grammar • \devanagarifont तुका (Tuka): This is the dative form of the pronoun Tu (You). In Konkani, when expressing an action that happened to someone, we use the dative case instead of the nominative. • \devanagarifont जेवण (Jevonn): While it literally means ”meal,” it is the standard term used for any kind of food or lunch in Goa. • \devanagarifont मेळ्ळें (Mell’llem): This comes from the root verb \devanagarifont मेळप (Melop) , which means ”to find” or ”to receive.” It is conjugated here in the past tense to match the neuter noun jevonn . Alternative Version If you want to be more specific about ”lunch” (often referred to as donparchem jenvon) or if you are speaking to someone older/formal, you might say: • \devanagarifont दोनपारचें जेवण खाल्लें? (Donparchem jevonn khall’lem?) Translation: Did you eat your lunch? Key Words: \devanagarifont दोनपारचें (Donparchem): Derived from Donpar (Afternoon). \devanagarifont खाल्लें (Khall’lem): The past tense of \devanagarifont खावप (Khavap), meaning ”to eat.”
qwen-2.5-14b
In standard Devanagari Konkani (Antruz dialect), the most natural and polite way to ask this is: \devanagarifont तुका जेवण जालें काय? (Tuka jevon zalem kai?) This translates literally to: “Has your meal happened?” Alternative Version: If you want to be more direct or use a slightly different phrasing often heard in casual conversation: • Alternative: \devanagarifont तूं जेवलो काय? • Translation: (Have you eaten?) Note: In Konkani, verbs change based on gender. If speaking to a male, say \devanagarifont जेवलो (jevlo); for a female, say \devanagarifont जेवली (jevli). For a general or respectful address, \devanagarifont जेवलो is commonly used.
qwen2.5-1b
In Devanagari Konkani (Standard Goan), the most natural way to ask this is: \devanagarifont तुका जेवण मेळ्ळें काय? Translation: Did you get your lunch? Detailed Explanation: 1. Subject: The word \devanagarifont तुका (Tuka) means ”to you” or ”you.” In Konkani, when asking about someone else’s experience, we use the dative case. 2. Object: \devanagarifont जेवण (Jevon) refers specifically to a meal or lunchtime food. 3. Verb Construction: \devanagarifont मेळ्ळें (Mell’lem) : This comes from the root verb \devanagarifont मेळप (Melop) meaning ”to find” or ”to receive.” \devanagarifont काय (Kay) : This is a question marker added at the end of a sentence to turn it into a polite inquiry. If you want to sound more casual or emphasize that you actually ate something, you can also say: • \devanagarifont तूं जेवलो/जेवली काय? • \devanagarifont तुका जेवण आयलें काय? (Literally: Did lunch come to you?)
Appendix
Table 8
Model
Response
gemma-3-4b
In the Goan and Mangalorean Catholic dialect of Konkani, the most authentic way to ask this is by using the word Jevon. 1. The Konkani Answer \kannadafont ತುಜೆಂ ಜೆವಣ್ ಆಸ್ಲೆಗಿ? (Tujeñ jevon asle-gi?) English Translation: Did you have your meal? 2. Key Vocabulary & Grammar • \kannadafont ತುಜೆಂ (Tujẽ): Matches neuter noun Jevon. • \kannadafont ಜೆವಣ್ (Jevon): Refers to a meal. • \kannadafont ಆಸ್ಲೆಗಿ (Asle-gi): Past tense verb matching subject. • \kannadafont ಖಾಲ್ಲೆಂ (Khal’lẽ): The act of eating. 3. Alternative Version • Alternative: \kannadafont ತುಜೆಂ ಖಾಣ್ ಪರ್ತಿ ಆಯ್ಲಾಂ? • Translation: (Is your food back again / Have you had some snacks?)
gemma-3-12b
In Kannada Konkani, this question is typically asked by asking if one has eaten or if one’s stomach/belly (pot) is full. 1. The Konkani Answer • Konkani Text: \kannadafont ತುಜೆಂ ಪೋಟ್ ಭರ್ಲಾಂ? • English Translation: Is your stomach full? • Note: Saying Pot bhorlam is the authentic way to inquire. 2. Key Vocabulary & Morphology • \kannadafont ತುಜೆಂ (Tujem): Your • \kannadafont ಪೋಟ್ (Pot): Stomach / Belly • \kannadafont ಭರ್ಲಾಂ (Bhorlam): Filled / Full • \kannadafont ಖಾಣ್ (Khan): Food / Meal 3. Alternative Version • Alternative: \kannadafont ತುಜೆಂ ಜೆವಣ್ ಜಾಲೆಂ? • Translation: (Was your meal done?)
gemma-3-27b
In Konkani, we use a construction that translates to ”Has your stomach-filling happened?” 1. The Standard Way (Polite/Formal) \kannadafont ತುಮ್ಗೆಂ ಜೆವಣ್ ಜಾಲಾ ಗಿ? (Tumgem jevonn jala gi?) 2. The Casual Way (Informal) \kannadafont ತುಜೆಂ ಜೆವಣ್ ಜಾಲೆಂ? (Tujem jevonn jalem?) 3. Common Variations • \kannadafont ಪೋಟ್ ಭರ್ಲಾಂ ಗಿ? (Is your stomach full?) • \kannadafont ಖೆಲ್ಲೆಂ? (Ate?) • \kannadafont ದನ್ಪಾರಾಚೆಂ ಜೆವಣ್ ತುಕಾ ಮೆಳ್ಳಾ? (Did you get your afternoon meal?)
llama-3.1-8b
In the Mangalorean Konkani tradition, the most authentic way to ask this is: \kannadafont ತುಕಾ ಜೆವಣ್ ಮೆಳ್ಳೆಂ? (Tuka jevon mell’lem?) Key Vocabulary & Grammar: • \kannadafont ತುಕಾ (Tuka): Dative form of ’You’. • \kannadafont ಜೆವಣ್ (Jevon): Traditional term for a full meal. • \kannadafont ಮೆಳ್ಳೆಂ (Mell’lem): From verb Mel (to get/find). Alternative Version: • Alternative: \kannadafont ಖಾಲೆಂ? • Translation: (Did you eat?) • Note: Often followed by Khay sab! (Eat, man!).
qwen-2.5-14b
People typically use \kannadafont ದುಷ್ಕರೆ (Dushkre) or \kannadafont ಜೇವಣ್ (Jevon). 1. Using Dushkre (Traditional/Catholic style) \kannadafont ತುಂ ದುಷ್ಕರೆ ಕೆಲೊಯ್ಗಿ? (Tum dushkre kelyogi?) 2. Using Jevon (General/Common style) \kannadafont ತುಂ ಜೇವಣ್ ಖಾಲ್ಯಾಯ್ಗಿ? (Tum jevon khalyaygi?) Word-by-Word Breakdown: • \kannadafont ತುಂ (Tum): You • \kannadafont ದುಷ್ಕರೆ (Dushkre): Lunch • \kannadafont ಕೆಲೊಯ್ಗಿ (Kelyogi): Did you eat?
qwen2.5-1b
In the context of asking about food or meals: \kannadafont ತುಂ ತುಮ್ಚೊ ಜೆವಣ್ ಖಾಲ್ಲೊಗೀ? Translation: Did you eat your lunch (meal)? Detailed Explanation: The sentence uses the interrogative pronoun \kannadafont ತುಂ (Tum) and the past tense verb \kannadafont ಖಾಲ್ಲೊಗೀ (Khallo-gi). The suffix -gi turns it into a question. Alternative (Formal): • \kannadafont ತುಂ ಮಧ್ಯಾನ್ ಜೆವಣ್ ಕೆಲ್ಲೆಂಗೀ? (Did you have lunch at noon?)
Appendix
Table 9
Model
Response
gemma-3-4b
In Goan Catholic Konkani, the most authentic way is using Jevon . 1. The Standard Question • Konkani Text: Jevon zalam? • Translation: Is your lunch over? / Have you had your meal? 2. Casual Variation • Casual: Jevon kela? • Translation: Did you eat lunch? 3. Cultural Context • Hungry: Tuka jevna-maka? • Translation: (Are you hungry for some food?)
gemma-3-12b
In Goan Catholic Konkani, the most natural way to ask this is: Tumi jevonn kela? 1. The Answer • Konkani Text: Tumi jevonn kela? • Translation: Have you had your lunch/meal? 2. Key Words • Jevonn : Meal / Lunch • Kela : Done / Had • Zalem : Happened / Finished 3. Alternative Version • Alternative: Tuka pottant bhorlam? • Translation: (Is your stomach full?)
gemma-3-27b
In Goan Catholic households, the most authentic way to say this in Romi Konxem is: 1. The Standard Way (Formal/Plural) Tumni xit khailam? Translation: Have you [plural/formal] eaten rice? 2. The Informal Version Tuvem xit khailam? Translation: (Informal) Have you eaten rice? 3. Casual / Shortened Version Xit khailam? Translation: Did you eat rice?
llama-3.1-8b
In Goan Catholic culture, the most authentic and polite way to ask this is: “Tum jenvlam kai?” • Translation: “Have you eaten [your] meal?” • Key words: Jenvlam (from Jeun - to eat a meal) and Kai (question particle). Alternative Version: “Jevon zalam kai?” • Translation: “Is your meal done/over?” Cultural Context: Asking if someone has “eaten rice” ( Xit khaleat ) is considered slightly blunt.
qwen-2.5-14b
In Romi Konkani, the most common way to ask this is: Tum jenvon kela? This translates literally to “Did you have your meal?” Alternative versions: • Tum jevonn zalem? (A more casual “did you finish eating”). • Jevun ailai? (Literally: “Did you come having eaten?”).
qwen2.5-1b
In Romi Konkani (Romi script): 1. The Question: “Have you eaten rice?” • Konkani Text: Tumchean khann ghetlam? • Translation: Have you taken your meal? 2. A more casual version: • Konkani Text: Khaunchem? • Translation: Did you eat? Common Lunchtime Phrases: • Do you want to eat? → Khatunk zai? • What time is lunch? → Jevnnacho vell kitem?
Large Language Models (LLMs) offer new potential for translation tasks but often experience performance degradation when handling low-resource languages. To address this limitation, we propose an approach for fine-tuning LLMs on a low-resource language, Kupang Malay. Our approach involves designing a set of instructions by leveraging explicit lexical and semantic features from a bilingual dictionary, and introducing Continual Instruction Tuning (CIT), a training paradigm that enables iterative instruction-based training. Experimental results demonstrate that our model, named Lius, yields notable improvements over standard instruction-tuned models by outperforming 4-6 points, and surpassing both Neural Machine Translation (NMT) and Multilingual LLM models by 10-13 points on several evaluation metrics. These findings highlight the potential of our approach to mitigate the reliance on large-scale parallel data in low-resource language translation.
Recent advances in Multilingual Large Language Models (MLLMs) have significantly enhanced cross-lingual conversational capabilities, yet modeling culturally nuanced and context-dependent communication remains a critical bottleneck. Specifically, existing state-of-the-art models exhibit a severe pragmatic gap when handling structural variations, regional idioms, and honorific consistencies in low-resource contexts like Bangla. To address this limitation, we introduce a novel, culturally aligned instruction-tuning dataset for BangLa Application and DialoguE generation - BLADE and benchmarking framework comprising 4,196 meticulously curated interaction pairs. We leverage this resource to systematically fine-tune and evaluate leading open-weight architectures, including DeepSeek-8B and LLaMA-3.2-3B, utilizing parameter-efficient fine-tuning via LoRA adapters in a 4-bit NormalFloat (NF4) quantization framework. Our empirical evaluations demonstrate that models fine-tuned on our dataset yield substantial improvements in structural fidelity and honorific alignment, providing a rigorous benchmark for bridging pragmatic disparities in low-resource multilingual text generation. Code and dataset: https://github.com/ashuvo25/Bangla_Application_LLM/tree/main
Constructed languages (conlangs) are intentionally created human languages with a rich tradition of linguistic creativity. Despite their potential for studying language learning in large language models (LLMs), existing conlangs remain largely underexplored in LLM research. We present ConlangBench, the first large-scale benchmark for evaluating and training LLMs on 21 existing conlangs. We collect over 21M conlang-English parallel sentence pairs (including 430K pairs across the 20 non-Esperanto conlangs) and 321K vocabulary entries. In bidirectional translation experiments, we find that models perform better on a posteriori conlangs, whose vocabularies are derived from natural languages, reflecting the design characteristics of conlangs. Training on ConlangBench also shows that models can learn all eight conlangs for which sufficient parallel corpora are available, while their learning curves vary depending on how the conlangs were created. Our findings suggest that conlangs provide a unique testbed for investigating how LLMs acquire low-resource languages.