When Less Is Enough: Context Selection and Prompting Strategies for Bengali News Headline Generation

Muhammad Ashad Kabir; Kawsar Ahmed; Md. Osama

cs.CLAug 16, 2026

When Less Is Enough: Context Selection and Prompting Strategies for Bengali News Headline Generation

Authors: Muhammad Ashad Kabir, Kawsar Ahmed, Md. Osama

Organizations: Charles Sturt University Bathurst, NSW, Australia · Chittagong University of Engineering and Technology Chittagong, Bangladesh · Bangladesh Army University of Science and Technology Nilphamari, Bangladesh

Abstract

Large language models (LLMs) have shown strong performance in text generation tasks, yet their effectiveness on headline generation remains sensitive to how input context is selected and presented. In this work, we investigate Bengali news headline generation as a document-level generation task that requires effective selection and presentation of salient contextual information from long-form articles. Using Gemini-2.0-Flash, Llama-3.3-70B, and GPT-4o, we systematically study the effects of context selection, prompting strategies, and in-context learning (i.e., few-shot) on the quality of headline generation. Our experiments show that providing the full article does not necessarily improve performance; instead, using selected lead paragraphs of the article can maintain, and in some cases improve, headline generation quality. We further compare Bengali Native Prompting (BNaP) and Cross-Lingual Prompting (XLP), and examine how each interacts with context-enriched prompt templates incorporating auxiliary contextual cues. Results demonstrate that prompting strategies substantially influence generation quality: XLP often yields stronger performance, particularly when combined with contextual enrichment, but its benefits are model-dependent. Additionally, few-shot prompting substantially improves Gemini, with most of the gain obtained from a single demonstration, whereas Llama shows limited benefit from additional examples. Overall, our findings highlight that effective Bengali news headline generation depends more on context relevance and prompt design than on increasing input length, offering practical insights for multilingual and low-resource LLM applications.

Figures & tables

Figure 1. Overview of our headline-generation workflow, illustrating paragraph selection, context conditioning, prompting strategies, contextual enrichment, and LLM evaluation.

Paragraph	Articles, n (%)	Cumulative, n (%)
1st	1138 (56.48)	1138 (56.48)
2nd	238 (11.81)	1376 (68.29)
3rd	155 (7.69)	1531 (75.98)
4th	104 (5.16)	1635 (81.14)
5th	74 (3.67)	1709 (84.81)
6th	82 (4.07)	1791 (88.88)

Table 1. Distribution of paragraph positions with the semantic similarity to the reference headline, showing frequency and cumulative percentage across the BeliN training set.

Figure 2. Paragraph-wise distribution of cosine similarity scores between reference headlines and article content, illustrating the positional concentration of salient headline-related information in early paragraphs.

Article	Gemini-2.0-Flash			Llama-3.3-70B
	BERTScore $\uparrow$	ROUGE-L $\uparrow$	METEOR $\uparrow$	BERTScore $\uparrow$	ROUGE-L $\uparrow$	METEOR $\uparrow$
$Full$	$0.719_{\color[rgb]{0.4,0.4,0.4}\pm 0.054}$ [0.715, 0.724]	$0.199_{\color[rgb]{0.4,0.4,0.4}\pm 0.146}$ [0.187, 0.212]	$0.224_{\color[rgb]{0.4,0.4,0.4}\pm 0.198}$ [0.207, 0.242]	$0.754_{\color[rgb]{0.4,0.4,0.4}\pm 0.064}$ [0.749, 0.760]	$0.243_{\color[rgb]{0.4,0.4,0.4}\pm 0.194}$ [0.227, 0.261]	$0.164_{\color[rgb]{0.4,0.4,0.4}\pm 0.203}$ [0.147, 0.183]
$paragraph_{1}$	$\textbf{0.727}_{\color[rgb]{0.4,0.4,0.4}\pm 0.064}$ [0.722, 0.732]	$0.209_{\color[rgb]{0.4,0.4,0.4}\pm 0.182}$ [0.194, 0.225]	$0.207_{\color[rgb]{0.4,0.4,0.4}\pm 0.210}$ [0.189, 0.226]	$0.747_{\color[rgb]{0.4,0.4,0.4}\pm 0.063}$ [0.742, 0.753]	$0.230_{\color[rgb]{0.4,0.4,0.4}\pm 0.201}$ [0.213, 0.248]	$0.154_{\color[rgb]{0.4,0.4,0.4}\pm 0.194}$ [0.138, 0.172]
$paragraph_{1\rightarrow{2}}$	$0.721_{\color[rgb]{0.4,0.4,0.4}\pm 0.057}$ [0.716, 0.726]	$0.204_{\color[rgb]{0.4,0.4,0.4}\pm 0.155}$ [0.191, 0.218]	$0.208_{\color[rgb]{0.4,0.4,0.4}\pm 0.193}$ [0.192, 0.225]	$0.750_{\color[rgb]{0.4,0.4,0.4}\pm 0.064}$ [0.744, 0.756]	$0.237_{\color[rgb]{0.4,0.4,0.4}\pm 0.194}$ [0.221, 0.255]	$0.159_{\color[rgb]{0.4,0.4,0.4}\pm 0.194}$ [0.144, 0.178]
$\textbf{{paragraph}}_{1\rightarrow{3}}$	$0.720_{\color[rgb]{0.4,0.4,0.4}\pm 0.056}$ [0.715, 0.725]	$0.204_{\color[rgb]{0.4,0.4,0.4}\pm 0.153}$ [0.191, 0.217]	$0.213_{\color[rgb]{0.4,0.4,0.4}\pm 0.199}$ [0.197, 0.231]	$\textbf{0.755}_{\color[rgb]{0.4,0.4,0.4}\pm 0.064}$ [0.749, 0.761]	$\textbf{0.243}_{\color[rgb]{0.4,0.4,0.4}\pm 0.191}$ [0.227, 0.261]	$\textbf{0.170}_{\color[rgb]{0.4,0.4,0.4}\pm 0.197}$ [0.153, 0.188]
$paragraph_{1\rightarrow{4}}$	$0.721_{\color[rgb]{0.4,0.4,0.4}\pm 0.058}$ [0.716, 0.727]	$0.206_{\color[rgb]{0.4,0.4,0.4}\pm 0.156}$ [0.193, 0.220]	$0.226_{\color[rgb]{0.4,0.4,0.4}\pm 0.206}$ [0.209, 0.245]	$0.752_{\color[rgb]{0.4,0.4,0.4}\pm 0.061}$ [0.746, 0.757]	$0.241_{\color[rgb]{0.4,0.4,0.4}\pm 0.189}$ [0.225, 0.257]	$0.158_{\color[rgb]{0.4,0.4,0.4}\pm 0.185}$ [0.143, 0.175]

Table 2. Performance of Gemini and Llama across varying context scopes (partial vs. full article input), illustrating the effect of selective context conditioning for the BNaP+Baseline prompt template. Values denote mean

\pm

SD, and brackets indicate 95% BCa bootstrap CI estimated over test instances.

BeliN test set	Count ( $n$ )	Input tokens
		Full article	First 3 paragraphs	Reduction
All articles	504	$1011.60_{\pm 665.89}$	$479.35_{\pm 398.29}$	52.6%
$>3$ paragraphs	378	$1089.44_{\pm 683.51}$	$379.82_{\pm 267.93}$	65.1%

Table 3. Input-token reduction achieved by first-three-paragraph selective context conditioning on the test set.

Shot	Gemini-2.0-Flash			Llama-3.3-70B
	BERTScore $\uparrow$	ROUGE-L $\uparrow$	METEOR $\uparrow$	BERTScore $\uparrow$	ROUGE-L $\uparrow$	METEOR $\uparrow$
Zero-shot	$0.720_{\color[rgb]{0.4,0.4,0.4}\pm 0.056}$ [0.715, 0.725]	$0.204_{\color[rgb]{0.4,0.4,0.4}\pm 0.153}$ [0.191, 0.217]	$0.213_{\color[rgb]{0.4,0.4,0.4}\pm 0.199}$ [0.197, 0.231]	$0.755_{\color[rgb]{0.4,0.4,0.4}\pm 0.064}$ [0.749, 0.761]	$0.243_{\color[rgb]{0.4,0.4,0.4}\pm 0.191}$ [0.227, 0.261]	$0.170_{\color[rgb]{0.4,0.4,0.4}\pm 0.197}$ [0.153, 0.188]
One-shot	$0.760_{\color[rgb]{0.4,0.4,0.4}\pm 0.068}$ [0.754, 0.766]	$0.281_{\color[rgb]{0.4,0.4,0.4}\pm 0.212}$ [0.263, 0.300]	$0.245_{\color[rgb]{0.4,0.4,0.4}\pm 0.243}$ [0.224, 0.267]	$\textbf{0.761}_{\color[rgb]{0.4,0.4,0.4}\pm 0.068}$ [0.755, 0.767]	$\textbf{0.257}_{\color[rgb]{0.4,0.4,0.4}\pm 0.209}$ [0.240, 0.276]	$\textbf{0.175}_{\color[rgb]{0.4,0.4,0.4}\pm 0.208}$ [0.158, 0.194]
Two-shot	$0.759_{\color[rgb]{0.4,0.4,0.4}\pm 0.069}$ [0.753, 0.765]	$0.283_{\color[rgb]{0.4,0.4,0.4}\pm 0.209}$ [0.266, 0.301]	$0.241_{\color[rgb]{0.4,0.4,0.4}\pm 0.235}$ [0.221, 0.262]	$0.756_{\color[rgb]{0.4,0.4,0.4}\pm 0.065}$ [0.750, 0.762]	$0.251_{\color[rgb]{0.4,0.4,0.4}\pm 0.203}$ [0.234, 0.269]	$0.163_{\color[rgb]{0.4,0.4,0.4}\pm 0.199}$ [0.147, 0.182]
Three-shot	$\textbf{0.760}_{\color[rgb]{0.4,0.4,0.4}\pm 0.069}$ [0.754, 0.766]	$\textbf{0.283}_{\color[rgb]{0.4,0.4,0.4}\pm 0.210}$ [0.266, 0.302]	$\textbf{0.251}_{\color[rgb]{0.4,0.4,0.4}\pm 0.246}$ [0.230, 0.273]	$0.754_{\color[rgb]{0.4,0.4,0.4}\pm 0.062}$ [0.749, 0.760]	$0.250_{\color[rgb]{0.4,0.4,0.4}\pm 0.195}$ [0.234, 0.268]	$0.164_{\color[rgb]{0.4,0.4,0.4}\pm 0.187}$ [0.149, 0.181]

Table 4. Impact of in-context learning (ICL) across zero-shot to three-shot settings for Gemini and Llama using the BNaP+Baseline prompting configuration with partial article input (

paragraph_{1\rightarrow 3}

Paradigm	Variant	Models	BERTScore $\uparrow$	ROUGE-L $\uparrow$	METEOR $\uparrow$
BNaP	Baseline	Llama-3.3	$0.754_{\color[rgb]{0.4,0.4,0.4}\pm 0.062}$ [0.749, 0.760]	$0.250_{\color[rgb]{0.4,0.4,0.4}\pm 0.195}$ [0.234, 0.268]	$0.164_{\color[rgb]{0.4,0.4,0.4}\pm 0.187}$ [0.149, 0.181]
		GPT-4o	$0.764_{\color[rgb]{0.4,0.4,0.4}\pm 0.067}$ [0.758, 0.770]	$0.275_{\color[rgb]{0.4,0.4,0.4}\pm 0.208}$ [0.258, 0.293]	$0.231_{\color[rgb]{0.4,0.4,0.4}\pm 0.243}$ [0.210, 0.253]
		Gemini-2.0	$0.760_{\color[rgb]{0.4,0.4,0.4}\pm 0.069}$ [0.754, 0.766]	$0.283_{\color[rgb]{0.4,0.4,0.4}\pm 0.210}$ [0.266, 0.302]	$0.251_{\color[rgb]{0.4,0.4,0.4}\pm 0.246}$ [0.230, 0.273]
	MultiGen	Llama-3.3	$0.746_{\color[rgb]{0.4,0.4,0.4}\pm 0.068}$ [0.740, 0.751]	$0.222_{\color[rgb]{0.4,0.4,0.4}\pm 0.198}$ [0.205, 0.239]	$0.146_{\color[rgb]{0.4,0.4,0.4}\pm 0.191}$ [0.130, 0.163]
		GPT-4o	$0.759_{\color[rgb]{0.4,0.4,0.4}\pm 0.062}$ [0.753, 0.764]	$0.253_{\color[rgb]{0.4,0.4,0.4}\pm 0.189}$ [0.237, 0.270]	$0.194_{\color[rgb]{0.4,0.4,0.4}\pm 0.212}$ [0.176, 0.213]
		Gemini-2.0	$0.758_{\color[rgb]{0.4,0.4,0.4}\pm 0.066}$ [0.753, 0.764]	$0.273_{\color[rgb]{0.4,0.4,0.4}\pm 0.206}$ [0.256, 0.292]	$0.230_{\color[rgb]{0.4,0.4,0.4}\pm 0.236}$ [0.211, 0.252]

Table 5. Performance comparison of prompting paradigms and contextual enrichment variants across different LLMs in a three-shot setting with partial article input (

paragraph_{1\rightarrow 3}

Study	Model	BERTScore $\uparrow$	ROUGE-L $\uparrow$	METEOR $\uparrow$
$\text{\cite[citet]{\@@bibref{Authors Phrase1YearPhrase2}{OSAMA2025100138}{\@@citephrase{(}}{\@@citephrase{)}}}}^{\dagger}$	mT5	0.717	0.169	0.109
	mT0	0.726	0.215	0.144
	BanglaT5	0.751	0.242	0.167
	mBART	0.746	0.226	0.146
This study	Gemini-2.0*	0.769 ( +0.018 )	0.292 ( +0.05 )	0.244 ( +0.077 )
† Scores are reproduced from the cited study

Table 6. Comparison between previously reported fine-tuned transformer models and the LLM-based prompting approach used in this study.

Model	BERTScore $\uparrow$	ROUGE-L $\uparrow$	METEOR $\uparrow$	BLEU $\uparrow$
$\text{BERT2BERT \cite[citep]{(\@@bibref{AuthorsPhrase1Year}{akash-etal-2023-shironaam}{\@@citephrase{, }}{})}}^{\dagger}$	0.831	0.503	0.435	0.318
Gemini-2.0* (this study)	$0.810_{\color[rgb]{0.4,0.4,0.4}\pm 0.081}$	$0.448_{\color[rgb]{0.4,0.4,0.4}\pm 0.251}$	$0.397_{\color[rgb]{0.4,0.4,0.4}\pm 0.293}$	$\textbf{0.373}_{\color[rgb]{0.4,0.4,0.4}\pm 0.248}$
	[0.808, 0.811]	[0.444, 0.452]	[0.393, 0.402]	[0.370, 0.378]
† Scores are reproduced from the cited study
* XLP+MultiGen $\cdot$ 3-shot

Table 7. Comparison between the best-performing fine-tuned model reported in prior work and the Gemini-based prompting approach used in this study on the Shironaam benchmark ( Akash et al., 2023 ) .

LLM Prompting

Large language model prompting studies how instructions, examples, context, and formats steer model behavior at inference time.

See all 429 papers

Bangla Key2Text: Text Generation from Keywords for a Low Resource Language
Apr 21, 2026Tonmoy Talukder, G M ShahariarLow-Resource Language Processing Text Generation
1Ahsanullah University of Science and Technology · University of California - Riverside
Register Shifts Break LLM Safety: A Bengali Benchmark with Culturally Grounded Harms
Aug 23, 2026Naymul Islam, Nusrat Jahan Lia, Shubhashis Roy Dipta +2LLM Safety Benchmarks Language Model Safety Evaluation
BanglaLLM · Institute of Information Technology, University of Dhaka · University of Maryland, Baltimore County +2
LLM-guided headline rewriting for clickability enhancement without clickbait
Mar 23, 2026Yehudit Aperstein, Linoy Halifa, Sagiv Bar +1Text Generation Controllable Text Generation
Intelligent Systems, Afeka Academic College of Engineering, Tel Aviv 69988, Israel · School of Computer Science, Faculty of Sciences, HIT-Holon Institute of Technology, Holon 58102, Israel

Apr 21, 2026J/K move · Enter open · S save

Bangla Key2Text: Text Generation from Keywords for a Low Resource Language

Tonmoy Talukder, G M Shahariar

1Ahsanullah University of Science and Technology · University of California - Riverside

Low-Resource Language Processing Text Generation Bengali NLP

Open paper AskPDF arXiv

This paper introduces Bangla Key2Text, a large-scale dataset of

2.6

million Bangla keyword--text pairs designed for keyword-driven text generation in a low-resource language. The dataset is constructed using a BERT-based keyword extraction pipeline applied to millions of Bangla news texts, transforming raw articles into structured keyword--text pairs suitable for supervised learning. To establish baseline performance on this new benchmark, we fine-tune two sequence-to-sequence models, mT5 and BanglaT5, and evaluate them using multiple automatic metrics and human judgments. Experimental results show that task-specific fine-tuning substantially improves keyword-conditioned text generation in Bangla compared to zero-shot large language models. The dataset, trained models, and code are publicly released to support future research in Bangla natural language generation and keyword-to-text generation tasks.

Abstract

LLM Prompting

Explore similar work

Bangla Key2Text: Text Generation from Keywords for a Low Resource Language

Register Shifts Break LLM Safety: A Bengali Benchmark with Culturally Grounded Harms

LLM-guided headline rewriting for clickability enhancement without clickbait