Text summarization is an important task for advancing natural language processing (NLP) research and applications in Indic languages. Despite progress in extractive and abstractive summarization, both monolingual summarization (MS) and cross-lingual summarization (CLS) remain relatively underexplored for these languages. Recent advances in Transformer architectures, pretrained language models (PLMs), and dataset availability offer new opportunities to address this gap. In this work, we construct cross-lingual training data from the monolingual Marathi MahaSum corpus and combine it with the smaller cross-lingual PMIndiaSum dataset to fine-tune IndicTrans2, an Indic neural machine translation model. We evaluate the resulting models using ROUGE, chrF and chrF++ metrics, focusing primarily on Marathi-to-English summarization due to computational constraints. We also discuss the applicability of our approach to the English-to-Marathi direction. Our experiments demonstrate that leveraging a pretrained translation model yields substantially better summarization performance than the IndicBART and mBART-50 based baselines reported in previous work on cross-lingual summarization.
Figures & tables
Metric
Mean
Median
Std. Dev.
chrF
56.34
56.88
11.70
chrF++
49.97
50.22
11.41
Character-length ratio
0.967
0.966
0.122
Word-length ratio
0.994
1.000
0.348
Table 1: Round-trip summary translation fidelity on the MahaSum validation split ( N=1,667 ). Scores compare original Marathi summaries with their Marathi back-translations.
Metric
Mean
Median
Std. Dev.
chrF
63.01
63.61
8.17
chrF++
58.69
59.15
7.71
Character-length ratio
0.981
0.909
0.713
Word-length ratio
0.971
0.916
0.718
Table 2: Round-trip text translation fidelity on the MahaSum validation split ( N=1,667 ). Scores compare original Marathi documents with their Marathi back-translations.
Dataset
Split
N
Text length
Summary length
Compression
Mean
Median
P95
Mean
Median
P95
Mean
Median
P95
MahaSum
Train
22000
316.99
300
511.00
23.32
22
42.0
16.67
13.75
36.94
MahaSum
Val
1667
320.70
302
528.10
23.10
22
42.0
17.08
14.20
37.64
MahaSum
Test
1667
91.03
71
187.00
18.48
18
32.0
6.21
4.47
17.25
XL-Sum
Train
10903
653.28
592
1424.90
32.57
30
60.0
23.34
18.76
58.15
XL-Sum
Val
1362
519.98
521
877.00
29.87
29
51.0
19.58
16.94
40.29
Table 3: Marathi-English dataset statistics for the training, validation, and test splits. Lengths are measured in words, and compression ratio is defined as source-text length divided by summary length.
Figure 1: Distributional comparison of PMIndiaSum, MahaSum, XL-Sum and CrossSum. Panels (a)–(c) show empirical cumulative distribution functions (ECDFs) of source-text length, summary length, and compression ratio, respectively; panel (d) shows source-text versus summary length. Colours denote PMIndiaSum (blue), MahaSum (orange), XL-Sum (red) and CrossSum (green). Solid and dashed lines denote training and validation splits, respectively.
Direction
Dataset
Original document
Original summary
Translated field
Data type
Role
MR → EN
MahaSum
MR
MR
SUMM
Synthetic
Training
MR → EN
XL-Sum-MR
MR
MR
Summary
Synthetic
Training
MR → EN
PMIndiaSum
EN
MR
None
Native
Evaluation
MR → EN
CrossSum
EN
MR
None
Native
Evaluation
EN → MR
MahaSum
MR
MR
TEXT
Synthetic
Training
EN → MR
XL-Sum-EN
EN
EN
Summary
Synthetic
Training
Table 4: Construction and use of datasets for the two cross-lingual summarization directions. “Translated field” identifies the field generated using IndicTrans2.
Hyperparameter
Value / Range
Description
Learning rate
1e-5 – 5e-5
Range explored; lower values can reduce catastrophic forgetting on smaller datasets
Batch size
2 – 8
Batch-size range considered; gradient accumulation may be used to increase the effective batch size
Optimizer
Adam ( β1=0.9 , β2=0.98 , ϵ=10−8 )
Standard optimizer for transformer models
Learning rate scheduler
Inverse square root / linear decay
Schedulers considered during fine-tuning
Dropout
0.1 – 0.3
Regularisation range explored during fine-tuning
Maximum sequence length
256 – 2048
Maximum number of tokens in each source–target pair
Table 5: Hyperparameters considered for fine-tuning the IndicTrans2 model
Training data
ROUGE-1
ROUGE-2
ROUGE-L
BERTScore-F1
PMIndiaSum
72.0151
53.2036
69.7104
99.0504
MahaSum
40.4998
22.3857
36.7866
98.5998
PMIndiaSum + MahaSum
72.4968
53.3091
69.8116
99.0485
PMIndiaSum + XL-Sum Marathi
72.2468
52.7246
69.4549
99.0442
XL-Sum Marathi
37.5122
20.8242
34.0955
98.5330
Table 9: MR → EN cross-lingual summarization results for the IndicTrans2 1B model under different training-data configurations.
Figure 2: Validation-metric trajectories during Marathi-to-English fine-tuning of indictrans2-200M model. Each line corresponds to training on PMIndiaSum, MahaSum, XL-Sum and their blended training set, as identified in the in-panel legends. The panels are enlarged to make the training dynamics and dataset labels legible; final held-out test-set scores should be reported separately in a results table.
Figure 3: Validation-metric trajectories during Marathi-to-English fine-tuning of rotary-indictrans2-200M model. Each line corresponds to training on PMIndiaSum, MahaSum, XL-Sum and their blended training set, as identified in the in-panel legends. The panels are enlarged to make the training dynamics and dataset labels legible; final held-out test-set scores should be reported separately in a results table.
Figure 4: Validation-metric trajectories during Marathi-to-English fine-tuning of 1B parameter models.