Organizations: Department of Information Technology, PICT, Pune, India · L3Cube Labs, Pune, India · Indian Institute of Technology Madras, Chennai, India
Named Entity Recognition (NER) is a core NLP task, but transformer-based sentence-level models struggle with long documents because of fixed input-length limits: truncation drops content, and non-overlapping chunking fragments entities at segment boundaries. We introduce an inference-only pipeline that extends a frozen NER model, MahaNER-BERT, fine-tuned on the MahaNER corpus, to document-level prediction via overlapping sliding windows that are merged into a single annotation, without any retraining or architectural change. We evaluate the pipeline on six document-level corpora built from the MahaNER test set using two strategies: Normal Repeat, which duplicates sentence sequences to extend length while preserving contextual continuity, and Random Repeat, which concatenates distinct sequences to produce longer, heterogeneous inputs, each instantiated at three length levels, across several sliding-window configurations. The model retains a macro F1-score of up to 0.8902, with variation staying below one percentage point regardless of document length or construction strategy. Compared with the conventional non-windowed approach, the sliding-window pipeline avoids the boundary-fragmentation errors introduced by non-overlapping segmentation, yielding consistently higher and more stable document-level F1-scores.
Figures & tables
Dataset
Construction Strategy
Objective
Normal Repeat 4x
Repeat each sentence sequence four times
Length evaluation
Normal Repeat 8x
Repeat each sentence sequence eight times
Length evaluation
Normal Repeat 10x
Repeat each sentence sequence ten times
Length evaluation
Random Repeat 4x
Concatenate four random sequences
Length eval., context diversity
Random Repeat 8x
Concatenate eight random sequences
Length eval., context diversity
Random Repeat 10x
Concatenate ten random sequences
Length eval., context diversity
Table 1: Document-level evaluation datasets generated from the MahaNER test set.
Window Size ( W )
Stride ( S )
Overlap
20
15
25%
40
30
25%
60
45
25%
80
60
25%
Table 2: Sliding window configurations used during document-level inference.
Window Size ( W )
Stride ( S )
Overlap
20
15
25%
40
30
25%
60
45
25%
80
60
25%
Table 3: Sliding window configurations used during document-level inference.
Dataset Variant
Macro F1-score
W=20,S=15
W=40,S=30
W=60,S=45
W=80,S=60
Normal Repeat 4x
0.8901
0.8878
0.8872
0.8872
Normal Repeat 8x
0.8902
0.8867
0.8856
0.8853
Normal Repeat 10x
0.8896
0.8867
0.8864
0.8850
Random Repeat 4x
0.8875
0.8848
0.8849
0.8849
Random Repeat 8x
0.8857
0.8824
0.8813
0.8789
Table 4: Macro F1-scores across all dataset variants and sliding window configurations.
Continual Learning for Named Entity Recognition (CLNER) enable models to incrementally learn new entity types without forgetting previously acquired ones. However, existing methods suffer from catastrophic forgetting and insufficient exploitation of shared information across tasks. This paper proposes FSE, a Fast-Slow Experts enhanced span-based NER model for CLNER. The shared fast expert learns token-level links to efficiently filter out unlikely spans, while the task-specific slow expert performs span classification only on the remaining candidates. It stabilizes learning by promoting knowledge sharing across tasks and maintains plasticity by reducing learning burden at each task. A length-decay negative sampling strategy to mitigate span imbalance is also introduced. Extensive experiments on OntoNotes and FewNERD synthestic datasets demonstrate that FSE achieves state-of-the-art performance in CLNER scenarios, with effectiveness of each component, empirical evidence of faster convergence and expected functionality of both experts.
Yunan Zhang, Yang Fan, Heng Li +2
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, Guangdong, China
Nested named entity recognition identifies entities contained within other entities, but requires expensive multi-level annotation. While flat NER corpora exist abundantly, nested resources remain scarce. We investigate whether models can learn nested structure from flat annotations alone, evaluating four approaches: string inclusions (substring matching), entity corruption (pseudo-nested data), flat neutralization (reducing false negative signal), and a hybrid fine-tuned + LLM pipeline. On NEREL, a Russian benchmark with 29 entity types where 21% of entities are nested, our best combined method achieves 26.37% inner F1, closing 40% of the gap to full nested supervision. Code is available at https://github.com/fulstock/Learning-from-Flat-Annotations.
Igor Rozhkov, Natalia Loukachevitch
Lomonosov Moscow State University Leninskie Gory, 1/4, Moscow, Russia
Named Entity Recognition (NER) is a key component in industrial information extraction pipelines, where systems must satisfy strict latency and throughput constraints in addition to strong accuracy. State-of-the-art NER accuracy is often achieved by span-based frameworks, which construct span representations from token encodings and classify candidate spans. However, many span-based methods enumerate large numbers of candidates and process each candidate with marker-augmented inputs, substantially increasing inference cost and limiting scalability in large-scale deployments. In this work, we propose SpanDec, an efficient span-based NER framework that targets this bottleneck. Our main insight is that span representation interactions can be computed effectively at the final transformer stage, avoiding redundant computation in earlier layers via a lightweight decoder dedicated to span representations. We further introduce a span filtering mechanism during enumeration to prune unlikely candidates before expensive processing. Across multiple benchmarks, SpanDec matches competitive span-based baselines while improving throughput and reducing computational cost, yielding a better accuracy-efficiency trade-off suitable for high-volume serving and on-device applications.
Andrea Maracani, Savas Ozkan, Junyi Zhu +2
Samsung Research UK, Staines-upon-Thames, United Kingdom