Analysing Lightweight Large Language Models for Biomedical Named Entity Recognition on Diverse Ouput Formats

Pierre Epron; Adrien Coulet; Mehwish Alam

cs.CLMar 27, 2026

Analysing Lightweight Large Language Models for Biomedical Named Entity Recognition on Diverse Ouput Formats

Authors: Pierre Epron, Adrien Coulet, Mehwish Alam

Organizations: HeKA | U1346, DIG · Inria, Inserm, Université Paris Cité, HeKA, UMR 1346, Paris, France · Télécom Paris, Institut Polytechnique de Paris, France · HeKA | U1346 ·

Abstract

Despite their strong linguistic capabilities, Large Language Models (LLMs) are computationally demanding and require substantial resources for fine-tuning, which is unadapted to privacy and budget constraints of many healthcare settings. To address this, we present an experimental analysis focused on Biomedical Named Entity Recognition using lightweight LLMs, we evaluate the impact of different output formats on model performance. The results reveal that lightweight LLMs can achieve competitive performance compared to the larger models, highlighting their potential as lightweight yet effective alternatives for biomedical information extraction. Our analysis shows that instruction tuning over many distinct formats does not improve performance, but identifies several format consistently associated with better performance.

Figures & tables

dataset	train	dev	test	nested	discont	labels
AnatEM	5861	2118	3830	-	-	1
bc2gm	12500	2500	5000	-	-	1
bc4chemd	30682	30639	26364	-	-	1
bc5cdr	4560	4581	4797	-	-	2
CADEC	5317	1140	1140	-	$\sim$ 6.5-8.5%	5
GENIA	15023	1669	1854	$\sim$ 2-2.5%	-	5

Table 1: Distribution of datasets used for this work. Columns nested and discount are a range approximation. Example for GENIA_NER, each split have between 2 and 2.5 percent of nested entities.

x	Qwen2.5-0.5B			Llama-3.2-1B
	P $\uparrow$	R $\uparrow$	F1 $\uparrow$	P $\uparrow$	R $\uparrow$	F1 $\uparrow$
all	0.65 ( $\pm$ 0.01)	0.52 ( $\pm$ 0.01)	0.58 ( $\pm$ 0.01)	0.67 ( $\pm$ 0.01)	0.54 ( $\pm$ 0.01)	0.60 ( $\pm$ 0.01)
7best	0.81 ( $\pm$ 0.01)	0.75 ( $\pm$ 0.01)	0.78 ( $\pm$ 0.01)	0.82 ( $\pm$ 0.00)	0.74 ( $\pm$ 0.01)	0.78 ( $\pm$ 0.00)
multi_triple	0.79 ( $\pm$ 0.02)	0.68 ( $\pm$ 0.01)	0.73 ( $\pm$ 0.01)	0.80 ( $\pm$ 0.01)	0.76 ( $\pm$ 0.01)	0.78 ( $\pm$ 0.01)
conv_term	0.81 ( $\pm$ 0.01)	0.79 ( $\pm$ 0.01)	0.80 ( $\pm$ 0.00)	0.81 ( $\pm$ 0.01)	0.79 ( $\pm$ 0.01)	0.80 ( $\pm$ 0.01)
term	0.81 ( $\pm$ 0.01)	0.71 ( $\pm$ 0.02)	0.76 ( $\pm$ 0.01)	0.81 ( $\pm$ 0.01)	0.73 ( $\pm$ 0.01)	0.77 ( $\pm$ 0.01)

Table 2: Precision (P), recall (R), f-score (F1) for each model and formats configuration. Bold is for best for each model. Underline is for best overall.

	Qwen2.5-0.5B			Llama-3.2-1B
	all $\uparrow$	7best $\uparrow$	only $\uparrow$	all $\uparrow$	7best $\uparrow$	only $\uparrow$
conv_term	0.78 ( $\pm$ 0.00)	0.78 ( $\pm$ 0.01)	0.80 ( $\pm$ 0.00)	0.79 ( $\pm$ 0.01)	0.79 ( $\pm$ 0.01)	0.80 ( $\pm$ 0.01)
multi_brat	0.13 ( $\pm$ 0.01)	-	-	0.17 ( $\pm$ 0.01)	-	-
multi_code	0.56 ( $\pm$ 0.01)	-	-	0.57 ( $\pm$ 0.01)	-	-
multi_span	0.10 ( $\pm$ 0.01)	-	-	0.12 ( $\pm$ 0.01)	-	-
multi_term	0.74 ( $\pm$ 0.01)	0.75 ( $\pm$ 0.01)	0.75 ( $\pm$ 0.01)	0.74 ( $\pm$ 0.01)	0.74 ( $\pm$ 0.01)	0.75 ( $\pm$ 0.01)

Table 3: F-score for each base model and formats with respect to their training configuration. Bold is for best for each model. Underline is for best overall.

	Qwen2.5-0.5B	Llama-3.2-1B
conv_term	0.81 ( $\pm$ 0.01)	0.82 ( $\pm$ 0.01)
multi_bio	0.44 ( $\pm$ 0.07)	0.60 ( $\pm$ 0.03)
multi_tag	0.82 ( $\pm$ 0.01)	0.83 ( $\pm$ 0.01)
single_tag	0.83 ( $\pm$ 0.01)	0.83 ( $\pm$ 0.01)

Table 4: F-score without discontinuous datasets for each base model and formats with respect to their training configuration. Bold is for best for each model. Underline is for best overall.

	Ours		Supervised			ZeroShot
	7best $\uparrow$	conv_term $\uparrow$	Base $\uparrow$	UIE $\uparrow$	UniNER $\uparrow$	UniNER $\uparrow$	GliNER $\uparrow$
AnatEM	0.87	0.87	0.86	0.89	0.89	0.25	0.34
GENIA	0.72	0.77	0.73	0.76	0.78	0.54	0.56
bc2gm	0.79	0.80	0.81	0.81	0.82	0.46	0.48
bc4chemd	0.85	0.87	0.87	0.88	0.89	0.48	0.43
bc5cdr	0.86	0.86	0.85	0.89	0.89	0.68	0.66

Table 5: F-score of our best model vs. the baselines for each dataset. Best results are in bold .

	Qwen2.5-0.5B			Llama-3.2-1B
	P $\uparrow$	R $\uparrow$	F1 $\uparrow$	P $\uparrow$	R $\uparrow$	F1 $\uparrow$
all	0.88 ( $\pm$ 0.03)	0.86 ( $\pm$ 0.03)	0.87 ( $\pm$ 0.03)	0.87 ( $\pm$ 0.02)	0.85 ( $\pm$ 0.02)	0.86 ( $\pm$ 0.01)
7best	0.87 ( $\pm$ 0.02)	0.87 ( $\pm$ 0.02)	0.87 ( $\pm$ 0.02)	0.88 ( $\pm$ 0.01)	0.86 ( $\pm$ 0.03)	0.87 ( $\pm$ 0.02)
only	0.80 ( $\pm$ 0.06)	0.03 ( $\pm$ 0.03)	0.06 ( $\pm$ 0.06)	0.84 ( $\pm$ 0.06)	0.78 ( $\pm$ 0.03)	0.81 ( $\pm$ 0.04)

Table 6: Precision (P), Recall (R), F1-score (F1) for each training configuration of the multi_triple format. Bold are worst results for each configuration.

		Qwen2.5-0.5B		Llama-3.2-1B
		$\Delta$ Nested $\downarrow$	$\Delta$ Discont $\downarrow$	$\Delta$ Nested $\downarrow$	$\Delta$ Discont $\downarrow$
all	conv_term	0.07	0.46	0.09	0.43
	multi_tag	0.39	0.81	0.32	0.82
	multi_term	0.34	0.48	0.36	0.43
	multi_triple	0.10	0.54	0.08	0.51
	single_code	0.07	0.43	0.06	0.39

Table 7: Difference of performances for nested/discontinuous and normal entities for each base model, training configuration, and format. Lowest difference for each model is in bold . Underlined values are lowest difference overall.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Training hyperparameters
#epoch	15
#GPU	4
train batch size	2
gradient accumulation	8
learning rate	1.00E-05
optimizer	adamw

Appendix

Table 8: Most important training hyperparameters

	Qwen2.5-0.5B			Llama-3.2-1B
	HH:MM:SS	gCO2e	kWh	HH:MM:SS	gCO2e	kWh
all	02:23:46	234.47	4.57	03:57:50	388.6	7.58
7best	02:11:45	214.8	4.19	03:47:43	372.2	7.26
conv_term	01:34:26	154.13	3.01	02:36:45	255.79	4.99
multi_triple	02:07:55	208.24	4.06	03:44:02	367.29	7.16
single_term	01:43:38	168.89	3.29	02:51:37	280.38	5.47

Appendix

Table 9: Estimation of the energy consumption of each experiments

Named Entity Recognition

Named Entity Recognition identifies and classifies people, places, organizations, and other entities in text, speech, and multimodal data.

See all 110 papers

Enhancing Biomedical Named Entity Recognition via Multiple Programming Languages Instruction Tuning and Ensemble Method
Oct 2, 2026Songtao Li, Yijia Zhang, Jianyuan Yuan +3Named Entity Recognition LLM Fine-Tuning
School of Information Science and Technology, Dalian Maritime University, Dalian, China · School of Automation, Beijing Institute of Technology, Beijing, China · School of Computer Science and Engineering, Northeastern University, Shenyang, China +1
Assessment of Generative Named Entity Recognition in the Era of Large Language Models
Jan 25, 2026Qi Zhan, Yile Wang, Hui HuangNamed Entity Recognition LLM Information Extraction
College of Computer Science and Software Engineering, Shenzhen University
Small LLMs for Medical NLP: a Systematic Analysis of Few-Shot, Constraint Decoding, Fine-Tuning and Continual Pre-Training in Italian
Feb 19, 2026Pietro Ferrazzi, Mattia Franzin, Alberto Lavelli +1Small Language Models Clinical NLP
Fondazione Bruno Kessler, Povo, Trento, Italy · University of Padova, Padova, Italy

Oct 2, 2026J/K move · Enter open · S save

Enhancing Biomedical Named Entity Recognition via Multiple Programming Languages Instruction Tuning and Ensemble Method

Songtao Li, Yijia Zhang, Jianyuan Yuan, Shidi Zhang+2

School of Information Science and Technology, Dalian Maritime University, Dalian, China · School of Automation, Beijing Institute of Technology, Beijing, China · School of Computer Science and Engineering, Northeastern University, Shenyang, China+1

Named Entity Recognition LLM Fine-Tuning Instruction Tuning Language Model Ensembles

Open paper AskPDF arXiv

Instruction tuning has become a common paradigm for applying large language models (LLMs) to biomedical named entity recognition (BioNER). However, existing instruction-tuning approaches still face two key challenges. First, conventional natural-language instructions typically serialize BioNER annotations as flat textual outputs, providing limited structural constraints for typed entity extraction. Second, high-quality biomedical annotations are limited, and learning from a single serialized output form may restrict structural diversity and reduce model robustness. Although external biomedical knowledge can be introduced to alleviate data scarcity, it often requires costly resource construction. To address these challenges, we propose MITE, a Multiple Programming Languages Instruction Tuning and Ensemble method for BioNER. MITE reformulates BioNER as a structure-to-structure generation task by representing both instructions and entity outputs in code-formatted representations. Specifically, each training instance is transformed into multiple programming-language formats, including Python, C++, and Java, while preserving the same underlying entity semantics. These language-specific representations provide structurally diverse supervision without requiring external biomedical knowledge or additional annotations. During inference, MITE aggregates predictions from different code formats through an entity-level voting strategy, reducing language-specific prediction variance and improving robustness. Experiments on three widely used BioNER datasets demonstrate that MITE achieves competitive performance against representative BERT-based and LLM-based baselines, attaining the best results on two benchmarks. Ablation studies further validate the effectiveness of multi-language representations.

Abstract

Named Entity Recognition

Explore similar work

Enhancing Biomedical Named Entity Recognition via Multiple Programming Languages Instruction Tuning and Ensemble Method

Assessment of Generative Named Entity Recognition in the Era of Large Language Models

Small LLMs for Medical NLP: a Systematic Analysis of Few-Shot, Constraint Decoding, Fine-Tuning and Continual Pre-Training in Italian