HalalBench: A Multilingual OCR Benchmark for Food Packaging Ingredient Extraction

Hasan Arief

cs.CVFeb 19, 2026

HalalBench: A Multilingual OCR Benchmark for Food Packaging Ingredient Extraction

Authors: Hasan Arief

Organizations: HalalLens Research

Abstract

No standardized benchmark exists for evaluating OCR on food packaging, despite its critical role in automated halal food verification. Existing benchmarks target documents or scene text, missing the unique challenges of ingredient labels: curved surfaces, dense multilingual text, and sub-8pt fonts. We present HalalBench, the first open multilingual benchmark for food packaging OCR, comprising 1,043 images (50 real, 993 synthetic) with 36,438 annotations in COCO format spanning 14 languages. We evaluate four engines: docTR achieves F1=0.193, ML Kit 0.180, EasyOCR 0.167, while all fail on Japanese (F1=0.000). A clustering ablation shows 36% F1 improvement from our post-processing algorithm. We validate findings through HalalLens (https://halallens.no), a production halal scanner serving 20+ countries. Dataset and code are released under open licenses.

Figures & tables

Language	Images	Annot.
English (en)	15	262
Norwegian (no)	8	140
French (fr)	6	108
Turkish (tr)	5	78
German (de)	3	47
Swedish (sv)	3	45

Table 1: Language distribution in the real-image subset.

Panel of Figure 1 — Figure 1: Four layout template families used in synthetic data generation.

	Real	Synth.	Total
Images	50	993	1,043
Annotations	829	35,609	36,438
Languages	12	14	14
Mean annot./image	16.6	35.9	35.0

Table 2: HalalBench dataset summary.

Engine	Architecture	Mode
ML Kit v2 ( Google, 2023 )	CNN + CTC	On-device
docTR ( Mindee, 2021 )	DBNet + CRNN	Server
EasyOCR ( JaidedAI, 2020 )	CRAFT + CRNN	Server
RapidOCR	PP-OCRv4/ONNX	Server

Table 3: Evaluated OCR engines. All benchmarked on a 6-core Intel Mac, 16 GB RAM, CPU only.

Figure 2: HalalBench evaluation pipeline. All engines are normalized to a common word-level format before clustering.

Engine	F1	Fuz.	P	R	Cat.%
ML Kit	0.180	0.229	0.152	0.259	33.3
docTR	0.193	0.234	0.167	0.259	36.1
EasyOCR	0.167	0.208	0.147	0.223	37.1
RapidOCR	0.044	0.080	0.038	0.061	75.0

Table 4: Multi-engine comparison on HalalBench (36 samples). Best values in bold .

Panel of Figure 3 — Figure 3: Engine comparison: exact and fuzzy F1 scores on HalalBench.

Lang	n	ML Kit	docTR	Easy	Rapid
de	2	0.531	0.655	0.621	0.000
no	4	0.360	0.393	0.412	0.164
en	13	0.218	0.216	0.172	0.059
da	1	0.263	0.154	0.146	0.065
fr	6	0.084	0.119	0.054	0.006
sv	2	0.082	0.136	0.056	0.000

Table 5: Per-language F1 scores. Bold = best engine.

Panel of Figure 4 — Figure 4: Best-engine F1 by language. Japanese represents complete failure across all engines.

	Strategy	F1	P	R
(a)	Raw OCR	0.129	0.088	0.330
(b)	Line-based	0.007	0.006	0.012
(c)	DBSCAN flat	0.134	0.096	0.299
(d)	DBSCAN+vote	0.176	0.151	0.242

Table 6: Clustering ablation on ML Kit output (36 samples).

Engine	ms	MB	F1	F1/s
RapidOCR	2,748	1,377	0.044	0.016
docTR	5,850	960	0.193	0.033
EasyOCR	10,135	2,006	0.167	0.016

Table 7: Resource usage for server-side engines (CPU-only).

Panel of Figure 5 — Figure 5: Speed-accuracy tradeoff for server-side engines.

Panel of Figure 6 — Figure 6: HalalLens production pipeline architecture.

Explore similar work

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
Apr 14, 2026Amir Hossein Kargaran, Nafiseh Nikeghbal, Jana Diesner +2Optical Character Recognition Multilingual VLM Evaluation
LMU Munich · MCML · TU Munich +1
LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR
Jan 20, 2026Said Taghadouini, Adrien Cavaillès, Baptiste AubertinVision-Language Models Optical Character Recognition
LightOn
OliveGemma: A 3 Billion Visual Language Model for Recognising the Mediterranean & European Diet
Aug 4, 2026Dimitrios I. Zaridis, Traianos Tsiokris, Vasileios C. Pezoulas +6Large Vision-Language Models Fine-Grained Image Classification
Unit of Medical Technology & Intelligent Information Systems University of Ioannina Ioannina, Greece · Department of Medical Physics, School of Medicine University of Patras Patras, Greece · Computational BioMedicine Laboratory Foundation for Research and Technology Hellas Heraklion, Greece +2

Apr 14, 2026J/K move · Enter open · S save

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts

Amir Hossein Kargaran, Nafiseh Nikeghbal, Jana Diesner, François Yvon+1

LMU Munich · MCML · TU Munich+1

Optical Character Recognition Multilingual VLM Evaluation

Open paper AskPDF arXiv

Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cluster of high- and mid-resource scripts. We introduce GlotOCR Bench, a comprehensive benchmark evaluating OCR generalization across 100+ Unicode scripts. Our benchmark comprises clean and degraded image variants rendered from real multilingual texts. Images are rendered using fonts from the Google Fonts repository, shaped with HarfBuzz and rasterized with FreeType, supporting both LTR and RTL scripts. Samples of rendered images were manually reviewed to verify correct rendering across all scripts. We evaluate a broad suite of open-weight and proprietary vision-language models and find that most perform well on fewer than ten scripts, and even the strongest frontier models fail to generalize beyond thirty scripts. Performance broadly tracks script-level pretraining coverage, suggesting that current OCR systems rely on language model pretraining as much as on visual recognition. Models confronted with unfamiliar scripts either produce random noise or hallucinate characters from similar scripts they already know. We release the benchmark and pipeline for reproducibility. Pipeline Code: https://github.com/cisnlp/glotocr-bench, Benchmark: https://hf.co/datasets/cis-lmu/glotocr-bench.

Abstract

Explore similar work

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts

LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR

OliveGemma: A 3 Billion Visual Language Model for Recognising the Mediterranean & European Diet