grapheme-kit: Grapheme-Level Metrics and Text Processing for Multilingual NLP

Izzath Nisfer; Ashini Kavindya; Ovindu Atukorala; Purushoth Velayuthan; Menan Velayuthan

cs.CLJul 24, 2026

grapheme-kit: Grapheme-Level Metrics and Text Processing for Multilingual NLP

Authors: Izzath Nisfer, Ashini Kavindya, Ovindu Atukorala, Purushoth Velayuthan, Menan Velayuthan

Organizations: Department of Computer Science & Engineering, University of Moratuwa, Sri Lanka · Computer Science and Engineering, University of Nevada, Nevada, USA · Department of Information & Computing Sciences, Utrecht University, Utrecht, The Netherlands

Abstract

Existing lexical distance, similarity, and evaluation metrics operate on Unicode code points, which can misrepresent errors in writing systems where a single grapheme is represented by multiple Unicode code points. We introduce grapheme-kit, an open-source Python library that extends these metrics to operate on grapheme clusters instead. The library also provides improved grapheme processing for Tamil and Sinhala, including accurate grapheme cluster identification and grapheme composition/decomposition utilities. Through an OCR case study, we demonstrate that grapheme-level metrics provide a more faithful evaluation of complex scripts.

Explore similar work

Rethinking Handwritten Character Recognition
Sep 11, 2026Ranjit Raut, Aarav Subedi, Ashim ShresthaHandwritten Text Recognition Regularity
Revisiting Lexicon Evaluation in Unsupervised Word Discovery
Jun 4, 2026Simon Malan, Danel Slabbert, Herman KamperLexicon Mispronunciation Detection
SPARCLE: SPeaker-aware Aligned Representations via Contrastive Language Embeddings
May 1, 2026Priyam Mazumdar, Yurii Halychanskyi, Steven Guo +2Phoneme-Level Analysis Discrete Speech Representations

Sep 11, 2026 · cs.CVJ/K move · Enter open · S save

Rethinking Handwritten Character Recognition

Ranjit Raut, Aarav Subedi, Ashim Shrestha

Non-Latin handwritten character recognition (HCR) remains understudied. Dominant methods consider it as generic image classification, which uses model scale to implicitly learn stroke structure. Structural-prior efficiency---the principle that explicitly encoding script-geometric regularities as architectural inductive biases can be both more accurate and require fewer parameters. We introduce GraphemeNet, a unified multi-script architecture, governed by two orthogonal binary axes. Axis 1 operationalises stroke-level geometric regularity via Persistent Scaffold Injection (PSI): a script-specific asymmetric convolution injects a stroke scaffold as a weighted residual at every encoder stage, continuously anchoring learned features to script geometry---distinct from skip connections, auxiliary losses, or attention reweighting. Axis 2 selects between global average pooling with gated fusion and cross-scale attention with a Stroke Topology Module (STM), depending on whether glyph discrimination requires spatial relational reasoning. A Linear Capsule Routing (LCR) with

O(n)

routing is shared universally. On fourteen benchmarks across eight writing systems, the architecture generalises with only scaffold and decoder topology varying per script, consistently challenging, outperforming published baselines, and establishing structural-prior efficiency as a broadly applicable principle for multi-script HCR.

Handwritten Text Recognition Regularity

Open paper Ask about it PDF

Abstract

Explore similar work

Rethinking Handwritten Character Recognition

Revisiting Lexicon Evaluation in Unsupervised Word Discovery

SPARCLE: SPeaker-aware Aligned Representations via Contrastive Language Embeddings