This paper presents new resources and baselines for Dependency Parsing in Pomak, an endangered Eastern South Slavic language with substantial dialectal variation and no widely adopted standard. We focus on the variety spoken in Turkey (Uzunköprü) and ask how well a dependency parser trained on the existing Pomak Universal Dependencies treebank, which was built primarily from the variety that is spoken in Greece, transfers across dialects. We run two experimental phases. First, we train a parser on the Greek-variety UD data and evaluate zero-shot transfer to Turkish-variety Pomak, quantifying the impact of phonological and morphosyntactic differences. Second, we introduce a new manually annotated Turkish-variety Pomak corpus of 650 sentences and show that, despite its small size, targeted fine-tuning substantially improves accuracy; performance is further boosted by cross-variety transfer learning that combines the two dialects.
Figures & tables
Figure 1: Workflow summary. Dataset A provides baseline supervision; Dataset B provides dialect-matched data for adaptation and evaluation.
Setup
Metric
Score
Train: Dataset A; Test: Dataset C
UAS
59%
Train: Dataset A; Test: Dataset C
LAS
51%
Table 1: Baseline cross-variety parsing performance. Dataset A is the existing Pomak UD treebank (primarily Greece variety), and Dataset C is the held-out Turkey-variety test set from a speaker not represented in Dataset B.
Training / Evaluation Setup
UAS
LAS
Dataset A → Dataset C
59%
51%
Dataset B-train → Dataset C
50%
44%
Table 2: Cross-variety baseline versus in-variety training. Dataset A is the existing Pomak UD treebank (primarily Greece variety). Dataset B-train and B-dev are splits of our Turkey-variety corpus, and Dataset C is the held-out Turkey-variety test set from a different speaker.
Model / Training Regime
UAS
LAS
Original Pomak UD (Greece variety)
59%
51%
Mini corpus
50%
44%
Transfer / adaptation (A → B)
68%
62%
Table 3: Parsing performance under three regimes: out-of-variety baseline, in-variety mini-corpus-only training, and transfer/adaptation leveraging both resources.
Pre-trained language models (PLMs) with encoder-based architectures have shown impressive capabilities in zero-shot cross-lingual transfer for various language understanding tasks. However, applying this technique to dependency parsing remains a significant challenge due to its syntactic nature. To boost model generalizability across linguistic typologies, we propose a cross-lingual unsupervised bootstrapping method to improve syntactic knowledge within the PLM. We show that our method achieves a significant improvement in zero-shot parsing performance in low-resource languages. Analysis of these bootstrapped models uncovers increased robustness in recognizing syntactic structures, evidenced by higher scores in parameter-free tree probing tests.
Lalita Lowphansirikul, Attapol Rutherford, Jian Gang Ngui +2
VISTEC, Thailand · Chulalongkorn University, Thailand · AI Singapore, Singapore
Dependency parsing resource development for Arabic has largely focused on Modern Standard Arabic, leaving dialectal Arabic comparatively under-resourced. We introduce MADARTB, the first large-scale parallel dependency treebank spanning five city dialects: Beirut, Cairo, Doha, Rabat and Tunis, and extending an existing parallel MSA version. In total MADARTB consists of 5,000 sentences annotated using the CATiB dependency scheme. We analyze syntactic similarity across all city dialects and MSA. Although global construction profiles exhibit high similarity across varieties, local syntactic structures frequently diverge, revealing substantial structural variation across dialects. We further establish a cross-dialect dependency parsing comparison using a supervised parser and four LLMs. The supervised parser achieves 87.5% LAS, while the best LLM setup trails at 85.7% LAS across all six varieties on the test set. We release our annotations and models publicly.
Noor Abo Mokh, Mohamed Adel, Jamila El-Gizuli +3
Mohamed bin Zayed University of Artificial Intelligence · Computational Approaches to Modeling Language Lab, New York University Abu Dhabi
Transformer-based models achieve state-of-the-art dependency parsing for high-resource languages, yet their advantage over simpler architectures in low-resource settings remains poorly understood. We evaluate four parsers---the Biaffine LSTM, Stack-Pointer Network, AfroXLMR-large, and RemBERT---across twelve typologically diverse languages, with a focus on low-resource African languages. We find that the Biaffine LSTM consistently outperforms transformer models in low-resource regimes, with transformers recovering their advantage as training data increases. The crossover falls within a resource range typical of treebanks for under-resourced languages. Morphological complexity (measured via MATTR) emerges as a significant secondary predictor of transformers' relative disadvantage after controlling for corpus size. These results indicate that the Biaffine LSTM may be better suited for syntactic tool development in low-resource regimes until sufficient annotated data is available to leverage the representational capacity of pre-trained transformers.