cs.CLNov 7, 2026

Enhancing Mathematical Reasoning in Turkish Large Language Models with Search-Based Inference and Model Merging

Authors: Ahmed Zeer, Osama Shbib, Muzaffer Kaan Yuce, Himmet Toprak Kesgin, Mehmet Fatih Amasyali

Organizations: Yildiz Technical University, Istanbul, Türkiye.

Abstract

We improve Turkish mathematical reasoning in Large Language Models by combining LoRA fine-tuning, weight-space model merging, and search-based inference. Using Turkish-Llama 8B-DPO with Turkish translations of MetaMath, NuminaMath, and TigerLab, we find data quality beats scale (Mix-30k best among fine-tuned; large MetaMath underperforms). Linear merging delivers the largest gains (GSM8K up to 0.63), while Ties is modest and Breadcrumbs/Slerp degrade. Search with MCTS and Tree of Thoughts improves mean LLM-judge scores for reasoning quality on a 0-100 scale. Limitations include machine translation, limited scale, and search stochasticity.

Figures & tables

Explore similar work

CardsList
  1. TUDUM: A Turkish-Thinking Reasoning Pipeline for Qwen3.5-27B

    Jul 2, 2026Baran Bingol, Bahaeddin TurkogluLLM Fine-TuningLLM Mathematical Reasoning

  2. Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning

    Sep 29, 2026Hongyang Li, Yiming Zhu, Xiao Li +3Language Model DistillationRL Post-Training