How Small Can You Go? LoRA Fine-Tuning 270M-8B Models for Merchant Information Extraction in Financial Transactions

Donghao Huang; Tomas Drietomsky; Benjamin Barrett; Zhaoxia Wang

cs.AIJun 6, 2026

How Small Can You Go? LoRA Fine-Tuning 270M-8B Models for Merchant Information Extraction in Financial Transactions

Authors: Donghao Huang, Tomas Drietomsky, Benjamin Barrett, Zhaoxia Wang

Organizations: Research and Development, Mastercard, Arlington, VA, USA · School of Computing and Information Systems, Singapore Management University, Singapore · Economic Intelligence, Mastercard, Prague, Czech Republic

Abstract

Merchant information extraction turns noisy financial transaction descriptors into structured fields at production scale. Our deployed LoRA-fine-tuned LLaMA~~3.1-8B reaches 96.95% F1, but its memory and throughput motivate smaller replacements. We evaluate 23 retained fine-tuning runs plus a separately trained production reference, spanning Gemma~~3 (270M--4B), Qwen~~3.5 (0.8B--4B), Aya~~3.35B, and LLaMA~~3.1-8B across LoRA ranks, prompts, training templates, and serving environments. A rank-8 LLaMA fine-tune reaches 96.75% F1, only 0.20 points below the rank-32 production reference. Qwen~~3.5~~4B with JSON-Only prompting reaches 96.60% F1 and strict record-level exact match of 91.67%, with a $3.8\times$ lower inverse-throughput time estimate than the rank-8 8B model. Qwen~~3.5~0.8B reaches 94.75% F1, and Qwen Think and Nothink templates differ by less than 0.004 F1. Across 14 Databricks endpoints, mean F1 change from local evaluation is $-0.0081$ ; Aya is the only family with a 2.7--5.1 point decline. These results show that compact fine-tuned models can preserve most extraction accuracy, but model selection must account for prompt choice, throughput, and serving-stack behavior.

Explore similar work

Domain-Adapted Small Language Models with Hybrid Post-Processing: Achieving Cost-Efficient, Low-Latency Multi-Label Structured Prediction via LoRA Fine-Tuning on Scarce Data
Jun 4, 2026Srinivasan Manoharan, Dilipkumar Nallusamy, Sachin Kumar +1Low-Rank Adaptation Data Scarcity
Internalizing Tool Knowledge in Small Language Models via QLoRA Fine-Tuning
May 18, 2026Yuval Shemla, Ayal Yakobe, Tanmay Agarwal +2Small Language Models Internalization
How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model
Jul 28, 2026Mahendra Singh Rathor, Anagheem AzzamLarge Language Model Quantization Parameter-Efficient Fine-Tuning

Jun 4, 2026 · cs.LGJ/K move · Enter open · S save

Domain-Adapted Small Language Models with Hybrid Post-Processing: Achieving Cost-Efficient, Low-Latency Multi-Label Structured Prediction via LoRA Fine-Tuning on Scarce Data

Srinivasan Manoharan, Dilipkumar Nallusamy, Sachin Kumar, Haifeng Wu

Deploying frontier large language models (LLMs) for domain-specific structured evaluation tasks incurs prohibitive latency, cost, and data-privacy overhead. We present a hybrid framework that fine-tunes a small language model (LLaMA 3.1 8B, 2.05% trainable parameters via LoRA) on only 219 curated examples and couples it with a deterministic rule-based postprocessing layer. Applied to multi-label compliance evaluation of conversational transcripts (18 heterogeneous output fields), our system achieves 100% JSON structural validity, 83.0% human-validated overall accuracy, and 100% accuracy on the most critical classification field in blind evaluation on 53 unseen production transcripts. On a single NVIDIA A100 GPU, inference completes in

\sim

2 seconds -- 2--5x faster than frontier APIs -- at USD 0.013 per evaluation versus USD 0.025--0.055 for proprietary alternatives, yielding 46--76% cost savings. We introduce targeted hard-negative augmentation for critical decision boundaries and formalize the hybrid neural-symbolic decomposition, demonstrating that domain-adapted small language models with postprocessing can match frontier model accuracy while dramatically reducing operational cost, latency, and privacy risk.

Low-Rank Adaptation Data Scarcity Multi-Label Classification Application Programming Interfaces Post-Processing

Open paper Ask about it PDF

Abstract

Explore similar work

Domain-Adapted Small Language Models with Hybrid Post-Processing: Achieving Cost-Efficient, Low-Latency Multi-Label Structured Prediction via LoRA Fine-Tuning on Scarce Data

Internalizing Tool Knowledge in Small Language Models via QLoRA Fine-Tuning

How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model