cs.LGNov 17, 2026

VRLLM: Turning LLMs into Powerful Tabular Classifiers

Authors: Yingtong Dou, Shubham Jain, Junpeng Wang, Hadi Abdullah, Vineeth Rakesh Mohan, Yuzhong Chen, Liang Wang, Yiwei Cai

Organizations: Visa Research

Abstract

Large Language Models (LLMs) have shown strong performance on text, but whether pretrained LLMs are effective for large-scale tabular classification remains unclear, as many previous works demonstrate that LLMs' tabular data classification performance falls behind tree-based models or neural architectures designed for tabular data. At Visa Research, we managed to fine-tune a pretrained Llama 3.2 1B LLM on billions of rows of labeled tabular data and make it outperform a wide array of tabular classifiers and even our competitive production deep learning model. For ease of reference, we refer to the model as the Visa Research LLM (VRLLM). In this paper, we share how we achieved this exciting result, covering data preparation, training recipe, ablation studies of design choices and deployment validation. We also benchmark frontier LLMs on the same task and analyze their prediction behavior across model families and generations. Our interpretability experiments suggest that VRLLM learns feature-specific predictive associations, while component ablations indicate that the earliest two FFN layers contribute to the emergence of linearly decodable task information. Our paper provides an engineering recipe and a focused empirical case study on adapting compact LLMs to industrial tabular prediction tasks.

Figures & tables

Explore similar work

CardsList
  1. Why Large Language Models Fail at Tabular Prediction

    Aug 3, 2026Marta Garnelo, Wojciech M. CzarneckiLLM EvaluationTabular Classification

  2. Strengthening LLMs for Tabular Prediction with Structural Priors

    Oct 20, 2025Pengxiang Cai, Zihao Gao, Wanchen Lian +2Tabular Foundation ModelsRL for Language Models