cs.LGSep 7, 2026

Streaming Hierarchical Inference with Tabular Foundation Models

Authors: Vitor CristaAfonso LourençoDiogo MartinhoGoreti Marreiros

Organizations: GECAD, Polytechnic of Porto, Portugal

Abstract

Tabular Foundation Models (TFMs) have recently demonstrated strong predictive performance through in-context learning, but their deployment in high-throughput data streams remains challenging due to communication overhead and latency. We propose \textit{HINT}, a hierarchical inference framework that combines edge-based retrieval with cloud-based TFM inference. A graph-based approximate nearest neighbor memory maintained over a sliding window provides local predictions and uncertainty estimates, allowing confident samples to be processed locally while uncertain instances are selectively offloaded, together with their retrieved context, to a cloud-hosted TFM. The framework exposes an offloading threshold and a neighborhood retrieval policy that can be varied to balance predictive performance and communication cost. Experiments show \textit{HINT} consistently identifies favorable trade-offs.

Explore similar work

CardsList
  1. Memory Efficient Tabular Foundation Models

    Jul 30, 2026Shuting Luo, Monika Mikhail Kanaan, Cameron Gordon +2Tabular Foundation ModelsTabular Prior-Data Fitted Network

  2. TabPFN-3.5: Technical Report

    Sep 15, 2026Benjamin Jäger, Nick Erickson, Léo Grinsztajn +44Tabular Foundation ModelsReal-Time Inference