Prior Data Fitted Network

Prior-data fitted networks (PFNs) represent a novel approach to machine learning that leverages pre-trained transformer models to perform in-context learning, enabling fast and accurate predictions on new tasks without retraining. Current research focuses on enhancing PFNs for specific data types, such as tabular data (TabPFN) and time series (LaT-PFN), and improving their scalability and interpretability. This paradigm shift offers significant potential for accelerating various machine learning applications, particularly in scenarios with limited data or a need for rapid inference, by bypassing traditional model selection and hyperparameter tuning.

Papers

November 15, 2024

Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data
Kai Helli, David Schnurr, Noah Hollmann, Samuel Müller, Frank Hutter
Tabular Data Different Context Temporal Distribution Shift Prior Data Fitted Network

September 9, 2024

Zero-shot Outlier Detection via Prior-data Fitted Networks: Model Selection Bygone!
Yuchen Shen, Haomin Wen, Leman Akoglu
Outlier Detection Model Selection Hyperparameter Selection Zero Shot Anomaly Outlier Data Prior Data Fitted Network Unsupervised Model Selection

June 11, 2024

Tokenize features, enhancing tables: the FT-TABPFN model for tabular classification
Quangao Liu, Wei Yang, Chen Liang, Longlong Pang, Zhuozhang Zou
Table Semantics Table Representation Prior Data Fitted Network Tabular Classification

May 16, 2024

LaT-PFN: A Joint Embedding Predictive Architecture for In-context Time-series Forecasting
Stijn Verdenius, Andrea Zerio, Roy L. M. Wang
Zero Shot Joint Framework Time Series Foundation Model Predictive Architecture Prior Data Fitted Network

March 16, 2024

Interpretable Machine Learning for TabPFN
David Rundel, Julius Kobialka, Constantin von Crailsheim, Matthias Feurer, Thomas Nagler, David Rügamer
Interpretable Machine Learning Posterior Predictive Prior Data Fitted Network

February 17, 2024

TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks
Benjamin Feuer, Robin Tibor Schirrmeister, Valeriia Cherepanova, Chinmay Hegde, Frank Hutter, Micah Goldblum, Niv Cohen, Colin White
Context Learning Tabular Datasets Hyper Tune Tabular Data Classification Context Optimization Prior Data Fitted Network Tabular Classification

November 17, 2023

Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks
Benjamin Feuer, Chinmay Hegde, Niv Cohen
Training Data Feature Selection Tabular Model Creative Sketching Prior Data Fitted Network Tabular Classification

October 31, 2023

Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks
Steven Adriaensen, Herilalaina Rakotoarison, Samuel Müller, Frank Hutter
Model Performance Extrapolation Framework Posterior Predictive Approximate Bayesian Inference Trend Extrapolation Prior Data Fitted Network Right Censored

May 27, 2023

PFNs4BO: In-Context Learning for Bayesian Optimization
Samuel Müller, Matthias Feurer, Noah Hollmann, Frank Hutter
Bayesian Optimization Bayesian Neural Network Surrogate Modeling Different Context Posterior Predictive Prior Data Fitted Network

May 18, 2023

Statistical Foundations of Prior-Data Fitted Networks
Thomas Nagler
Machine Learning Pre Trained Model Scientific Inference Statistical Model Untrained Neural Network Prior Data Fitted Network

July 5, 2022

TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second
Noah Hollmann, Samuel Müller, Katharina Eggensperger, Frank Hutter
Synthetic Data Transformer Based Tabular Datasets Tabular Data Classification End to End AutoML Prior Data Fitted Network

December 20, 2021

Transformers Can Do Bayesian Inference
Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, Frank Hutter
Transformer Megatron Decepticons Bayesian Neural Network Bayesian Inference Posterior Approximation Efficient Bayesian Inference Prior Data Fitted Network

Prior Data Fitted Network

Papers

Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data

Zero-shot Outlier Detection via Prior-data Fitted Networks: Model Selection Bygone!

Tokenize features, enhancing tables: the FT-TABPFN model for tabular classification

LaT-PFN: A Joint Embedding Predictive Architecture for In-context Time-series Forecasting

Interpretable Machine Learning for TabPFN

TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks

Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks

Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks

PFNs4BO: In-Context Learning for Bayesian Optimization

Statistical Foundations of Prior-Data Fitted Networks

TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

Transformers Can Do Bayesian Inference