Autoregressive Model

Autoregressive models are a class of generative models that predict sequential data by probabilistically modeling the next element based on preceding ones. Current research focuses on enhancing their controllability for tasks like image and 3D shape generation, improving their efficiency through parallel processing and non-autoregressive techniques, and applying them to diverse domains including time series forecasting, speech recognition, and even scientific modeling of physical systems. These advancements are driving improvements in various applications, from more accurate weather forecasting and efficient speech recognition to novel approaches in drug discovery and materials science.

Papers

December 22, 2023

Generative Pretraining at Scale: Transformer-Based Encoding of Transactional Behavior for Fraud Detection
Ze Yu Zhao, Zheng Zhu, Guilin Li, Wenhan Wang, Bo Wang
Anomaly Detection Visual Analogue Scale Autoregressive Model Fraud Detection Unsupervised Pre Training Transformer Encoder Generative Pre Training

December 16, 2023

Semantic-Aware Autoregressive Image Modeling for Visual Representation Learning
Kaiyou Song, Shan Zhang, Tong Wang
Semantic Segmentation Autoregressive Model Visual Representation Learning Autoregressive Image Semantic Patch

December 6, 2023

A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints
Kareem Ahmed, Kai-Wei Chang, Guy Van den Broeck
Autoregressive Model Neuro Symbolic Semantic Loss Logical Constraint

November 26, 2023

Utilizing Multiple Inputs Autoregressive Models for Bearing Remaining Useful Life Prediction
Junliang Wang, Qinghua Zhang, Guanhua Zhu, Guoxi Sun
Autoregressive Model Useful Life Rolling Bearing

November 9, 2023

Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities
AJ Piergiovanni, Isaac Noble, Dahun Kim, Michael S. Ryoo, Victor Gomes, Anelia Angelova
Audio Visual Multimodal Phenomenon Multimodal Model Multimodal Learning Autoregressive Model Multimodal Benchmark Multimodal Context

November 2, 2023

October 30, 2023

October 25, 2023

General Point Model with Autoencoding and Autoregressive
Zhe Li, Zhangyang Gao, Cheng Tan, Stan Z. Li, Laurence T. Yang
Large Language Model Autoregressive Model Auto Encoder Model Point Cloud Transformer Point Model

October 23, 2023

October 19, 2023

Exploring Invariance in Images through One-way Wave Equations
Yinpeng Chen, Dongdong Chen, Xiyang Dai, Mengchen Liu, Yinan Feng, Youzuo Lin, Lu Yuan, Zicheng Liu
Convolutional Neural Network Autoregressive Model Visual Encoder Wave Equation

October 17, 2023

Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression
Adam Block, Dylan J. Foster, Akshay Krishnamurthy, Max Simchowitz, Cyril Zhang
Stochastic Gradient Descent Autoregressive Model Behavior Cloning Autoregressive Text Generation Closed Loop Stability Butterfly Effect Error Simulation Minibatch SGD

October 16, 2023

Population-based wind farm monitoring based on a spatial autoregressive approach
W. Lin, K. Worden, E. J. Cross
Autoregressive Model Structural Health Monitoring Wind Turbine Structural Characterization Population Based Structural Health Monitoring Population Level

October 5, 2023

Sparse Deep Learning for Time Series Data: Theory and Applications
Mingxuan Zhang, Yan Sun, Faming Liang
Financial Application Recurrent Neural Network Theoretical Understanding Many Sparse Autoregressive Model Time Series Data Prediction Uncertainty Sparse Deep

September 26, 2023

Segment-Level Vectorized Beam Search Based on Partially Autoregressive Inference
Masao Someki, Nicholas Eng, Yosuke Higuchi, Shinji Watanabe
Automatic Speech Recognition Autoregressive Model Beam Search Attention Based Architecture Attention Based Encoder Decoder

September 24, 2023

Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve
R. Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, Thomas L. Griffiths
Large Language Model Autoregressive Model Evidence Piece Right Problem Next Word Prediction LLM Accuracy Word Sequence

September 16, 2023

Music Generation based on Generative Adversarial Networks with Transformer
Ziyi Jiang, Ruoxue Wu, Zhenghan Chen, Xiaoxuan Liang
Generative Adversarial Network Transformer Based Autoregressive Model Music Generation Adversarial Loss

September 7, 2023

A Tutorial on the Non-Asymptotic Theory of System Identification
Ingvar Ziemann, Anastasios Tsiamis, Bruce Lee, Yassir Jedra, Nikolai Matni, George J. Pappas
Autoregressive Model Least Square Tutorial Review System Identification Non Asymptotic Nonlinear System Identification Martingale Difference

Autoregressive Model

Papers

Generative Pretraining at Scale: Transformer-Based Encoding of Transactional Behavior for Fraud Detection

Semantic-Aware Autoregressive Image Modeling for Visual Representation Learning

A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints

Utilizing Multiple Inputs Autoregressive Models for Bearing Remaining Useful Life Prediction

Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities

Time-series Generation by Contrastive Imitation

Non-Autoregressive Diffusion-based Temporal Point Processes for Continuous-Time Long-Term Event Prediction

Bayesian Simulation-based Inference for Cosmological Initial Conditions

AMLNet: Adversarial Mutual Learning Neural Network for Non-AutoRegressive Multi-Horizon Time Series Forecasting

General Point Model with Autoencoding and Autoregressive

An Efficient Imbalance-Aware Federated Learning Approach for Wearable Healthcare with Autoregressive Ratio Observation

Meaning Representations from Trajectories in Autoregressive Models

Exploring Invariance in Images through One-way Wave Equations

Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression

Population-based wind farm monitoring based on a spatial autoregressive approach

Sparse Deep Learning for Time Series Data: Theory and Applications

Segment-Level Vectorized Beam Search Based on Partially Autoregressive Inference

Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve

Music Generation based on Generative Adversarial Networks with Transformer

A Tutorial on the Non-Asymptotic Theory of System Identification