Transformer Neural

Transformer neural networks are a powerful class of deep learning models leveraging attention mechanisms to capture long-range dependencies in sequential data, achieving superior performance in various tasks compared to traditional methods. Current research focuses on enhancing their efficiency and applicability through architectural innovations like pseudo-token-based transformers and integrating them with meta-learning frameworks such as neural processes for improved generalization and uncertainty quantification. These advancements are driving significant improvements in diverse fields, including load forecasting, robotic pose estimation, and material science, by enabling more accurate and efficient modeling of complex systems.

Papers

October 20, 2024

Amortized Probabilistic Conditioning for Optimization, Simulation and Inference
Paul E. Chang, Nasrulloh Loka, Daolang Huang, Ulpu Remes, Samuel Kaski, Luigi Acerbi
Optimization Purpose Scientific Inference Meta Learning Simulation Study Latent Variable Amortized Bayesian Inference Amortized Optimization Transformer Neural Meta Transformer

October 9, 2024

Gridded Transformer Neural Processes for Large Unstructured Spatio-Temporal Data
Matthew Ashman, Cristiana Diaconu, Eric Langezaal, Adrian Weller, Richard E. Turner
Neural Process Temporal Data Spatiotemporal Datasets Unstructured Information Large Scale Spatio Temporal Transformer Neural Forecasting Pipeline

June 19, 2024

In-Context In-Context Learning with Transformer Neural Processes
Matthew Ashman, Cristiana Diaconu, Adrian Weller, Richard E. Turner
Context Learning Neural Process Transformer Neural

June 13, 2024

Deep Transformer Network for Monocular Pose Estimation of Ship-Based UAV
Maneesha Wickramasuriya, Taeyoung Lee, Murray Snyder
Unmanned Aerial Vehicle Deep Transformer 6D Object Pose Transformer Neural Ship Landing

July 28, 2023

Differential Evolution Algorithm based Hyper-Parameters Selection of Transformer Neural Network Model for Load Forecasting
Anuvab Sen, Arul Rhik Mazumder, Udayon Sen
Load Forecasting Differential Evolution Transformer Neural

May 26, 2023

How Powerful are Decoder-Only Transformer Neural Models?
Jesse Roberts
Transformer Megatron Decepticons Neural Model General Transformer Transformer Neural

May 25, 2023

End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes
Alexandre Maraval, Matthieu Zimmer, Antoine Grosnit, Haitham Bou Ammar
Reinforcement Learning Bayesian Optimisation Hyperparameter Optimisation Transformer Neural Meta Learning Bayesian Optimization

April 16, 2023

A Neural Network Transformer Model for Composite Microstructure Homogenization
Emil Pitz, Kishore Pochiraju
3D Microstructures Microstructural Feature Homogenization Effect Transformer Neural

July 9, 2022

Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling
Tung Nguyen, Aditya Grover
Gaussian Process Meta Learning Neural Process Transformer Neural