General Neural Operator Transformer

General Neural Operator Transformers leverage the power of transformer architectures, particularly the attention mechanism, to learn mappings between function spaces, primarily focusing on solving and approximating solutions to partial differential equations (PDEs). Current research emphasizes improving efficiency and interpretability through novel attention mechanisms like position-attention, which reduces computational cost compared to standard self-attention, and incorporating multi-scale time-stepping for enhanced accuracy and speed. This approach offers a powerful and flexible framework for surrogate modeling of complex systems, accelerating scientific computation and potentially impacting diverse fields requiring efficient PDE solutions.

Papers

June 10, 2024

Continuum Attention for Neural Operators
Edoardo Calvello, Nikola B. Kovachki, Matthew E. Levine, Andrew M. Stuart
Neural Operator Attention Operator Attention Trait General Neural Operator Transformer

May 15, 2024

Positional Knowledge is All You Need: Position-induced Transformer (PiT) for Operator Learning
Junfeng Chen, Kailiang Wu
Neural Operator Operator Learning Positional Information Position Detection Transformer General Neural Operator Transformer

November 3, 2023

Multi-scale Time-stepping of Partial Differential Equations with Transformers
AmirPouya Hemmasian, Amir Barati Farimani
Transformer Megatron Decepticons Transformer Based Transformer Architecture Partial Differential Equation Fourier Neural Operator Fast Surrogate Time Stepping General Neural Operator Transformer

February 28, 2023

GNOT: A General Neural Operator Transformer for Operator Learning
Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, Jun Zhu
Transformer Architecture Attention Layer Operator Learning Effective Transformer General Neural Operator Transformer

May 26, 2022

Transformer for Partial Differential Equations' Operator Learning
Zijie Li, Kazem Meidani, Amir Barati Farimani
Deep Learning Model Transformer Based Partial Differential Equation Operator Learning Data Driven Learning General Neural Operator Transformer

General Neural Operator Transformer

Papers

Continuum Attention for Neural Operators

Positional Knowledge is All You Need: Position-induced Transformer (PiT) for Operator Learning

Multi-scale Time-stepping of Partial Differential Equations with Transformers

GNOT: A General Neural Operator Transformer for Operator Learning

Transformer for Partial Differential Equations' Operator Learning