Simple MLP

Simple Multilayer Perceptrons (MLPs) are undergoing a resurgence in machine learning, driven by efforts to create efficient and effective alternatives to more complex architectures like transformers and convolutional neural networks. Current research focuses on enhancing MLPs' ability to capture spatial relationships within data, leading to novel architectures like Strip-MLP and SiT-MLP that incorporate sophisticated token interaction mechanisms and dynamic mixing strategies. These advancements demonstrate that carefully designed MLPs can achieve competitive performance on various tasks, including image classification and time series analysis, while offering significant advantages in terms of computational cost and ease of training.

Papers

May 5, 2024

Spatial Transfer Learning with Simple MLP
Hongjian Yang
Transfer Learning Full Potential Limited Field First Step Simple MLP

August 30, 2023

SiT-MLP: A Simple MLP with Point-wise Topology Feature Learning for Skeleton-based Action Recognition
Shaojie Zhang, Jianqin Yin, Yonghao Dang, Jiajun Fu
Skeleton Based Action Recognition Topological Learning Simple MLP

July 21, 2023

Strip-MLP: Efficient Token Interaction for Vision MLP
Guiping Cao, Shengda Luo, Wenjian Huang, Xiangyuan Lan, Dongmei Jiang, Yaowei Wang, Jianguo Zhang
MLP Architecture MLP Layer Simple MLP

July 18, 2023

U-shaped Transformer: Retain High Frequency Context in Time Series Analysis
Qingkui Chen, Yiqin Zhang
Time Series Time Series Analysis Long Range Context U Shaped Transformer Backbone Simple MLP

April 7, 2022

A Pathology-Based Machine Learning Method to Assist in Epithelial Dysplasia Diagnosis
Karoline da Rocha, José C. M. Bermudez, Elena R. C. Rivero, Márcio H. Costa
Convolutional Neural Network Multi Layer Neural Network Simple MLP Epithelial Dysplasia

March 7, 2022

HyperMixer: An MLP-based Low Cost Alternative to Transformers
Florian Mai, Arnaud Pannatier, Fabio Fehr, Haolin Chen, Francois Marelli, Francois Fleuret, James Henderson
Transformer Megatron Decepticons Language Understanding MLP Like Model Static Deep Neural Network Simple MLP

January 28, 2022

DynaMixer: A Vision MLP Architecture with Dynamic Mixing
Ziyu Wang, Wenhao Jiang, Yiming Zhu, Li Yuan, Yibing Song, Wei Liu
Global Mixing MLP Architecture Multi Level Fusion MLP Like Model Simple MLP