Multi Layer

Multi-layer architectures are a central theme in contemporary machine learning, aiming to improve the efficiency and accuracy of various models by strategically organizing computational units into multiple layers. Current research focuses on optimizing these architectures, exploring alternatives to traditional multilayer perceptrons (MLPs) such as Kolmogorov-Arnold Networks (KANs) and Fourier Analysis Networks (FANs), and investigating techniques like layer distillation and frequency shifting for improved performance and reduced computational cost. These advancements have significant implications for diverse applications, including music generation, image processing, natural language processing, and scientific computing, by enabling faster, more accurate, and more efficient models.

Papers

February 22, 2024

In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization
Ruiqi Zhang, Jingfeng Wu, Peter L. Bartlett
Context Learning Multi Layer Complementary Benefit Linear Attention Multi Layer Perceptron

February 21, 2024

February 20, 2024

Transformer tricks: Precomputing the first layer
Nils Graef
Scientific Inference Multi Layer Transformer Layer Early Layer

February 18, 2024

Why Lift so Heavy? Slimming Large Language Models by Cutting Off the Layers
Shuzhou Yuan, Ercong Nie, Bolei Ma, Michael Färber
Natural Language Processing Multi Layer Model Pruning Text Classification Task Prompt Based Fine Tuning Floor Lift

February 14, 2024

End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Training
Keitaro Sakamoto, Issei Sato
Comparative Study Multi Layer Information Bottleneck Deep Understanding Layer Wise Intermediate Representation End to End Training

February 12, 2024

Chain-of-Layer: Iteratively Prompting Large Language Models for Taxonomy Induction from Limited Examples
Qingkai Zeng, Yuyang Bai, Zhaoxuan Tan, Shangbin Feng, Zhenwen Liang, Zhihan Zhang, Meng Jiang
Context Learning Multi Layer Fewer Example Taxonomy Construction Ensemble Score Filter

January 30, 2024

Layered and Staged Monte Carlo Tree Search for SMT Strategy Synthesis
Zhengyang Lu, Stefan Siemer, Piyush Jha, Joel Day, Florin Manea, Vijay Ganesh
Multi Layer Monte Carlo Tree Search SMT Solver

January 29, 2024

Neural Network Training on Encrypted Data with TFHE
Luis Montero, Jordan Frery, Celia Kherfallah, Roman Bredehoft, Andrei Stoian
Neural Network Multi Layer Neural Network Model Homomorphic Encryption

January 25, 2024

Manifold GCN: Diffusion-based Convolutional Neural Network for Manifold-valued Graphs
Martin Hanik, Gabriele Steidl, Christoph von Tycowicz
Convolutional Neural Network Multi Layer Riemannian Geometry Graph Diffusion Latent Manifold Neural Network Layer Manifold Graph

January 21, 2024

LR-CNN: Lightweight Row-centric Convolutional Neural Network Training for Memory Reduction
Zhigang Wang, Hangyu Yang, Ning Wang, Chuanfei Xu, Jie Nie, Zhiqiang Wei, Yu Gu, Ge Yu
Convolutional Neural Network Multi Layer Convolution Layer Dataflow Architecture Memory Reduction

January 19, 2024

Quantised Neural Network Accelerators for Low-Power IDS in Automotive Networks
Shashwat Khandelwal, Anneliese Walsh, Shanker Shreejith
Multi Layer Field Programmable Gate Array Intrusion Detection System Traditional ECU Quantised Neural Network

January 18, 2024

Imitation Learning Inputting Image Feature to Each Layer of Neural Network
Koki Yamane, Sho Sakaino, Toshiaki Tsuji
Neural Network Training Data Non Humanoid Robot Imitation Learning High Dimensional Multi Layer

January 15, 2024

Improved Implicit Neural Representation with Fourier Bases Reparameterized Training
Kexuan Shi, Xingyu Zhou, Shuhang Gu
Multi Layer Implicit Neural Representation Multi Layer Perceptron Fourier Feature

January 12, 2024

Dynamic Behaviour of Connectionist Speech Recognition with Strong Latency Constraints
Giampiero Salvi
Multi Layer Connectionist Temporal Classification Connectionist Model Dynamic Behavior Latency Constraint

January 6, 2024

When To Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networks
Haihang Wu, Wei Wang, Tamasha Malepathirana, Damith Senanayake, Denny Oetomo, Saman Halgamuge
Deep Neural Network Multi Layer Regularization Effect

December 31, 2023

An Analysis of Embedding Layers and Similarity Scores using Siamese Neural Networks
Yash Bingi, Yiqiao Yin
Large Language Model General Analysis Multi Layer Word Embeddings Siamese Network Similarity Score Encoding Layer

December 28, 2023

Intelligent Parsing: An Automated Parsing Framework for Extracting Design Semantics from E-commerce Creatives
Guandong Li, Xian Yang
Multi Layer Human Parsing Naming Variation Material Recognition Design Framework

December 15, 2023

Improving Cross-domain Few-shot Classification with Multilayer Perceptron
Shuanghao Bai, Wanqi Zhou, Zhirong Luan, Donglin Wang, Badong Chen
Multi Layer Shot Classification Transferable Representation Discriminative Power Shot Classifier Domain Few Shot

December 13, 2023

Read Between the Layers: Leveraging Intra-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models
Kyra Ahrens, Hans Hergen Lehmann, Jae Hee Lee, Stefan Wermter
Continual LEArning Pre Trained Model Multi Layer Read V Continual Learning Benchmark Rehearsal Free Continual Learning

Multi Layer

Papers

In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization

Cracking Factual Knowledge: A Comprehensive Analysis of Degenerate Knowledge Neurons in Large Language Models

An Effective Networks Intrusion Detection Approach Based on Hybrid Harris Hawks and Multi-Layer Perceptron

Transformer tricks: Precomputing the first layer

Why Lift so Heavy? Slimming Large Language Models by Cutting Off the Layers

End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Training

Chain-of-Layer: Iteratively Prompting Large Language Models for Taxonomy Induction from Limited Examples

Layered and Staged Monte Carlo Tree Search for SMT Strategy Synthesis

Neural Network Training on Encrypted Data with TFHE

Manifold GCN: Diffusion-based Convolutional Neural Network for Manifold-valued Graphs

LR-CNN: Lightweight Row-centric Convolutional Neural Network Training for Memory Reduction

Quantised Neural Network Accelerators for Low-Power IDS in Automotive Networks

Imitation Learning Inputting Image Feature to Each Layer of Neural Network

Improved Implicit Neural Representation with Fourier Bases Reparameterized Training

Dynamic Behaviour of Connectionist Speech Recognition with Strong Latency Constraints

When To Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networks

An Analysis of Embedding Layers and Similarity Scores using Siamese Neural Networks

Intelligent Parsing: An Automated Parsing Framework for Extracting Design Semantics from E-commerce Creatives

Improving Cross-domain Few-shot Classification with Multilayer Perceptron

Read Between the Layers: Leveraging Intra-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models