Neural Network Architecture

Neural network architecture research focuses on designing efficient and effective network structures for various machine learning tasks. Current efforts concentrate on automating architecture design through techniques like evolutionary algorithms and neural architecture search (NAS), exploring diverse architectures such as transformers, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and their hybrid combinations, often tailored for specific data types and hardware constraints. This field is crucial for advancing machine learning capabilities across diverse applications, from image recognition and natural language processing to scientific modeling and medical diagnosis, by improving model accuracy, efficiency, and fairness.

Papers

November 15, 2024

Y-MAP-Net: Real-time depth, normals, segmentation, multi-label captioning and 2D human pose in RGB images
Ammar Qammaz, Nikolaos Vasilikopoulos, Iason Oikonomidis, Antonis A. Argyros
Semantic Segmentation Human Pose Neural Network Architecture RGB Image Depth Video

November 4, 2024

How Analysis Can Teach Us the Optimal Way to Design Neural Operators
Vu-Anh Le, Mehmet Dik
Strong Generalization General Analysis Neural Operator Neural Network Architecture Near Optimal Exponential Convergence

October 30, 2024

Estimating Neural Network Robustness via Lipschitz Constant and Architecture Sensitivity
Abulikemu Abuduweili, Changliu Liu
Native Robustness Neural Network Architecture Neural Network Robustness Robust Neural Network Lipschitz Constant Lipschitz Bound Robust Robot

October 25, 2024

Notes on the Mathematical Structure of GPT LLM Architectures
Spencer Becker-Kahn
Neural Network Architecture Short Note Level Mathematics Mathematical Structure Expository Text

October 1, 2024

nGPT: Normalized Transformer with Representation Learning on the Hypersphere
Ilya Loshchilov, Cheng-Ping Hsieh, Simeng Sun, Boris Ginsburg
Transformer Based Representation Learning Jina Embeddings Neural Network Architecture Attention Matrix

September 30, 2024

Cartesian Genetic Programming Approach for Designing Convolutional Neural Networks
Maciej Krzywda, Szymon Łukasik, Amir Gandomi H
Convolutional Neural Network Neural Architecture Search Neural Network Architecture Genetic Programming Cartesian Genetic Programming

September 12, 2024

Open Source Infrastructure for Automatic Cell Segmentation
Aaron Rock Menezes, Bharath Ramsundar
Neural Network Architecture Cell Segmentation UNet Based Manual Segmentation

September 8, 2024

From Computation to Consumption: Exploring the Compute-Energy Link for Training and Testing Neural Networks for SED Systems
Constance Douwes, Romain Serizel
Neural Network Training Data Machine Learning Model Neural Network Architecture Computation Method Memory Management GIT Net

August 11, 2024

A Comparative Study of Convolutional and Recurrent Neural Networks for Storm Surge Prediction in Tampa Bay
Mandana Farhang Ghahfarokhi, Seyed Hossein Sonbolestan, Mahta Zamanizadeh
Recurrent Neural Network Comparative Study Deep Convolutional Neural Network LSTM Network Neural Network Architecture LSTM Model CNN LSTM Storm Surge

August 9, 2024

rule4ml: An Open-Source Tool for Resource Utilization and Latency Estimation for ML Models on FPGA
Mohammad Mehdi Rahimifar, Hamza Ezzaoui Rahali, Audrey C. Therrien
Machine Learning Model Field Programmable Gate Array Neural Network Architecture Inference Latency Resource Utilization High Level Synthesis VIS4ML Research

July 18, 2024

Data-Algorithm-Architecture Co-Optimization for Fair Neural Networks on Skin Lesion Dataset
Yi Sheng, Junhuan Yang, Jinyang Li, James Alaina, Xiaowei Xu, Yiyu Shi, Jingtong Hu, Weiwen Jiang, Lei Yang
Neural Network Neural Network Architecture Skin Lesion Co Design Medical AI Contextual Biasing Medical Artificial Intelligence Fairness Aware Training

July 17, 2024

On Diversity in Discriminative Neural Networks
Brahim Oubaha, Claude Berrou, Xueyao Ji, Yehya Nasser, Raphaël Le Bidan
Neural Network Machine Learning Diversity Awareness Neural Network Architecture

June 20, 2024

Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecture
Zhengxin Yang, Wanling Gao, Luzhou Peng, Yunyou Huang, Fei Tang, Jianfeng Zhan
Neural Network Neural Architecture Benchmark Dataset Neural Network Architecture Synthetic to Real First Dataset Architectural Layout

June 19, 2024

Fair Differentiable Neural Network Architecture Search for Long-Tailed Data with Self-Supervised Learning
Jiaming Yan
Deep Learning Self Supervised Learning Neural Architecture Search Neural Network Architecture Long Tailed Long Tailed Data

June 13, 2024

Hadamard Representations: Augmenting Hyperbolic Tangents in RL
Jacob E. Kooi, Mark Hoogendoorn, Vincent François-Lavet
Neural Network Architecture Fast Learning Human Like RL Differentiable Activation Function

May 30, 2024

Growing Tiny Networks: Spotting Expressivity Bottlenecks and Fixing Them Optimally
Manon Verbockhaven, Sylvain Chevallier, Guillaume Charpiat
Gradient Flow Neural Network Architecture Hyper Parameter Tiny Network

May 27, 2024

Time Elastic Neural Networks
Pierre-François Marteau
Neural Architecture LSTM Network Neural Network Architecture Multivariate Time Series Classification Elastic Net

May 23, 2024

Scalable Optimization in the Modular Norm
Tim Large, Yang Liu, Minyoung Huh, Hyojin Bahng, Phillip Isola, Jeremy Bernstein
Neural Network Architecture P$ Norm Rapid Norm Growth Modern Deep Learning

May 15, 2024

Improving Transformers using Faithful Positional Encoding
Tsuyoshi Idé, Jokin Labaien, Pin-Yu Chen
Transformer Megatron Decepticons Time Series Classification Neural Network Architecture Positional Encoding Novel Encoding

April 13, 2024

Advanced Neural Network Architecture for Enhanced Multi-Lead ECG Arrhythmia Detection through Optimized Feature Extraction
Bhavith Chandra Challagundla
Feature Extraction Neural Network Architecture Cardiac Arrhythmia Electrocardiogram Arrhythmia Arrhythmia Type

Neural Network Architecture

Papers

Y-MAP-Net: Real-time depth, normals, segmentation, multi-label captioning and 2D human pose in RGB images

How Analysis Can Teach Us the Optimal Way to Design Neural Operators

Estimating Neural Network Robustness via Lipschitz Constant and Architecture Sensitivity

Notes on the Mathematical Structure of GPT LLM Architectures

nGPT: Normalized Transformer with Representation Learning on the Hypersphere

Cartesian Genetic Programming Approach for Designing Convolutional Neural Networks

Open Source Infrastructure for Automatic Cell Segmentation

From Computation to Consumption: Exploring the Compute-Energy Link for Training and Testing Neural Networks for SED Systems

A Comparative Study of Convolutional and Recurrent Neural Networks for Storm Surge Prediction in Tampa Bay

rule4ml: An Open-Source Tool for Resource Utilization and Latency Estimation for ML Models on FPGA

Data-Algorithm-Architecture Co-Optimization for Fair Neural Networks on Skin Lesion Dataset

On Diversity in Discriminative Neural Networks

Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecture

Fair Differentiable Neural Network Architecture Search for Long-Tailed Data with Self-Supervised Learning

Hadamard Representations: Augmenting Hyperbolic Tangents in RL

Growing Tiny Networks: Spotting Expressivity Bottlenecks and Fixing Them Optimally

Time Elastic Neural Networks

Scalable Optimization in the Modular Norm

Improving Transformers using Faithful Positional Encoding

Advanced Neural Network Architecture for Enhanced Multi-Lead ECG Arrhythmia Detection through Optimized Feature Extraction