Performant Neural Network

Creating highly performant neural networks is a central challenge in machine learning, focusing on efficient architecture design and training optimization. Current research emphasizes automated architecture search (NAS) techniques, including novel cellular encoding methods and differentiable architecture learning, to discover optimal network structures without exhaustive manual design. These advancements, coupled with improved training strategies like efficient convolution paradigms and dynamic neuron addition, aim to reduce computational costs and enhance model accuracy across diverse applications, from image recognition to natural language processing. The resulting improvements in efficiency and performance have significant implications for deploying large-scale models in resource-constrained environments and for advancing our understanding of AI systems.

Papers

August 16, 2024

NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance
Raphael T. Husistein, Markus Reiher, Marco Eckhoff
Neural Network Machine Learning Neural Architecture Search Training Free Neural Network Layer Performant Neural Network

July 18, 2024

From Words to Worlds: Compositionality for Cognitive Architectures
Ruchira Dhar, Anders Søgaard
Large Language Model World Event Word List Cognitive Architecture Compositional Ability Compositional Language Compositional Approach Performant Neural Network

December 12, 2023

XC-NAS: A New Cellular Encoding Approach for Neural Architecture Search of Multi-path Convolutional Neural Networks
Trevor Londt, Xiaoying Gao, Peter Andreae, Yi Mei
Neural Architecture Search CNN Architecture Multi Path Performant Neural Network Population Encoding

October 25, 2023

June 25, 2023

Im2win: An Efficient Convolution Paradigm on GPU
Shuai Lu, Jun Chu, Luanzheng Guo, Xu T. Liu
Direct Convolution Single GPU Convolution Technique Benchmarking Deep Learning Fast Convolution CUDA Kernel Performant Neural Network

June 24, 2023

Computron: Serving Distributed Deep Learning Models with Model Parallel Swapping
Daniel Zou, Xinchen Jin, Xueyang Yu, Hao Zhang, James Demmel
Deep Learning Model Model Parallelism Multi GPU GPU Cluster Performant Neural Network

February 10, 2023

DNArch: Learning Convolutional Neural Architectures by Backpropagation
David W. Romero, Neil Zeghidour
Convolutional Neural Network Back Propagation Neural Architecture Convolutional Architecture Differentiable Neural Patch Masking Performant Neural Network

January 25, 2023

Modelling Long Range Dependencies in $N$D: From Task-Specific to a General Purpose CNN
David M. Knigge, David W. Romero, Albert Gu, Efstratios Gavves, Erik J. Bekkers, Jakub M. Tomczak, Mark Hoogendoorn, Jan-Jakob Sonke
CNN Model Task Specific Convolutional Kernel Long Range Dependency Performant Neural Network Continuous Convolutional Neural Network

February 25, 2022

Extracting Effective Subnetworks with Gumbel-Softmax
Robin Dupont, Mohammed Amine Alaoui, Hichem Sahbi, Alice Lebois
lightWeight Network Performant Neural Network

February 17, 2022

When, where, and how to add new neurons to ANNs
Kaitlin Maile, Emmanuel Rachelson, Hervé Luga, Dennis G. Wilson
Neuron Model Ann Bp Performant Neural Network