Fundamental Limitation

Fundamental limitations in artificial intelligence research currently focus on identifying and addressing bottlenecks in model capabilities and performance. Active research areas include exploring the limitations of large language models (LLMs) in reasoning, particularly compositional abilities and handling complex tasks; analyzing the inherent quadratic time complexity of transformer architectures and the challenges of developing subquadratic alternatives; and investigating the impact of data quality and size on model performance and safety. Understanding these limitations is crucial for improving the reliability, safety, and efficiency of AI systems and for developing more robust and generalizable models across various applications.

Papers

November 6, 2022

Understanding the properties and limitations of contrastive learning for Out-of-Distribution detection
Nawid Keshtmand, Raul Santos-Rodriguez, Jonathan Lawry
Contrastive Learning Distribution Detection Fundamental Limitation Many Property

October 22, 2022

On the Limitations of Reference-Free Evaluations of Generated Text
Daniel Deutsch, Rotem Dror, Dan Roth
Machine Translation Evaluation Metric Fundamental Limitation Metric Library Generated Text Reference Free Metric Reference Free Evaluation

October 17, 2022

Deepfake Text Detection: Limitations and Opportunities
Jiameng Pu, Zain Sarwar, Sifat Muhammad Abdullah, Abdullah Rehman, Yoonjin Kim, Parantapa Bhattacharya, Mobin Javed, Bimal Viswanath
Emerging Opportunity Fundamental Limitation Machine Generated

October 11, 2022

Embeddings as Epistemic States: Limitations on the Use of Pooling Operators for Accumulating Knowledge
Steven Schockaert
Jina Embeddings Knowledge Based Fundamental Limitation Epistemic State Pooling Operator Non Negative Knowledge Pooling

October 6, 2022

When not to use machine learning: a perspective on potential and limitations
M. R. Carbone
Machine Learning Artificial Intelligence Data Driven Visual Perspective Fundamental Limitation Risk Sensitive Data Driven Modeling

October 3, 2022

Limitations of neural network training due to numerical instability of backpropagation
Clemens Karner, Vladimir Kazeev, Philipp Christian Petersen
Neural Network Deep Neural Network Gradient Descent Back Propagation Natural Gradient Fundamental Limitation Numerical Stability Independent Rate

September 25, 2022

Algorithms that Approximate Data Removal: New Results and Limitations
Vinith M. Suriyakumar, Ashia C. Wilson
Machine Learning Model Practical Algorithm Empirical Risk Minimization Fundamental Limitation Approximate Unlearning New Finding Unlearning Algorithm Data Deletion

August 13, 2022

On the Limitations of Continual Learning for Malware Classification
Mohammad Saidur Rahman, Scott E. Coull, Matthew Wright
Continual LEArning Continual Learning Fundamental Limitation Malware Classification Malware Datasets

August 9, 2022

Limitations of Language Models in Arithmetic and Symbolic Induction
Jing Qian, Hong Wang, Zekun Li, Shiyang Li, Xifeng Yan
Language Model Natural Language Processing Reasoning Task Fundamental Limitation Symbolic Computation Arithmetic Operation

August 1, 2022

July 16, 2022

Progress and limitations of deep networks to recognize objects in unusual poses
Amro Abbas, Stéphane Deny
Convolutional Neural Network Native Robustness Deep Network Arbitrary Object Fundamental Limitation Much Progress Noise Robust Learning Unusual Pose

July 2, 2022

The Parallelism Tradeoff: Limitations of Log-Precision Transformers
William Merrill, Ashish Sabharwal
Transformer Based Transformer Architecture Fundamental Limitation Trade Offs Computational Power Precision Transformer Threshold Circuit

June 21, 2022

On the Limitations of Elo: Real-World Games, are Transitive, not Additive
Quentin Bertrand, Wojciech Marian Czarnecki, Gauthier Gidel
Fundamental Limitation Strong Transitivity Relation Elo Rating Transitive Game Real World Game

June 20, 2022

Limitations of the NTK for Understanding Generalization in Deep Learning
Nikhil Vyas, Yamini Bansal, Preetum Nakkiran
Deep Learning Strong Generalization Neural Tangent Kernel Fundamental Limitation Kernel Size Layer NTK Scaling Deep Network

June 19, 2022

On the Limitations of Stochastic Pre-processing Defenses
Yue Gao, Ilia Shumailov, Kassem Fawaz, Nicolas Papernot
Adversarial Example Adversarial Robustness Fundamental Limitation Stochastic Way Adversarial Input Stochastic Defense

June 9, 2022

Multiple Instance Learning for Digital Pathology: A Review on the State-of-the-Art, Limitations & Future Potential
Michael Gadermayr, Maximilian Tschuchnig
Deep Neural Network Full Potential State of the Art Multiple Instance Learning Digital Pathology Fundamental Limitation Deep Multiple Instance

June 5, 2022

Sequential Counterfactual Decision-Making Under Confounded Reward
Erik Skalnes
Fundamental Limitation Randomized Experiment Soft Intervention

May 22, 2022

Limitations of a proposed correction for slow drifts in decision criterion
Diksha Gupta, Carlos D. Brody
Fundamental Limitation Language Correction Real Drift Systematic Bias Generative Trajectory Trial and Error

May 16, 2022

Power and limitations of single-qubit native quantum neural networks
Zhan Yu, Hongshun Yao, Mujin Li, Xin Wang
Neural Network Real Power Quantum Physic Quantum Circuit Fundamental Limitation Quantum Neural Network Quantum Neuron