Fundamental Limitation

Fundamental limitations in artificial intelligence research currently focus on identifying and addressing bottlenecks in model capabilities and performance. Active research areas include exploring the limitations of large language models (LLMs) in reasoning, particularly compositional abilities and handling complex tasks; analyzing the inherent quadratic time complexity of transformer architectures and the challenges of developing subquadratic alternatives; and investigating the impact of data quality and size on model performance and safety. Understanding these limitations is crucial for improving the reliability, safety, and efficiency of AI systems and for developing more robust and generalizable models across various applications.

Papers

July 5, 2023

Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, Yoon Kim
Language Model Complex Reasoning Fundamental Limitation High Quality Counterfactuals Abstract Reasoning Mixed Task Counterfactual Task

June 22, 2023

On Addressing the Limitations of Graph Neural Networks
Sitao Luan
Convolutional Neural Network Graph Neural Network Graph Drawing Future Direction Fundamental Limitation Heterophilic Graph

June 15, 2023

Overcoming the Limitations of Localization Uncertainty: Efficient & Exact Non-Linear Post-Processing and Calibration
Moussa Kassem Sbeyti, Michelle Karg, Christian Wirth, Azarm Nowzad, Sahin Albayrak
High Efficiency Object Detector Calibration Performance Fundamental Limitation Localization Performance Calibration Error Localization Task Localization Uncertainty

June 5, 2023

Representational Strengths and Limitations of Transformers
Clayton Sanford, Daniel Hsu, Matus Telgarsky
Transformer Megatron Decepticons Attention Layer Fundamental Limitation Modern Deep Learning Prototype Attention

June 1, 2023

On the Limitations of Temperature Scaling for Distributions with Overlaps
Muthu Chidambaram, Rong Ge
Fundamental Limitation Temperature Scaling Image Classification Benchmark Calibration Metric Train Time Calibration

May 30, 2023

May 22, 2023

Observations on LLMs for Telecom Domain: Capabilities and Limitations
Sumit Soman, Ranjani H G
Large Language Model Chatbot Response Generative Artificial Intelligence Fundamental Limitation Capability Evolution Evolving Observation Conversational Interface Telecom Domain

May 21, 2023

On the Limitations of Simulating Active Learning
Katerina Margatina, Nikolaos Aletras
Active Learning Fundamental Limitation Human Annotation Human in the Loop ActIve Learning algORithm

May 12, 2023

A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects
Leo Pauly, Wassim Rharbaoui, Carl Shneider, Arunkumar Rathinam, Vincent Gaudilliere, Djamila Aouada
Timely Survey Computer Vision Fundamental Limitation Monocular Camera Visionary ProSpect Computer Vision Application Pose Optimization Spacecraft Pose Estimation Vision Based Autonomous System

May 11, 2023

Promise and Limitations of Supervised Optimal Transport-Based Graph Summarization via Information Theoretic Measures
Sepideh Neshatfar, Abram Magner, Salimeh Yasaei Sekeh
Fundamental Limitation Promise Unsupervised Extractive Summarization Graph Summarization Information Theoretic Measure

May 9, 2023

On the Limitations of Model Stealing with Uncertainty Quantification Models
David Pape, Sina Däubener, Thorsten Eisenhofer, Antonio Emanuele Cinà, Lea Schönherr
Training Data Fundamental Limitation Model Stealing Quantified Uncertainty

April 19, 2023

Fundamental Limitations of Alignment in Large Language Models
Yotam Wolf, Noam Wies, Oshri Avnery, Yoav Levine, Amnon Shashua
Large Language Model Language Model Alignment Problem Fundamental Limitation Adversarial Agent Alignment Approach

April 14, 2023

Domain shifts in dermoscopic skin cancer datasets: Evaluation of essential limitations for clinical translation
Katharina Fogelberg, Sireesha Chamarthi, Roman C. Maron, Julia Niebling, Titus J. Brinker
Global Evaluation Domain Shift Fundamental Limitation Unseen Domain Dermoscopic Image Clinical Evidence

April 11, 2023

Machine learning for structure-property relationships: Scalability and limitations
Zhongzheng Tian, Sheng Zhang, Gia-Wei Chern
Machine Learning New Machine Potential Scalability Fundamental Limitation Many Body Ising Model Structure Property Relationship Scaling Relation

March 22, 2023

February 28, 2023

Beyond the limitations of any imaginable mechanism: large language models and psycholinguistics
Conor Houghton, Nina Kazanina, Priyanka Sukumaran
Large Language Model Full Model Fundamental Limitation Psycholinguistic Research Language Processing Animal Model

February 23, 2023

On the Limitations of Physics-informed Deep Learning: Illustrations Using First Order Hyperbolic Conservation Law-based Traffic Flow Models
Archie J. Huang, Shaurya Agarwal
Fundamental Limitation Physic Informed Deep Learning Conservation Law Traffic State Estimation Hand Drawn Illustration Hyperbolic Partial Differential Equation Nonlocal Model

February 14, 2023

Data pruning and neural scaling laws: fundamental limitations of score-based algorithms
Fadhel Ayed, Soufiane Hayou
Fundamental Limitation Score Based Data Pruning Neural Scaling Law Random Pruning

Fundamental Limitation

Papers

Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks

On Addressing the Limitations of Graph Neural Networks

Overcoming the Limitations of Localization Uncertainty: Efficient & Exact Non-Linear Post-Processing and Calibration

Representational Strengths and Limitations of Transformers

On the Limitations of Temperature Scaling for Distributions with Overlaps

GPT Models in Construction Industry: Opportunities, Limitations, and a Use Case Validation

Universality and Limitations of Prompt Tuning

Observations on LLMs for Telecom Domain: Capabilities and Limitations

On the Limitations of Simulating Active Learning

A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects

Promise and Limitations of Supervised Optimal Transport-Based Graph Summarization via Information Theoretic Measures

On the Limitations of Model Stealing with Uncertainty Quantification Models

Fundamental Limitations of Alignment in Large Language Models

Domain shifts in dermoscopic skin cancer datasets: Evaluation of essential limitations for clinical translation

Machine learning for structure-property relationships: Scalability and limitations

The power and limitations of learning quantum dynamics incoherently

Neuro-Symbolic Reasoning Shortcuts: Mitigation Strategies and their Limitations

Beyond the limitations of any imaginable mechanism: large language models and psycholinguistics

On the Limitations of Physics-informed Deep Learning: Illustrations Using First Order Hyperbolic Conservation Law-based Traffic Flow Models

Data pruning and neural scaling laws: fundamental limitations of score-based algorithms