Fundamental Limitation

Fundamental limitations in artificial intelligence research currently focus on identifying and addressing bottlenecks in model capabilities and performance. Active research areas include exploring the limitations of large language models (LLMs) in reasoning, particularly compositional abilities and handling complex tasks; analyzing the inherent quadratic time complexity of transformer architectures and the challenges of developing subquadratic alternatives; and investigating the impact of data quality and size on model performance and safety. Understanding these limitations is crucial for improving the reliability, safety, and efficiency of AI systems and for developing more robust and generalizable models across various applications.

Papers

November 21, 2023

The Obscure Limitation of Modular Multilingual Language Models
Muhammad Farid Adilazuarda, Samuel Cahyawijaya, Ayu Purwarianti
Language Model Fundamental Limitation Language Identification Multilingual Scenario Multilingual Evaluation Cross Lingual Natural Language Inference

November 20, 2023

On the Potential and Limitations of Few-Shot In-Context Learning to Generate Metamorphic Specifications for Tax Preparation Software
Dananjay Srinivas, Rohan Das, Saeid Tizpaz-Niari, Ashutosh Trivedi, Maria Leonor Pacheco
Context Learning Full Potential Fundamental Limitation First Order Metamorphic Testing Metamorphic Relation

November 17, 2023

Exploring Machine Learning Models for Federated Learning: A Review of Approaches, Performance, and Limitations
Elaheh Jafarigol, Theodore Trafalis, Talayeh Razzaghi, Mona Zamankhani
Machine Learning Model Constructive Approach System Performance Fundamental Limitation Privacy Preserving Machine Learning Federated Learning Application Learning Based Point

November 16, 2023

On Retrieval Augmentation and the Limitations of Language Model Training
Ting-Rui Chiang, Xinyan Velocity Yu, Joshua Robinson, Ollie Liu, Isabelle Lee, Dani Yogatama
Language Model Information Retrieval Fundamental Limitation Softmax Function Retrieval Augmentation Language Model Generalization

October 30, 2023

When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations
Aleksandar Petrov, Philip H. S. Torr, Adel Bibi
Fine Tuning Context Learning Theoretical Understanding Prompt Tuning Fundamental Limitation Capability Evolution Attention Pattern

October 29, 2023

Worst-case Performance of Popular Approximate Nearest Neighbor Search Implementations: Guarantees and Limitations
Piotr Indyk, Haike Xu
Nearest Neighbor Fundamental Limitation Formal Guarantee Approximate Nearest Neighbor Search Graph Based Approximate Nearest Neighbor Worst Case Performance

October 24, 2023

AI Alignment and Social Choice: Fundamental Limitations and Policy Implications
Abhilash Mishra
AI Agent Fundamental Limitation AI Alignment Social Choice Agent Alignment Human AI Alignment Policy Recommendation

October 20, 2023

Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification
Amr Keleg, Walid Magdy
Multi Label Classification Fundamental Limitation Single Label Dialect Classification Arabic Dialect Identification

October 11, 2023

Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations
Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, Ming Yin
Large Language Model Synthetic Data Text Classification Full Potential Synthetic Data Generation Fundamental Limitation Text Classification Model

October 5, 2023

September 20, 2023

September 15, 2023

September 13, 2023

Limitations of Face Image Generation
Harrison Rosenberg, Shimaa Ahmed, Guruprasad V Ramesh, Ramya Korlakai Vinayak, Kassem Fawaz
Generative Model Image Generation Text to Image Diffusion Model Fundamental Limitation Face Generation

August 31, 2023

Laplacian-Former: Overcoming the Limitations of Vision Transformers in Local Texture Detection
Reza Azad, Amirhossein Kazerouni, Babak Azad, Ehsan Khodapanah Aghdam, Yury Velichko, Ulas Bagci, Dorit Merhof
Vision Transformer Attention Mechanism Fundamental Limitation Self Attention Layer Efficient Attention Laplacian Canonization

August 12, 2023

Distributionally Robust Optimization and Invariant Representation Learning for Addressing Subgroup Underrepresentation: Mechanisms and Limitations
Nilesh Kumar, Ruby Shrestha, Zhiyuan Li, Linwei Wang
Fundamental Limitation Invariant Representation Functional Mechanism Distributionally Robust Optimization Underrepresented Group Subgroup Performance

July 27, 2023

Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, Tony Wang, Samuel Marks, Charbel-Raphaël Segerie, Micah Carroll, Andi Peng, Phillip Christoffersen, Mehul Damani, Stewart Slocum, Usman Anwar, Anand Siththaranjan, Max Nadeau, Eric J. Michaud, Jacob Pfau, Dmitrii Krasheninnikov, Xin Chen, Lauro Langosco, Peter Hase, Erdem Bıyık, Anca Dragan, David Krueger, Dorsa Sadigh, Dylan Hadfield-Menell
Large Language Model Reinforcement Learning Human Feedback AI System Fundamental Limitation Open Problem Safe AI

Fundamental Limitation

Papers

The Obscure Limitation of Modular Multilingual Language Models

On the Potential and Limitations of Few-Shot In-Context Learning to Generate Metamorphic Specifications for Tax Preparation Software

Exploring Machine Learning Models for Federated Learning: A Review of Approaches, Performance, and Limitations

On Retrieval Augmentation and the Limitations of Language Model Training

When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations

Worst-case Performance of Popular Approximate Nearest Neighbor Search Implementations: Guarantees and Limitations

AI Alignment and Social Choice: Fundamental Limitations and Policy Implications

Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification

Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations

How toxic is antisemitism? Potentials and limitations of automated toxicity scoring for antisemitic online content

Exploring DINO: Emergent Properties and Limitations for Synthetic Aperture Radar Imagery

Potential and limitations of random Fourier features for dequantizing quantum machine learning

Examining the Limitations of Computational Rumor Detection Models Trained on Static Datasets

Limitations in odour recognition and generalisation in a neuromorphic olfactory circuit

On some limitations of data-driven weather forecasting models

Understanding the limitations of self-supervised learning for tabular anomaly detection

Limitations of Face Image Generation

Laplacian-Former: Overcoming the Limitations of Vision Transformers in Local Texture Detection

Distributionally Robust Optimization and Invariant Representation Learning for Addressing Subgroup Underrepresentation: Mechanisms and Limitations

Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback