Fundamental Limitation

Fundamental limitations in artificial intelligence research currently focus on identifying and addressing bottlenecks in model capabilities and performance. Active research areas include exploring the limitations of large language models (LLMs) in reasoning, particularly compositional abilities and handling complex tasks; analyzing the inherent quadratic time complexity of transformer architectures and the challenges of developing subquadratic alternatives; and investigating the impact of data quality and size on model performance and safety. Understanding these limitations is crucial for improving the reliability, safety, and efficiency of AI systems and for developing more robust and generalizable models across various applications.

Papers

February 3, 2024

A Closer Look at the Limitations of Instruction Tuning
Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Ramaneswaran S, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, Dinesh Manocha
Large Language Model Instruction Tuning Fundamental Limitation Open Domain Glance Annotation LoRA Fine Tuning Instruction Response Pair

February 2, 2024

Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions
Pouya Pezeshkpour, Eser Kandogan, Nikita Bhutani, Sajjadur Rahman, Tom Mitchell, Estevam Hruschka
Large Language Model Technical Challenge Complex Reasoning Multi Agent System Reasoning Capability Fundamental Limitation Human Centric Real World Task

January 26, 2024

On the Limitations of Markovian Rewards to Express Multi-Objective, Risk-Sensitive, and Modal Tasks
Joar Skalse, Alessandro Abate
Reinforcement Learning Fundamental Limitation Risk Sensitive Multi Objective Reinforcement Learning Procedural Activity Markovian Reward

January 22, 2024

Hallucination is Inevitable: An Innate Limitation of Large Language Models
Ziwei Xu, Sanjay Jain, Mohan Kankanhalli
Large Language Model Ground Truth Fundamental Limitation Content Hallucination Mitigating Hallucination

January 15, 2024

A Study on Large Language Models' Limitations in Multiple-Choice Question Answering
Aisha Khatun, Daniel G. Brown
Study Feature Fundamental Limitation Multiple Choice Open Source Model Choice Question Answering AI Task

January 8, 2024

Limitations of Data-Driven Spectral Reconstruction -- Optics-Aware Analysis and Mitigation
Qiang Fu, Matheus Souza, Eunsue Choi, Suhyun Shin, Seung-Hwan Baek, Wolfgang Heidrich
General Analysis Fundamental Limitation Spectral Feature Faulty Negative Mitigation Hyperspectral Imaging Spectral Imaging Spectral Signature Spectral Reconstruction

January 2, 2024

Fundamental Limitation of Semantic Communications: Neural Estimation for Rate-Distortion
Dongxu Li, Jianhao Huang, Chuan Huang, Xiaoqi Qin, Han Zhang, Ping Zhang
Semantic Communication Fundamental Limitation Rate Distortion Neural Estimator Rate Distortion Function

December 6, 2023

Invariance & Causal Representation Learning: Prospects and Limitations
Simon Bing, Jonas Wahl, Urmi Ninad, Jakob Runge
Causal Model Fundamental Limitation Alpha Invariance Causal Representation Learning Visionary ProSpect Latent Causal Causal Representation Causal Variable

December 5, 2023

Inherent limitations of LLMs regarding spatial information
He Yan, Xinyao Hu, Xiangpeng Wan, Chengyu Huang, Kai Zou, Shiqi Xu
Medical LLM Fundamental Limitation Spatial Reasoning Spatial Information Spatial Understanding Interactive Navigation

November 30, 2023

November 23, 2023

Expanding the deep-learning model to diagnosis LVNC: Limitations and trade-offs
Gregorio Bernabé, Pilar González-Férez, José M. García, Guillem Casas, Josefa González-Carrillo
Deep Learning Model Fundamental Limitation Left Ventricular Nonlinear Elasticity

November 21, 2023

November 20, 2023

On the Potential and Limitations of Few-Shot In-Context Learning to Generate Metamorphic Specifications for Tax Preparation Software
Dananjay Srinivas, Rohan Das, Saeid Tizpaz-Niari, Ashutosh Trivedi, Maria Leonor Pacheco
Context Learning Full Potential Fundamental Limitation First Order Metamorphic Testing Metamorphic Relation

November 17, 2023

Exploring Machine Learning Models for Federated Learning: A Review of Approaches, Performance, and Limitations
Elaheh Jafarigol, Theodore Trafalis, Talayeh Razzaghi, Mona Zamankhani
Machine Learning Model Constructive Approach System Performance Fundamental Limitation Privacy Preserving Machine Learning Federated Learning Application Learning Based Point

November 16, 2023

On Retrieval Augmentation and the Limitations of Language Model Training
Ting-Rui Chiang, Xinyan Velocity Yu, Joshua Robinson, Ollie Liu, Isabelle Lee, Dani Yogatama
Language Model Information Retrieval Fundamental Limitation Softmax Function Retrieval Augmentation Language Model Generalization

October 30, 2023

When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations
Aleksandar Petrov, Philip H. S. Torr, Adel Bibi
Fine Tuning Context Learning Theoretical Understanding Prompt Tuning Fundamental Limitation Capability Evolution Attention Pattern

October 29, 2023

Worst-case Performance of Popular Approximate Nearest Neighbor Search Implementations: Guarantees and Limitations
Piotr Indyk, Haike Xu
Nearest Neighbor Fundamental Limitation Formal Guarantee Approximate Nearest Neighbor Search Graph Based Approximate Nearest Neighbor Worst Case Performance

October 24, 2023

AI Alignment and Social Choice: Fundamental Limitations and Policy Implications
Abhilash Mishra
AI Agent Fundamental Limitation AI Alignment Social Choice Agent Alignment Human AI Alignment Policy Recommendation

Fundamental Limitation

Papers

A Closer Look at the Limitations of Instruction Tuning

Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions

On the Limitations of Markovian Rewards to Express Multi-Objective, Risk-Sensitive, and Modal Tasks

Hallucination is Inevitable: An Innate Limitation of Large Language Models

A Study on Large Language Models' Limitations in Multiple-Choice Question Answering

Limitations of Data-Driven Spectral Reconstruction -- Optics-Aware Analysis and Mitigation

Fundamental Limitation of Semantic Communications: Neural Estimation for Rate-Distortion

Invariance & Causal Representation Learning: Prospects and Limitations

Inherent limitations of LLMs regarding spatial information

Which way is `right'?: Uncovering limitations of Vision-and-Language Navigation model

Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web

Expanding the deep-learning model to diagnosis LVNC: Limitations and trade-offs

Fair Enough? A map of the current limitations of the requirements to have fair algorithms

The Obscure Limitation of Modular Multilingual Language Models

On the Potential and Limitations of Few-Shot In-Context Learning to Generate Metamorphic Specifications for Tax Preparation Software

Exploring Machine Learning Models for Federated Learning: A Review of Approaches, Performance, and Limitations

On Retrieval Augmentation and the Limitations of Language Model Training

When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations

Worst-case Performance of Popular Approximate Nearest Neighbor Search Implementations: Guarantees and Limitations

AI Alignment and Social Choice: Fundamental Limitations and Policy Implications