Task Utility
Task utility, in the context of machine learning and AI, focuses on evaluating the usefulness and effectiveness of algorithms and models in achieving specific tasks, often considering trade-offs with other factors like privacy, fairness, and computational efficiency. Current research emphasizes developing robust benchmarking methods and algorithms to measure and improve utility across diverse applications, including data synthesis, optimization problems, and human-AI collaboration, employing techniques like generative adversarial networks, spectral graph learning, and temporal point processes. Understanding and maximizing task utility is crucial for responsible AI development, ensuring that models not only perform well but also deliver practical value and align with ethical considerations in real-world deployments.
Papers
On the Utility of External Agent Intention Predictor for Human-AI Coordination
Chenxu Wang, Zilong Chen, Angelo Cangelosi, Huaping Liu
Assessing and Verifying Task Utility in LLM-Powered Applications
Negar Arabzadeh, Siqing Huo, Nikhil Mehta, Qinqyun Wu, Chi Wang, Ahmed Awadallah, Charles L. A. Clarke, Julia Kiseleva