Paper ID: 2212.01415

Measuring Competency of Machine Learning Systems and Enforcing Reliability

M. Planer, J. M. Sierchio, for BAE Systems

We explore the impact of environmental conditions on the competency of machine learning agents and how real-time competency assessments improve the reliability of ML agents. We learn a representation of conditions which impact the strategies and performance of the ML agent enabling determination of actions the agent can make to maintain operator expectations in the case of a convolutional neural network that leverages visual imagery to aid in the obstacle avoidance task of a simulated self-driving vehicle.

Submitted: Dec 2, 2022