Must AI Spell It Out? Pragmatic Comprehension of Alerts in a Time-Critical Task
Organizations: Saarland Informatics Campus, Saarland University, Germany
Abstract
Assistive systems typically tell users everything, often spending time and attention on what a user already knows, leaving less for what happens next. People, on the other hand, leave out what they expect the listener can supply. Can people use their prior knowledge to understand an incomplete message when the speaker is a machine though? We studied this pragmatic comprehension in a monitoring task controlling users' prior knowledge and the available time to detect critical situations. In a preregistered study (N=114), we compared a pragmatic planner that reasons over a model of the listener's knowledge to decide what to say, when, and how explicitly, with a baseline specifying problems fully in onset order. Participants resolved underspecified alerts almost as reliably as fully specified ones and sooner, detected more problems, and rated the pragmatic assistant helpful more often, with no increase in reported mental demand. Our findings show that people can recover what the system left implicit and that pragmatic communication can support effective assistance under the constraints of a changing world.
Figures & tables
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Scene | Critical events (onset frame; * briefed) | pp schedule (start frame, alert) | Baseline schedule (start frame, alert) |
|---|---|---|---|
| HA_nc3_1 | D1 Altitude (2); D2 Battery (2); D2 No-fly zone (2)* | 1 –; 2 Battery; 4 Altitude; 6 No-fly zone | 1 –; 2 D1 Altitude; 5 D2 Battery |
| HA_nc3_3 | D3 Altitude (1); D4 Wind speed (1); D4 Distance (1) | 1 D3 Altitude; 4 D4 Distance; 7 – | 1 D4 Distance; 4 D4 Wind speed; 7 beep |
| HA_nc3_4 | D1 Rotor (1); D2 Rotor (2); D2 Battery (3) | 1 Drone 1; 3 D2 Battery; 6 Rotor | 1 D1 Rotor; 4 D2 Rotor; 7 beep |
| HA_nc3_6 | D1 Wind speed (2); D3 Rotor (2); D4 Altitude (2) | 1 –; 2 Altitude; 4 D1 Wind speed; 7 beep | 1 –; 2 D1 Wind speed; 5 D3 Rotor |
| HA_nc3_7 | D2 Distance (2); D3 No-fly zone (3); D4 Wind speed (4) | 1 –; 2 Distance; 4 D4 Wind speed; 7 beep | 1 –; 2 D2 Distance; 5 D3 No-fly zone |
| HA_nc4_1 | D1 Altitude (1); D2 Battery (1); D2 Distance (1); D4 Wind speed (1)* | 1 D2 Battery; 4 Altitude; 6 Wind speed | 1 D1 Altitude; 4 D2 Battery; 7 beep |
| Analysis (unit; model) | Status | Estimate [95% CI] | Test | Registration: prediction; one-sided ; outcome | |
|---|---|---|---|---|---|
| Assistance | |||||
| Detection, pp baseline Event; logistic GLMM (M1) | Registered test | (OR 1.73) | H1b; pp higher; one-sided ; as registered | ||
| Detection under low awareness M1 contrast | Follow-up | (OR 2.28) | – | ||
| Detection under high awareness M1 contrast | Follow-up | (OR 1.31) | – | ||
| Detection, agent awareness M1 term | Registered model term | opposite to H2a’s prediction | |||
| Criterion (clicks on non-critical cells) Cell; probit GLMM | Registered companion | H1b companion | |||
| Term | Estimate | SE | 95% CI | Statistic | exp(estimate) [95% CI] | |
|---|---|---|---|---|---|---|
| M1: Critical event caught (event) | ||||||
| Intercept | ||||||
| Agent ( pp ) | ||||||
| Awareness (HA ) | ||||||
| Complexity (four events ) | ||||||
| Trial position (centred) | ||||||
Human-AI Interaction
Human-AI interaction studies how people and AI systems communicate, collaborate, make decisions, and shape one another over time.
See all 354 papers