When Does Test-Time Physical Diagnosis Pay?
Organizations: University of Southern California
Abstract
When a robot faces unfamiliar physical conditions, a common approach is to collect evidence about what changed and adapt. For such diagnosis to improve behavior, six ordered empirical conditions must hold: a meaningful reference, identifiability of the physical condition, use of the acquired evidence, decision value, selection value over a fixed alternative, and safe realization. We test this chain in controlled and public environments. In our controlled environments the frozen policy chooses correctly on every decision, but which evidence it buys is fixed by how the panels assign decisions to evidence strata. On three mechanisms absent from training, the frozen decoder fails its registered transfer gate. On all 96 decisions of the transfer panel, its full-trace input lies far outside its training range, and on such inputs it returns one answer whatever the trace contains. All 15 of its errors fall in the 30 decisions from which cheaper evidence was removed by design, a split hidden by aggregate accuracy. None of three primary public tasks passes its registered qualification gates. Before attributing a failure to how a frozen model uses evidence, evaluation should therefore check whether its deployment inputs lie inside its training range.
Figures & tables
| Task | Protocol | Ranking reversal | Winner diversity | Utility separation | Evidence recovery | Early sufficiency | No deletion | Decision |
| Door | pass | fail | pass | pass | fail | pass | n/c | not admitted |
| Drawer | pass | fail | fail | pass | fail | pass | n/c | not admitted |
| Push | pass | pass | pass | pass | fail | fail | n/c | not admitted |
| Ant | pass | pass | fail | pass | pass | pass | n/c | replication only |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Test-Time Adaptation
Test-time adaptation updates or recalibrates pretrained models at inference to handle distribution shifts, often without target labels.
See all 322 papers