cs.CLNov 3, 2026

Contextual Pictograms for Repairing Multilingual ASR Errors

Authors: Mariana Hernandez, Hector Luna, Carlos Bruno, Luis Ricardo Santos, Elena Simoes

Organizations: Department Signal Theory, Telematics and Communications, University of Granada · Brain, Mind, and Behavior Research Center (CIMCYC), University of Granada · Faculty of Psychology and Education Sciences, University of Coimbra · Institute of Intelligent Systems and Robotics, CNRS, Sorbonne Université

Abstract

Speech transcription errors can be difficult to recognize when competing word hypotheses are unfamiliar or visually similar. We describe an interface that combines alternative transcripts, replay and contextually checked pictograms for selected uncertain concepts. The system will initially support French and Spanish functional utterances. A counterbalanced study will assess whether pictorial feedback helps listeners recover the speaker's intended action beyond text alternatives alone. We will also quantify semantic mismatch and additional interaction time, since an inappropriate image can reinforce the wrong interpretation rather than repair it.

Explore similar work

CardsList
  1. SCRIBE: Diagnostic Evaluation and Rich Transcription Models for Indic ASR

    May 20, 2026Kavya Manohar, Arghya Bhattacharya, Kush Juvekar +1ASR EvaluationAutomatic Speech Recognition

  2. Text-Only LLM Correctors Break Correct Transcripts: A Repair-Damage Analysis of Audio Grounding in ASR Error Correction

    Oct 21, 2026Vasiliy Kudryavtsev, Kirill Borodin, Nikita Vasiliev +2LLM Self-CorrectionAutomatic Speech Recognition