Visual Speaker Recognition

Visual speaker recognition uses facial appearance and motion, especially lip movements, to identify who is speaking, recognize speech, and associate faces with voices. Methods include Conformer and transformer models, pose-conditioned feature modulation, event-based motion encoders, and audio-visual fusion, evaluated on datasets such as LRS2, LRS3, and DVS-Lip as well as cross-language identity benchmarks. These systems can help when audio is noisy or unavailable, and lightweight or neuromorphic designs target real-time use on edge devices. A key tension is whether models rely on visual identity and articulation or exploit shortcuts such as language, word frequency, and gender; performance also varies with pose, lighting, and individual speakers.