cs.LGMay 9, 2026

Communicating Sound Through Natural Language

Authors: Emanuele RossiEmanuele Rodolà

Organizations: Sapienza University of Rome · Sapienza University of Rome / Paradigma

Abstract

Natural language is widely used to describe, prompt, and control audio systems, but rarely serves as the representation carrying audio itself. We introduce lexical acoustic coding (LAC), a framework in which pre-trained LLM sender and receiver agents transmit sound through natural language. Under fixed system prompts, the agents write their own analysis and synthesis code, communicating only through a lexical sentence, shared vocabulary, and optional symbolic music structure. The sender analyzes an input waveform into interpretable, non-learned acoustic descriptors, quantizes each with a feature-specific interval vocabulary, and verbalizes the lexical code as English. The receiver parses the sentence back into lexical-acoustic constraints and renders a waveform through closed-loop refinement. The transmitted text serves as both a rich caption and as the transport representation itself. We frame LAC as a finite-rate lossy quantizer, exposing trade-offs between vocabulary size, rate, and fidelity. Experiments on short sounds and symbolic music transfer show that plain text preserves measurable acoustic structure while remaining interpretable, editable, and native to LLM-mediated communication.

Explore similar work

CardsList
  1. LILAC: An Idempotent Neural Speech Codec

    Aug 6, 2026June Young Yi, Dongwook Lee, Jiheum Yeom +1Neural Audio CodecsLibrispeech

  2. Taming Long-form Text-to-Speech

    Sep 15, 2026Rongxiang Wang, Berkin Durmus, Aysegul Orhon +2Word Error Rate