Rel2Traj: Inter-Source Relation-Aware Trajectory Generation for Text-To-Spatial-Audio
Organizations: Waseda University, Tokyo, Japan · University of Houston, Houston, USA
Abstract
"A car drives from front left to front right while a dog follows it." Such a description specifies no particular trajectory, yet imposes a constraint on the joint motion of both sources that must hold at every frame. Existing text-to-spatial-audio methods can position and move each sound source according to its description, but none of them ensures or verifies that such inter-source relations hold. We therefore propose Rel2Traj, which parses the description into sources, relations, and anchors, jointly generates the trajectories of all sources using a relation-conditioned diffusion model, and finally renders sounds from a pretrained text-to-audio model along these trajectories to produce binaural audio. For training and evaluation, we construct a synthetic dataset from a controlled language of source motions and inter-source relations. A single frame-wise relational geometry defines both our evaluation metric, the Relation Satisfaction Rate (RSR), and an auxiliary training loss. On motion-relation combinations held out from training, Rel2Traj maintains all relation types, whereas every baseline fails on at least one. A listening test further confirms that listeners recognize the described relations. To our knowledge, this is the first text-to-spatial-audio work to model relations among moving sources as explicit geometric constraints.
Figures & tables
| RSR | Jerk | Div. | ||
|---|---|---|---|---|
| System | val | comp | (m) | |
| Ground truth (ref.) | 0.996 | 0.995 | 0.5 | – |
| Rule-based solver | 0.963 | 0.919 | 8.0 | single |
| LLM-direct | 0.610 | 0.530 | 9.7 | single |
| Indep. per-source | 0.896 | 0.869 | 9.2 | 1.56 |
| Rel2Traj | 0.987 | 0.983 | 9.8 | 0.44 |
| RSR | Div. | ||||
|---|---|---|---|---|---|
| Variant | val | comp | follow | behind | (m) |
| Rel2Traj (full) | 0.987 | 0.983 | 0.992 | 0.780 | 0.44 |
| w/o | 0.988 | 0.987 | 0.991 | 0.923 | 0.45 |
| w/o coupling | 0.983 | 0.969 | 0.692 | 0.929 | 0.45 |
| w/o anchors | 0.952 | 0.938 | 0.999 | 0.053 | 1.30 |
| Criterion | Test | Result |
|---|---|---|
| Rel2Traj chance | -test | 86.0% , |
| pts | TOST | pts, |
| ABX chance | binomial | 46.9%, |
Spatial Audio
Spatial audio studies how sound is captured, represented, generated, and perceived with direction, distance, motion, and room acoustics.
See all 118 papers