PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation

Tianxin Xie; Wentao Lei; Kai Jiang; Guanjie Huang; Pengfei Zhang; Chunhui Zhang; Fengji Ma; Haoyu He

cs.SDDec 30, 2025

PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation

Authors: Tianxin Xie, Wentao Lei, Kai Jiang, Guanjie Huang, Pengfei Zhang, Chunhui Zhang, Fengji Ma, Haoyu He,

Organizations: HKUST(GZ) · Tencent

Abstract

Text-to-audio-video (T2AV) generation is central to applications such as filmmaking and world modeling. However, current models often fail to produce physically plausible sounds. Previous benchmarks primarily focus on audio-video temporal synchronization, while largely overlooking explicit evaluation of audio-physics grounding, thereby limiting the study of physically plausible audio-visual generation. To address this issue, we present PhyAVBench, the first benchmark that systematically evaluates the audio-physics grounding capabilities of T2AV, image-to-audio-video (I2AV), and video-to-audio (V2A) models. PhyAVBench offers PhyAV-Sound-11K, a new dataset of 25.5 hours of 11,605 audible videos collected from 184 participants to ensure diversity and avoid data leakage. It contains 337 paired-prompt groups with controlled physical variations that drive sound differences, each grounded with an average of 17 videos and spanning 6 audio-physics dimensions and 41 fine-grained test points. Each prompt pair is annotated with the physical factors underlying their acoustic differences. Importantly, PhyAVBench leverages paired text prompts to evaluate this capability. We term this evaluation paradigm the Audio-Physics Sensitivity Test (APST) and introduce a novel metric, the Contrastive Physical Response Score (CPRS), which quantifies the acoustic consistency between generated videos and their real-world counterparts. We conduct a comprehensive evaluation of 17 state-of-the-art models. Our results reveal that even leading commercial models struggle with fundamental audio-physical phenomena, exposing a critical gap beyond audio-visual synchronization and pointing to future research directions. We hope PhyAVBench will serve as a foundation for advancing physically grounded audio-visual generation. Prompts, ground-truth, and generated video samples are available at https://github.com/imxtx/PhyAVBench.

Figures & tables

Figure 1. The proposed audio-physics sensitivity test.

	Acoustic Physical Dimensions						Controlled Setting	Newly Collected	#GT Videos per Prompt	Evaluation Metric
	Sound Source Mechanism	Fluid & Aerodynamics	Sound Propagation	Observer Physics	Time & Causality	Complex Coupling	Controlled Setting	Newly Collected	#GT Videos per Prompt	Evaluation Metric
TAVGBench ( Mao et al., 2024 )	✗	✗	✗	✗	✗	✗	✗	✗	1	AV-Align
SAVGBench ( Shimada et al., 2024 )	✗	✗	✗	✗	✓	✗	✗	✗	-	AV & Spatial-Align
Verse-Bench ( Wang et al., 2025a )	✗	✗	✗	✗	✗	✗	✗	✓	1	AV-Align
JavisBench ( Liu et al., 2026a )	✗	✗	✗	✗	✗	✗	✗	✓ (partial)	1	AV-Align
VABench ( Hua et al., 2025 )	✓	✓	✗	✗	✓	✗	✗	✗	0	AV & Stereo Align

Table 1. Comparison with the existing audio-video generation benchmarks.

Figure 2. The data curation pipeline of PhyAVBench.

Model	AV Quality			Semantic Consistency				AV Synchronization				Physical
Model	FVD $\downarrow$	KVD $\downarrow$	FAD $\downarrow$	TV-IB $\uparrow$	TA-IB $\uparrow$	CLIP $\uparrow$	CLAP $\uparrow$	AV-IB $\uparrow$	DeSync $\downarrow$	AVH $\uparrow$	Javis $\uparrow$	CPRS $\uparrow$
Text-to-Audio-Video (T2AV)
Commercial (Closed-source)
Sora 2 ( OpenAI, 2025 )	578.70	72.04	4.25	0.3415	0.2028	0.2713	0.3592	0.2137	0.0370	0.2118	0.1803	0.4512
Wan 2.6 ( Wanxiang, 2025 )	656.62	108.25	1.48	0.3434	0.1820	0.2792	0.3858	0.2143	0.2645	0.2141	0.1846	0.4120
Seedance 1.5 Pro ( Seedance et al., 2025 )	968.03	315.38	3.09	0.3274	0.2258	0.2663	0.3600	0.2794	0.0240	0.2754	0.2412	0.4044

Table 2. Quantitative evaluation on PhyAVBench. We categorize models by task (T2AV, I2AV, V2A) and source type. Bold and underlined values denote the best and second-best results within each task category, respectively.

Figure 3. Qualitative comparison of generated audio-visual results on PhyAVBench.

Figure 4. The audio-physics grounding performance of SOTA T2AV, I2AV, and V2A models measured by CPRS.

Metric	$-\mathrm{FAD}$	TA-IB	CLAP	AV-IB	$-\mathrm{DeSync}$	AVH	Javis	CPRS
P.C.	0.51	0.77	0.86	0.76	0.60	0.72	0.71	0.92

Table 3. Pearson correlation (P.C.) across T2AV models, using per-model PVR-MOS means. For lower-is-better metrics, the score is defined as

-\mathrm{DeSync}

and

-\mathrm{FAD}

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Major Dimension	m Code	Sub Category	c Code	Test Point	t Code	Example
Sound Source Mechanics	m01	Material Properties	c01	Hardness / Damping	t01	Wrench dropping on concrete vs. carpet
				Density	t02	Solid vs. hollow ball impact
				Surface Texture	t03	Scrapping wood vs. metal
		Object Geometry	c02	Size	t04	Dropping large vs. small bottoles
				Shape	t05	Tapping long tubes vs. flat plates
				Thickness	t06	Thin vs. thick steel plates

Appendix

Table 4. The fine-grained test points and illustrative examples of PhyAVBench.

Figure 6. Prompt groups and newly recorded video examples in PhyAV-Sound-11K.

Panel of Figure 7 — Figure 7. The duration distribution of PhyAV-Sound-11K.

Panel of Figure 8 — Figure 8. The difficulty distribution of PhyAV-Sound-11K.

Rank	Task	Model	CPRS $\pm$ std	Cos $\pm$ std	$p$ $\pm$ std	$f(p)$ $\pm$ std
01	I2AV	MOVA	0.3613 $\pm$ 0.1891	0.0853 $\pm$ 0.2335	0.172 $\pm$ 0.526	0.1799 $\pm$ 0.2841
02	I2AV	LTX	0.3285 $\pm$ 0.1544	0.0652 $\pm$ 0.1712	0.148 $\pm$ 0.579	0.1244 $\pm$ 0.2452
03	I2AV	Ovi	0.3198 $\pm$ 0.1429	0.0580 $\pm$ 0.1840	0.104 $\pm$ 0.386	0.1106 $\pm$ 0.2177
04	I2AV	UniVerse-1	0.2729 $\pm$ 0.0812	0.0084 $\pm$ 0.1172	0.018 $\pm$ 0.283	0.0416 $\pm$ 0.1236
01	T2AV	Sora 2	0.4512 $\pm$ 0.2149	0.1694 $\pm$ 0.2213	0.389 $\pm$ 0.580	0.3177 $\pm$ 0.3432
02	T2AV	Wan 2.6	0.4120 $\pm$ 0.2173	0.1209 $\pm$ 0.2261	0.275 $\pm$ 0.643	0.2635 $\pm$ 0.3455

Appendix

Table 5. CPRS ranking based on the LAION-CLAP embedding model.

Rank	Task	Model	CPRS $\pm$ std	Cos $\pm$ std	$p$ $\pm$ std	$f(p)$ $\pm$ std
01	I2AV	MOVA	0.3688 $\pm$ 0.2001	0.0501 $\pm$ 0.1925	0.128 $\pm$ 0.712	0.2125 $\pm$ 0.3274
02	I2AV	LTX	0.3125 $\pm$ 0.1422	0.0203 $\pm$ 0.1297	0.062 $\pm$ 0.485	0.1149 $\pm$ 0.2377
03	I2AV	Ovi	0.3005 $\pm$ 0.1269	0.0238 $\pm$ 0.1322	0.050 $\pm$ 0.394	0.0892 $\pm$ 0.2070
04	I2AV	UniVerse-1	0.2844 $\pm$ 0.1038	0.0061 $\pm$ 0.1066	0.010 $\pm$ 0.378	0.0657 $\pm$ 0.1711
01	T2AV	Sora 2	0.4288 $\pm$ 0.2214	0.1148 $\pm$ 0.1883	0.350 $\pm$ 0.756	0.3003 $\pm$ 0.3716
02	T2AV	Wan 2.6	0.3850 $\pm$ 0.1961	0.0766 $\pm$ 0.2145	0.286 $\pm$ 0.947	0.2317 $\pm$ 0.3206

Appendix

Table 6. CPRS ranking based on the ImageBind embedding model.

Rank	Task	Model	CPRS $\pm$ std	Cos $\pm$ std	$p$ $\pm$ std	$f(p)$ $\pm$ std
01	I2AV	MOVA	0.3677 $\pm$ 0.1972	0.0879 $\pm$ 0.2245	0.197 $\pm$ 0.666	0.1914 $\pm$ 0.3075
02	I2AV	LTX	0.3098 $\pm$ 0.1307	0.0602 $\pm$ 0.1834	0.096 $\pm$ 0.395	0.0895 $\pm$ 0.1961
03	I2AV	Ovi	0.2983 $\pm$ 0.0935	0.0760 $\pm$ 0.1701	0.085 $\pm$ 0.251	0.0586 $\pm$ 0.1318
04	I2AV	UniVerse-1	0.2720 $\pm$ 0.0794	0.0226 $\pm$ 0.1375	0.014 $\pm$ 0.218	0.0326 $\pm$ 0.1173
01	T2AV	Wan 2.6	0.3595 $\pm$ 0.1690	0.1179 $\pm$ 0.2207	0.162 $\pm$ 0.409	0.1600 $\pm$ 0.2519
02	T2AV	Kling-v2-6	0.3482 $\pm$ 0.1356	0.1347 $\pm$ 0.2054	0.185 $\pm$ 0.315	0.1291 $\pm$ 0.1962

Appendix

Table 7. CPRS ranking based on the CAV-MAE-Sync embedding model.

Metric	LAION-CLAP	ImageBind	CAV-MAE-Sync
P.C.	0.9230	0.9457	0.8927

Appendix

Table 8. Pearson correlation (P.C.) between CPRS metrics based on three embedding models and the PVR-MOS.

Figure 9. The example questionnaire for human evaluation.

Figure 10. The prompt for cross-model validation.

Figure 11. The prompt used to generate prompt groups based on the taxonomy of PhyAVBench.

Audio-Video Generation

Audio-video generation creates or edits synchronized sound and moving images from text, media, or other multimodal inputs.

See all 155 papers

Benchmarking Single-Factor Physical Video-to-Audio Generation
May 28, 2026Tingle Li, Siddharth Gururani, Kevin J. Shih +6Audio-Video Generation Audio-Visual Alignment
UC Berkeley · NVIDIA · University of Washington
AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation
Apr 9, 2026Ziwei Zhou, Zeyuan Lai, Rui Wang +6Audio-Video Generation Generative AI Evaluation
Fudan University · University of Science and Technology of China · Microsoft Research Asia
Do Joint Audio-Video Generation Models Understand Physics?
May 8, 2026Zijun Cui, Xiulong Liu, Hao Fang +8Physical Consistency in Video Generation Audio-Video Generation
University of Texas at Dallas · University of Washington · University of California, Los Angeles

May 28, 2026J/K move · Enter open · S save

Benchmarking Single-Factor Physical Video-to-Audio Generation

Tingle Li, Siddharth Gururani, Kevin J. Shih, Gantavya Bhatt+5

UC Berkeley · NVIDIA · University of Washington

Audio-Video Generation Audio-Visual Alignment

Open paper AskPDF arXiv

Generative video-to-audio (V2A) models produce highly plausible soundtracks, but it remains unclear whether they capture the underlying physical processes. Existing evaluations emphasize perceptual realism and overlook physical correctness under controlled interventions. In this paper, we introduce FlatSounds, a benchmark that audits the physical reasoning of V2A models through: 1) controlled counterfactual pairs in which a single physical factor is varied, and 2) single-video pattern tests that probe internal consistency and directional trends. These settings test whether the generated audio correctly reflects specific physical properties and timings. Our evaluation of state-of-the-art models reveals a consistent trade-off: models rely more on text captions than the visual stream to infer physics and semantics. Captions generally improve physical and semantic accuracy, but paradoxically degrade temporal alignment. Our results highlight the need to move beyond audio quality toward learning physical processes directly from pixels. Finally, we find that our physics-based metrics correlate strongly with human preference tests on our own data. Project webpage: https://research.nvidia.com/labs/cosmos-lab/flatsounds/

Abstract

Audio-Video Generation

Explore similar work

Benchmarking Single-Factor Physical Video-to-Audio Generation

AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation

Do Joint Audio-Video Generation Models Understand Physics?