Reusing Multimodal Agents: Efficient Evaluation of Sound Designs in Games
Authors: Thai Nguyen Van, Ibrahim Khan, Mury Fajar Dewantoro, Pratch Suntichaikul, Hai Van Pham, Ruck Thawonmas
Organizations: Graduate School of Information Science and Engineering, Ritsumeikan University, Japan · College of Information Science and Engineering, Ritsumeikan University, Japan · Independent Researcher · School of Information and Communications Technology, Hanoi University of Science and Technology, Vietnam
This paper introduces a novel framework for efficiently evaluating in-game sound designs using multimodal learning and reinforcement learning. Sound design significantly influences gameplay and accessibility; however, objective evaluation methods, defined here as automated assessments based on agent performance rather than human ratings, often require retraining agents from scratch for each sound design. Our approach first pre-trains a multimodal encoder that encodes visual and auditory information into a shared latent space. A reinforcement learning agent is then trained using this latent representation. We fine-tune the multimodal encoder for each sound design while keeping the trained policy fixed, allowing us to isolate the effects of sound changes on agent behavior. To evaluate the proposed method, we conducted experiments on games from multiple genres, comparing the performance of blind agents (agents using only sound input) under various sound designs. Experimental results suggest that this approach can detect differences in agent performance caused by degraded or altered sound design. Moreover, we significantly reduce the computational cost compared to retraining the agent from scratch. Experimental results suggest that this approach provides a practical and scalable solution for evaluating in-game sound designs. The source code, data and supplementary document are available: https://github.com/thainv0212/tog-reusing-agents.
Figures & tables
Fig. 1: Our method pipeline and agent evaluation setting. Only audio is used in the blind setting, while both images and audio are used for the standard setting.
Fig. 3: Multimodal encoder architecture. Audio, Image, and fused modalities are processed by dedicated feature extractors and MLPs before converging into a shared Common MLP module.
Fig. 4: PENDULUM sound designs. (a) Design 1: three listeners positioned at the bottom left and right, and middle top. (b) Design 2: same as (a) but the three listeners are moved to the top left and right, and bottom middle. (c) Design 3: same as (b) but regions where the sound cue is indistinguishable are shaded. (d) Design 4: same as (b) but regions where the sound cue is indistinguishable are shaded.
Fig. 5: HYPERHOT sound designs. (a) Design 1: four listeners (white dots) receive distinct sound cues from enemies (yellow) and bullets (green/blue). (b) Design 2: same as (a), but listener positions are changed. (c) Design 3: enemies and bullets emit the same pitch, and only one listener remains at the bottom left. (d) Design 4: only two enemies and two bullets emit sound, enemies and bullets emit the same pitch, and two listeners remain at the bottom left and bottom right.
Fig. 6: VizDoom basic scenario sound designs. The listener is shown as a green dot at the bottom. (a) Design 1: enemy emits a sound cue whose volume decreases linearly with distance. (b) Design 2: same as (a) but the emitted volume decreases quadratically with distance. (c) Design 3: the player can only hear the enemy’s sound if it is on the left side of the player’s view. (d) Design 4: there is no sound cues indicating the presence of enemy.
Game
Design 1
Design 2
p -value
95% CI
Pendulum
-1.020 ± 0.033
-1.028 ± 0.048
0.273
[-0.016, 0.044]
Hyperhot
1.728 ± 0.183
1.780 ± 0.124
0.364
[-0.299, 0.160]
ViZDoom
0.798 ± 0.014
0.810 ± 0.011
0.076
[-0.027, 0.006]
TABLE I: Comparison of finetuned agent performance between the baseline (Design 1) and Design 2 across three games. Statistical significance is evaluated using the Mann-Whitney U test, and the 95% confidence intervals (CI) for the difference in medians are calculated using a two-sample independent percentile bootstrap method (number of resamples: 9999).
Game
Method
Design 2
Design 3
Design 4
PENDULUM
Ours
-1.028 ± 0.048
-1.382 ± 0.043
-2.464 ± 0.109
No fine-tuning
-5.897 ± 0.298
-5.807 ± 0.057
-5.437 ± 0.065
RL training under blind setting
-1.005 ± 0.091
-2.236 ± 0.068
-2.719 ± 0.244
Finetuned + Policy Training
-1.030 ± 0.090
-1.198 ± 0.047
-2.298 ± 0.152
HYPERHOT
Ours
1.780 ± 0.124
0.953 ± 0.088
0.904 ± 0.081
No fine-tuning
1.701 ± 0.157
0.877 ± 0.091
0.877 ± 0.091
TABLE II: Comparison of agent performance across Designs 2, 3, and 4 in different games. No fine-tuning: The multimodal encoder is trained only on Design 1 and is not fine-tuned.
Game
Method
D2 vs D3
D3 vs D4
D2 vs D4
Average Effect size
PENDULUM
Ours
1.8×10−4/0.84
1.8×10−4/0.84
1.8×10−4/0.84
0.84
RL training under blind setting
1.8×10−4/0.84
3.3×10−4/0.80
1.8×10−4/0.84
0.83
Finetuned + Policy Training
4.4×10−4/0.79
1.8×10−4/0.84
1.8×10−4/0.84
0.82
HYPERHOT
Ours
1.7×10−4/0.84
0.29/0.24
1.8×10−4/0.84
0.64
RL training under blind setting
4.5×10−2/0.44
1.8×10−4/0.84
1.8×10−4/0.84
0.70
Finetuned + Policy Training
1.9×10−2/0.52
2.4×10−4/0.82
1.8×10−4/0.84
0.73
TABLE III: Pairwise performance comparisons ( p -values/effect size) using Mann-Whitney U test of sound designs. D2: Design 2, D3: Design 3, D4: Design 4.
Game
Method
Samples
Trainable Parameters
Backward FLOPs
Training Steps
Total FLOPs
PENDULUM
Ours
20,000
9,664
0.25 GFLOP
70,499
17 TFLOP
RL training under blind setting
150,000
340,268
0.19 GFLOP
150,000
29 TFLOP
Finetuned + Policy Training
20,000 + 150,000
9,664 + 275,972
0.25 GFLOP + 0.36 GFLOP
70,499 + 150,000
17 TFLOP + 54 TFLOP
HYPERHOT
Ours
32,000
9,140,288
2.58 GFLOP
56,250
145 TFLOP
RL training under blind setting
2,000,000
9,248,220
4.8 GFLOP
2,000,000
9632 TFLOP
Finetuned + Policy Training
32,000 + 2,000,000
9,248,220 + 48,140
2.58 GFLOP + 0.024 GFLOP
56,250 + 2,000,000
145 TFLOP + 48 TFLOP
TABLE IV: Comparison of training cost in PENDULUM, HYPERHOT and Vizdoom.
Game
Sound Design
Image-Sound
Image-Joint
Sound-Joint
Average
PENDULUM
Design 1
1.00
1.00
1.00
1.00
Design 2
0.99
1.00
0.99
0.99
Design 3
0.83
0.98
0.87
0.89
Design 4
0.80
0.98
0.85
0.88
HYPERHOT
Design 1
0.68
0.44
0.52
0.55
Design 2
0.64
0.45
0.50
0.53
TABLE V: Latent representation similarities across sound designs in PENDULUM, HYPERHOT and VizDoom.
Game
Design 3
Design 4
Pendulum
-1.081 (+0.301)
-1.893 (+0.571)
Hyperhot
1.538 (+0.585)
1.073 (+0.169)
VizDoom
0.272 (+0.73)
-1.631 (+0.941)
TABLE VI: Performance of fine-tuned policy agents using Design 3 and Design 4 across three games. Values in parentheses denote the performance improvement relative to baseline agents where only the multimodal encoder was fine-tuned