Speech Prefix-Tuning with RNNT Loss for Improving LLM Predictions [2406.14701]