Existing approaches to evasion detection in earnings calls focus on textual transcripts, treating evasion as a single-dimensional phenomenon. We argue that evasion in spoken communication is inherently multidimensional: beyond what executives say, how they say it carries independent and complementary information. To study these dimensions jointly, we introduce DualEvasion, a benchmark for evasion detection across text and audio in earnings call Q&A. The benchmark contains 505 annotated question-answer pairs from 60 earnings calls, each with two independent labels: textual evasion (direct vs. evasive) and vocal cues operationalized as speaker confidence (confident vs. unconfident). Our experiments show that state-of-the-art multimodal models struggle to detect vocal confidence, particularly on unconfident responses. Our analysis suggests these models interpret acoustic cues in isolation rather than relative to each speaker's baseline. Providing speaker-level references yields modest improvements, but a substantial gap with human performance remains.
Figures & tables
Figure 1: The challenge of DualEvasion : an LLM detects textual evasion but misses vocal unconfidence in the same response.
SubjECTive-QA
Nuaimi
EvasionBench
DualEvasion
Period
’07–’21
’19–’22
Multi-yr
’23–’25
Audio
✗
✗
✗
✓
Evasion
△
✓
✓
✓
Label
Text
Text
Text
Text+Vocal
Annotation
Human
H+LLM
LLM
Domain Expert
Table 1: Comparison with earnings call Q&A datasets Pardawala et al. (2024) ; Nuaimi et al. (2025) ; Ma et al. (2026) . Existing datasets label evasion from text only; ours additionally captures vocal cues annotated by finance professionals. △ = partial.
Model
Direct/Confident
Evasive/Unconfident
Overall
Precision
Recall
F1
Precision
Recall
F1
Macro F1
Accuracy
Textual Evasion: Direct vs. Evasive
Qwen2-7B-Instruct
79.3
91.6
85.0
60.8
35.3
44.7
64.8
76.4
Llama-3.1-8B-IT
84.9
80.8
82.8
53.9
61.0
57.2
70.0
75.4
Gemini-2.5-Flash
93.2
74.8
83.0
55.5
85.3
67.2
75.1
77.6
Gemini-3-Flash
88.7
97.6
92.9
90.9
66.2
76.6
84.7
89.1
Table 2: Zero-shot performance on textual evasion and vocal confidence detection. Results for commercial audio models are averaged over three runs, while all other results are from a single run. Bold indicates best, underline indicates second best per column within each section.
Text
Vocal
Count
%
Direct
Confident
302
59.8
Direct
Unconfident
67
13.3
Evasive
Confident
95
18.8
Evasive
Unconfident
41
8.1
Cross-modal disagreement
162
32.1
Table 3: Joint distribution of textual evasion and vocal confidence labels. Shaded rows indicate cross-modal disagreement.
Text
Vocal
LLM–Annotator
0.813
0.225
Annotator–Annotator
0.866
0.774
Table 4: Cohen’s κ agreement with human annotators, averaged over two annotators, and human–human agreement. GPT-5 is used for text and Gemini-3-Flash for vocal confidence.
Figure 2: Correlation between acoustic features and unconfident predictions for Gemini-2.5-Flash, the model with the largest gap, measured using absolute values and speaker-relative deviations (response value minus the speaker’s mean).
∣rraw∣
∣rcent∣
Gap ( p )
Models
Gemini-2.5-Flash
.195
.032
.164 (.005)
Gemini-3-Flash
.179
.045
.135 (.003)
GPT-Audio
.142
.030
.113 (.025)
Gemini-2.5-Flash-lite
.093
.013
.080 (.234)
Audio-Flamingo-3
.040
.015
.025 (.522)
Table 5: Mean absolute point-biserial correlation between acoustic features and the unconfident label, before ( ∣rraw∣ ) and after ( ∣rcent∣ ) speaker-centering. Gap =∣rraw∣−∣rcent∣ ; p -values from a cluster bootstrap over the 60 calls. Bold indicates gaps significant at p<.05 .
Model
Baseline
+Speaker Norm.
Δ F1
Gemini-2.5-Flash
38.5
43.2
+4.7
Gemini-3-Flash
38.7
41.6
+2.9
GPT-Audio
38.2
39.2
+1.0
Table 6: Effect of speaker-level normalization on unconfident F1. The model infers the speaker’s vocal baseline before classifying the target response.
Figure 3: R2 of text-only, audio-only, and combined regression models for post-earnings stock volatility across Q&A coverage thresholds. Coverage is the fraction of each call’s Q&A pairs included in DualEvasion ; at each threshold, only calls with coverage at or above that level are retained. Annotations indicate the number of retained calls ( n ); † : p<0.10 .
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Composition
Earnings calls
60
Q&A pairs
505
Unique tickers
49
Call period
2023–2025
Market cap range
56M–189B
Market Group
Appendix
Table 7: Dataset composition.
Med.
Min
Max
Response duration (sec)
66.9
0.2
296.9
Response length (tokens)
186
2
918
Q&A pairs per call
10
4
10
Total audio duration
10.9 hrs
Appendix
Table 8: Response duration and length statistics.
A1
A2
A3
A4
A5
A1
–
.915
.706
.651
.917
A2
–
.623
.651
.834
A3
–
.442
.713
A4
–
.662
A5
–
Appendix
Table 9: Pairwise Cohen’s κ for vocal confidence labels on the validation subset ( N = 52).
Gem-2.5
Gem-3.0
GPT-Aud
Gem-2.5
–
.483
.309
Gem-3.0
–
.407
GPT-Aud
–
Appendix
Table 10: Inter-model agreement on vocal confidence labels measured by pairwise Cohen’s κ ( N = 52). Gem-2.5 denotes Gemini-2.5-Flash, Gem-3.0 denotes Gemini-3-Flash, and GPT-Aud denotes GPT-Audio.
Figure 4: Annotation interface for vocal confidence.
Model
Thr.
Conf. F1
Unconf. F1
Macro F1
Audio-Flamingo-3
0.40 M
82.7
30.0
56.4
0.95 U
2.5
35.5
19.0
Qwen2-Audio
0.05 M
65.2
34.3
49.7
0.45 U
24.4
36.7
30.5
Appendix
Table 13: Threshold sweep for the two open models that expose logits. M : threshold maximizing macro F1; U : threshold maximizing unconfident F1. Bold marks the maximized metric.
Model
Setting
Unconf. F1
Macro F1
Qwen2-Audio
zero-shot
36.2
28.0
fine-tuned
32.4
(-3.8)
48.6
(+20.6)
Audio-Flamingo-3
zero-shot
30.8
51.7
fine-tuned
34.7
(+3.9)
53.5
(+1.8)
VibeVoice-ASR
fine-tuned
9.9
48.8
Appendix
Table 14: Supervised fine-tuning with five-fold cross-validation. Values in parentheses indicate changes from zero-shot performance. Fine-tuning improves overall performance but does not consistently improve unconfident detection.
Model
Methods
Confident
Unconfident
Macro-F1
Precision
Recall
F1
Precision
Recall
F1
Gemini-2.5-Flash
baseline
83.7
72.0
77.4
32.0
48.4
38.5
57.9
fewshot( n=1 )
83.6
77.1
80.2
34.5
44.4
38.9
59.5
fewshot( n=2 )
81.1
85.6
83.3
34.1
26.9
30.1
56.7
fewshot( n=3 )
81.1
91.9
86.2
41.8
21.3
28.2
57.2
Gemini-3-Flash
baseline
84.0
67.8
75.0
30.7
52.5
38.7
56.8
Appendix
Table 15: Effect of few-shot same-speaker reference audio on vocal confidence detection. Bold indicates the best score and shaded indicates the lowest score within each model.
Table 16: Prompt for textual evasion classification.
Table 17: Prompt for vocal confidence classification (audio-only).
Table 18: Prompt for few-shot vocal confidence classification with same-speaker reference audio.
Table 19: Prompt for speaker-normalized vocal confidence classification with explicit speaker-baseline inference.
Earnings conference calls are a primary channel through which managers disclose information under analyst scrutiny. Prior work has linked vocal and lexical cues to future adverse outcomes, but often pools features over an entire call and underuses the interactive structure of Q&A. We propose a two-branch late-fusion framework for detecting managerial evasiveness as a predictor of extrinsic SEC events (primarily late filings): (i) an LLM-as-a-judge that maps Q&A text to an interpretable call-level vector X_text via a structured binary rubric, and (ii) a frozen conversational encoder whose temporal hidden states are read by a DeepVoice-style sequential reader to produce an audio representation h. Late fusion of (X_text, h) yields a call-level risk score p. On n=1,039 calls (212 late filings) with firm-grouped 5-fold CV, fusion reaches AUROC approx. 0.89, versus 0.55 for the text judge and 0.71 for duration alone. These results show that conversational audio dynamics encode managerial evasiveness beyond lexical content and call length, yielding a stronger early-warning signal of adverse SEC outcomes.
Huizhong Chen, Huan Zhang
Beijing Normal-Hong Kong Baptist University · Individual researcher
We introduce Earnings25, a finance-domain benchmark for evaluating automatic speech recognition (ASR) on English-language earnings calls under realistic conditions. Earnings25 comprises two complementary test sets: (i) testset-full, 498 hours of full English-language S&P 500 earnings calls from Q4 2025, and (ii) testset-segmented, a 46-hour industry-balanced set of 290 segments sampled from English-language U.S. earnings calls in 2025. The benchmark provides aligned transcripts and structured metadata, including speaker roles, industry labels, and call structure, enabling speaker- and industry-aware evaluation beyond aggregate word error rate (WER). We report reproducible baselines for Whisper and Parakeet-TDT using standardized scoring.
In computational paralinguistics, detecting cognitive load and deception from speech signals is a heavily researched domain. Recent efforts have attempted to apply these acoustic frameworks to corporate earnings calls to predict catastrophic stock market volatility. In this study, we empirically investigate the limits of acoustic feature extraction (pitch, jitter, and hesitation) when applied to highly trained speakers in in-the-wild teleconference environments. Utilizing a two-stream late-fusion architecture, we contrast an acoustic-based stream with a baseline Natural Language Processing (NLP) stream. The isolated NLP model achieved a recall of 66.25% for tail-risk downside events. Surprisingly, integrating acoustic features via late fusion significantly degraded performance, reducing recall to 47.08%. We identify this degradation as Acoustic Camouflage, where media-trained vocal regulation introduces contradictory noise that disrupts multimodal meta-learners. We present these findings as a boundary condition for speech processing applications in high-stakes financial forecasting.
Dhruvin Dungrani, Disha Dungrani
Department of Information Systems, Independent Researchers