Understanding human intent in complex multi-turn interactions remains a fundamental challenge in human-computer interaction and behavioral analysis. While existing intent recognition datasets focus mainly on single utterances or simple dialogues, real-world scenarios often involve sophisticated strategic interactions where participants must maintain complex deceptive narratives over extended periods. To address this gap, we introduce MISID, a comprehensive multimodal, multi-turn, and multi-participant benchmark for intent recognition. Sourced from high-stakes social strategy games, MISID features a fine-grained, two-tier multi-dimensional annotation scheme tailored for long-context discourse analysis and evidence-based causal tracking. Our systematic evaluation of state-of-the-art Multimodal Large Language Models (MLLMs) on MISID reveals critical deficiencies in complex scenarios, including text-prior visual hallucination, impaired cross-modal synergy, and limited capacity in chaining causal cues. Consequently, we propose FRACTAM as a baseline framework. Using a ``Decouple-Anchor-Reason'' paradigm, FRACTAM reduces text bias by extracting pure unimodal factual representations, employs two-stage retrieval for long-range factual anchoring, and constructs explicit cross-modal evidence chains. Extensive experiments demonstrate that FRACTAM enhances mainstream models' performance in complex strategic tasks, improving hidden intent detection and inference while maintaining robust perceptual accuracy. Our dataset is available at https://naislab.cn/datasets/MISID.
Figures & tables
Figure 1. An overview of the MISID benchmark. (Top) A multi-participant strategic dialogue timeline exhibiting hidden tactics. (Bottom) Our multi-dimensional annotation scheme and fact-based reasoning paradigm for deducing hidden intents. Top: A multi-turn interaction timeline where participants engage in deceptive strategies, such as identity theft and bluffing, to mask their true alignments. Bottom: The multi-dimensional annotation scheme of MISID. It systematically progresses from utterance-level micro-states (e.g., emotion, subjectivity) and turn-level discourse analysis (e.g., modal inconsistency, key events) to a comprehensive causal reasoning chain, guiding models to decipher hidden intents from long-contextual clues.
Dataset
Depth
Text
Audio
Video
CSE
FCA
Length
MCIC ( Yuan et al., 2022 )
Explicit
✓
✗
✗
✗
✗
10-30
MSAIRS ( Shi et al., 2025 )
Explicit
✓
✗
✗
✗
✗
1-10
SLURP ( Bastianelli et al., 2020 )
Explicit
✓
✓
✗
✗
✗
1-10
FSC ( Qian et al., 2021 )
Explicit
✓
✓
✗
✗
✗
1-10
MINDS-14 ( Gerz et al., 2021 )
Explicit
✓
✓
✗
✗
✗
1-10
MIntRec ( Zhang et al., 2022 )
Explicit
✓
✓
✓
✗
✗
1-10
Table 1. Comparison of MISID with existing multimodal dialogue and intent recognition benchmarks. Depth: Literal (Explicit) or Underlying (Implicit) Intentions. CSE: Complex Strategic Environment. FCA: Fact-based Causal Annotation. Length: Dialogue Turn Ranges.
Dimension
Annotation Distribution
Total Participants
15
Subjectivity: Subj
3,259
Avg. Participants
12
Subjectivity: Obj
703
Role Instances
120
Emotion: Calm
2802
Total Utterances
3,962
Emotion: Others
1160
Total Duration
9.15 hours
Deception: Truth
2191
Max Turns
555
Deception: Lie
1771
Table 2. Dataset Statistics. Physical statistics (left) and multimodal annotation distributions (right) of the MISID dataset. VAD: Voice Activity Detection. Inconsistency: Cross-modal Incongruence During Interactions.
Figure 2. Distribution of multi-dimensional annotations in the MISID dataset. The x-axis denotes the annotation score, and the y-axis represents the occurrence frequency. Description: A set of four bar charts presented in a 2x2 grid, illustrating the statistical distribution of annotations. For all charts, the vertical axis represents count and the horizontal axis ranges from 1 to 5. \parThe top-left chart, Emotion Intensity, shows a strong right-skewed or L-shaped distribution, where score 1 has the highest count (approx. 2000) and counts decrease significantly for subsequent scores. \parThe top-right chart, Subjectivity, depicts a complex distribution with peaks at 1 (approx. 500) and 5 (approx. 1800), with low counts for scores 2, 3, and 4, indicating an overall increase after an initial moderate count. \parThe bottom-left chart, Confidence, displays a complex, multimodal shape with prominent peaks at score 2 (approx. 1000) and particularly score 5 (approx. 1200), while scores 1, 3, and 4 have lower values. \parThe bottom-right chart, Modality Inconsistency, features an L-shaped distribution with a dominant peak at score 1 (approx. 2300), followed by very low counts for scores 2, 3, and 4, and a secondary moderate peak at score 5 (approx. 1300).
Figure 3. Emotion category distribution (left) and speech duration distribution (right) in MISID dataset. Figure 4 contains two subplots. The left subplot is a donut chart showing the distribution of emotion categories, heavily dominated by "Calm" (70.7%), followed by "Disgust" (15.6%), and minor percentages for Joy, Anger, Surprise, Sadness, and Fear. The right subplot is a radar chart comparing the average performance of VLLM, LLM, and FRACTAM across eight metrics (ESA, EIS, SA, IJA, IRS, LDA, LRS, HIS). The red polygon representing FRACTAM encompasses the largest area, indicating superior performance across all evaluation dimensions compared to the baselines.
Figure 4. Overall architecture of the FRACTAM framework. The pipeline standardizes multimodal inputs into objective text, retrieves historical evidence via dual-stage hybrid search, and constructs explicit logical chains to deduce underlying intents for dual-phase evaluation. A flowchart illustrating the FRACTAM framework, divided into four sequential stages from left to right. \parUnimodal Fact Decoupling and Textualization: Text, Visual, and Audio signals are independently processed by Multimodal Large Models under constrained decoupling to form a Unified Fact Set, which is then archived in a Historical Fact Repository. \parHybrid Long-range Fact Anchoring: A two-phase retrieval process. Phase 1 performs a Dual Track Rough Recall combining lexical and semantic features via Reciprocal Rank Fusion (RRF) to generate a Top-M candidate subset. Phase 2 uses a Cross-Encoder to rerank this subset into Top-K relevant facts. \parChain-of-Evidence Causal Reasoning: The current fact set and Top-K retrieved facts prompt an Inference Model to construct a Causal Chain of Thought (Step 1). This chain, guided by a system prompt, drives the final evidence-based reasoning result (Step 2). \parDual-phase Evaluation Task: The final outputs address Subtask 1 (Ground Truth Extraction for emotion and subjectivity) and Subtask 2 (Fact-based Intent Reasoning for role perception, deception detection, and hidden intent).
Model
Emotion State Accuracy
Emotion Intensity Score
Subjectivity Accuracy
Identity Judgment Accuracy
Identity Reasoning Score
Lie Detection Accuracy
Lie Reasoning Score
Hidden Intent Inference Score
VideoLLM
GPT-4o ( Hurst et al., 2024 )
76.43
73.75
77.14
46.15
43.77
50.16
45.77
39.77
Grok-4-Fast ( xAI, 2025 )
86.21
79.89
90.80
53.37
41.88
53.82
39.46
32.69
Qwen3-VL-235B-A22B ( Bai et al., 2025 )
79.05
76.19
87.14
28.85
31.69
42.31
31.46
21.69
Qwen3-VL-Flash
69.23
46.70
87.91
46.15
41.54
46.15
41.58
28.27
Qwen3-VL-Max
74.05
75.57
91.22
30.77
34.85
46.15
35.54
23.42
Qwen3-VL-Plus
72.83
61.78
90.58
15.38
27.69
34.62
34.77
23.27
Table 3. Performance of LLMs on various tasks. The scores are reported in percentage (%). The best and the second best results are denoted by pink and yellow. LLM implies the exclusive use of the text modality. FRACTAM refers to LLMs assisted by the FRACTAM framework.