While Multimodal Large Language Models (MLLMs) excel in general video understanding, their capability in fine-grained and motion-centric tasks remains limited. This limitation is particularly critical in micro-gesture recognition (MGR), where micro-gestures (MGs) - subtle, short-duration, and spatially localized human movements - serve as key discriminative signals for implicit affective analysis, yet are easily neglected following common prompting practices. Although MGR has been intensively studied by many discriminative approaches, the use of MLLMs for MGR is underexplored, with notably poor performance. We hypothesize that the motion-sensitive representation ability of MLLMs is constrained by their inherent single-pass forward inference, which can be substantially enhanced through carefully designed test-time guidance. Motivated by this, building on our prior findings regarding temporal insensitivity in Video LLMs, we diagnose zero-shot MGR errors in the Negative Log-Likelihood (NLL) space. We observe that MLLMs suffer from two bottlenecks: 1) insufficient localized evidence and 2) severe score biases driven by language and motion-agnostic appearances. Thus, we propose a novel test-time evidence calibration framework that improves both reasoning details and prediction reliability. Specifically, we introduce a tree search mechanism to progressively acquire localized, fine-grained visual evidence, coupled with a test-time calibration module to mitigate score biases. The multi-cue fusion module then integrates evidence from multiple cues without relying on a single cue for final prediction. Our framework achieves mean-class accuracies of 26.84% on iMiGUE and 22.10% on MA-52, significantly outperforming the Qwen2.5-VL baseline, which produces 16.15% and 10.20%, respectively. The code will be available at https://zero-melo.github.io/Zero-MELO.
Figures & tables
Figure 1. Comparison between existing methods and Zero-MELO. Given the input video clips and question, the target is to predict the correct label in MGR. Compared to the single-pass forward inference of other MLLMs, Zero-MELO can utilize the relevant local evidence and predict the correct label via calibration at inference time.
Figure 2. The overall algorithm of Zero-MELO. It mainly consists of three key modules: the cue-guided search (Sec. 3.3 ), the test-time calibration (Sec. 3.4 ), and the multi-cue fusion (Sec. 3.5 ). The global video input will first be fed to the MLLM for the global inference. Then, the model will acquire local evidence via the cue-guided tree search. Next, the test-time calibration will be conducted, and the calibrated scores will be yielded. Based on the global and local calibrated scores, the multi-cue fusion contributes to the final prediction.
Figure 3. The TAM visualization of a failure case for the gesture Pressing lips . The wrong prediction is Bulging face, deep breath . We show the token-wise activation maps for three successive generated tokens during free-form generation in (a), (b), and (c), respectively. Although the model suggests the GT in Top-5, the highlighted regions are not consistently grounded on the lip area or the most discriminative motion phase, indicating that the naive single-pass MLLM fails to capture compact motion-centric local evidence.
Figure 4. Failure analysis on iMiGUE of the prompt-based free-form generation. (a) The comparison of distributions of Qwen2.5-VL predictions and the ground-truths (GTs) of iMiGUE . Due to the space limitations, only the appearing frequency of labels is larger than 1% in either GT or Top-1 prediction is listed. (b) The comparison of the severity of the prediction-collapse across different vanilla MLLMs .
Method
iMiGUE
MA-52
mCA@1
mCA@5
MF1
mCA@1
mCA@5
MF1
Prompt-based Zero-shot
GPT-4o mini ( OpenAI, 2024 )
0.1113
0.3392
0.0699
0.0576
0.2362
0.0363
Gemini2.5-Flash ( Comanici et al., 2025 )
0.1917
0.3689
0.1367
0.1666
0.3072
0.1217
MiMo-VL-7B ( Team et al., 2025 )
0.2445
0.5643
0.1506
0.1869
0.3814
0.1042
InternVL3.5-8B ( Wang et al., 2025c )
0.2034
0.4358
0.0927
0.1796
0.3677
0.0914
Table 1. Zero-shot performance comparison on iMiGUE testset and MA-52 valid set.
Method
mCA@1
mCA@5
MF1
VQA
0.177
0.4201
0.0842
Label Scoring
0.0899
0.2545
0.0623
Ours
0.2210
0.4359
0.1221
Table 2. Comparison between our method and the SFT method on MA-52 dataset.
Figure 5. Case study of Zero-MELO. Each panel shows the global video clip, the zoomed-in view of the most supportive cue, and the trajectory of the prediction Top-1 across global inference, calibration, and final multi-cue fusion. The (a) , (b) , (c) correspond to the successful samples, while (d) is the failure case.
Method
mCA@1
mCA@5
MF1
Vanilla (ablation format)
0.0357
0.2084
0.0094
+ TS
0.0358
0.2629
0.0100
+ Cali (Blank)
0.2040
0.4943
0.0855
+ Cali (Aggregate)
0.2363
0.5436
0.1097
+ TS + Cali (Aggregate, Single-Cue)
0.2244
0.5342
0.1204
+ TS + Cali (Blank)
0.2131
0.5078
0.0924
Table 3. Ablation study of different components of Zero-MELO on iMiGUE dataset. Vanilla (ablation format) uses the unified paired-input format adopted by all component ablations, differing from Table 1 . TS, Cali, and F denote tree search, calibration, and fusion; Max and Average denote lowest-NLL cue selection and uniform cue averaging.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Model
Total clips
Clips with mismatched labels
iMiGUE
GPT-4o mini
4534
36 (0.79%)
iMiGUE
Gemini2.5-Flash
4534
280 (6.18%)
MA-52
GPT-4o mini
5586
441 (7.89%)
MA-52
Gemini2.5-Flash
5586
195 (3.49%)
Appendix
Table 4. Simple statistics of exact-name mismatches in free-form generation. A clip is counted if at least one returned top- k label is not an exact-string-matched label.
Dataset
Model
Representative misaligned labels
iMiGUE
GPT-4o mini
Touching or scratching jaw; Touching forehead; Touching or adjusting hair
iMiGUE
Gemini2.5-Flash
Holding back arms; Adjusting tie; Touch jaw; Touchng jaw
MA-52
GPT-4o mini
Bowling head; Scratching or touching head; Raising up; Touching face
MA-52
Gemini2.5-Flash
Scratching face; Touching face; Touching chest; Tipping head
Appendix
Table 5. Representative misaligned labels observed in prompt-based free-form generation.
Parameter
Qwen2.5-VL
InternVL3.5
MiMo-VL
τd
0.9
0.7
1.1
πg
0.5
0.5
0.5
πc
0.5
0.5
0.5
αy
0
0.85
0.97
whead
0.2
1.0
1.5
whand
1.0
0.8
1.12
Appendix
Table 6. Parameters of multi-cue fusion in iMiGUE.
Parameter
Qwen2.5-VL
τd
0.5
πg
0.5
πc
0.5
αy
0.25
whead
0.7
wupper−limbs
0.1
Appendix
Table 7. Parameters of multi-cue fusion in MA-52.
Label y
Label-Conditioned Cue Set C(y)
Label y
Label-Conditioned Cue Set C(y)
Turtle neck
{neck}
Folding arms
{body-hand,arms}
Bulging face, deep breath
{face}
Dusting off clothes
{body-hand,arms}
Touching hat
{head-hand}
Putting arms behind body
{arms}
Touching or scratching head
{head-hand}
Moving torso
{torso}
Touching or scratching forehead
{head-hand,face}
Sitting straightly
{torso}
Covering face
{head-hand,face}
Scratching or touching arms
{arms,hands}
Appendix
Table 8. Label-conditioned cue sets C(y) for iMiGUE.
Figure 6. A failure case of free-form generation, analogous to Fig. 3. The GT of this sample is Pressing lips . (a) : When only the original video clip is provided, the vanilla MLLM predicts Touching jaw . (b) : When additional local evidence of the mouth is provided, the vanilla MLLM will predict Pressing lips .
Figure 7. Class-wise effect of blank and lowrank calibration in NLL-based label scoring. Left: the ground-truth class share of samples that are misclassified by the vanilla model ( vanilla wrong ). Right: per-class calibration effect, where fix means that a sample is originally misclassified by the vanilla model but becomes correct after the corresponding calibration method, and harm means that a sample is originally correct under the vanilla model but becomes wrong after calibration. The rates are normalized by class size.
Figure 8. Sample-level overlap of the two calibrations on vanilla-wrong samples. Both means that the sample is corrected by both blank and lowrank calibration; Blank only and Lowrank only mean that only one calibration corrects it; Neither means that neither method corrects it. Each cell reports the corresponding percentage within the vanilla-wrong subset.
Figure 9. Per-class overlap composition on vanilla-wrong samples. For each class, the bar is decomposed into four groups: Both , Blank only , Lowrank only , and Neither , using the same definitions as in Fig. 8 . Thus, each bar shows how the two calibrations overlap, or fail to overlap, when correcting errors in that class.
Figure 10. Bias diagnosis by overlap group. The four groups ( Both , Blank only , Lowrank only , and Neither ) are defined in the same way as in Fig. 8 . Ablank measures language-prior bias revealed by the blank reference, while Alowrank measures appearance bias revealed by the lowrank reference relative to the blank reference.
Method
mCA@1
mCA@5
MF1
Qwen2.5-VL
0.1667
0.3561
0.0563
Qwen2.5-VL w/ label list
0.0523
0.2524
0.0072
Appendix
Table 9. Comparison of NLL-based Qwen2.5-VL with and without the label list in the prompt.
Dataset
Model
Mode
mCA@1
#Top-1 labels
Dominant ratio
Hnorm
JSD
Dominant Top-1 label
iMiGUE
Qwen2.5-VL-7B
Prompt-based
0.1615
19
37.6%
0.503
0.478
Touching or scratching forehead
iMiGUE
Qwen2.5-VL-7B
Vanilla NLL
0.0523
6
89.9%
0.116
0.643
Buckle button, pulling shirt collar, adjusting tie
iMiGUE
Qwen2.5-VL-7B
Blank-cal. NLL
0.1363
23
38.6%
0.614
0.430
Crossing fingers
iMiGUE
MiMo-VL-7B
Prompt-based
0.2445
27
54.1%
0.491
0.339
Rubbing or holding hands
iMiGUE
MiMo-VL-7B
Vanilla NLL
0.0325
2
99.7%
0.006
0.673
Buckle button, pulling shirt collar, adjusting tie
iMiGUE
MiMo-VL-7B
Blank-cal. NLL
0.1573
26
28.9%
0.623
0.310
Touching or scratching facial parts
Appendix
Table 10. iMiGUE prediction-distribution comparison among prompt-based free-form generation, raw NLL-based scoring, and blank-calibrated NLL-based scoring. For prompt-based and raw NLL-based, mCA@1 is aligned with Table 1 in the main paper; the remaining statistics are computed from the corresponding Top-1 histograms. Lower dominant ratio and JSD are better, while higher #Top-1 labels and Hnorm indicate less concentrated predictions.
Method
mCA@1
mCA@5
MF1
BlockerGCN(J)
0.2324
0.6080
0.2375
BlockerGCN(B)
0.2362
0.6040
0.2407
BlockerGCN(V)
0.2375
0.5820
0.2467
BlockerGCN(3s)
0.2658
0.6469
0.2809
PoseC3D
0.3402
0.7159
0.3583
MMN(J)
0.2991
0.6626
0.3062
Appendix
Table 11. Full comparison between Zero-MELO and supervised discriminative models on the iMiGUE dataset.
InternVL3.5
iMiGUE
mCA@1
mCA@5
MF1
Prompt-based zero-shot
0.2034
0.4358
0.0927
NLL-based
0.0329
0.2408
0.0007
Zero-MELO (Ours)
0.2425
0.4375
0.0974
Appendix
Table 12. Comparison between Zero-MELO and the corresponding vanilla MLLM under the InternVL3.5 backbone on iMiGUE.
MiMo-VL
iMiGUE
mCA@1
mCA@5
MF1
Prompt-based zero-shot
0.2445
0.5643
0.1506
NLL-based
0.0325
0.2480
0.0008
Zero-MELO (Ours)
0.2977
0.5834
0.1534
Appendix
Table 13. Comparison between Zero-MELO and the corresponding vanilla MLLM under the MiMo-VL backbone on iMiGUE.
Method
mCA@1
mCA@5
MF1
Vanilla
0.0357
0.2084
0.0094
+ Cali (Lowrank)
0.1455
0.3717
0.0569
+ Cali (Blank)
0.2040
0.4943
0.0855
+ Cali (Aggregation)
0.2363
0.5436
0.1097
Appendix
Table 14. Comparison experiments across different calibration strategies.
Method
Acc@1
M
Train
Method
Acc@1
M
Train
SynSE [ICIP’21]
0.576
S
✓
SKI-FROSTER [AAAI’25]
0.685
R+T+S
✓
FS-VAE [ICCV’25]
0.679
S
✓
SKI-ViFiCLIP [AAAI’25]
0.775
R+T+S
✓
SkeletonContext [CVPR’26]
0.655
S+T
✓
Qwen2.5-VL-7B [5]
0.663
R
×
XCLIP [ECCV’22]
0.578
R+T
×
Zero-MELO ( Ours )
0.694
R
×
Appendix
Table 15. Directly transfer Zero-MELO to the zeroshot task on NTU120-110/10 without manual intervention. M : modality. Train : trained on seen split. ACC@1 is reported for consistency with other methods. R/T/S denote RGB/text/skeleton.
Method
Time
Passes
Method
Time
Passes
Prompt-based Qwen2.5-VL
2.71
1
Zero-MELO ( Ours )
399.82
585
NLL-based Qwen2.5-VL
17.12
96
- TS
389.63
510
ZoomEye [47]
325.29
690
- Cali
9.52
75
- Context Learning
2.03
10
- Fusion
0.59
0
- ZoomEye Search
308.98
648
- Answer
14.18
32
Appendix
Table 16. Average runtime per iMiGUE sample. Time is in seconds, and Passes denotes MLLM calls.
Label y
Label-Conditioned Cue Set C(y)
Label y
Label-Conditioned Cue Set C(y)
Shaking body
{torso,legs}
Spread legs
{legs}
Sitting straightly
{torso}
Closing legs
{legs}
Shrugging
{torso}
Crossing legs
{legs}
Turning around
{torso,legs}
Stretching feet
{feet,legs}
Rising up
{torso,legs}
Retracting feet
{feet,legs}
Bowing head
{head,neck}
Tiptoe
{feet,legs}
Appendix
Table 17. Label-conditioned cue sets C(y) for MA-52.
Prompt type
Qwen2.5-VL
InternVL3.5 / MiMo-VL
Wrapper Prompts
Global wrapper
<video>
Main video frames: <video>
Zoom wrapper
<video> This is the main video, and the section enclosed by the red rectangle in each frame is the focus region. <video> This is the zoomed-in view of the focus region.
Main video frames (the section enclosed by the red rectangle in each frame is the focus region): <video> Zoomed-in video frames of the focus region: <video>
Single Cue Prompts
CQ (region)
In the current view, is the {cue} clearly visible and large enough to follow its motion over time? Answer Yes or No.
Based only on visible evidence, is the person’s {cue} visible in this view? Answer Yes or No.
Existence (region)
Is there a {cue} in the current view? Answer Yes or No.
Based only on visible evidence, can you see the person’s {cue} in this view? Answer Yes or No.
Appendix
Table 18. Prompt templates for region cues used by different backbones. Here, {cue} denotes the queried region cue.
Dataset
Label
Description
iMiGUE
Turtle neck
The movement of drawing the head and neck inward toward the shoulders, making the neck appear shortened.
iMiGUE
Bulging face, deep breath
The movement of taking a noticeable deep breath with visible expansion around the face and upper chest.
iMiGUE
Touching hat
The movement of bringing the hand close to and touching or adjusting the hat or its brim.
iMiGUE
Touching or scratching head
The movement of bringing the hand close to and touching or scratching the scalp or top of the head.
iMiGUE
Touching or scratching forehead
The movement of bringing the hand close to and touching or scratching the forehead.
iMiGUE
Covering face
The movement of covering a large portion of the face with one or both hands.
Appendix
Table 19. Descriptions used for verbalizer marginalization in iMiGUE and MA-52.
Micro-gesture recognition (MGR) is challenging due to subtle inter-class variations. Existing methods rely on category-level supervision, which is insufficient for capturing subtle and localized motion differences. Thus, this paper proposes a Fine-Grained Semantic Guidance Learning (FG-SGL) framework that jointly integrates fine-grained and category-level semantics to guide vision--language models in perceiving local MG motions. FG-SA adopts fine-grained semantic cues to guide the learning of local motion features, while CP-A enhances the separability of MG features through category-level semantic guidance. To support fine-grained semantic guidance, this work constructs a fine-grained textual dataset with human annotations that describes the dynamic process of MGs in four refined semantic dimensions. Furthermore, a Multi-Level Contrastive Optimization strategy is designed to jointly optimize both modules in a coarse-to-fine pattern. Experiments show that FG-SGL achieves competitive performance, validating the effectiveness of fine-grained semantic guidance for MGR.
Jinsheng Wei, Zhaodi Xu, Guanming Lu +2
Nanjing University of Posts and Telecommunications, China · University of Oulu, Finland
Micro-gestures (MGs) are spontaneous and subtle body movements that frequently convey hidden human emotions. Recognizing MGs in untrimmed videos remains highly challenging due to their extremely low signal-to-noise ratio, severe long-tailed class distribution, and the inherent domain shift encountered in cross-subject evaluation scenarios. In this paper, we propose a comprehensive multi-modal framework for Track 1 of the 4th MiGA-IJCAI Challenge. To capture fine-grained representations, we design a saliency-guided multi-modal extraction pipeline integrating 68-keypoint skeleton joint coordinates, 3D heatmap volumes, and high-resolution RGB visual features. We introduce a gentle square-root smoothed weighting mechanism paired with an Orthogonal Semantic Embedding Loss to protect tail classes without compromising overall recognition capabilities. More importantly, to bridge the cross-subject generalization gap, we propose a Cross-Modal Pseudo-Labeling (CMPL) strategy for unsupervised domain adaptation, which significantly boosts single-modal robustness. A temperature-scaled soft-voting mechanism is finally utilized to alleviate overconfidence during late fusion. Extensive experiments demonstrate that our framework achieves a competitive F1-score of 68.13%, securing the 4th place.
Haoran Zhang, Haokun Zhang, Pengyu Liu +3
School of Computer Science and Information Engineering, Hefei University of Technology (HFUT), Hefei, China · School of Computer Science, University of Auckland(UOA), Auckland, New Zealand
With the rapid development of Multimodal Large Language Models (MLLMs), their potential in Micro-Action understanding, a vital role in human emotion analysis, remains unexplored due to the absence of specialized benchmarks. To tackle this issue, we present MA-Bench, a benchmark comprising 1,000 videos and a three-tier evaluation architecture that progressively examines micro-action perception, relational comprehension, and interpretive reasoning. MA-Bench contains 12,000 structured question-answer pairs, enabling systematic assessment of both recognition accuracy and action interpretation. The results of 23 representative MLLMs reveal that there are significant challenges in capturing motion granularity and fine-grained body-part dynamics. To address these challenges, we further construct MA-Bench-Train, a large-scale training corpus with 20.5K videos annotated with structured micro-action captions for fine-tuning MLLMs. The results of Qwen3-VL-8B fine-tuned on MA-Bench-Train show clear performance improvements across micro-action reasoning and explanation tasks. Our work aims to establish a foundation benchmark for advancing MLLMs in understanding subtle micro-action and human-related behaviors. Project Page: https://MA-Bench.github.io
Kun Li, Jihao Gu, Fei Wang +3
CVLab, College of Information Technology, United Arab Emirates University · University College London · Hefei University of Technology +2