Multimodal large language models (MLLMs) are increasingly used as evaluators, yet their reliability in professional assessment tasks that require expert judgment remains unclear. We investigate this challenge in the context of assessing psychotherapeutic interpersonal skills and introduce MIS-Bench, a Multimodal Interpersonal Skills (MIS) benchmark comprising 996 psychotherapy response videos annotated across 8 dimensions of Facilitative Interpersonal Skills. Across 9 MLLMs with multiple modality and prompting settings, we find that current models show only modest agreement with human experts, inconsistent gains from multimodal input, and limited benefits from reasoning-based prompting. To mitigate this gap, we propose MIS-RAFT, a regression-aware fine-tuning method inspired by RAFT and tailored to fine-grained interpersonal skill scoring at one-decimal precision. MIS-RAFT addresses the mismatch between autoregressive token prediction and scalar-valued expert assessment, significantly improving agreement with human ratings. Overall, MIS-Bench reveals a clear gap between general multimodal capability and expert-level interpersonal judgment, while MIS-RAFT offers a promising path toward more reliable model-based assessment.
Figures & tables
Figure 1: Overview of MIS-Bench. Each benchmark instance is built from a psychotherapy response video and associated transcript and rubric information. Models are evaluated with 4 modality configurations and 2 prompting styles, and predict one scalar score for each FIS dimension. Performance is measured by ICC(3) and MSE.
Modality
Feature
Shape / Type
Face
Action units
T×8
Face
Facial landmarks (98 points, x/y)
T×196
Face
Gaze direction
T×2
Face
Detection confidence
T×2
Audio
Wav2Vec 2.0 frame embeddings
T×768
Audio
Prosody
T×15
Table 1: Released feature structure of MIS-Bench. The public release contains de-identified feature representations rather than raw audiovisual recordings.
VF
H/PE
Per.
Emo.
WAU
Emp.
ABC
ARRR
Human ICC(3,1)
64.0
53.0
54.0
51.0
50.0
43.0
51.0
53.0
95% CI
[61.0, 68.0]
[49.0, 58.0]
[49.0, 58.0]
[46.0, 55.0]
[45.0, 54.0]
[38.0, 48.0]
[46.0, 55.0]
[48.0, 57.0]
Table 2: Human single-rater agreement ceiling on the 625 fully rated videos. Values are single-measure ICC(3,1) scores reported in percentage points; brackets denote 95% confidence intervals. Higher is better.
Model
LLM Size
A
V
T
VF
H/PE
Per.
Emo.
WAU
Emp.
ABC
ARRR
Open-source MLLMs
Qwen2-Audio-7B-Instruct ( Chu et al., 2024 )
7B
✓
✓
-6.0
17.1
8.4
0.9
8.4
2.3
9.1
0.1
Qwen2.5-VL-7B-Instruct ( Bai et al., 2025 )
7B
✓
✓
2.8
39.9
29.9
16.5
36.4
28.8
33.0
27.4
Qwen2.5-Omni-7B ( Xu et al., 2025a )
7B
✓
✓
✓
-3.1
19.4
15.3
11.0
18.6
23.0
26.1
20.1
Qwen3-Omni-30B-A3B-Instruct ( Xu et al., 2025b )
30B
✓
✓
✓
21.2
24.6
25.1
14.8
22.8
24.6
18.6
22.7
InternVL3.5-8B-HF ( Wang et al., 2025 )
8B
✓
✓
5.9
32.4
16.6
14.7
28.9
22.9
26.8
17.6
Table 3: Overall performance comparison on MIS-Bench using dimension-level ICC (%). For each model, we report the dimension-level ICC values from the configuration that achieves the highest overall score. This overall score is computed by averaging the eight predicted dimension scores for each video and comparing the resulting average against the corresponding average of the human ratings. If a model output is unparseable for any of the eight dimensions, that video is excluded when determining the overall score, with a mean exclusion rate of 1.5%. The top three performing results in each dimension are highlighted in red (1st), blue (2nd), and yellow (3rd) backgrounds, respectively.
Figure 2: Dimension-level ICC heatmaps on MIS-Bench. We report every supported model × modality × prompting combination across four modality configurations and two prompting strategies. In each panel, rows correspond to models and columns correspond to the eight FIS dimensions. Higher ICC indicates better agreement with human ratings, and blank cells indicate configurations unsupported by a given model. The complete grid enables controlled comparisons among models within a common configuration.
Figure 3: MIS-RAFT scoring pipeline for one-decimal score prediction.
FIS Dimension
Raw
Fine-tuning
MIS-RAFT
VF
−3.8 [ −12.5 , 2.0 ]
1.2 [ −16.3 , 18.9 ]
−0.7 [ −16.0 , 12.6 ]
H/PE
37.6 [ 24.0 , 49.5 ]
21.7 [ 12.3 , 31.2 ]
50.7 [ 38.0 , 62.3 ]
Per.
−2.0 [ −6.6 , 0.0 ]
1.8 [ −15.7 , 16.6 ]
33.6 [ 20.2 , 45.9 ]
Emo.
−0.0 [ −0.0 , 0.0 ]
−13.5 [ −29.4 , 4.1 ]
17.0 [ 0.6 , 31.1 ]
WAU
12.9 [ −1.6 , 27.3 ]
−12.2 [ −22.9 , 0.0 ]
31.4 [ 14.9 , 46.6 ]
Emp.
43.6 [ 28.2 , 55.9 ]
3.7 [ −7.6 , 15.3 ]
35.5 [ 19.1 , 50.7 ]
Table 4: ICC(3,1) by FIS dimension under raw inference, standard fine-tuning, and MIS-RAFT fine-tuning for the Qwen2.5-VL backbone. Values are reported in percentage points; brackets report 95% percentile bootstrap confidence intervals computed over videos using 10,000 bootstrap resamples. Higher is better, and the best ICC in each row is shown in bold.
Figure 4: Radar chart of ICC scores across eight FIS dimensions.