MuDD: A Multimodal Deception Detection Dataset and GSR-Guided Progressive Distillation for Non-Contact Deception Detection

Peiyuan Jiang; Yao Liu; Yanglei Gan; Jiaye Yang; Lu Liu; Daibing Yao; Qiao Liu

cs.CVMar 27, 2026

MuDD: A Multimodal Deception Detection Dataset and GSR-Guided Progressive Distillation for Non-Contact Deception Detection

Authors: Peiyuan Jiang, Yao Liu, Yanglei Gan, Jiaye Yang, Lu Liu, Daibing Yao, Qiao Liu

Organizations: School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China · School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China · Yizhou Prison, Sichuan Province, Chengdu, Sichuan, China

Abstract

Non-contact automatic deception detection remains challenging because visual and auditory deception cues often lack stable cross-subject patterns. In contrast, galvanic skin response (GSR) provides more reliable physiological cues and has been widely used in contact-based deception detection. In this work, we leverage stable deception-related knowledge in GSR to guide representation learning in non-contact modalities through cross-modal knowledge distillation. A key obstacle, however, is the lack of a suitable dataset for this setting. To address this, we introduce MuDD, a large-scale Multimodal Deception Detection dataset containing recordings from 130 participants over 690 minutes. In addition to video, audio, and GSR, MuDD also provides Photoplethysmography, heart rate, and personality traits, supporting broader scientific studies of deception. Based on this dataset, we propose GSR-guided Progressive Distillation (GPD), a cross-modal distillation framework for mitigating the negative transfer caused by the large modality mismatch between GSR and non-contact signals. The core innovation of GPD is the integration of progressive feature-level and digit-level distillation with dynamic routing, which allows the model to adaptively determine how teacher knowledge should be transferred during training, leading to more stable cross-modal knowledge transfer. Extensive experiments and visualizations show that GPD outperforms existing methods and achieves state-of-the-art performance on both deception detection and concealed-digit identification.

Figures & tables

Figure 1. Motivation of this work. (a) Visual deception cues are often subtle and subject-dependent, making it difficult to identify stable cross-subject patterns, whereas GSR exhibits more consistent deception-related responses across individuals. (b) Illustration of the cross-modal knowledge distillation paradigm, where GSR serves as the teacher modality to transfer feature-level and logit-level knowledge to non-contact modalities through a teacher–student framework.

Dataset	Subs.	Dur.	Setting	Modalities	Labels
Without physiological signals
CSC Deceptive Speech ( Hirschberg et al., 2005 )	32	7920 min	Interview	A	Bin.
CXD Corpus ( Levitan et al., 2015 )	340	7200 min	False interview	A+T	Bin.
Real-life Trial ( Pérez-Rosas et al., 2015 )	56	$<60$ min	Courtroom	V+T	Bin.
MU3D ( Lloyd et al., 2019 )	80	$<200$ min	Social interview	V+T	Bin.
Truth-Tellers ( Monaro et al., 2020 )	60	$<480$ min	Simulated interview	V	Bin.

Table 1. Comparison of MuDD with existing deception-related datasets. Abbreviations: Subs. = subjects, Dur. = duration, V = video, A = audio, T = text, Pers. = personality traits, Bin. = binary label, and T-10cls = trial-level 10-class label.

Figure 2. Overview of the proposed GSR-guided Progressive Distillation (GPD) framework.

Method	Params (M)	FLOPs/sample (M)	Video modality				Audio modality
Method	Params (M)	FLOPs/sample (M)	Top-1	Top-2	F1	AUC	Top-1	Top-2	F1	AUC
GENLIE	0.908	36.75	26.8 $\pm$ 5.0	35.3 $\pm$ 5.7	16.2 $\pm$ 2.8	54.4 $\pm$ 1.4	29.9 $\pm$ 6.7	39.8 $\pm$ 5.1	18.1 $\pm$ 4.8	56.1 $\pm$ 4.2
XKD	1.237	49.32	23.6 $\pm$ 6.5	38.4 $\pm$ 4.5	13.9 $\pm$ 4.9	53.3 $\pm$ 1.9	28.9 $\pm$ 9.7	39.1 $\pm$ 8.6	15.3 $\pm$ 4.2	55.1 $\pm$ 6.4
SemBridge	1.456	34.16	23.7 $\pm$ 5.7	32.9 $\pm$ 5.1	12.9 $\pm$ 7.1	52.5 $\pm$ 3.6	28.4 $\pm$ 6.7	39.9 $\pm$ 6.3	18.6 $\pm$ 5.2	58.3 $\pm$ 2.9
HKD-Emotion	1.075	34.16	24.4 $\pm$ 7.3	33.7 $\pm$ 5.1	8.1 $\pm$ 6.9	52.3 $\pm$ 1.8	29.9 $\pm$ 9.0	36.1 $\pm$ 11.5	7.6 $\pm$ 7.8	58.0 $\pm$ 2.0
DecomKD	1.535	61.78	24.5 $\pm$ 6.0	32.1 $\pm$ 8.8	5.8 $\pm$ 5.7	52.8 $\pm$ 2.5	30.0 $\pm$ 7.2	39.3 $\pm$ 8.1	14.5 $\pm$ 7.6	58.5 $\pm$ 3.8

Table 2. Main comparison with state-of-the-art CMKD methods on MuDD. Results are reported as mean

\pm

std over five runs with different random seeds. The best results are shown in bold, and the second-best results are underlined.

Method	Video modality				Audio modality
Method	Top-1	Top-2	F1	AUC	Top-1	Top-2	F1	AUC
GENLIE	26.92	35.38	16.30	53.14	29.86	40.77	18.94	59.03
w/o $\mathcal{L}_{\mathrm{logit\mbox{-}kd}}$	26.92	37.69	17.00	52.13	30.77	41.27	18.50	57.48
w/o $\mathcal{L}_{\mathrm{feat\mbox{-}kd}}$	25.38	36.92	16.70	51.46	32.31	43.08	19.70	57.66
w/o prog. wt.	23.08	35.38	16.72	51.89	30.77	42.47	20.05	59.25
GPD	32.31	40.00	17.43	53.55	35.38	43.84	20.63	59.30

Table 3. Ablation study of different components in GPD on MuDD. Prog. wt. denotes progressive weighting.

Schedule	Video modality				Audio modality
Schedule	Top-1	Top-2	F1	AUC	Top-1	Top-2	F1	AUC
Linear	30.77	41.54	17.27	52.67	33.08	38.46	18.97	57.07
Step	30.00	41.54	17.74	52.80	34.62	40.76	19.22	57.41
Cosine	30.00	39.23	17.53	52.70	33.08	40.00	19.06	58.20
Sigmoid	32.31	40.00	17.43	53.55	35.38	43.84	20.63	59.30

Table 4. Comparison of different progressive weighting schedules on MuDD.

Route	Video modality				Audio modality
Route	Top-1	Top-2	F1	AUC	Top-1	Top-2	F1	AUC
No-feat.	30.77	41.54	17.68	52.69	34.62	41.54	19.75	57.47
Feat.-first	31.31	39.23	17.31	52.81	33.08	42.31	19.42	58.14
Logit-first	31.54	40.77	17.47	52.46	33.08	39.23	19.14	56.73
Joint	30.77	39.31	17.46	52.66	32.31	42.31	17.92	58.13
Dyn.	32.31	40.00	17.43	53.55	35.38	43.84	20.63	59.30

Table 5. Comparison of different distillation routes on MuDD.

Wt.	Rt.	Video modality				Audio modality
Wt.	Rt.	Top-1	Top-2	F1	AUC	Top-1	Top-2	F1	AUC
Sig.	On	32.31	40.00	17.43	53.55	35.38	43.84	20.63	59.30
	Off	30.77	42.31	17.37	52.66	33.08	42.31	19.42	58.14
Lin.	On	30.77	41.54	17.27	52.67	33.08	38.46	18.97	57.07
	Off	31.54	40.77	17.26	52.52	31.54	41.54	17.77	56.68
Step	On	30.00	41.54	17.74	52.80	34.62	40.76	19.22	57.41

Table 6. Decoupled ablation of routing and weighting on MuDD. “Wt.” and “Rt.” denote weighting schedule and route configuration, while “On” and “Off” indicate whether dynamic routing is enabled. For each weighting schedule, the better result between On and Off is highlighted in bold.

Panel of Figure 3 — Figure 3. Sensitivity analysis of dynamic routing. Left: varying the EMA momentum $\mu$ . Right: uniformly shifting all routing thresholds by $\Delta$ .

Panel of Figure 4 — Figure 4. Effect of distillation on student behavior in the video and audio modalities.

Multimodal Deception Detection

See all 19 papers

DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning
Mar 25, 2026Jiajian Huang, Dongliang Zhu, Zitong YU +4Multimodal Robustness Cross-Domain Generalization
Great Bay University, China · Wuhan University, China · Dongguan Key Laboratory for Intelligence and Information Technology, China +1
ThinkDeception: A Progressive Reinforcement Learning Framework for Interpretable Multimodal Deception Detection
Jun 17, 2026Jinhao Song, Shan Liang, Yiqun Yue +2RL for Language Model Reasoning Multimodal Large Language Models
Xi’an Jiaotong-Liverpool University · Suzhou, China
DeceptionX: Explainable Deception Detection with Multimodal Large Language Models
Jun 9, 2026Jiayu Zhang, Shuo Ye, Jiajian Huang +7Multimodal Large Language Models Multimodal Reasoning
Great Bay University Dongguan, Guangdong, China · Hong Kong Polytechnic University HongKong, China

Mar 25, 2026J/K move · Enter open · S save

DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning

Jiajian Huang, Dongliang Zhu, Zitong YU, Hui Ma+3

Great Bay University, China · Wuhan University, China · Dongguan Key Laboratory for Intelligence and Information Technology, China+1

Multimodal Robustness Cross-Domain Generalization Multimodal Deception Detection

Open paper AskPDF arXiv

Multimodal deception detection aims to identify deceptive behavior by analyzing audiovisual cues for forensics and security. In these high-stakes settings, investigators need verifiable evidence connecting audiovisual cues to final decisions, along with reliable generalization across domains and cultural contexts. However, existing benchmarks provide only binary labels without intermediate reasoning cues. Datasets are also small with limited scenario coverage, leading to shortcut learning. We address these issues through three contributions. First, we construct reasoning datasets by augmenting existing benchmarks with structured cue-level descriptions and reasoning chains, enabling models to output auditable reports. Second, we release T4-Deception, a multicultural dataset based on the unified ``To Tell the Truth'' television format implemented across four countries. With 1695 samples, it is the largest non-laboratory deception detection dataset. Third, we propose two modules for robust learning under small-data conditions. Stabilized Individuality-Commonality Synergy (SICS) refines multimodal representations by combining learnable global priors with sample-adaptive residuals and applying polarity-aware recalibration. Distilled Modality Consistency (DMC) aligns modality-specific predictions with the fused multimodal predictions via knowledge distillation to prevent unimodal shortcut learning. Experiments on three established benchmarks and our novel dataset demonstrate that our method achieves state-of-the-art performance in both in-domain and cross-domain scenarios, while exhibiting superior transferability across diverse cultural contexts. The datasets and code are available at this link.

Abstract

Multimodal Deception Detection

Explore similar work

DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning

ThinkDeception: A Progressive Reinforcement Learning Framework for Interpretable Multimodal Deception Detection

DeceptionX: Explainable Deception Detection with Multimodal Large Language Models