Is Seeing Believing? Evaluating Human Sensitivity to Synthetic Video

David Wegmann; Emil Stevnsborg; Søren Knudsen; Luca Rossi; Aske Mottelson

cs.HCMar 14, 2026

Is Seeing Believing? Evaluating Human Sensitivity to Synthetic Video

Authors: David Wegmann, Emil Stevnsborg, Søren Knudsen, Luca Rossi, Aske Mottelson

Organizations: Department of Media and Journalism Studies, School of Communication and Culture, Aarhus University, Aarhus, Denmark · Department of Computer Science, University of Copenhagen, Copenhagen, Denmark · IT University of Copenhagen, Copenhagen, Denmark

Abstract

Advances in machine learning have enabled the creation of realistic synthetic videos known as deepfakes. As deepfakes proliferate, concerns about rapid spread of disinformation and manipulation of public perception are mounting. Despite the alarming implications, our understanding of how individuals perceive synthetic media remains limited, obstructing the development of effective mitigation strategies. This paper aims to narrow this gap by investigating human responses to visual and auditory distortions of videos and deepfake-generated visuals and narration. In two between-subjects experiments, we study whether audio-visual distortions affect cognitive processing, such as subjective credibility assessment and objective learning outcomes. A third study reveals that artifacts from deepfakes influence credibility. The three studies show that video distortions and deepfake artifacts can reduce credibility. Our research contributes to the ongoing exploration of the cognitive processes involved in the evaluation and perception of synthetic videos, and underscores the need for further theory development concerning deepfake exposure.

Figures & tables

Panel of Figure 1 — Figure 1: Stills of the two stimulus videos used in Study I illustrating the differences between the experimental conditions: (a) a still from the unaltered, recorded video shown to participants in the baseline condition, and (b) a still from the distorted video shown to participants in the reduced visual clarity condition

Panel of Figure 2 — Figure 2: Stills showing the stimulus videos used in Study III illustrating the visual differences between the experimental conditions: (a) a still from the unaltered, recorded video shown to participants in the conditions baseline and synthesized narration , and (b) a still from the deepfake video shown to participants in the synthesized video and synthesized video and narration conditions.

Panel of Figure 3 — Figure 3: Measurements of Message Credibility (a) and Processing Fluency (b) in Study I divided by experimental condition. Dots represent mean values, horizontal lines indicate bootstrapped confidence intervals. Significant differences are indicated with brackets and asterisks: One asterisk () $p<0.05$ , two asterisks (**) $p<0.01$ , three asterisks (**) $p<0.0001$ .

Figure 4: Percentage of responses to the alteration detection question by treatment group in Study I.

Panel of Figure 5 — Figure 5: Measurements of Message Credibility (a), Processing Fluency (b), Factual and conceptual learning (c), and Source Vividness (d) in Study II divided by experimental condition. Dots represent mean values, horizontal lines indicate bootstrapped confidence intervals. Significant differences according to Bonferroni-corrected tests are indicated with brackets and asterisks: One asterisk (*) $p<0.05$ , two asterisks (**) $p<0.01$ .

Figure 6: Percentage of responses to the alteration detection question in Study II by treatment group.

Panel of Figure 7 — Figure 7: Distribution of credible differences between the baseline video and the fully synthesized video. (a) shows the distribution of message credibility differences, (b) the distribution for processing fluency in Study III. Bayesian estimation [ 48 ] was used to calculate distributions of mean processing fluency and message credibility values for the baseline and synthesized video and narration conditions. The differences between the means are expressed in fractions of the pooled standard deviation (i.e., in units comparable to Cohen’s $d$ ). The 95% HDIs (High Density Intervals) indicate where the bulk of the credible differences fall. For processing fluency (b), the HDI is close to or includes 0, consistent with no meaningful difference between conditions. For message credibility (a), the HDI lies substantially above 0, and essentially all credible effect sizes exceed a small-effect threshold (Cohen’s $d=0.10$ ), indicating that the distance between the means is too large to accept $H_{0}$ .

Panel of Figure 8 — Figure 8: Measurements of Message Credibility (a), Processing Fluency (b), Learning (factual and conceptual combined) (c), and Source Vividness (d) in Study III divided by experimental condition. Dots represent mean values, horizontal lines indicate bootstrapped confidence intervals. Significant differences according to Bonferroni-corrected tests are indicated with brackets and asterisks: One asterisk (*): $p<0.05$ , two asterisks (**): $p<0.01$ .

Figure 9: Percentage of responses to the alteration detection question in Study III by treatment group.

Study	Manipulation	Key findings	Link to others
I	Visual distortion of a real recorded video.	Distortion sharply reduced processing fluency and message credibility, despite unchanged video content.	Establishes that non-probative visual quality alone can lower credibility, motivating the broader manipulation space in II and III.
II	Reduced visual clarity, reduced speech clarity, and audio–visual asynchrony.	Visual degradation and AV asynchrony lowered credibility and source vividness more than mild speech distortion; learning outcomes were unchanged.	Shows that visual and timing cues systematically affect credibility, and that these effects occur without harming learning; provides the bridge to ecologically realistic deepfake cues in III.
III	Synthetic face and/or synthetic narration (deepfake video).	Fully synthetic face+voice reduced credibility and vividness most strongly. Visual deepfake artifacts drove larger credibility drops than synthetic narration; fluency and learning remained stable.	Demonstrates that the credibility penalties from I and II generalize to realistic deepfake artifacts, with strongest effects when both modalities are synthetic. Together, the studies trace a progression from abstract distortions to real-world deepfake cues.

Table 1: Summary of key findings across Studies I–III on how visual distortions, audio distortions, and deepfake artifacts shape video credibility.

Appendix figures & tables38 assets

Supplementary material from the paper’s appendix.

Appendix

Dependent Variable	Instrument	Type	Prompt
Subjective Processing Fluency	Single item scale adapted from Graf et al. [44]	Slider anchored with “difficult” and “easy”	“The process of watching this video was…”
Message Credibility	Message credibility scale adapted from Appelman & Sundar [ 34 ]	7-point Likert-type items anchored with “describes very poorly” and “describes very well”	How well does the word ‘accurate’ describe the video?
			How well does the word ‘authentic’ describe the video?
			How well does the word ‘believable’ describe the video?
Digital Manipulation Detection	Multiple choice item adapted from Nightingale et al. [49]	Multiple choice question with the answer options: “Yes”, “I don’t know”, and “No”	“Do you think the video has been digitally altered?”

Appendix

Table S1: Dependent measures used in Study I.

Dependent Variable	Instrument	Type
Conceptual Learning	Conceptual learning scale adapted from Petersen et al. [45]	Eight multiple choice questions, with four answer options each, applied before and after stimulus presentation
Factual Learning	Factual learning scale adapted from Petersen et al. [45]	Eight multiple choice questions, with four answer options each, applied before and after stimulus presentation
Subjective Processing Fluency	Single item scale adapted from Graf et al. [44]	Slider anchored with “difficult” and “easy”
Message Credibility	Message credibility scale adapted from Appelman and Sundar [34]	Three 7-point Likert-type items anchored with “describes very poorly” and “describes very well”
Source Vividness	Source Vividness scale adapted from Lee and Shin [7]	Four 7-point Likert-type items anchored with “strongly disagree” and “strongly agree”
Liking	Online news liking scale adapted from Sundar [50]	Five 7-point Likert-type items anchored with “describes very poorly” and “describes very well”

Appendix

Table S2: Dependent measures employed in Study II and Study III.

Test	p-value	Cohen’s d
Processing Fluency	< 0.0001	1.06

Appendix

Table S3: Study I: Summary of T-test Results for Processing Fluency

Test	p-value	Cohen’s d
Processing Fluency	= 0.0006	0.38

Appendix

Table S4: Study I: Summary of T-test Results for Message Credibility

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	7458	2486.2	2.844	0.0377*
Residuals	351	306806	874.1

Appendix

Table S5: Study II: ANOVA Results for Processing Fluency Across Conditions

Comparison	P-value
Reduced Visual Clarity vs. Baseline	0.35
Audi-Visual Incoherence vs. Baseline	0.34
Reduced Auditive Clarity vs. Baseline	1.00
Reduced Auditive Clarity vs. Reduced Visual Clarity	0.17
Audi-Visual Incoherence vs. Reduced Visual Clarity	1.00
Audi-Visual Incoherence vs. Reduced Auditive Clarity	0.17

Appendix

Table S6: Study II: Pairwise Comparisons for Processing Fluency Across Conditions. P-values adjusted using Bonferroni method

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	10.3	3.450	2.954	0.0326*
Residuals	351	410.0	1.168

Appendix

Table S7: Study II: ANOVA Results for Message Credibility Across Conditions

Comparison	P-value
Reduced Visual Clarity vs. Baseline	0.677
Audi-Visual Incoherence vs. Baseline	0.313
Reduced Auditive Clarity vs. Baseline	1.000
Reduced Auditive Clarity vs. Reduced Visual Clarity	0.190
Audi-Visual Incoherence vs. Reduced Visual Clarity	1.000
Audi-Visual Incoherence vs. Reduced Auditive Clarity	0.075

Appendix

Table S8: Study II: Pairwise Comparisons for Message Credibility Across Conditions (P-values adjusted using Bonferroni method)

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	21	6.992	1.694	0.168
Residuals	351	1448	4.126

Appendix

Table S9: Study II: ANOVA Results for Factual Learning Across Conditions

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	4.6	1.55	0.787	0.502
Residuals	351	691.3	1.97

Appendix

Table S10: Study II: ANOVA Results for Conceptual Learning Across Conditions

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	36.0	11.998	6.478	0.000281***
Residuals	351	650.1	1.852

Appendix

Table S11: Study II: ANOVA Results for Condition on Source Vividness

Comparison	P-value
Reduced Visual Clarity vs. Baseline	0.0066
Audi-Visual Incoherence vs. Baseline	0.0040
Reduced Auditive Clarity vs. Baseline	1.0000
Reduced Auditive Clarity vs. Reduced Visual Clarity	0.0372
Audi-Visual Incoherence vs. Reduced Visual Clarity	1.0000
Audi-Visual Incoherence vs. Reduced Auditive Clarity	0.0239

Appendix

Table S12: Study II: Pairwise Comparisons for Source Vividness Across Conditions (P-values adjusted using Bonferroni method)

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	10.03	3.344	3.761	0.0111*
Residuals	351	312.09	0.889

Appendix

Table S13: Study II: ANOVA Results for Condition on Liking

Comparison	P-value
Reduced Visual Clarity vs. Baseline	0.151
Audi-Visual Incoherence vs. Baseline	0.031
Reduced Auditive Clarity vs. Baseline	1.000
Reduced Auditive Clarity vs. Reduced Visual Clarity	0.414
Audi-Visual Incoherence vs. Reduced Visual Clarity	1.000
Audi-Visual Incoherence vs. Reduced Auditive Clarity	0.102

Appendix

Table S14: Study II: Pairwise Comparisons for Liking Across Conditions (P-values adjusted using Bonferroni method)

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	8.9	2.976	1.046	0.372
Residuals	351	998.3	2.844

Appendix

Table S15: Study II: ANOVA Results for Condition on Attentional Focus

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	1679	559.7	0.823	0.482
Residuals	361	245374	679.7

Appendix

Table S16: Study III: ANOVA Results for Processing Fluency Across Conditions

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	15.1	5.040	4.229	0.00589**
Residuals	361	430.2	1.192

Appendix

Table S17: Study III: ANOVA Results for Message Credibility Across Conditions

Comparison	P-value
RecVid-SynAud vs. RecVid-RecAud	0.0582
SynVid-RecAud vs. RecVid-RecAud	0.2727
SynVid-SynAud vs. RecVid-RecAud	0.0044
SynVid-RecAud vs. RecVid-SynAud	1.0000
SynVid-SynAud vs. RecVid-SynAud	1.0000
SynVid-SynAud vs. SynVid-RecAud	1.0000

Appendix

Table S18: Study III: Pairwise Comparisons for Message Credibility Across Conditions (P-values adjusted using Bonferroni method)

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	40.3	13.424	6.739	0.000196***
Residuals	361	719.1	1.992

Appendix

Table S19: Study III: ANOVA Results for Source Vividness Across Conditions

Comparison	P-value
RecVid-SynAud vs. RecVid-RecAud	1.0000
SynVid-RecAud vs. RecVid-RecAud	0.0032
SynVid-SynAud vs. RecVid-RecAud	0.0015
SynVid-RecAud vs. RecVid-SynAud	0.0797
SynVid-SynAud vs. RecVid-SynAud	0.0458
SynVid-SynAud vs. SynVid-RecAud	1.0000

Appendix

Table S20: Study III: Pairwise Comparisons for Source Vividness Across Conditions (P-values adjusted using Bonferroni method)

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	5.45	1.8161	2.183	0.0896
Residuals	361	300.26	0.8318

Appendix

Table S21: Study III: ANOVA Results for Liking Across Conditions

Source	Df	Sum Sq	Mean Sq	F value	Pr(>F)
Condition	3	2.5	0.8494	0.314	0.815
Residuals	361	977.3	2.7072

Appendix

Table S22: Study III: ANOVA Results for Attentional Focus Across Conditions

Figure S1: Distribution of credible differences between the mean processing fluency of the baseline video and the synthesized narration video.

Figure S2: Distribution of credible differences between the mean processing fluency of the baseline video and the synthesized video.

Figure S3: Distribution of credible differences between the mean message credibility of the baseline video and the synthesized narration video.

Figure S4: Distribution of credible differences between the mean message credibility of the baseline video and the synthesized video.

Figure S5: Distribution of credible differences between the mean correct answers (Learning) of the baseline video and synthesized video and narration condition.

Figure S6: Distribution of credible differences between the mean correct answers (Learning) of the baseline video and synthesized video condition.

Figure S7: Distribution of credible differences between the mean correct answers (Learning) of the baseline video and synthesized narration condition.

Figure S8: Distribution of credible differences between the mean source vividness of the baseline video and synthesized video and narration condition.

Figure S9: Distribution of credible differences between the mean source vividness of the baseline video and synthesized video condition.

Figure S10: Distribution of credible differences between the mean source vividness of the baseline video and synthesized narration condition.

Figure S11: Distribution of credible differences between the mean liking measurement of the baseline video and synthesized video and narration condition.

Figure S12: Distribution of credible differences between the mean liking measurement of the baseline video and synthesized video condition.

Figure S13: Distribution of credible differences between the mean liking measurement of the baseline video and synthesized narration condition.

Figure S14: Distribution of credible differences between the mean attentional focus of the baseline video and synthesized video and narration condition.

Figure S15: Distribution of credible differences between the mean attentional focus of the baseline video and synthesized video condition.

Figure S16: Distribution of credible differences between the mean attentional focus of the baseline video and synthesized narration condition.

Beyond Seeing Is Believing: On Crowdsourced Detection of Audiovisual Deepfakes
May 6, 2026Michael Soprano, Andrea Cioci, Stefano MizzaroAudio Deepfake Detection Explainable Deepfake Detection
Department of Mathematics, Computer Science and Physics University of Udine, Udine, Friuli-Venezia Giulia, Italy
Eroding Trust in Real Speech: A Large-Scale Study of Human Audio Deepfake Perception
May 21, 2026Nicolas M. Müller, Wei Herng ChoongAudio Deepfake Detection Speech Processing
Fraunhofer AISEC & Resemble AI · Germany · Fraunhofer AISEC
I Hear, Therefore I Trust: A Socio-Technical Investigation of Humans as Synthetic Speech Detectors
May 27, 2026Lelia Erscoi, Tomi KinnunenAudio Deepfake Detection Trust in AI
Computational Speech Group, University of Eastern Finland · Computational Speech Group, University of Eastern Finland, Joensuu, Finland

May 6, 2026J/K move · Enter open · S save

Beyond Seeing Is Believing: On Crowdsourced Detection of Audiovisual Deepfakes

Michael Soprano, Andrea Cioci, Stefano Mizzaro

Department of Mathematics, Computer Science and Physics University of Udine, Udine, Friuli-Venezia Giulia, Italy

Audio Deepfake Detection Explainable Deepfake Detection Deepfake Detection

Open paper AskPDF arXiv

Deepfakes are increasingly realistic and easy to produce, raising concerns about the reliability of human judgments in misinformation settings. We study audiovisual deepfake detection by measuring how consistently crowd workers distinguish authentic from manipulated videos and, when they flag a video as manipulated, how accurately they identify the manipulation type (audio-only, video-only, or audio-video) and how consistently they report manipulation timestamps. We run two matched crowdsourcing studies on Prolific using AV-Deepfake1M and the Trusted Media Challenge (TMC) dataset. We sample 48 videos per dataset (96 total) and collect 960 judgments (10 per video). Results show that crowd workers rarely misclassify authentic videos as manipulated, but they miss many manipulations, and agreement remains limited across videos. Aggregating multiple judgments per video stabilizes the authenticity signal, but it cannot recover manipulations that most workers consistently miss. Manipulation type identification is substantially noisier than authenticity detection even when workers detect a manipulation, with joint audio-video cases being particularly hard to recognize. Overall, these findings suggest that crowdsourcing can provide a scalable screening signal for audiovisual authenticity, while reliable modality attribution remains an open challenge.

Abstract

Explore similar work

Beyond Seeing Is Believing: On Crowdsourced Detection of Audiovisual Deepfakes

Eroding Trust in Real Speech: A Large-Scale Study of Human Audio Deepfake Perception

I Hear, Therefore I Trust: A Socio-Technical Investigation of Humans as Synthetic Speech Detectors