Missing-by-Design: Certifiable Modality Deletion for Revocable Multimodal Sentiment Analysis

Rong Fu; Ziming Wang; Chunlei Meng; Jiekai Wu; Kangan Qian; Hao Zhang; Simon Fong

cs.CLFeb 18, 2026

Missing-by-Design: Certifiable Modality Deletion for Revocable Multimodal Sentiment Analysis

Authors: Rong Fu, Ziming Wang, Chunlei Meng, Jiekai Wu, Kangan Qian, Hao Zhang, Simon Fong

Organizations: Independent Researcher

Abstract

As multimodal systems increasingly process sensitive personal data, the ability to selectively revoke specific data modalities has become a critical requirement for privacy compliance and user autonomy. We present Missing-by-Design (MBD), a unified framework for revocable multimodal sentiment analysis that combines structured representation learning with a certifiable parameter-modification pipeline. Revocability is critical in privacy-sensitive applications where users or regulators may request removal of modality-specific information. MBD learns property-aware embeddings and employs generator-based reconstruction to recover missing channels while preserving task-relevant signals. For deletion requests, the framework applies saliency-driven candidate selection and a calibrated Gaussian update to produce a machine-verifiable Modality Deletion Certificate. Experiments on benchmark datasets show that MBD achieves strong predictive performance under incomplete inputs and delivers a practical privacy-utility trade-off, positioning surgical unlearning as an efficient alternative to full retraining.

Figures & tables

Figure 1: Overview of the Missing-by-Design (MBD) framework for certifiable modality deletion. The architecture is organized into two primary stages: a property-informed joint training phase and a weight surgery pipeline. In the training phase, multimodal inputs ( $X^{L},X^{A},X^{V}$ ) are integrated via a fusion network $\mathcal{F}$ for sentiment prediction, while auxiliary generator networks $\mathcal{G}_{m}$ and property embeddings $P^{m}$ are optimized to enforce modality-specific reconstruction and property alignment. Upon a revocation request for modality $m^{\star}$ , the surgery pipeline utilizes a calibration batch $\mathcal{B}$ to compute the modality saliency $s_{q}^{(m^{\star})}$ and a SwiftPrune-inspired importance proxy $L_{q}$ . These metrics guide the candidate selection and thresholding process ( $\eta_{s},\eta_{L}$ ), followed by a differential-privacy calibrated Gaussian mechanism ( $\varepsilon_{\mathrm{mod}},\delta_{\mathrm{mod}}$ ) for parameter modification. The pipeline ultimately outputs the modified model parameters $\mathcal{W}^{\prime}$ alongside a machine-verifiable Modality Deletion Certificate (MDC).

Method	CMU-MOSI					CMU-MOSEI
Method	Acc7	Acc2	F1	MAE $\downarrow$	Corr	Acc7	Acc2	F1	MAE $\downarrow$	Corr
HyCon [ 46 ]	46.6	85.2	85.1	0.741	0.779	52.8	85.4	85.6	0.554	0.751
UniMSE [ 13 ]	48.7	86.9	86.4	0.691	0.809	54.4	87.5	87.5	0.523	0.773
ConFEDE [ 26 ]	42.3	85.5	85.5	0.742	0.782	54.9	85.8	85.8	0.522	0.780
MGCL [ 25 ]	49.3	86.7	86.7	0.685	0.707	53.9	86.4	86.4	0.535	0.772
HyDiscGAN [ 47 ]	43.2	86.7	86.3	0.749	0.782	54.4	86.3	86.2	0.533	0.761

Table 1: CMU-MOSI [ 44 ] and CMU-MOSEI [ 45 ] full-modality comparison. Metrics: Acc7 (%), Acc2 (%), F1 (%), MAE (lower better), Corr. Best entries are bold . Mean

\pm

std over 3 runs; all MBD improvements significant at

p<0.01

vs runner-up (two-tailed paired t-test).

Figure 2: Privacy–utility trade-off after certified audio deletion. Plotted curves show binary accuracy (Acc2) together with attack success rate (ASR, white-box) as functions of $\varepsilon_{\mathrm{mod}}$ . Lower ASR and higher Acc2 are preferred.

Figure 3: Training trajectories for the principal loss terms (averaged across three seeds).

Figure 4: t-SNE visualization of reconstructed embeddings (left: without property embedding pathway; right: with property embedding pathway).

Model	WA (%) $\uparrow$	UA (%) $\uparrow$
TwoStageFT [ 53 ]	74.9	76.1
AdaptiveMixup [ 54 ]	75.4	76.0
EmoAug [ 55 ]	72.7	73.8
MoMKE [ 56 ]	77.9	77.1
APIN [ 57 ]	77.8	78.2
IAM [ 58 ]	74.8	75.6

Table 2: IEMOCAP [ 52 ] full-modality comparison. WA: weighted accuracy; UA: unweighted accuracy. Best entries are bold . Mean

\pm

std over 3 runs; all MBD improvements significant at

p<0.01

vs runner-up (two-tailed paired t-test).

Available	GCNet [ 6 ]	IMDer [ 5 ]	MoMKE [ 56 ]	LNLN [ 61 ]	EUAR [ 18 ]	CIDer [ 62 ]	MBD (ours)
{t}	83.7/83.6/42.3	84.8/84.7/44.8	86.2/86.1/38.1	84.9/84.7/45.1	86.0/86.0/46.1	83.7/83.6/41.3	88.9/88.9/48.9
{v}	56.1/55.7/16.9	61.3/60.8/22.2	54.1/53.7/17.0	52.2/58.9/18.8	64.9/64.9/23.6	57.8/42.3/15.5	67.0/67.0/23.0
{a}	56.1/54.5/16.6	62.0/62.2/22.0	59.3/59.0/18.4	52.2/58.9/18.0	63.0/62.3/23.2	57.8/43.2/15.2	67.5/67.5/23.5
{t,v}	84.3/84.2/43.4	85.5/85.4/45.3	86.5/86.4/37.5	84.3/84.6/44.6	86.2/86.2/45.5	83.8/83.8/42.1	89.5/89.5/50.0
{t,a}	84.3/84.2/43.4	85.4/85.3/45.0	86.5/86.4/38.6	84.9/85.2/45.1	86.1/86.1/44.7	83.8/83.8/41.7	90.0/90.0/50.5
{v,a}	62.0/61.9/17.2	63.6/63.4/23.8	59.6/59.6/20.1	52.2/58.9/18.8	66.1/65.8/24.2	57.8/44.0/15.5	69.0/69.0/24.0

Table 3: Fixed missing-modality results on CMU-MOSI [ 44 ] . Each cell reports Acc2 / F1 / Acc7. Column ‘Available‘ indicates which modalities are available (t: text, a: audio, v: visual). Best results are bold . Mean

\pm

std over 3 runs; all MBD improvements significant at

p<0.01

vs runner-up (two-tailed paired t-test).

Missing rate $\eta$	GCNet [ 6 ]	IMDer [ 5 ]	MoMKE [ 56 ]	LNLN [ 61 ]	EUAR [ 18 ]	CIDer [ 62 ]	MBD (ours)
0.1	82.4/82.2/41.9	83.3/83.2/43.0	83.6/83.6/35.5	81.1/82.0/42.0	84.1/84.1/43.8	81.1/79.6/39.4	88.9/88.9/48.5
0.2	79.6/79.3/38.9	80.9/80.8/40.7	80.7/80.7/33.7	78.0/79.5/39.5	81.9/81.9/41.5	78.5/75.6/36.7	86.9/86.9/46.5
0.3	76.7/76.5/35.9	78.5/78.4/38.4	77.8/77.7/31.9	74.8/77.0/36.9	79.8/79.7/39.2	76.0/71.6/34.0	84.6/84.6/44.1
0.4	73.6/73.2/33.0	76.0/75.9/36.0	74.7/74.6/29.8	71.6/74.4/34.3	77.4/77.3/36.9	73.4/67.4/31.2	82.2/82.2/41.2
0.5	70.4/69.9/30.1	73.5/73.4/33.6	71.6/71.4/27.8	68.4/71.8/31.7	75.1/74.9/34.7	70.8/63.3/28.5	79.8/79.8/38.5
0.6	67.3/66.7/27.2	71.0/70.9/31.2	68.5/68.3/25.8	65.2/69.2/29.0	72.8/72.6/32.5	68.2/59.1/25.8	77.4/77.4/36.3

Table 4: Varying global missing rate

\eta

on CMU-MOSI [ 44 ] . Each cell reports Acc2 / F1 / Acc7. Best results are bold . Mean

\pm

std over 3 runs; all MBD improvements significant at

p<0.01

vs runner-up (two-tailed paired t-test).

Variant	FIX (Acc2 / F1 / Acc7)	MR (Acc2 / F1 / Acc7)
w/o property embedding pathway	75.0 / 74.8 / 31.8	77.7 / 77.4 / 36.3
w/o reconstruction module	76.2 / 76.0 / 33.6	78.9 / 78.8 / 37.6
w/o fusion module	75.6 / 75.4 / 32.9	78.3 / 78.0 / 36.9
w/o controlled unlearning module	77.8 / 77.7 / 35.4	79.8 / 79.8 / 38.6
w/o property pathway + reconstruction	73.9 / 73.6 / 30.7	76.1 / 75.8 / 34.6
w/o all modules	72.1 / 71.8 / 29.0	74.2 / 73.9 / 31.5

Table 5: Ablation study on CMU-MOSI [ 44 ] . Each entry reports Acc2 / F1 / Acc7 for FIX and MR regimes.

Condition	FIX	MR
With corruption	77.9 / 77.9 / 36.9	79.9 / 80.0 / 39.2
Clean (no corruption)	78.7 / 78.8 / 37.8	80.9 / 80.9 / 40.6

Table 6: Robustness to synthetic corruption on CMU-MOSI [ 44 ] (Acc2 / F1 / Acc7).

$\varepsilon_{\mathrm{mod}}$	ASR (white-box)	ASR (black-box)	Acc2 (%)	F1 (%)	$\Delta\mathcal{L}_{\mathrm{re}}^{(a)}$
Full model (no deletion)	$78.4\pm 0.3$	$72.1\pm 0.4$	$89.9\pm 0.2$	$89.9\pm 0.2$	0.000
0.5	$50.2\pm 0.5$	$51.8\pm 0.6$	$88.6\pm 0.3$	$88.4\pm 0.3$	0.012
1	$49.9\pm 0.4$	$50.7\pm 0.5$	$88.9\pm 0.2$	$88.8\pm 0.2$	0.009
2	$49.7\pm 0.6$	$50.5\pm 0.5$	$89.1\pm 0.3$	$89.0\pm 0.3$	0.006
4	$49.6\pm 0.5$	$50.4\pm 0.4$	$89.3\pm 0.2$	$89.2\pm 0.2$	0.004

Table 7: Certified-deletion diagnostics on CMU-MOSI (audio) [ 44 ] . ASR = attack success rate; Acc2 = binary sentiment accuracy; F1 = binary F1;

\Delta\mathcal{L}_{\mathrm{re}}^{(a)}

= post-surgery minus from-scratch reconstruction loss. All values are means ± std (three seeds).

Hyper-parameter	Acc2 (%)	$\varepsilon_{\mathrm{mod}}$
$r=0.01$	$88.4\pm 0.3$	0.49
$r=0.02$	$88.7\pm 0.2$	0.50
$r=0.03$ (default)	$88.6\pm 0.3$	0.50
$r=0.05$	$89.0\pm 0.2$	0.51
$\eta_{s}=0.05$	$88.5\pm 0.3$	0.49
$\eta_{s}=0.10$ (default)	$88.6\pm 0.3$	0.50

Table 8: Sensitivity scan on CMU-MOSI (audio deletion). Acc2 measured on the full test set;

\varepsilon_{\mathrm{mod}}

computed via zCDP composition.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Cumulative privacy budget under sequential modality deletions. The solid curve shows the theoretical $(\varepsilon,\delta)$ conversion obtained from zCDP composition, while the dashed curve reports the empirical quantile estimated by Monte Carlo sampling of the privacy-loss random variable.

Figure 6: SwiftPrune proxy $\widehat{L}_{q}$ versus the true leave-one-out increment $\Delta L_{q}$ . Each point corresponds to a candidate parameter $q$ . The dashed line is $y=x$ and the shaded band indicates $\pm 8\%$ around $y=x$ . Spearman $\rho=0.87$ and 98% of points fall inside the $\pm 8\%$ band, which visually confirms that the proxy tracks the true increments closely and does not systematically over-estimate them.

Dataset	$\chi_{\max}$	Max rel. error	98% within bound?	OverDelete $(\gamma=0.1)$	Spearman $\rho$
CMU-MOSI	0.99	$7.9\%$	$\checkmark$	$1.8\%$	0.87
CMU-MOSEI	0.99	$8.2\%$	$\checkmark$	$2.1\%$	0.86
IEMOCAP	0.99	$8.0\%$	$\checkmark$	$1.9\%$	0.88

Appendix

Table A2. SwiftPrune proxy error summary and over-delete measurements (clipping

\chi_{\max}=0.99

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 351 papers

Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
Jul 8, 2026Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu +1VLM Unlearning MLLM Unlearning
University of Maryland, Baltimore County · University of Notre Dame · UNC Chapel Hill
Should Missing Modalities Always Be Necessary to Repair for Multi-modal Sentiment Analysis?
Jul 19, 2026Yubo Gao, Haotian Wu, Xiaoyu Xu +9Missing-Modality Learning Multimodal Sentiment Analysis
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology · Nanyang Technological University +2
SALMUBench: A Benchmark for Sensitive Association-Level Multimodal Unlearning
Mar 27, 2026Cai Selvas-Sala, Lei Kang, Lluis GomezVLM Unlearning Machine Unlearning Evaluation
Computer Vision Center · Universitat Polit`ecnica de Catalunya · Universitat Aut`onoma de Barcelona

Jul 8, 2026J/K move · Enter open · S save

Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks

Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu, Vaidehi Patil

University of Maryland, Baltimore County · University of Notre Dame · UNC Chapel Hill

VLM Unlearning MLLM Unlearning Multimodal Foundation Models Machine Unlearning

Open paper AskPDF arXiv

With the growing adoption of VLMs, DMs, LLMs, and AFMs, these multimodal foundation models can inadvertently encode sensitive, copyrighted, biased, or unsafe cross-modal associations that originate from their training data. Retraining after deletion requests or policy updates is often impractical, and targeted forgetting remains difficult because knowledge is distributed across shared representations. Multimodal unlearning addresses this challenge by enabling selective removal across modalities while retaining overall utility. This survey offers a unified, system-oriented view of multimodal unlearning across vision, language, audio, and video, grounded in recent advances, emerging applications, and open problems. Our taxonomy enables systematic comparison across model architectures and modalities, clarifying trade-offs among deletion strength, retention, efficiency, reversibility, and robustness. This survey highlights open problems and practical considerations to support future research and deployment of multimodal unlearning. We release a curated repository: https://smsnobin77.github.io/Awesome-Multimodal-Unlearning/

Abstract

Machine Unlearning

Explore similar work

Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks

Should Missing Modalities Always Be Necessary to Repair for Multi-modal Sentiment Analysis?

SALMUBench: A Benchmark for Sensitive Association-Level Multimodal Unlearning