Learned Parametric Emotion Editing: Real-Time Affective Filtering for On-Device Social Media Video

Musa Rochi; Marcel Schubert; Christoph Gebhardt

cs.CVSep 18, 2026

Learned Parametric Emotion Editing: Real-Time Affective Filtering for On-Device Social Media Video

Authors: Musa Rochi, Marcel Schubert, Christoph Gebhardt

Organizations: Eastern Switzerland University of Applied Sciences (OST)

Abstract

Problematic internet use affects a growing share of the population, yet common interventions, e.g., time limits, blocking, forced breaks, are coercive and easily circumvented. We explore a less restrictive alternative: adapting the emotional intensity of visual content. Prior work has shown that optimization can steer an image's affective content, but its per-image optimization cost makes it impractical for real-time deployment. We instead learn a model that predicts this transformation in a single forward pass: a MobileNetV4 backbone with FiLM-based emotion conditioning outputs parameters for differentiable global transformations. This replaces prior iterative optimization (80 s per image) with a single 3.7 ms forward pass. In a user study (N = 54), the model reduced viewer-reported arousal relative to unedited images, comparably to the grayscale well-being filter, while being rated higher in perceived quality. We integrate the model into an Android app that adapts Instagram video in real time, sustaining 60 fps on a Samsung Galaxy S23.

Figures & tables

Figure 1: Overview of our approach : An original social media feed frame (1) is captured and processed by our learned filter (2), which predicts parameters for a sequence of differentiable global transformations in a single forward pass. The adapted frame is rendered back into the live feed (3), reducing viewer-reported arousal while preserving perceived quality.

Figure 2: Conditional parametric generator. The conditioning signal $\alpha$ and the input image $I$ are encoded separately and fused via three FiLM blocks before three prediction heads regress the parameters $p$ of the global transformation $T$ .

Figure 3: End-to-end pipeline. Render Frame executes on every animation frame; inference ( Send Frame

\to

Parameters ) is triggered only every

n

-th frame and runs asynchronously off the main thread. Predicted parameters are smoothed ( Eq. 3 ) before being applied to subsequent frames.

Configuration	Loss	Arousal MSE	CLIP loss
FiLM Sigmoid	0.0405	0.0341	0.1271
FiLM Sigmoid (128)	0.0407	0.0342	0.1311
FiLM TanH	0.0422	0.0360	0.1236
No FiLM (Sigmoid)	0.0431	0.0373	0.1160
No FiLM (TanH)	0.0432	0.0374	0.1162
$\lambda_{\mathrm{clip}}=0.0$	0.0362	0.0362	0.1920

Table 1: Validation metrics at epoch 25 across five architecture configurations (top) and five different

\lambda_{\mathrm{clip}}

-values for the FiLM Sigmoid architecture (bottom); the two blocks come from separate training runs. Loss

=

Arousal MSE

+\lambda_{\mathrm{clip}}\cdot

CLIP loss ( Eq. 1 ); CLIP loss as defined in Eq. 2 .

Figure 4: Qualitative results. Example edits for (a) decreasing $\alpha$ (vintage, desaturated shift) and (b) increasing $\alpha$ (magenta/pink shift), annotated with regressor-predicted valence (V) and arousal (A) per image.

Panel of Figure 5 — Figure 5: Baseline comparison. (a) Achieved $\Delta$ arousal vs. requested $\alpha$ ; our model’s response is closer to the identity line $\Delta\text{arousal}=\alpha$ than baselines. (b) Content preservation (DINO) vs. $\alpha$ ; our model achieves higher similarity near $\alpha=0$ but drops more sharply than baselines beyond $|\alpha|\approx 0.1$ – $0.2$ .

Figure 6: Example stimulus of user study. Stimulus across the five study conditions: original images, our approach at $\alpha=-0.11$ , $\alpha=-0.12$ , and $\alpha=-0.13$ , and the grayscale baseline.

Condition	Arousal	Quality	Valence
original	4.55 (1.98)	3.97 (0.68)	6.34 (1.33)
alpha-1	4.36 (1.94) ∗	3.79 (0.76) ∗†	6.18 (1.43) ∗
alpha-2	4.34 (1.98) ∗	3.71 (0.78) ∗∗	6.12 (1.52) ∗
alpha-3	4.28 (2.10) ∗	3.40 (0.92) ∗∗∗†	6.01 (1.56) ∗∗∗
grayscale	4.19 (1.97) ∗∗	3.64 (0.74) ∗∗∗	6.05 (1.28) ∗∗

Table 2: Rating statistics. Mean ratings (SD in parentheses) for arousal, perceived image quality, and valence per condition. Asterisks mark differences from original and daggers from grayscale (

{}^{*}p<.05

{}^{**}p<.01

{}^{***}p<.001

{}^{\dagger}p<.05

). The shaded row marks our recommended operating point.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Training procedure. Solid arrows show the forward pass: a sampled conditioning value $\alpha$ and the frozen regressor’s estimate of $I$ define the target score $a^{\prime}=\mathrm{clip}(R_{a}(I){+}\alpha,0,1)$ ; the generator $f_{\theta}$ predicts transform parameters $p$ , producing $\hat{I}=T(I,p)$ , which is re-scored by $R_{a}$ and compared against $a^{\prime}$ ( $\mathcal{L}_{\text{affect}}$ ), while a frozen CLIP encoder supplies a semantic-preservation term ( $\mathcal{L}_{\text{CLIP}}$ ) against the original image. The dashed red arrow shows the backward pass: gradients of the combined objective $\mathcal{L}(\theta)$ propagate back through $R_{a}$ , the CLIP encoder, and $T$ into $f_{\theta}$ ; $R_{a}$ and the CLIP encoder (boxes with locks) remain frozen throughout.

Operation	Dim.	Raw Range	Range $[m,M]$	Identity
Exposure	1	$(-\infty,\infty)$	$[-0.1,1]$	0.0
Saturation	1	$[0,\infty)$	$[0.5,3]$	1.0
Tone curve	$K$	$(-\infty,\infty)$	$[0,3]$	1.0
Color curve	$3K$	$(-\infty,\infty)$	$[0,3]$	1.0
Contrast	1	$[0,\infty)$	$[0.5,3]$	1.0
Sharpness	1	$[0,\infty)$	$[0,3]$	1.0

Appendix

Table 3: Sub-transforms composing

T

. Raw range is the domain of the sub-transform; range is the interval

[m,M]

enforced by

f_{\mathrm{sig}}

( Eq. 4 ) on the values predicted by

f_{\theta}

. Parameters are applied in the fixed order listed above.

Setting	Value
Optimizer	Adam [ 32 ]
Learning rate	$1\times 10^{-4}$
Batch size	32 (4 $\times$ 8 GPUs)
Hardware	8 $\times$ NVIDIA Tesla P100
Dataset	MS-COCO [ 38 ]
$\lambda_{\mathrm{clip}}$	0.05

Appendix

Table 4: Training configuration for the parametric generator.

Panel of Figure 8 — Figure 8: Full training curves for the FiLM ablation. Validation overall loss, arousal MSE term, and CLIP preservation loss over training, for all five configurations described in Sec. C.1.1 .

Panel of Figure 9 — Figure 9: Full training curves for the $\lambda_{\mathrm{clip}}$ ablation. Validation overall loss, arousal MSE term, and CLIP preservation loss over training, for all five $\lambda_{\mathrm{clip}}$ values described in Sec. C.1.2 .

		Prior literature
Attribute	Computation	Valence	Arousal	References
Brightness	Mean grayscale pixel intensity.	$(+)$	–	[ 61 , 23 ]
Saturation	Mean of the HSV saturation channel.	$(+)$	$(+)$	[ 60 , 55 ]
Contrast	Std. dev. of grayscale pixel intensities.	$(+)$	–	[ 65 ]
Colorfulness	Std. dev. plus weighted mean of distances from the mean in the LAB $a$ / $b$ channels.	$(+)/(-)$ a	$(+)$	[ 5 , 12 ]
Blur	Laplacian variance of grayscale image (high $=$ less blur).	$(-)$	$(-)$	[ 17 ]

Appendix

Table 5: Image attribute metrics and their affective associations. Associations are reported from prior literature, where

(+)

(-)

denote a positive/negative association and “–” indicates none reported.

Panel of Figure 10 — Figure 10: Regressor-predicted arousal and valence as a function of $\alpha$ . (a) Arousal increases monotonically with $\alpha$ and saturates beyond $|\alpha|\gtrsim 0.2$ . (b) Valence, not directly controlled, shows similar turning points, indicating an arousal influence.

Figure 11: Preservation metrics over

\alpha

. FID, KID and L1 reconstruction error as a function of the conditioning value

\alpha

, showing the parabolic increase near

\alpha=0

followed by a sharp transition and subsequent flattening.

Figure 12: Evolution of image attributes over

\alpha

. All six attributes from Tab. 5 , plotted against the conditioning value

\alpha

Method	Median	Mean	p95	Min	Max
Ours (ms)	3.71	3.72	4.34	3.05	4.99
[ 24 ] (s)	80.92	80.71	83.01	78.81	83.10

Appendix

Table 6: Per-frame latency on training hardware. Ours is a forward pass (

n=5{,}000

); [ 24 ] is a 300-step edit (

n=250

Stage	Median	Mean	p95	Min	Max
Copy to canvas	0.00	0.05	0.10	0.00	3.30
Extract pixels	7.90	8.98	17.00	2.70	42.20
Inference	37.90	40.48	61.60	21.80	114.50
Total	46.00	49.52	77.80	24.60	138.50

Appendix

Table 7: Per-frame latency (ms) on the target device. Computed on a Samsung Galaxy S23 using

n=1{,}500

frames.

Component	Mean $\Delta$ (mAh)	SD
Total	+31.86	10.65
CPU	+25.13	13.78
Screen	+5.74	8.73
Video	+0.24	0.65
WiFi	$-$ 0.00	0.02

Appendix

Table 8: On-device energy cost. Paired delta (filter-on minus filter-off) in app-attributed power use, per hardware component, across five matched 30-minute sessions on a Samsung Galaxy S23.

Panel of Figure 13 — Figure 13: Rating differences. Per-participant rating differences between each condition and original for (a) arousal, (b) perceived image quality, and (c) valence. Each box summarizes one difference score per participant; values below the dashed zero line indicate lower ratings than for the original images. Significance notation as in Tab. 2 .

Explore similar work

MooD: Perception-Enhanced Efficient Affective Image Editing via Continuous Valence-Arousal Modeling
May 4, 2026Xinyi Yin, Yiduo Wang, Tingqi Hu +6Valence-Arousal Modeling Image Editing
School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, China · School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China · School of Journalism and New Media, Xi’an Jiaotong University, Xi’an 710049, China +1
What We Talk About When We Talk About Emotion Editing
Oct 9, 2026Xingzu Zhan, Lin Gu, Ruogu FangImage Editing Affective Computing
Department of Biomedical Engineering, Vanderbilt University, Nashville, Tennessee, USA · Research Institute of Electrical Communication, Tohoku University, Sendai, Japan
EMOVIS: Emotion-Optimized Image Processing
May 4, 2026Dor Barber, Rony Zatzarinni, Hava Matichin +1Image Enhancement
Intel Corporation

May 4, 2026J/K move · Enter open · S save

MooD: Perception-Enhanced Efficient Affective Image Editing via Continuous Valence-Arousal Modeling

Xinyi Yin, Yiduo Wang, Tingqi Hu, Meicong Si+5

School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, China · School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China · School of Journalism and New Media, Xi’an Jiaotong University, Xi’an 710049, China+1

Valence-Arousal Modeling Image Editing Controllable Image Generation Affective Computing

Open paper AskPDF arXiv

Affective Image Editing (AIE) aims to modify visual content to evoke targeted emotions. Although current approaches achieve impressive editing quality, they often overlook inference efficiency, which limits their applicability in computational social scenarios. Moreover, most methods depend on discrete emotion representations, which hinder the continuous modeling of complex human emotions and constrain expressive capabilities in interactive scenarios. To tackle these gaps, we propose MooD, the first framework that directly leverages continuous Valence-Arousal (VA) values as editing instruction for fine-grained and efficient AIE in computational social systems. Specifically, we first introduce a VA-Aware retrieval strategy to bridge vague affective values and detailed visual semantics. Building upon this, MooD integrates visual transfer and perception-enhanced semantic guidance to achieve controllable AIE. Furthermore, considering that existing VA-annotated datasets mainly focus on social scenarios and largely overlook natural scenes, we therefore construct AffectSet, a comprehensive VA-annotated dataset covering diverse scenarios, to support model optimization and evaluation. Extensive qualitative and quantitative experimental results demonstrate that our MooD achieves superior performance in both affective controllability and visual fidelity while maintaining high efficiency. A series of ablation studies further reveal the crucial factors of our design.

Abstract

Explore similar work

MooD: Perception-Enhanced Efficient Affective Image Editing via Continuous Valence-Arousal Modeling

What We Talk About When We Talk About Emotion Editing

EMOVIS: Emotion-Optimized Image Processing