Spatial-Temporal Multi-scale Network for Screen Content Video Quality Enhancement

Ziyin Huang; Sik-Ho Tsang; Xinyuan Qin; Yui-Lam Chan; Xueling Zhou; Feiyu Chen

cs.CVSep 30, 2026

Spatial-Temporal Multi-scale Network for Screen Content Video Quality Enhancement

Authors: Ziyin Huang, Sik-Ho Tsang, Xinyuan Qin, Yui-Lam Chan, Xueling Zhou, Feiyu Chen

Organizations: School of Artificial Intelligence, Shenzhen Polytechnic University, Shenzhen, China · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, China · Department of Computer Science, Hong Kong Chu Hai College, Hong Kong, China · College of Eilte Engineers, Dongguan University of Technology, Dongguan, China · Department of Computer Science, City University of Hong Kong, Hong Kong, China

Abstract

Different from natural videos, Screen Content Videos (SCVs) are characterized by abrupt motion, scene switches, and high-frequency details such as text and graphics. Conventional video enhancement methods, which rely heavily on temporal continuity, often suffer from performance degradation when processing SCVs due to the disruption of temporal correlations. To address these challenges, we propose the Spatial-Temporal Multi-scale Network (STM-Net), a novel framework specifically tailored for compressed SCV enhancement. Our approach integrates three complementary components: a Prior-Guided Spatio-Temporal Dispatcher (PG-STD) that routes input into three parallel streams to avoid feature contamination, a Bidirectional Temporal Feature Extraction (BTFE) module that adaptively handles abrupt transitions without explicit detection, and a Cascaded Multi-scale Feature Distillation (CMFD) module that preserves critical high-frequency details. Experimental results demonstrate that STM-Net outperforms state-of-the-art methods in both objective metrics and subjective visual quality, providing a robust solution for screen content artifacts. Code is available at https://github.com/HUANGZiyin1/STM-Net.

Figures & tables

Fig. 1: The proposed STM-Net structure, comprising Prior-Guided Spatio-Temporal Dispatcher (PG-STD), Bidirectional Temporal Feature Extraction (BTFE) module, and Cascaded Multi-scale Feature Distillation (CMFD) module.

Seq. (QP=37)	STDF-R3 [ 14 ]		QECF [ 15 ]		CAT [ 27 ]		CF-STIF [ 22 ]		STDR [ 23 ]		TGAF [ 24 ]		EAST-LITE [ 25 ]		STM-Net		STM-Net-L
Seq. (QP=37)	$\Delta$ P	$\Delta$ S	$\Delta$ P	$\Delta$ S	$\Delta$ P	$\Delta$ S	$\Delta$ P	$\Delta$ S	$\Delta$ P	$\Delta$ S	$\Delta$ P	$\Delta$ S	$\Delta$ P	$\Delta$ S	$\Delta$ P	$\Delta$ S	$\Delta$ P	$\Delta$ S
BigBuck	0.327	3.18	0.325	3.23	0.318	4.19	0.369	3.98	0.408	4.33	0.432	4.11	0.411	4.82	0.438	4.61	0.485	5.04
ChineseEditing	0.273	2.22	0.244	1.63	0.200	1.24	0.360	2.41	0.321	1.77	0.48	3.79	0.525	4.17	0.544	3.68	0.65	4.68
EnglishDoc	0.867	2.78	0.770	2.70	0.951	3.27	1.243	3.41	1.266	4.01	1.101	3.63	0.991	3.36	1.194	3.46	1.275	4.04
MissionControl1	0.492	4.17	0.503	4.12	0.477	4.00	0.625	4.72	0.546	4.59	0.721	5.73	0.647	6.02	0.760	5.94	0.782	6.39
MissionControl2	0.569	5.33	0.563	5.19	0.568	5.18	0.672	5.67	0.689	6.08	0.677	5.38	0.612	5.55	0.700	5.48	0.748	5.94

TABLE I:

\Delta

PSNR (

\Delta

P) (dB) and

\Delta

SSIM (

\Delta

S) (

\times 10^{-3}

) at QP=37, average results for QP=32, 27, 22, and BD-Rate comparison.

Fig. 2:

\Delta

PSNR curves for mixvideo ; dashed lines mark scene switches.

Fig. 3: Subjective visual quality comparison at QP=37. Top: Textual content enhancement on Sephora . Bottom: Graphical content enhancement on ChineseEditing . STM-Net restores clearer texts and sharper edges compared to state-of-the-art methods.

Model Structure	STM-Net	STM-Nscale	STM-NHF	STM-Net-S
Multi-scale Feature Extraction	✓	$\times$	✓	✓
High-Frequency Processing	✓	✓	$\times$	✓
Number of RBs	3	3	3	2
Number of MSFDBs	3	3	3	2
$\Delta$ PSNR (dB)	0.875	0.839	0.834	0.798

TABLE II: Ablation Study of Different Architecture Components in STM-Net at QP=37

Fig. 4: Visualization of Intermediate Features on Scene Switch. ( Left : BTFE Features Suppressed for Previous Frames; Middle : BTFE Features Emphasized for Future Frames, e.g.: left menu changed; Right : CMFD Features Emphasized for Screen Content Edges at Current Frame)

Model	STDF-R3	QECF	CAT	CF-STIF	STDR	STM-Net-S	STM-Net	STM-Net-L
$\Delta$ PSNR (dB)	0.632	0.615	0.641	0.800	0.777	0.798	0.875	0.949
Parameters (KB)	364.51	773.31	848.55	1242.10	1521.13	674.37	1009.75	1345.13

TABLE III: COMPARISON OF MODEL SIZES

Explore similar work

NeR-SC: Adapting Neural Video Representation to Screen Content
May 26, 2026Ruohan Shi, Jiaoyan Zhao, Haogang FengVideo Compression Neural Representations
Management school The University of Sheffield · Undergraduate School of Artificial Intelligence Shenzhen Polytechnic University · School of Artificial Intelligence Shenzhen University of Information Technology
Frequency Decoupled Framework for Screen Content Image Super-Resolution
Jun 8, 2026Xufei Wang, Qicheng Zhang, Qi Wu +2
School of Electronic and Information Engineering, Anhui University, Hefei 230039, China
DiffCVE: Diffusion-based Compressed Video Enhancement
Jul 8, 2026Wenqiang Xiao, Wenzhuo Ma, Junxi Zhang +1Video Compression Perceptual Quality
School of Remote Sensing and Information Engineering, Wuhan University

May 26, 2026, cs.CVJ/K move · Enter open · S save

NeR-SC: Adapting Neural Video Representation to Screen Content

Ruohan Shi, Jiaoyan Zhao, Haogang Feng

Management school The University of Sheffield · Undergraduate School of Artificial Intelligence Shenzhen Polytechnic University · School of Artificial Intelligence Shenzhen University of Information Technology

Video Compression Neural Representations

Open paper AskPDF arXiv

Implicit neural representations have emerged as a promising paradigm for video compression, with recent methods achieving competitive performance on natural video. However, screen content video -- common in remote desktop, online education, and cloud gaming -- exhibits distinct statistics: sharp edges, limited color palettes, and strong temporal redundancy. Existing neural representation methods, designed for natural scenes, lack mechanisms to exploit these properties, leaving substantial room for improvement. In this paper, we propose NeR-SC, a neural representation framework tailored for screen content video. Building on the SNeRV backbone, NeR-SC introduces three screen-content-specific modules: (i) a learnable color palette that models the discrete color structure of screen content by restricting the low-frequency sub-band to a learned color set; (ii) a multi-gate dense fusion module that replaces sequential feature fusion with dense, attention-gated cross-stage interaction; and (iii) an embedding-level frame skip strategy that bypasses redundant decoder invocations for static frames, with zero training overhead. Experiments on DSCVC and VCD show that NeR-SC achieves 40.32~~dB and 41.73~~dB average PSNR, outperforming representative neural video representation methods and, at low bitrates, surpassing H.264 and H.265. The skip strategy enables real-time decoding with no loss in quality.

Abstract

Explore similar work

NeR-SC: Adapting Neural Video Representation to Screen Content

Frequency Decoupled Framework for Screen Content Image Super-Resolution

DiffCVE: Diffusion-based Compressed Video Enhancement