Shadow Feature Refinement Network: Progressive Feature Refinement based on Knowledge Distillation for Effective Shadow Removal

Donghyun Han; Byoung-Dai Lee

cs.CVOct 4, 2026

Shadow Feature Refinement Network: Progressive Feature Refinement based on Knowledge Distillation for Effective Shadow Removal

Authors: Donghyun Han, Byoung-Dai Lee

Organizations: Kyonggi University, Korea

Abstract

In the field of deep learning, has seen significant advancements; however, shadow removal remains a persistent challenge owing to the variable sizes and colors of shadows influenced by lighting conditions. This study proposes a novel shadow feature refinement network (SFR-Net), which leverages supervised learning, feature refinement loss, and knowledge distillation to enhance shadow removal performance. A dedicated post-processing algorithm is further introduced to restore natural color consistency in the generated shadow-free images. We evaluated our method on two public datasets: the adjusted image shadow triplet dataset (ISTD+) and the shadow removal dataset (SRD), which demonstrate strong generalization capabilities under diverse conditions. On ISTD+, our model achieved a root mean square error (RMSE) of 3.4627 and structural similarity index measure (SSIM) of 0.9382 across the entire image. On SRD, it recorded an RMSE of 4.3781 and an SSIM of 0.9341. These comprehensive results show that our approach performs competitively across both shadow and non-shadow regions while setting a promising direction for robust and perceptually natural shadow removal. Code is available at https://github.com/DongHyun99/SFRNet.

Figures & tables

Figure 1: Network architecture of the proposed shadow feature refinement network.

Figure 2: Architecture of FFC block and spectral transform.

Figure 3: Comparison of SFR-Net’s shadow-removed feature maps. (a) Without $L_{fr}$ , (b) With $L_{fr}$ .

Panel of Figure 4 — Figure 4: Qualitative evaluation results of SFR-Net and state-of-the-art shadow removal methods for the ISTD+ dataset. (a) Input, (b) ShadowDiffusion [ 14 ] , (c) HomoFormer [ 43 ] , (d) ShadowFormer [ 13 ] , (e) BMNet [ 46 ] , (f) Ours, (g) InstanceShadowDiffusion [ 31 ] , (h) GT.

Method	Shadow Region			Non-Shadow Region			All
Method	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
Shadow Image	30.4831	0.9265	20.8095	3.1493	0.9552	33.8855	12.4080	0.8747	20.2209
ShadowDiffusion [ 14 ]	5.8667	0.9782	35.8192	3.2892	0.9253	32.8163	3.8768	0.8995	30.3881
HomoFormer [ 43 ]	5.0098	0.9840	37.4862	3.1282	0.9350	33.3300	3.5245	0.9161	31.3985
Inpaint4Shadow [ 27 ]	6.5203	0.9621	35.2136	4.4683	0.8234	29.8008	4.9124	0.7783	28.2474
ShadowFormer [ 13 ]	5.0755	0.9864	38.0692	2.8663	0.9550	35.1463	3.3461	0.9387	32.7827

Table 1: Quantitative evaluation results of SFR-Net and state-of-the-art shadow removal methods on the ISTD+ dataset.

Panel of Figure 5 — Figure 5: Qualitative evaluation results of SFR-Net and state-of-the-art shadow removal methods for the SRD dataset. (a) Input, (b) ShadowDiffusion [ 14 ] , (c) HomoFormer [ 43 ] , (d) ShadowFormer [ 13 ] , (e) BMNet [ 46 ] , (f) Ours, (g) InstanceShadowDiffusion [ 31 ] , (h) GT.

Method	Shadow Region			Non-Shadow Region			All
Method	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
Shadow Image	17.3859	0.8694	18.7869	2.7657	0.9683	30.2704	16.3193	0.8206	18.0297
ShadowDiffusion [ 14 ]	4.1636	0.9662	31.7264	2.5340	0.9651	31.2735	5.3763	0.9222	27.8394
HomoFormer [ 43 ]	3.4217	0.9676	33.1020	2.2344	0.9502	31.9545	4.5635	0.8988	28.8686
Inpaint4Shadow [ 27 ]	3.8828	0.9713	32.2222	2.1370	0.9739	32.7109	4.8368	0.9324	28.7194
ShadowFormer [ 13 ]	3.5833	0.9739	33.0057	1.9617	0.9758	33.7010	4.4366	0.9374	29.6462

Table 2: Quantitative evaluation results of SFR-Net and state-of-the-art shadow removal methods on the SRD dataset.

Method	ISTD+ Dataset			SRD Dataset
Method	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
ShadowDiffusion [ 14 ]	2.495	0.9424	34.4869	2.7020	0.9755	30.3506
HomoFormer [ 43 ]	1.6334	0.9863	38.7659	3.0385	0.9457	28.8426
Inpaint4Shadow [ 27 ]	3.8885	0.8237	30.2485	2.5566	0.9617	30.0065
ShadowFormer [ 13 ]	1.2962	0.9909	41.0936	2.4080	0.9649	30.6063
BMNet [ 46 ]	1.4613	0.9928	40.0063	2.6261	0.9633	29.9894

Table 3: Quantitative evaluation results of SFR-Net and state-of-the-art shadow removal methods for the non-shadow region of the original shadow image

I_{S}

on the ISTD+ and SRD datasets.

Figure 6: Division of layers into three stages for analyzing the effectiveness of knowledge distillation

Panel of Figure 7 — Figure 7: Matte images when $L_{fr}$ is applied to SFR-Net and when $L_{fr}$ is not applied to SFR-Net. (a) $I_{s}$ , (b) With $L_{fr}$ , (c) Without $L_{fr}$ , (d) $I_{gt}$

Method	Shadow Region			Non-Shadow Region			All
Method	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
Config. #1 (Shallow Only)	6.0479	0.9852	35.9776	3.6449	0.9515	32.5604	4.1722	0.9332	30.2424
Config. #2 (Middle Only)	5.5084	0.9858	36.8616	3.4796	0.9542	33.7787	3.9229	0.9369	31.4653
Config. #3 (Deep Only)	5.2107	0.9862	37.4321	3.2502	0.9542	34.0711	3.6675	0.9372	31.8471
Config. #4 (Shallow & Middle)	5.6130	0.9849	36.5651	3.3170	0.9545	33.6416	3.8253	0.9360	31.3321
Config. #5 (Middle & Deep)	5.4761	0.9857	37.1288	3.3211	0.9518	34.0083	3.7688	0.9341	31.7260

Table 4: Evaluation of shadow removal performance when feature refinement loss is applied to each stage of SFR-Net.

Method			Shadow Region			Non-Shadow Region			All
$L_{LaMa}$	$L_{mix}$	$L_{fr}$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
$\checkmark$			7.4366	0.9774	34.0190	5.1019	0.9249	29.7530	5.6161	0.8976	27.8767
$\checkmark$	$\checkmark$		6.0975	0.9849	35.4867	3.7747	0.9533	31.7673	4.2855	0.9347	29.6694
$\checkmark$		$\checkmark$	6.4203	0.9827	35.6343	4.1685	0.9472	31.5645	4.6311	0.9263	29.7403
$\checkmark$	$\checkmark$	$\checkmark$	5.2107	0.9862	37.4321	3.2502	0.9542	34.0711	3.6675	0.9372	31.8471

Table 5: Evaluation of shadow removal performance of SFR-Net when

L_{LaMa}

L_{mix}

, and

L_{fr}

are applied.

Method	Shadow Region			Non-Shadow Region			All			Processing Time (second)
	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
SFR-Net (default)	5.2107	0.9862	37.4321	3.2502	0.9542	34.0711	3.6675	0.9372	31.8471	–
BGR Space
+ linear regression	4.9750	0.9865	37.8747	3.0631	0.9550	34.5793	3.4627	0.9382	32.3872	0.068
+ polynomial regression	5.2444	0.9860	37.6492	3.2386	0.9553	34.3712	3.6599	0.9380	32.1919	0.187
+ histogram matching	10.8398	0.9698	32.9278	3.2208	0.9541	34.2324	5.0366	0.9185	29.9032	0.050

Table 6: Quantitative comparison of different post-processing techniques on the ISTD+ dataset.

Method	Shadow Region			Non-Shadow Region			All			Processing Time (second)
	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	RMSE $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
SFR-Net (default)	3.7840	0.9719	32.9293	2.5129	0.9690	33.5797	5.2807	0.9266	29.4810	–
BGR Space
+ linear regression	3.5184	0.9744	33.1909	1.9665	0.9727	33.7399	4.3781	0.9341	29.8287	0.046
+ polynomial regression	3.8751	0.9719	32.8250	2.4020	0.9703	33.6360	5.1542	0.9277	29.4965	0.146
+ histogram matching	6.8171	0.9445	28.2236	1.9761	0.9743	33.7197	6.9081	0.9037	26.4355	0.030

Table 7: Quantitative comparison of different post-processing techniques on the SRD dataset.

Panel of Figure 8 — Figure 8: Comparison of color histograms when color restoration post-processing is applied to SFR-Net. (a): Blue Space Pixel Histogram, (b): Green Space Pixel Histogram, (c): Red Space Pixel Histogram, (d): $I_{gt}$ , (e): $I_{s}$ , (f): $I_{sf}$ , (g): $I_{sf-pp}$

Panel of Figure 9 — Figure 9: Qualitative evaluation results of post-processing algorithm for color restoration. (a): $I_{sf}$ , (b): linear regression, (c): polynomial regression, (d): Reinhard et al. [ 33 ] , (e): gradient boosting, (f): gamma correction, (g): $I_{s}$

Explore similar work

ODDR: One-Step Deshadow Diffusion via Reward Guidance
Oct 1, 2026Junseong Shin, Kijun Kim, Minseong Kim +2Video Object Removal Diverse Occlusion-And-Revelation Scenarios
Department of Artificial Intelligence, Hanyang University · AX Future Technology Institute, KT · Department of Computer Science, Hanyang University
Consistency as Regularization for Unsupervised Shadow Removal
Sep 1, 2026Anh-Kiet Duong, Petra Gomez-Krämer, Jean-Michel CarozzaVideo Object Removal Unsupervised
L3i Laboratory, La Rochelle University, 17042 La Rochelle Cedex 1, France · LIENSs Laboratory, La Rochelle University, 17042 La Rochelle Cedex 1, France
FreeShadow: Training-Free Shadow Removal via Illumination Transfer and Selective Content Preservation in Diffusion Models
Jul 29, 2026Yinan Wang, Yan Huang, Yong Xu +1Diffusion Models Illumination Variations
School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China · Pazhou Lab, Guangzhou 510005, China · Polytech Nantes, Universit´e de Nantes, Nantes 44306, France

Oct 1, 2026, cs.CVJ/K move · Enter open · S save

ODDR: One-Step Deshadow Diffusion via Reward Guidance

Junseong Shin, Kijun Kim, Minseong Kim, Dongjin Kim+1

Department of Artificial Intelligence, Hanyang University · AX Future Technology Institute, KT · Department of Computer Science, Hanyang University

Video Object Removal Diverse Occlusion-And-Revelation Scenarios Synthetic Data

Open paper AskPDF arXiv

Recent advances in deep learning for shadow removal have significantly enhanced image quality and realism. However, most approaches rely on real-world paired datasets, which are costly to collect and often limited in scene diversity, leading to limited generalization. To address these limitations, we propose One-step Deshadow Diffusion via Reward guidance (ODDR), a new framework that achieves efficient and high-fidelity shadow removal without relying on real-world paired supervision. Our method begins with One-step Deshadow Diffusion (ODD), a baseline model trained on synthetic shadow data for efficient one-step shadow-free reconstruction. We further adapt ODD into ODDR using ShadowReward. In contrast to traditional, annotation-heavy approaches, ShadowReward is the first reward model for shadow removal trained entirely without human annotation. It learns to mimic human perceptual judgments by ranking synthetically generated images with controlled degradations, such as texture distortion and boundary artifacts. This reward-guided fine-tuning enables ODDR to close the synthetic-to-real domain gap. Extensive experiments show that ODD achieves strong performance without relying on real-world paired supervision, and ODDR further improves the results, narrowing the gap to fully supervised methods trained on real-world paired data while maintaining higher computational efficiency as a single-step model.

Abstract

Explore similar work

ODDR: One-Step Deshadow Diffusion via Reward Guidance

Consistency as Regularization for Unsupervised Shadow Removal

FreeShadow: Training-Free Shadow Removal via Illumination Transfer and Selective Content Preservation in Diffusion Models