cs.CVApr 5, 2026

High-Fidelity Mural Restoration via a Unified Hybrid Mask-Aware Transformer

Authors: Jincheng Jiang, Qianhao Han, Chi Zhang, Zheng Zheng

Organizations: Northeastern University (Toronto Campus), Toronto, Canada · The Bishop Strachan School, Toronto, Canada

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Ancient murals are valuable cultural artifacts, but many have suffered severe degradation due to environmental exposure, material aging, and human activity. Restoring these artworks is challenging because it requires both reconstructing large missing structures and preserving authentic, undamaged regions. We present the Hybrid Mask-Aware Transformer (HMAT), a unified framework for high-fidelity mural restoration that addresses both structural completion and authentic-region preservation. HMAT integrates Mask-Aware Dynamic Filtering for robust local texture modeling with a Transformer bottleneck for long-range structural inference, enabling recovery of continuous line patterns and coherent mural structures under irregular damage. To handle diverse degradation morphologies, we introduce a mask-conditional style fusion module that adapts the generative process according to the shape and extent of missing regions. We also propose a fidelity-oriented training objective that combines hole-normalized reconstruction, discriminator feature matching, and high-receptive-field perceptual supervision to improve damaged-region fidelity, texture consistency, and boundary quality. In addition, we analyze a Teacher-Forcing Decoder with hard-gated skip connections as a feature-space boundary-conditioning strategy. Experiments show that HMAT matches or outperforms representative convolutional, transformer-based, and edge-guided inpainting baselines, with especially strong gains in perceptual realism and severe-mask settings. Ablation studies further identify the proposed objective, MADF-based mask-aware encoding, and mask-conditioned synthesis as the main contributors to restoration quality. These results demonstrate that HMAT provides an effective and competitive solution for cultural heritage mural restoration.

Figures & tables

Figure 1: Overview of the proposed Hybrid Mask-Aware Transformer (HMAT). The core architecture integrates Hybrid Encoder (MADF + Transformer) for robust feature extraction, Mask-Conditional Style Fusion (SF) to dynamically guide synthesis, and a Teacher-Forcing Decoder (TFD) with hard-gated skip fusion, whose empirical effect is analyzed in the ablation study. The resulting coarse structural completion is composited with the observed pixels and then processed by a second-stage synthesis network to produce the final restored image.
ConfigurationPSNR ↑\uparrowSSIM ↑\uparrowFID ↓\downarrowℓ1\ell_{1} ↓\downarrow
Equal Capacity27.220.90711.160.014
Heavy Semantic Bias26.580.89613.790.016
Baseline (Ours)28.470.91910.900.012
Table 1: Quantitative ablation on the dimensionality of the Mask-Conditional Style Fusion module.
Figure 2: Qualitative comparison of style dimensionality configurations on the Nine-Colored Deer dataset. To evaluate capacity distribution, we compare our Baseline ( simg=360,slatent=180,smask=64s_{img}=360,s_{latent}=180,s_{mask}=64 ) against Equal Capacity ( simg=180,slatent=180,smask=180s_{img}=180,s_{latent}=180,s_{mask}=180 ) and Heavy Semantic Bias ( simg=360,slatent=64,smask=16s_{img}=360,s_{latent}=64,s_{mask}=16 ).
 Moderate Mask (20%–30%)Severe Mask (40%–50%)
MethodPSNR ↑\uparrowSSIM ↑\uparrowFID ↓\downarrowℓ1\ell_{1} ↓\downarrowPSNR ↑\uparrowSSIM ↑\uparrowFID ↓\downarrowℓ1\ell_{1} ↓\downarrow
MADF24.030.88718.060.02219.720.78531.200.049
MAT22.530.86233.300.02718.950.74242.500.054
LaMa-Fourier25.300.89817.770.01921.350.81228.400.042
EdgeConnect23.750.88132.540.02419.300.76644.800.053
HMAT (Ours)25.300.89912.540.01921.850.82522.300.039
Table 2: Quantitative comparison with state-of-the-art methods on the DHMural and Nine-Colored Deer datasets. Metrics are computed on composited full images.
Figure 3: Qualitative comparison with state-of-the-art methods on the DHMural dataset. Red boxes indicate representative regions where competing methods exhibit broken contours, boundary artifacts, or texture degradation.
ComponentΔ\Delta Band PSNRΔ\Delta Band SSIMΔ\Delta Hole PSNRΔ\Delta Hole SSIM
Full HMAT0.00000.00000.00000.0000
w/o MADF-0.3991-0.0039-0.1996-0.0085
w/o Teacher Forcing+0.0016+0.0002+0.0009+0.0002
w/o Mask-Guided Style-0.2187-0.0024-0.1249-0.0062
w/o Proposed Objective-1.1467-0.0161-1.1108-0.0450
Table 3: Ablation study on the DHMural test set. We report metric changes relative to the full HMAT model. Positive values indicate improvement, while negative values indicate degradation.
More in this topic

Image Restoration

Image restoration uses computational methods to recover useful visual content from noisy, blurred, low-resolution, or otherwise degraded images.

See all 372 papers

Explore similar work

CardsList
Mar 16, 2026

M2IR: Proactive All-in-One Image Restoration via Mamba-style Modulation and Mixture-of-Experts

While Transformer-based architectures have dominated recent advances in all-in-one image restoration, they remain fundamentally reactive: propagating degradations rather than proactively suppressing them. In the absence of explicit suppression mechanisms, degraded signals interfere with feature learning, compelling the decoder to balance artifact removal and detail preservation, thereby increasing model complexity and limiting adaptability. To address these challenges, we propose M2IR, a novel restoration framework that proactively regulates degradation propagation during the encoding stage and efficiently eliminates residual degradations during decoding. Specifically, the Mamba-Style Transformer (MST) block performs pixel-wise selective state modulation to mitigate degradations while preserving structural integrity. In parallel, the Adaptive Degradation Expert Collaboration (ADEC) module utilizes degradation-specific experts guided by a DA-CLIP-driven router and complemented by a shared expert to eliminate residual degradations through targeted and cooperative restoration. By integrating the MST block and ADEC module, M2IR transitions from passive reaction to active degradation control, effectively harnessing learned representations to achieve superior generalization, enhanced adaptability, and refined recovery of fine-grained details across diverse all-in-one image restoration benchmarks. Our source codes are available at https://github.com/Im34v/M2IR.

Shiwei Wang, Yongzhen Wang, Bingwen Hu +3

School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China · School of Computer Science and Technology, Anhui University of Technology, Ma’anshan 243032, China · Shenzhen Ubiquitous Data Enabling Key Laboratory, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China +1

Degradation-Aware Image RestorationAll-in-One Image RestorationImage RestorationMixture of Experts
AskPDF
Jul 23, 2026

DART: A Degradation-Aware Recurrent Transformer for Archival Film Restoration

Archival film restoration is a challenging problem because historical footage contains compound degradations such as scratches, dust, blur, noise, flicker, and photometric aging, while clean reference videos are unavailable. Existing video restoration methods largely treat these degradations implicitly, reconstructing frames without explicit knowledge of where damage occurs or how severe it is. We propose DART, a degradation-aware recurrent transformer for archival film restoration. DART predicts and propagates a soft defect mask through time, using it to guide temporal fusion and condition the restoration network on both damage location and severity. This makes the restoration process explicitly aware of film artifacts rather than relying only on reconstruction losses. Experiments on real archival benchmarks show that DART improves no-reference perceptual quality over prior restoration architectures while remaining compact and efficient, producing cleaner and more temporally consistent restorations of structured film damage.

Mikołaj Jastrzębski, Wojciech Kozłowski, Kamil Adamczewski

Wrocław University of Science and Technology

Recurrent TransformersVideo RestorationDegradation-Aware Image Restoration
AskPDF
Aug 1, 2026

MDTD-ArtIR: Benchmarking Image Editing and Restoration Models for Art Image Restoration under Texture-Overlay Degradations

Restoring severely degraded visual media still remains a formidable challenge, as existing methods often hallucinate unnatural textures and contents, struggle with preserving color and texture, or fail to leverage partially retained image information. Existing restoration benchmarks assume known degradation operators and fail to capture the complex characteristics of artistic damage such as cracks, stains, and color/texture deviation. We introduce a controlled benchmark for blind restoration of semantic, semi-transparent image media degradations, accompanied by a new, publicly open degradation alpha texture mask dataset MDTD-Art. We present a new dataset and benchmark evaluating state-of-the-art universal restoration models against image editing and vision-language models across varying mask opacity levels. Our experiments demonstrate that image editing models consistently outperform specialized restoration architectures for arbitrary degradations, with performance gains amplified by structured prompt engineering emphasizing detail preservation and structural consistency. These findings position recoverable semantic information and prompt controllability as critical factors in art image restoration.

Mridula Vijendran, Shuang Chen, Hubert P. H. Shum

Department of Computer Science, Durham University, Durham, United Kingdom

Image Editing EvaluationImage Restoration
AskPDF