Authors: Freddie Åström, Michael Felsberg, George Baravdish
Organizations: Heidelberg University Heidelberg Collaboratory for Image Processing (HCI) Im Neuenheimer Feld 205 69 120 Heidelberg, Germany · Linköping University Department of Electrical Engineering 581 83 Linköping, Sweden · Linköping University Department of Science and Technology 581 83 Linköping, Sweden
In this work, we introduce a novel tensor-based functional for targeted image enhancement and denoising. Via explicit regularization, our formulation incorporates application dependent and contextual information using first principles. Few works in literature treat variational models that describe both application dependent information and contextual knowledge of the denoising problem. We prove the existence of a minimizer and present results on tensor symmetry constraints, convexity, and geometric interpretation of the proposed functional. We show that our framework excels in applications where nonlinear functions are present such as in gamma correction and targeted value range filtering. We also study general denoising performance where we show comparable results to dedicated PDE-based state of the art methods.
Figures & tables
Figure 1: The presented framework studies image denoising from the viewpoint that the image data is not only corrupted by “noise” but also pertubated by some nonlinear function. In this figure, the nonlinear function corrects for the overexposed image. We incorporate this nonlinear function called a “mapping” as a feedback component in the iterative update scheme. For additional examples of the mapping function see Figure 2 .
Figure 2: In this work we introduce a mapping function in the energy ( 2 ). We consider three applications, each with different definitions of the mapping. First we perform gamma correction, illustrated on the first row. In the second application (second row) we perform targeted filtering. In the third application we perform general denoising by estimating the mapping from the image data via an oversegmentation map.
Figure 3: Synthetic example where an image has been corrupted with additive Gaussian noise with standard deviation 20 in a specific value range. By appropriately defining the mapping in ( 2 ) we are able to preserve image details oversmoothed by MRF, TGV and BM3D. In this example the mapping function was of sigmoidal type, ( 3 ) with a=100 and c=1/2 .
Applications
I . Gamma correction
Global mapping applied pixel-wise
II . Targeted filtering
Local mapping applied globally (on the value range)
III . Oversegmentation
Local mappings applied locally (within one segment)
Table 1: Overview of the different applications considered in this work and their relation to local and global mappings. A discussion on the difference between these two classes of mappings is given in Section 10 . Figure 2 shows examples of the different applications. For additional details, we refer to the respective section in the experimental evaluation.
Methods
Regularizer
M in E-L equation
Comment
Without mapping
u−u0−λdiv(M∇u)=0
Isotropic
∫Ω∣∇u∣2dx
I
TV Rudin et al. (1992)
∫Ω∣∇u∣dx
I/∣∇u∣
EAD Åström et al. (2012)
∫Ω∇u⊤W(∇u)∇udx
W+W⊤+Ws
GETV Åström et al. (2015)
∫Ω∣∇u∣(λ1(v⋅ξ)2+λ2(w⋅ξ)2)dx
S
see ( 63 ) with m(u)=u
With mapping
u−u0−λm′(u)div(M∇u)=0
Table 2: Examples of E-L equations for different choices of mapping functions and tensors. We have listed linear diffusion, total variation (TV), extended anisotropic diffusion (EAD), gradient energy total variation (GETV), targeted iterative filtering (TIF) and density driven diffusion (D3). The corresponding boundary conditions can be computed using ( 30a ) and ( 30b ). The methods TVm and GETVm are extensions, introduced in this work, of TV and GETV with a mapping function.
Figure 4: (a) Illustration of eigenvector basis at coordinate (x,y) . Dashed (red) arrows indicate the eigenvectors of W and S and thick (black) arrows ∇u∇u⊤ . (b) Illustration of the paraboloid ( 47 ) where we set c=τ1q12+τ2q22 and τ1,τ2≥0 .
Figure 5: Visualization of estimated local orientations for (a) the structure tensor and (b) the gradient energy tensor.
Figure 6: Results of sensitivity analysis of orientation estimation by using the structure tensor and the gradient energy tensor. We used the radially symmetric pattern (a) for different noise levels in Figure 7 . (a) shows the mean absolute angular error. (b) illustrates the percentage of pixels which has an angular error smaller than 1 degree. (c) shows the histogram of angular errors divided into 10 bins from 0 to π radians. See text for details.
Figure 8: Mapping function for gamma correction. Illustration of different gamma parameters (dash-dotted), corresponding compensation curves (dashed) and the compensation curve with 10% error (thick). The dash-dotted curve is used to obtain the noisy images and the thick curve is used to obtain the denoised results in Figure 9 .
ISO/m , TV/m
GETV/m
λ
0.1
0.05
τ
λ/8
β
10−12
γ
1
k
100
Table 3: Parameters of the numerical scheme were fixed for all applications. “ /m ” denotes with and without mapping. The abbreviations stand for isotropic diffusion (ISO), isotropic diffusion with a mapping function (ISOm) Åström et al. (2013b) , total variation (TV) Rudin et al. (1992) , total variation with a mapping function (TVm), see ( 52 ), gradient energy tensor total variation (GETV) Åström et al. (2015) and GETV with a mapping function (GETVm), see ( 65 ).
Figure 9: Examples of combined gamma correction (GC) and denoising compared to “denoise and then gamma correct” and “gamma correct and then denoise”. If the degradation parameter is set to γ=1 , ( 77 ) is a one-to-one mapping and ( 78 ) is the identity. Instead of using the true gamma correction parameter 1/γ we use 1/(1.1γ) to represent the uncertainty in the estimate of an unknown inverse mapping. With a stronger nonlinearity, it can be seen that the mapping-based filter performs better than the other approaches, both visually and with respect to error measures.
Figure 10: Denoising results for the mapping-based filter when γ=0.4 . In each instance we accurately preserve the dominant features while suppressing the main parts of the noise. The noise is multiplicative and σ was set to 40 levels of the intensity range in the noise model ( 76 ).
Figure 11: Evaluation of gamma correction of overexposed images Cameraman, Barbara, Lena and Boat. As γ becomes smaller, i.e. , the nonlinearity is more pronounced, the mapping-based filter consistently performs better than the two alternatives “denoise and gamma correct” and “gamma correct and denoise”. We set σ=40 of the intensity range in ( 76 ).
Figure 12: Two examples of targeted value range filtering from user selected regions. On the first row we focus on the text. As visible in the close-up in the Result-column the noise is removed in the white text whereas and preserved in the red background. On the second row we instead choose to filter the red color of the soda can, and as visible in the Result-column the red is denoised whereas the noise remains in the white text. The column labeled “Mapping” is computed from the fitted (normalized to 1) density functions in the right-most column. See text for details.
Figure 13: Image “Boat”. Top left original image, right is the degraded image corrupted with additive Gaussian noise with standard deviation 20. Down left, oversegmentation map and down right, is the obtained mapping function.
Figure 14: Comparison of visual quality. The Boat’s bow appears more clear where we have applied the mapping function, compared to ISO, TV and GETV. Visually, TGV appear similar to TVm and GETVm. While ISO and ISOm are oversmoothed, MRF and GETV exhibits color artifacts in the blue sky. Although, BM3D has good visual and good error values (Table 5 ), its approach to denoising is fundamentally different from the PDE-based method introduced in this work.
Table 17
Figure 15: Image “Lena”. Top left original image, top right shows the degraded image corrupted with additive Gaussian noise of standard deviations 20. Down left shows the oversegmentation map of the noisy image and down right illustrates the obtained mapping function.
Figure 16: Comparison of visual quality. The brim of Lena’s hat appears more clear where we have applied the mapping function, compared to ISO, TV and GETV. As expected, ISO and ISOm are oversmoothed. TVm, GETVm and TGV all have similar visual appearance. In this example image, there are no apparent color artifacts. Visually, Lenas hair retains more details in BM3D, TGV and MRF compared with TVm and GETVm.
The case when a partial differential equation (PDE) can be considered as an Euler-Lagrange (E-L) equation of an energy functional, consisting of a data term and a smoothness term is investigated. We show the necessary conditions for a PDE to be the E-L equation for a corresponding functional. This energy functional is applied to a color image denoising problem and it is shown that the method compares favorably to current state-of-the-art color image denoising techniques.
Freddie Åström, George Baravdish, Michael Felsberg
Computer Vision Laboratory, Department of E.E., Linköping University · Center for Medical Image Science and Visualization (CMIV), Linköping University · Department of Science and Technology, Linköping University
We present a novel variational approach to a tensor-based total variation formulation which is called gradient energy total variation, GETV. We introduce the gradient energy tensor [6] into the GETV and show that the corresponding Euler-Lagrange (E-L) equation is a tensor-based partial differential equation of total variation type. Furthermore, we give a proof which shows that GETV is a convex functional. This approach, in contrast to the commonly used structure tensor, enables a formal derivation of the corresponding E-L equation. Experimental results suggest that GETV compares favourably to other state of the art variational denoising methods such as extended anisotropic diffusion (EAD)[1] and total variation (TV) [18] for gray-scale and colour images.
Freddie Åström, George Baravdish, Michael Felsberg
Computer Vision Laboratory, Linköping University, Sweden · Center for Medical Image Science and Visualization (CMIV), Linköping University · Department of Science and Technology, Linköping University, Sweden
The assessment of image denoising results depends on the respective application area, i.e. image compression, still-image acquisition, and medical images require entirely different behavior of the applied denoising method. In this paper we propose a novel, nonlinear diffusion scheme that is derived from a linear diffusion process in a value space determined by the application. We show that application-driven linear diffusion in the transformed space compares favorably with existing nonlinear diffusion techniques.
Freddie Åström, Michael Felsberg, George Baravdish +1
Computer Vision Laboratory, Linköping University, Sweden · Center for Medical Image Science and Visualization, Linköping University, Sweden · Department of Science and Technology, Linköping University, Sweden +1