cs.CVAug 23, 2026
SaveOn the Choice of Tensor Estimation for Corner Detection, Optical Flow and Denoising
Organizations: Computer Vision Laboratory, Linköping University, Sweden · Center for Medical Image Science and Visualization (CMIV), Linköping University
Abstract
Many image processing methods such as corner detection, optical flow and iterative enhancement make use of image tensors. Generally, these tensors are estimated using the structure tensor. In this work we show that the gradient energy tensor can be used as an alternative to the structure tensor in several cases. We apply the gradient energy tensor to common image problem applications such as corner detection, optical flow and image enhancement. Our experimental results suggest that the gradient energy tensor enables real-time tensor-based image enhancement using the graphical processing unit (GPU) and we obtain 40% increase of frame rate without loss of image quality.
Figures & tables
Figure 1: An illustration that the Gradient Energy Tensor performs comparably to the Structure tensor in the two applications of corner detection and optical flow estimation (sequence Teddy frames 10 and 11 [ 1 ] ). The gradient energy tensor does not (necessarily) require a post-convolution of its tensor-components, which is the case in the case of corner detection. See text for details.
Figure 2: Illustration of the resulting tensor-fields. The structure tensor require additional post-smoothing to form a rank-2 tensor compared to the gradient energy tensor which is defined without post-smoothing. Note that the size of the ellipses has been scaled for improved visualization.
Figure 3: Examples of corner detection for the structure tensor (with post-smoothing) and the gradient energy tensor (without post-smoothing) using Good Features to Track. Both tensors detect corners accurately but not always the same corners.
Figure 4: Optical flow estimated from frames 7 and 8 of the Schefflera sequence in the Middleburry optical flow dataset [ 1 ] . Top row illustrate the vector field. On the bottom we show the obtained flow fields where the direction is colour coded and intensity corresponds to the magnitude. The graph to the right show the mean squared error between the warped image and the reference image in ( 11 ).
Figure 5: Mean squared error (MSE) graphs between the first and second image after warping the first image with the estimated motion field. The sequences are part of the Middlebury dataset [ 1 ] . The estimate of the motion field in the Wooden sequence for the gradient energy tensor diverge on the finest scale, but in the other sequences the MSE yield comparable final displacement estimate.
| Anisotropic Diffusion [ 20 ] | Gradient Energy Diffusion |
Table 1: Algorithm pseudo-code for the main CUDA kernels. Left: Standard approach to adaptive image diffusion using the structure tensor. Right: adaptive image diffusion using the gradient energy tensor. The convolutions are separable Gaussian functions of size with standard deviation of .
Figure 6: We use tiles of size (blue regions) with padding of 2 pixels (red region) for the convolution and computation of the gradient energy tensor. We use corresponding tile layouts for computing the image gradient and filter update.
Figure 7: Colour test images and the corresponding image sizes in pixels that are used in the evaluation.
Figure 8: Obtained PSNR (a), SSIM (b) and fps (c) for the considered image sizes. The PSNR and SSIM values are similar for the two tensors but the obtained fps is significantly higher for the gradient energy tensor.
Figure 9: Two patches from the resolution image and the corresponding denoised images at iteration 10. The applied noise was 20 standard deviation of normal distributed additive noise. Note that the difference in visual appearance is not discernible but the fps count is improved by 25% with the gradient energy tensor.
| Ratio | PSNR | SSIM | fps |
|---|---|---|---|
| 0.991 | 0.995 | 1.35 | |
| 0.982 | 0.997 | 1.27 | |
| 0.968 | 0.999 | 1.40 |
Table 3: Ratios are computed as the measure obtained by the gradient energy tensor divided by the measure obtained by the structure tensor and averaged over iterations. A ratio of 1.0 show that the performance is identical. The SSIM and PSNR ratios are nearly identical but at up to 40% higher fps values in the largest image resolution, the PSNR show a marginal loss in image quality for the gradient energy tensor.