Complex Wavelet SSIM

---

The complex wavelet structural similarity (CW-SSIM) index is an extension of the structural similarity index that utilizes a complex continuous wavelet transform in order to determine the perceptual similarity between two digital images: a reference image and a distorted image. This model was developed in the Laboratory for Computational Vision (LCV) at New York University.

Motivation

The SSIM was devised with the intent of being a structural similarity measurement that should provide an adequate measurement of perceived image similarity. The SSIM does this by taking advantage of strong correlations between neighboring pixels in an image. The SSIM works very well for many types of image distortions that affect the image's structural information such as contrast stretching, luminance shifting, or the addition of Gaussian noise. However, the SSIM has a major drawback: it is sensitive to image rotations and image translations.:

This becomes particularly troublesome in cases when trying to compare images where one of the image-taking devices is off-centered and leads to erroneous image quality measurements. The CW-SSIM was developed to make the SSIM insensitive to these aforementioned distortions.

The CW-SSIM takes advantage of the fact that phase contains more structural information than magnitude does in natural images and the fact that translations in images lead to a consistent phase shift in the complex wavelet domain. This allows it to deal with image rotations since small image rotations can be locally approximated as translations in an image.

Algorithm

The CW-SSIM index is calculated on various subbands of an image in the complex wavelet domain. The resultant indices are then averaged over the whole image to get a resultant quality measure for the whole images.

The similarity measure between two images over a window of size 'N' 'x' 'N' is given as

${\tilde {S}}(c_{x},c_{y})={\frac {2\sum _{i=1}^{N}|c_{x,i}||c_{y,i}|+K}{\sum _{i=1}^{N}|c_{x,i}|^{2}+\sum _{i=1}^{N}|c_{y,i}|^{2}+K}}\cdot {\frac {2|\sum _{i=1}^{N}c_{x,i}c_{y,i}^{*}|+K}{2\sum _{i=1}^{N}|c_{x,i}c_{y,i}^{*}|+K}}$

where:

• $c_{x},i$ is the zero mean complex wavelet coefficient taken at spatial location $x$ for window $i$
• $c_{y},i$ is the zero mean complex wavelet coefficient taken at spatial location $y$ for window $i$
• $c^{*}$ is the complex conjugate of $c$
• $K$ is a small positive constant used to keep the CW-SSIM from becoming unbounded

The CW-SSIM index has the same properties as the SSIM index. Notably symmetry: ${\tilde {S}}(c_{x},c_{y})={\tilde {S}}(c_{y},c_{x})$ and boundedness from -1 to 1.

Formula components

The CW-SSIM formula can be understood more fully by breaking up the the formula into a maginitude component (${\tilde {m}}(c_{x},c_{y})$) and a phase component (${\tilde {p}}(c_{x},c_{y})$).

The magnitude component is given by ${\tilde {m}}(c_{x},c_{y})={\frac {2\sum _{i=1}^{N}|c_{x,i}||c_{y,i}|+K}{\sum _{i=1}^{N}|c_{x,i}|^{2}+\sum _{i=1}^{N}|c_{y,i}|^{2}+K}}$.

This term is only dependent on the magnitudes of the complex wavelet transform coefficients and reaches its upper bound of 1 if and only if $|c_{x,i}|=|c_{y,i}|$.

The phase component is given by ${\tilde {p}}(c_{x},c_{y})={\frac {2|\sum _{i=1}^{N}c_{x,i}c_{y,i}^{*}|+K}{2\sum _{i=1}^{N}|c_{x,i}c_{y,i}^{*}|+K}}$.

This term depends only on the phase difference between the components $c_{x,i}$ and $c_{y,i}$. It reaches its upper bound of 1 if the difference in phase between $c_{x,i}$ and $c_{y,i}$ is uniform over all windows i.

Comparison of the CW-SSIM With Other Fidelity Measures

Above is a comparison of visual similarity measurements for images that have been distorted altered by different types of distortions. The images are as follows: (a) Original Image,(b) Luminance shift, (c) Contrast Stretch (d) Zoom In, (e) Zoom Out, (f) Left Shift, (g) Right shift, (h) Clockwise Rotation.

Like the SSIM, the CW-SSIM method returns values closer to 1 for distortions caused by luminance and contrast changes. In addition, it returns high values in response to small translations and subsequently small rotations, and small scalings. It works well with the latter two since small rotations and small scalings can be locally approximated as small translations in a given window.. Furthermore, additions that degrade the structural information contained within the magnitudes of the pixel values but do not degrade the information in the phase by a lot are awarded higher similarity scores by the CW-SSIM then by the SSIM.

An example of this comparison is shown in the figure to the right. This file shows seven different non-structural distortions on a picture of stuffed cow. As seen by the captions, the MSE (Mean Squared Error) gives very mixed results when comparing the visual similarity of the altered images to the original image. The SSIM on the other hand, performs quite well for the images (b) and (c), demonstrating its robustness to luminance and contrast changes. It shows very poor results the remaining altered images which demonstrates its sensitivity to scaling (d and e), shifts (f and g) and rotation (h). The CW-SSIM is shown to work better at recognizing the perceptual similarity of the images above in all cases when compared to the SSIM. work very well for nonstructural distortions such as contrast stretching. We can see though that the CW-SSIM correctly awards the translated images with a higher scores than the MSE and the SSIM due to its phase component which attributes the shift of the image to a phase shift in the complex wavelet domain. The CW-SSIM also is shown to be effective when faced with scaling and rotations due to approximating both of those locally as small translations in the spatial domain.

Despite being much better than the SSIM and MSE when utilized for image similarity comparisons, the CW-SSIM still suffers from a limitation that it only performs well if the shift, rotation, or scaling distortions are small enough.

Applications

In the paper "Translation insensitive image similarity in complex wavelet domain" by Zhou Wang and Eero P. Simoncelli, the CW-SSIM was compared to the MSE and the SSIM through character recognition. In the results of the paper, the CW-SSIM was shown to have a much greater character recognition rate than methods based off of the MSE and SSIM. In another paper titled "Complex Wavelet Structural Similarity: A New Image Similarity Index", the CW-SSIM has also been tested as a method for facial recognition and it was shown to perform this task very well compared to existing methods based off of the mean squared error and the Partial Hausdorff Distance Metric (PHD). The CW-SSIM has also been utilized as in a method of palm print verification. The method based off of the CW-SSIM was shown to perform better than one of the most effective palmprint verification algorithms based off of the competitive coding scheme.

Variations

Weighted CW-SSIM

One weakness of the CW-SSIM is its poor performance in dealing with low-frequency errors that cause spatial distortions in images. The Weighted Subband Extension of CWSSIM (WCWSSIM) is a proposed extension of the CW-SSIM that works to combine the results of multiple wavelet scales in a manner reminiscent of the technique used in the MS-SSIM. This technique is shown to work better than the CW-SSIM when dealing with images affected by local mean distortions.

References

This article "Complex Wavelet SSIM" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Complex Wavelet SSIM. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.

---