Super-resolution algorithm in Python

Abstract

Super-resolution algorithms is a family of algorithms used to obtain higher resolution image from set of images in lower resolutions. It's important to notice that by using those methods the higher resolution image is more detailed in opposition to resizing and debluring.

It may be applied on many different data, including spying and meteorological satellites, telescopes, tomographs and X-ray machines. Super-resolution algorithms can be also applied to 3D sceneries or sounds.

In this article, I will describe a super resolution algorithm proposed by Irani and Peleg and its simple implementation in Python.

One of low resolution images (for easier comparison it's resized to fit the estimated image):

The image estimated by using super resolution algorithm:

Introduction

The most important role in presented super resolution algorithm plays imaging process, so I will start from its explanation. Let's assume following notation:

• HR – original, high resolution image
• LR(n) - n'th lower resolution image
• s↓ - downsampling factor
• h^psf – point spread function
• N – noise
• T – movement of constant vector [x,y]

Image process of capturing n-th image can be described, by using below formula.

LR(n) = ((T(HR) + N)*hpsf) s↓

It means that any LR image is captured by taking HR image, 1) moving it by T vector, 2) blurring it by noises and h^psf and 3) downsizing it. Multiple different LR images may be obtained from one HR image because T and N may vary.

The goal is, to take this equitation, known factors and low resolution images (its left side) and try to extract HR from it. This may be achieved by below algorithm.

Input:

• number of iterations
• upsampling factor
• set of low resolution pictures with know movement from the original HR image
• point spread function

Algorithm:

• Create initial upsampled image. Simple resizing algorithm with optional debluring may be also used in this step. Let's name this image E.
• For all LR images simulate image process from E; Different images are taken with different offsets, so simulated LR images will differ.
• For each LR image get the difference between simulated and captured image, backproject this difference to E.
• if not extended number of iteration, then back to 2.

Output:

• E is the SR-image

It's useful to have a function that tells how much estimated image E differs from the original HR image in each iteration. For this purpose we add up for all LR images the differences between values of simulated pixels and captured pixels. This factor is normalized by product of numbers of pictures, width of LR image and height of LR image. This information may be used to tweak the input constants or to compare, how algorithm behaves on different sets of input images.

Software

Python was used due to its easy syntax and support for functional programming. Detailed info, how to use this software was presented on project's wiki page

For easier implementation noises weren't included and psf was hardcoded. More details of motion estimation algorithm used in this project can be found on a separate article.

The super-resolution is available on GitHub

Results

Conclusion

There are plenty of articles about super resolution algorithms, but almost none of them includes implementation in one of popular programming language. In presented here simple implementation that works correctly and can be used as a base for further researches.