Printable Version of Topic

Click here to view this topic in its original format

Unmanned Spaceflight.com _ New Horizons _ Super-resolution challenge

Posted by: john_s Nov 1 2008, 12:16 AM

Greetings:

Here's another chance to maybe help out the New Horizons science team! We are planning the Pluto imaging sequence, and are considering the best strategy for squeezing the maximum resolution out of our images, including "super resolution" techniques, particularly for the side of Pluto that faces away from us at close approach, which we will see with a disk diameter of only about 120 pixels in our high-resolution camera LORRI. As some members of this forum have done impressive work with super-resolution processing of MER images, maybe you can try your hand with some synthetic "Pluto" images to see what the potential is? This may be more challenging than for MER, because our PSF is relatively broad, about 2 pixels wide.

We want to know how many images to take of a given face of Pluto to get maximum benefit from super resolution techniques, if indeed they are useful at all.

I plan to generate a bunch of synthetic images with slightly different pixel positioning and smear, and with realistic noise levels, and then make them available for experiments to see how well they can be combined and sharpened to improve the resolution.

If you are interested in giving this a shot, let me know. Also let me know if you can work with 12-bit or 16-bit images, and if so what format is most convenient. I can easily make 16-bit FITS files, for instance. It may be that 8-bit PNGs will be adequate, too.

Thanks,
John.

Posted by: tedstryk Nov 1 2008, 02:33 AM

I am definitely interested. I will say this. Although some sets showed some improvements with just three or four images, it starts to reach its peak around five or six images. Beyond about 10 images (I am basing this on MER and Pathfinder) and there isn't any real improvement. I have worked a little with deconvoluting some of the images of the Galileans, but there are of course no sets to stack. I don't know how such images would work in a super-res sequence, but it is definitely worth a try.

Posted by: john_s Nov 1 2008, 03:25 AM

Cool! Tell me your preferred image format.

John.

Posted by: tedstryk Nov 1 2008, 12:47 PM

QUOTE (john_s @ Nov 1 2008, 04:25 AM) *
Cool! Tell me your preferred image format.

John.



While 16-bit tiff files might have some improvement, I am not so sure it would be significant when compared to a 12-bit file. The greater bit depth might help with the low contrast features, but it is hard to say for sure - A lot depends on LORRI and whether there is real information in the images that could be better plucked out of 16-bit images. I suspect that 12-bit imaging would be adequate. 8-bit would suffer a lot, especially given the fact that they would have to be deconvoluted to reduce the effects of the wide PSF.

Posted by: john_s Nov 1 2008, 03:00 PM

So do you prefer TIFF files? If so, I have to figure out how to make 12- or 16-bit TIFFs. If you can read FITS files (or 16-bit PNGs) we're all set, though.

John.

Posted by: tedstryk Nov 1 2008, 03:04 PM

QUOTE (john_s @ Nov 1 2008, 03:00 PM) *
So do you prefer TIFF files? If so, I have to figure out how to make 12- or 16-bit TIFFs. If you can read FITS files (or 16-bit PNGs) we're all set, though.

John.

Yes, I can read FITS files.

Ted

Posted by: john_s Nov 1 2008, 04:19 PM

Great, here's a file with 6 simulated images of Ganymede in 16-bit FITS format. I'll send another six in my next mail.

 superresolutiontest1.zip ( 649.71K ) : 1073

Posted by: john_s Nov 1 2008, 04:27 PM

And here's the other six...

John.

 superresolutiontest2.zip ( 650.01K ) : 893


Posted by: john_s Nov 1 2008, 06:46 PM

Oops, there's a bug in those image sets- they are variably smeared (at a realistic sub-pixel level), but the image *positions* don't vary randomly at the pixel level as they should. I'll fix the problem and re-post when I can- in the mean time the current image sets should be good for practice.

John.

Posted by: tedstryk Nov 1 2008, 06:58 PM

As far as stacking goes, these might be interesting. Without position variations, there will be no super-res effect. I have a real test question. It also might be interesting to take a test sequence using Saturn.

Posted by: siravan Nov 2 2008, 10:07 PM

QUOTE (john_s @ Nov 1 2008, 11:19 AM) *
Great, here's a file with 6 simulated images of Ganymede in 16-bit FITS format. I'll send another six in my next mail.

 superresolutiontest1.zip ( 649.71K ) : 1073


Here is my first pass on the data as a practice for the actual test. Of course, as there is no shift in the data set, no superresolution effect in expected. The images are enlarged four times, stacked together and then unsharped. The main effect is improved SNR by stacking which allows for a more aggressive high-pass filtering.


 

Posted by: john_s Nov 3 2008, 09:10 PM

OK, here's a new and improved set of six FITS files- they are now shifted randomly as well as having random smear at the levels expected for the actual LORRI images (typically ~0.7 pixels).

The other six will follow next.

John.
 lorrirestest1_fits.zip ( 634.54K ) : 908



And the next six...

 lorrirestest2_fits.zip ( 634.44K ) : 878


Posted by: john_s Nov 3 2008, 09:15 PM

Not sure why those two ZIP files ended up in the same message...

Anyway, here are the first 6 images in 16-bit PNG format in case that's easier to read. I've added a bias of 20 DN to the PNGs, to avoid negative numbers (there are some negative sky values in the original images, due to random noise).

John.

 lorrirestest1_png.zip ( 536.32K ) : 861


Posted by: john_s Nov 3 2008, 09:19 PM

And the rest of the images in PNG format...

 lorrirestest2_png.zip ( 535.53K ) : 891


Posted by: john_s Nov 3 2008, 09:24 PM

By the way, the PSF in these simulations (not counting the motion smear) is a Gaussian with a 2-pixel FWHM, in case you want to do some deconvolving. That's fairly close to the actual LORRI PSF, though not a perfect match.

John.

Posted by: ugordan Nov 3 2008, 11:11 PM

Here's a flicker gif between a single frame and manual stack in Photoshop of the 12 frames, just for fun:

http://i108.photobucket.com/albums/n15/ugordan/comparison.gif

Magnified 2x from original pixel scale. Most likely much better than this can be done. If anything, the many frames allow heavier deconvolution/sharpening to be performed.

Posted by: john_s Nov 4 2008, 12:14 AM

Nice! It looks like you did some sharpening- what kind of sharpening?

John.

Posted by: ugordan Nov 4 2008, 08:40 AM

I used the Smart Sharpen filter in Photoshop, using the "more accurate" lens blur removal. It produces tighter (and noisier) results than simple unsharp filtering. It's hard to force oneself not to overdo the sharpening, but in this case I do believe it brings out details that aren't resolvable in single frames.

Posted by: siravan Nov 8 2008, 09:59 PM

Here is the effect of subpixel shift-and-add on the dataset. The image on the right is the first raw image, enlarged twice and unsharped. To make the one on the left, all the raw images are scaled x8, then the amount of shift for each one compared to the first image (in the x8 setting, i.e. 0.125 pixels resolution in the original dataset) is found and they are co-added taking into account the shift value, and finally the co-added image is rescaled to x2 and processed using the same unsharp filter as the one applied to the image on the right.

The main difference in that unsharping introduced lots of noise to the raw image, whereas the processed image tolerated it much better and some previously unseen features are now detectable.

 

Posted by: john_s Nov 10 2008, 08:41 PM

Thanks to both ugordan and siravan for those images! We're also interested in what can be done with a smaller number of images, 4, for example. Could you try the same thing using just four of the input images?

Thanks,
John.


Posted by: Fran Ontanaya Nov 13 2008, 09:54 AM

I had to try. smile.gif



http://www.franontanaya.com/gimp/superresolutiontest-x2.jpg
I used 5 images plus one to reduce noise. Mostly was about scaling up without interpolation, aligning the images, do some blending and several steps of gaussian blur and unsharp mask before scaling down --all very empirical. A real pro should be able to do better. tongue.gif

Posted by: tedstryk Nov 13 2008, 01:07 PM

Here is my take. I have versions with 4 frames, 6 frames, and 12. First, I deconvoluted the images to combat the broad PSF. I then blew the images up to 5x and sharpened them based on the new artificial point spread the enlargement created. I selectively stacked them, weighting them based on quality (I could probably do a bit better, but I was trying to hurry). After merging the image, I applied a light round of deconvolution based on a 4 pixel PSF and then reduced the images to 1.9x. A slight bit of sharpening was applied at this point.

4 Frames


6 Frames

12 Frames

Posted by: tedstryk Nov 14 2008, 05:00 AM

Here is a view of Triton when it was 120 pixels across (shown here at about 2.5x). The inset shows two apparent plumes visible near the bottom of the terminator.



Now that I have worked on the Ganymede stack, might I ask if the image used is the NH frame from the same angle or generated from something else? The reason I ask is that the detail cutoff seems a bit odd compared to the Triton set and other real sets I have worked on. Namely, the technique does much better with high contrast details than low contrast details for obvious reasons. With the Ganymede images, the cutoff seems even. If it is from the lone NH frame, the reason is simple - I am smacking into the resolution limit of the image.

Posted by: john_s Nov 15 2008, 04:22 PM

This is a synthetic image using a Galileo photomosaic, but generated with the same geometry as the best New Horizons Ganymede image (for reasons that seemed like a good idea at the time).

Very interesting Triton image- that plume looks fairly convincing! It's a good analog for what we might see on approach to Pluto, where the approach phase angle (15 degrees) is similar.

John.


Posted by: Andrei Nov 19 2008, 01:23 PM

I'll take the challenge...I only hope that I'll have some results before NH reaches its target unsure.gif...

Posted by: tedstryk Nov 20 2008, 01:34 PM

QUOTE (john_s @ Nov 15 2008, 04:22 PM) *
This is a synthetic image using a Galileo photomosaic, but generated with the same geometry as the best New Horizons Ganymede image (for reasons that seemed like a good idea at the time).

I figured out what the issue is. Most stacks I have worked with include some severely underexposed images. However, the well exposed images were more likely to be smeared because of the longer exposure time. Hence, the high contrast features (the ones visible in even the underexposed but very sharp images) ended up looking better than the fainter features. Given the nature of LORRI (not having color filters), the image sets it produces will likely be more like the Ganymede sample than what I am used to.

Posted by: S_Walker Oct 16 2009, 04:43 PM

Sorry I'm late to this test, just joined the sight.

Attached is four images resized 2x, aligned then drizzle combined in MaxIm DL 5.02.
The result is deconvolved using Richardson-Lucy algorithm, PSF of 1.5, 20 iterations.


 

Posted by: S_Walker Oct 16 2009, 04:45 PM

Next is all 12 images resized 2x, aligned and drizzle stacked in MaxIm DL 5.02. THis was deconvolved the same way, but only 10 iterations (more just added noise artifacts).


 

Posted by: S_Walker Oct 16 2009, 04:51 PM

My final version upsamples all 12 images, align then sum combined in Maxim. I then deconvolved the image twice; the first used a PSF of 1.6, 20 iterations, then a PSF of 1.2, 4 iterations.

 

Powered by Invision Power Board (http://www.invisionboard.com)
© Invision Power Services (http://www.invisionpower.com)