Printable Version of Topic

Posted by: Magnus Lundstedt Mar 14 2006, 03:41 PM

I am toying around with some raw image data from MGS in the form of pds files. (EDIT: MOLA instrument images). I have downloaded the data from here:

http://pds-geosciences.wustl.edu/missions/mgs/megdr.html

Each image consists of a pair of files: ".img" and ".lbl". The img-file contains the data and the lbl file contains metadata. In this metadata file it says that the sample size is 16 bits. But in the program NASAView, which one uses to generate images from pds files, I have only managed to create 8 bit images.

Is it possible to create images with the full 16 bit data with NASAView? Or should I use some other program? Or do I have to write it myself?

Thanks for any suggestions!

/Magnus (magnus @ #space)

Posted by: djellison Mar 14 2006, 04:04 PM

Your best bet is going to be using the MOC Gallery
http://www.msss.com/moc_gallery/ as then you'll have the projected images, processed just about as well as they can be, in a comparatively lossless format (gif) - AND - they're easier to find

MC might correct me, but I believe MOC is only 8 bits. MER cameras are 12, but MOC is just 8.

Doug

Posted by: Magnus Lundstedt Mar 14 2006, 04:16 PM

I dont know. That would mean that they have 255 steps in -8 km to +21 km - 110 m resolution - not too good Also the meda datafile says 16 bits, but its maybe something else? The instrument in question is the "MOLA", the laser altimeter instrument, I dont know if it can be compared to ordinary cameras? I thought the MOLA had sub meter vertical resolution. (i saw 30 cm somewhere)

If I go to moc-gallery and pick already processede images, I dont get them the way I want. Besides if i get them in gif (max 8 bits per pixel) its no better than what the NASAView program does

/Magnus

Posted by: mcaplinger Mar 14 2006, 04:21 PM

That's true, but he was talking about MOLA gridded products, which are 16 bits.

It depends on what you want to do with the data as to how many bits you need. More recent versions of Photoshop do 16 bits. For altimetry, many applications make a shaded relief map in 8 bits from the original topo data.

Posted by: Magnus Lundstedt Mar 14 2006, 04:48 PM

Ok, but there is no imageformat with 16 bits of indexed colors? Or even possible to export in NASAView as a 32 bit image, where there is plenty of space for 16 bit data for each pixel? In that case I must do my own program I guess. Ah well.

Thanks for the help!

Posted by: helvick Mar 14 2006, 05:43 PM

ImageMagick supports 16bit PNG greyscale so you can work with that format using the various ImageMagick libraries but it's a new format so support is rare and you have to use the 16bit ImageMagick variant.

You could work the data into 16,24 or 32 bit image format but none of them would display 16bits of greyscale properly.

Posted by: Magnus Lundstedt Mar 14 2006, 07:05 PM

I have done what I wanted with 8 bit vertical resolution instead. That is to make images for a mars with a hypothetical oceans with several different ocean heights. To compose in a video afterwards. It was pretty fun. Will probably try somehow to do it with 16 bit vertical resolution instead and far more images, or perhaps dynamically generate images or parts of images based on user input data.

Another thing I have learned is that 11000x5500 px videos does not work so well. I managed to encode it, but when I played it in VLC, my computer froze. heh. So I had to settle for 1280x640 for the video. Ah well.

This is the result of what I did:

http://magnus.infidyne.com/mars/water/

Posted by: algorimancer Mar 16 2006, 01:37 PM

I like the 1300-1400 meter water depth versions. That looks like a nice world to live on

Posted by: Bjorn Jonsson Mar 22 2006, 03:35 PM

16 bit PNGs work well for this purpose and Photoshop CS can handle them. I have been using 16 bit PNGs to do renders of Mars based on MOLA data (an old and rather bad experimental one can be seen here: http://www.mmedia.is/bjj/3dtest/mars_vm_pt3rt.jpg ). 8 bits are usually insufficient for this kind of work.

What I did was to take all of the IMG MOLA files and convert/merge them into a single, huge 'RAW' file using a small program I wrote. I then wrote a small program that can extract a selected latitude/longitude range from the 'RAW' file and write the result to a 16 bit PNG file. I then use that file to render a limited area at high resolution.

I don't remember if my IMG2PNG utility can convert a MOLA PDS file to a PNG (I'm at work so I can't check it ) but it could be modified fairly easily to do so if it doesn't work.

Posted by: djellison Dec 22 2007, 05:36 PM

Peaaaaawwwwwwwwwwwwwwwwww.... 21 month forum time warp. Wadda ya know, googling about MOLA and PNG's - I ended up back here again

MOLA+IMG2PNG - It doesn't like them - and I'd really like to go to the 64 or 256 pix/deg resolution from here -http://pds-geosciences.wustl.edu/missions/mgs/megdr.html

NASA View loads the 4 chunks for the 64 fine, and saves GIF's fine - but they get stretched, unequally, in the process, and downed to 8bit.

A 20k or 40k 16bit PNG bump/displacement map for Mars rendering would be rather good when I'm rendering out of 3ds max at 4000x4000 pixels

I've got a fairly 'flat' 16k colour map which isn't too bad....but more is ALWAYS better when 16 megapixels are required

Doug

Posted by: djellison Dec 26 2007, 03:22 PM

YEah - here's the 8 bit v 16 bit problem - I'm getting steps in my terrain. Looks awesome at the moment, but that extra resolution for altitude would take it to that level where you couldn't see those 256 steps.

Doug

Posted by: mcaplinger Dec 26 2007, 04:26 PM

The problem is that the GIF format is inherently limited to 8 bits.

For the maps of Mars I was working on in 2000-2002, I used the GMT tools and GMT-format files that could be obtained from the MOLA website (since they used GMT for a lot of their products.) But GMT is really hard to use and I think predates the widespread use of PNG.

Later versions of Photoshop can load 16-bit images in raw format, which would involve using NASAview to determine the image dimensions and the header size, and then telling Photoshop to load those dimensions and skip the header. It might then be possible to save as a 16-bit PNG. But I've never tried this.

Your best best is to use some kind of raw file import capability.

Posted by: djellison Dec 31 2007, 03:31 PM

NEARLY works with Photoshop.

The dimensions are in the LBL, but I'm getting an odd clipping whereby (I think) I'm getting two hacks at 16 bits, with the depths resetting from black to white to go deeper. Odd.

Works a treat for CTX...not so good for MOLA.

Doug

Posted by: hendric Dec 31 2007, 03:58 PM

Doug,
I'm just guessing, but it is it possible for you to change the DEM image from 8 to 16 bit in Photoshop, and then add some Gaussian noise for the lower 8 bits? It would get rid of the stairstepping effect at least.

Posted by: djellison Dec 31 2007, 04:03 PM

That would help a bit, for now - but...

These things take 2 or 3 days to render out a 4k res sequence of 1800 frames

What I'd really like is to get both the 16 bit sorted AND use the 64 ppd data. That means getting the four panels processed the same to mosaic them up as one roughly 20k x 10k mosaic and the way NasaView loads them, it stretches each, unequally.

So I'll hang tight till I can figure out the processing at 16 bit, mosaic them, then get re-rendering ( I think a cheap render node or two might be usefull in the meantime!)

Doug

Posted by: hendric Jan 2 2008, 06:55 AM

Well, if there was some way to divvy up the work, couldn't we create a virtual renderfarm for you? Although, I was looking into it, and for ~$600 I could build a quad-core 2GB system. Would be easy to create a a renderfarm with one or more of those puppies.

Posted by: djellison Jan 2 2008, 09:08 AM

That's exactly what I'm planing I did a spreadsheet comparing total system cost to render speed (basically frames per hour per pound purchase prices) for various options. I've always had this belief that the best cpu in performance/£ isn't always the best component because it's only a part of the whole system...and indeed and for £300 I can do exactly the sort of rig you suggest - little asus barebone boxes, Q6600's, 2gb and a small SATA drive. I'm looking at the various asus boxes carefully as I used Win XP 64 and would like the same OS on the nodes for rendering consistency - and if possible, I'd like to do what I've done on my desktop rig which is overclock the 2.4 gig Q6600 to 3 gig - 25% extra free, and it's been 100% stable on my desktop rig

I literally, set the fsb from 266 to 300 by accident when I built it, so it was at 2.7 all the time before I noticed - and seeing as 3 gig is the fastest quadchip they'll sell you, all be it at 3x the £150 price of the Q6600, I thought it was worth a stab, so I set the FSB to 333 and it's been 100% perfectly stable ever since - even when rendering for 3 days straight over Christmas. 3ds Max is the only thing I can give it to 'fill' all four cores

However - I don't really need a farm just yet - so I'm going to hold off as long as I can, and then maybe sort something out in a few months time. For £1k today, I could have 12 cores - and technically quadrouple the rendering performance of the shed.

I'm also looking at ISIS again on my Macbook. Last time around I gave up before I even got any app working. Last night I got vicar2isis working for HRSC dem's - but in trying to shortcut the data rsync, I'd missed some template it needed so I've got a day of that to get thru yet before I can try that, and I'll try MOLA thru it as well.

Doug

Posted by: Bjorn Jonsson Jan 2 2008, 12:18 PM

The problem is that the 16 bit numbers need to be byteswapped (bigendian/smallendian stuff; take a look at http://en.wikipedia.org/wiki/Endianness).

Obviously I *really* should try to find the time to modify IMG2PNG to handle these files properly. The problem is that I'm spending huge amounts of time making DEMs of Saturn's satellites - I'm sort of relieved to see I'm not the only one needing a supercomputer. 10-20 hour calculation runs are a routine for me now ;-).

Posted by: hendric Jan 2 2008, 02:06 PM

Bjorn,
Would you be interested in sharing the source for IMG2PNG? I'm not really a SW guy, but I can hum a few bars and fake it pretty well. Byteswapping would be an easy option to add, and it would be nice to calibrate & decompand the CTX IMGs directly.

Posted by: djellison Jan 23 2008, 09:32 PM

ImageJ can handle this lot perfectly - I'm now working on mosaicing the 128ppd set - but basically importing at 16 bit signed, then exporting a TIFF - the final tif is fine.

Doug

Posted by: djellison Jul 10 2009, 07:44 AM

Just for fun - I've revisited my stock Mars model in 3DS Max. It needed work, and I think I've improved it. I'll post some obscure low sun low altitude renders later, but meanwhile, a 72 second animation that has two rotations of Mars - one at normal topography, one with the Topography massively exaggerated.

The finished 1080 res MOV is available here : http://vimeo.com/5529718 (link at the bottom to download it)

and also in You Tube's idea of 'HD'

http://www.youtube.com/watch?v=o5QlvD3bn_w

Posted by: Art Martin Jul 10 2009, 01:51 PM

Wow, that's incredible Doug. Truly gives you perspective on how varied Mars topography is.

Art

Posted by: djellison Jul 10 2009, 02:02 PM

A bunch more random renders. One is especially for Stu and I, at 2kx2k, the resolution of OSIRIS on Rosetta.

Posted by: eoincampbell Jul 10 2009, 03:05 PM

Love the Mars Blowfish! Thanks for sharing.
Eoin

Posted by: SFJCody Jul 10 2009, 07:19 PM

Ahh, beautiful stuff, even though I do prefer a Mars that looks more ochre.

Posted by: djellison Jul 19 2009, 06:24 PM

Another animation just for fun.

http://vimeo.com/5666678
http://www.youtube.com/watch?v=0OQWxe6UhNg

Put it full-screen, put your laptop on the floor, and watch standing up

Posted by: nprev Jul 19 2009, 06:55 PM

Great stuff!!!!


Took your suggestion & it damn near gave me vertigo.