Here's something of great interest to Doug

Now how's that book coming?

http://pdsimg.jpl.nasa.gov/data/mer/

Woo - and indeed yay I believe the first 30 sols is due 6 months after landing - which is right about now.

Now if only we had an IMG imported for Photoshop

Doug

In looking through all the files there, I'm curious if this is the 'work in progress' of the PDS. It does contain many of the raw files (at least i'm assuming they are raw, since they are in the EDR folder, as opposed to the 'prepared' products which would be RDR's) and their PDS tags. But it is interesting to see all the jpg's, in the ops folders, though they are downsized (understandable, considering the...tremendous volume of files in those folders.)


But we're still going to have to wait for the begining of August for the full release, which will have the calibrated RDR files I think will be the most interesting (and capable of making the most accurate color i'd figure).

I'm looking forward to it!

A Comparison

JPEG Image on the MER Website:



IMG File converted to JPEG in NASA View

I take it you used NasaView to read the img?

I must admit - it's so good to see un-butchered imagery

I'm very busy on a 3D project at the moment ( involving an electric land speed record attempt ) - but once the rush for that is over - i'll be looking at the raw imageyr properly

Doug

Yep I used NASA View, it allows you to save the files as GIF or JPEG. The original IMG file was 2MB that JPEG is 182K. The GIF I saved from it is about 840K.

Only problem is - that Gif or JPG - you're limited to 8bit greyscale data - and some of the images will be 12 bit Photoshop can handle 12 bit which is just SO frustrating -ARHSAghhgeghaheahfhgh

Surely someone should do an .img Photoshop importer

I think I might email Dr Parker again

Doug

I do not have Photoshop CS yet (which I understand has vastly improved 16 bit image support compared to earlier versions) but I think it should be trivial to import the files that have 16 bits/pixel as RAW and enter the header size when importing.

In my case this is not needed though since I am able to convert the IMG files to 16 bit PNGs.

Helloooo - would you like to step into my office and tlel me how

Doug

I'm using software written by myself to read the PDS stuff and do some initial processing/conversion. However, it is in a (probably permanent) state of 'not ready for release'.

However, I might be able to easily and quickly 'cannibalize' it into a standalone utility for converting IMG files to PNGs if someone is interested. However, the user interface (UI) would be crude (probably a command prompt, at least in version 1.0) since for small, quick-and-dirty utilities like this I'm usually too lazy to do a nice UI.

I for one would much appreciate it. Just something that'd take all the imgs in a folder and PNG them would be perfection!!!

Doug

I made a very quick and dirty Windows command line utility that will convert PDS IMG files to PNG files, either 8 or 16 bit PNGs depending on the source data.

It's available at http://www.mmedia.is/bjj/misc/img2png.zip

It is run by opening a command prompt window and typing something like

img2png *.img

which will create PNG files from all IMG files in the directory. This might also work for more file types, e.g. Voyager and even Mariner 9/10 but I haven't tested it yet.

Please note that this is a *very* quick and dirty utility and I do not guarantee it to work at all. I have not tested it thoroughly although it seemed to work. One problem I know of is that IMGs containing floating point data (4 bytes/pixel - the 4 MB files) get converted but the result looks strange. I'll fix this later, however, I will be away from my computer for several days starting tomorrow.

Also note that the resulting PNGs will often look very dark. The reason is that 16 bits yield an intensity range of 0-65535 but in all cases I have seen the maximum intensity is roughly 3000 or lower.

I now pronounce you IMG god!



All thats left now really is a way to un-adjust the 'auto levels' type affair that's been done on-board

Doug

Now that I'm back at my computer I've been considering some minor improvements to IMG2PNG if needed:

(1) Does anyone know what the files that have a size of 4 MB contain (I'm too lazy to spend a lot of time figuring it out since I suspect someone here knows). They contain floating point data that I think I have been able to read but the image I tested looks weird but recognizable. Also the data contains lots of zeros which suggests to me these are not regular images. I found these files at http://pdsimg.jpl.nasa.gov/data/mer/

(2) It's trivial to add the capability to dump the labels to separate text files if that's of interest.

(3) In a different thread someone mentioned scaling the intensity values using RADIANCE_SCALING_FACTOR and RADIANCE_OFFSET_FACTOR. Doing so is trivial but it's not something I need for my purposes. Also what I saw in that thread seemed to imply this was not needed.

(4) Any interesting ideas ?

Bjorn

To be honest - I dont see the need for the radiance scaling either. Then again - I dont understand the relevance now we have the fully geometric and radiance calibrated files.

The only problem I've had is that, as you said, the PNG's are so dark, that you cant really see them at first till a big tweak. So you cant browse around What would be MIGHT usefull is tiny, stretched PNG thumbnail type file to go with each IMG turned to PNG?

LIke, 256x256 maybe - something of that size. It'd make things HUGELY easier when browsing for an image set.

But - as it is - the tool is UBER usefull Many thanks for it!!

Doug

The scaling factors in the Cornell radiance files (RAD) are completely necessary, but that is the only one that needs it. None of the other file types have the scaling factors and/or have them mean anything.

Without the factors, they will be quite off


For example:

Sol 9 for spirit
Scale factors for horizon looking panorama shot for the series starting with
2p127173379rad0211p2354l2C1.IMG

L2 Offset factor -0.0001046761 Scale factor 3.197851e-006
L3 Offset factor 0.006277401 Scale factor 2.722607e-006
L4 Offset factor 0.005301316 Scale factor 2.959709e-006
L5 Offset factor 0.00194943 Scale factor 1.9689e-006
L6 Offset factor 0.001033983 Scale factor 1.633268e-006
L7 Offset factor 0.0004936097 Scale factor 1.156879e-006


Scale factors for panorama shot just below the horizon, no sky showing for the series starting with
2p127173774rad0211p2354l2C1.IMG

L2 Offset factor -6.133495e-005 Scale factor 1.875653e-006
L3 Offset factor 0.005767861 Scale factor 1.425623e-006
L4 Offset factor 0.0049567 Scale factor 1.469256e-006
L5 Offset factor 0.001754055 Scale factor 8.729979e-007
L6 Offset factor 0.0006809833 Scale factor 7.102834e-007
L7 Offset factor 0.0002937913 Scale factor 5.460266e-007



Without the scaling factors, the value of a rock in the bottom of the first frame overlapping with the second frame won't match (for the above image L2 would be off by a factor of 1.7, L3 by a factor of 1.9...)

If the image you're reading out has these factors (only the Cornell radiometric calibration images do, the others you don't have to worry about)

Ahhh - this is the factor that makes the difference in that side my side I did

Doug

Talking of this - anyone know which file type are colorifically and radiance sorted - but NOT geometrically adjusted.

The geometric adjustment allows you to stitch images directly in photoshop - but I'd rather have the full square frame, not the slightly warped frame.

Doug

The RAD files don't have geometric adjustment.
THere is a potential type of Cornell radiance file that is supposed to have the geometric adjustment, the RAL file, but i've yet to see any...I dont believe they made it into this PDS release.

I have now uploaded a new version of this utility. It is identical to the previous version except for two things:

(1) It is possible to add a "-t" option, e.g.

img2png *.img -t

This will create thumbnail files in addition to the full size files. The thumbnails are 1/4 the size of the full size images and are contrast stretched.

(2) I corrected a whole lot of minor bugs which AFAIK didn't affect the conversion of the MER files. However, it should now be possible to also convert files from Mariner 9 and 10, the Viking orbiters and landers, the Voyagers, Magellan, Clementine, Galileo, MPF, MGS and probably more.

I may add the ability to dump the labels to text files later on. Doing so would be particularly useful in the case of the Mariner 9 and 10 files which do not have ASCII headers (the headers are in EBCDIC).

I'd appreciate any feedback, especially if bugs show up.

Bjorn

Not got a chance to try it at the moment - but a MAJOR MAJOR Thankyou for this - if I ever manage to get a book published of Mer imagery - the credits will include Steve Squires, Jim Bell, Justin Maki, and You.

Doug

Spirit's Sol 30-90 data is scheduled for release today.
Hopefully the Analyst's Notebook and the PDS Imaging Node will be updated soon to give us a calibrated look at Adirondack, the drive to and analysis of Bonneville Crater and the first few sols of the drive for the Columbia Hills.

Well, even though I feel its still a work in progress, I've got my gallery of calibrated color images up now for Spirit sol 1-70 and Opportunity sol 1-30.

http://www.lyle.org/~markoff/

I'm still looking for a psychological model for saturation and brightness balancing, but as it stands, i'm fairly confident with the hue of the colors after chromatic adaptation. The whitepoints for Spirit and Opportunity came out quite differently (xyz of 97,100,75 for Spirit and 97,100,85 for Opportunity) when I looked at the calibration target images, and the images are adjusted according to their own whitepoint.

Also, the brightness is fixed for each set (every image is scaled identically, leading to bright sky / sun images being washed out), but different between the rovers. The scaling is 0 - 0.12 watts/m^2/sr as input for Spirit, and 0 - 0.1 for Opportunity (Opportunity is displayed brighter than Spirit). Opportunity, ground and sky, is significantly darker (at least through sol 30), so I adjusted Oppy's brightness up to give a larger range to be viewed.

I'd love to hear any feedback/suggestions people might have for this little project of mine.

Daniel Crotty

I have now finally uploaded a new version of img2png which can convert lots of different PDS files to PNGs, including 16 bit PNGs when applicable. This new version can perform radiometric correction of MER RAD files. Doug has been testing/using this version for some time.

It's available at http://www.mmedia.is/bjj/misc/img2png.zip

It is run by opening a command prompt window and typing something like

img2png *.img

which will create PNG files from all IMG files in the directory. This also works for more file types, e.g. Voyager and even Mariner 9/10.

There are three optional command line options:

(1) -t tells img2png to create thumbnail files in addition to full size files. Example:

img2png *.img -t

This will create thumbnail files in addition to the full size files. The thumbnails are 1/4 the size of the full size images and are automatically contrast stretched.

(2) -r tells img2png to perform radiometric correction (the default is not to do so). You'll want to do this when working with RAD files.

(3) -sNNNN where NNNN is a floating point number (e.g. -s12: This only applies when -r is used. The image is multiplied (scaled) by NNNN after radiometric correction. The reason for this is that the images are often very dark without this. The default value if -s is not used is 8 which seems to work well in most cases.


Note that the resulting PNGs will often look very dark. The reason is that 16 bits yield an intensity range of 0-65535 but in all cases I have seen the maximum intensity is roughly 3000 or lower. An exception is RAD files, see (3) above.

As before, this should work for lots of different PDS files, e.g. MER, Pathfinder, Voyager, Galileo, Mariner 9/10 (!), Viking etc.

Just a reminder, the next major PDS release for both Spirit and Opportunity, Sols 90-180, will be next monday, the 31st. sorta...

The Analyst's Notebook had said January 31st for several months now, but was recently updated to:



Here's hoping the RAD files make it into the first distribution.

If you're cunning, you can actually find much of the data for the next 90 sols already

It would appear that it's mainly engineering camera data that isnt in yet - I've already put together an Opportunity mosaic from sol 114





Doug

Nice mosaic!!

<Rude and impatient>So... tell us about the process of putting together another 180 sols worth of Radiometrically Calibrated Imagery.</Rude and impatient>

From the MER Analysts notebook site:
"Third data releases for Spirit and Opportunity have been postponed until the end of February"

If you're cunning - you can find almost all of the pancam imagery anyway - just none of the usefull navigation pages, or the other usefull stuff.

Doug

Yep, all the way to sols 349 (MER-B) and 369 (MER-A).

djellison,
your references to "cunning" bring a smile on my face every time, since they remind me of Black Adder : "I have a cunning plan ....."

Actually - that I havnt found...

i.e. this has something in it
http://anserver1.eprsl.wustl.edu/navigator...rover=B&sol=180

and this doesnt
http://anserver1.eprsl.wustl.edu/navigator...rover=B&sol=181

ditto

http://anserver1.eprsl.wustl.edu/navigator...rover=A&sol=180 is full and
http://anserver1.eprsl.wustl.edu/navigator...rover=A&sol=181 is empty

Doug

It's there (or some of it is).

The official PDS release of sols 91-180 for both rovers finally made it out. http://anserver1.eprsl.wustl.edu/ has the new info.

I've updated my calibrated 6 filter color galleries to include the new sols :
Spirit sol 91-180
Opportunity sol 91-180

Great new galleries slinted!! Those colors look superb!

Incredible as always slinted, thanks!

Slinted: I was eager to see true colors images of Columbia Hills and Endurance crater, many thanks. But i'm also amazed with last images of Spirit's Sundial: these true color pictures allow for a nice perception of Dusty Martian world...

Beautiful, Slinted! It's nice to see that the basalt blocks indeed have the same color as I imagined. The basalt-block plains of Mars IMO should look similar to those on the moon, except with a rust-layer on top:

http://www.lyle.org/~markoff/pdsprocessed/...58L234567C1.JPG

Looks like the special URL's for RAD images ahead of release have been shutdown

Doug

Maybe someone read it here and closed the loophole.
Well. it was cunning while it lasted ....

Frankly - it should have been closed all the time - it was quite worrying that we could snoop around the APXS and Mossbauer data 6 months ahead of time.

Sols 180 - 270 are due in a month anyway

Doug

Well, it would have been more worrying if they had the Sol 500-600 data up there already....

Yes It was FUN while it lasted...! sol 213- 'Cahokia' images are awesome!
One set of L256 for the horizon (Grissom Hill)only had operations data though...
Real pitty they've vanished..

I have quite a bit of the Larrys Lookout pan - but not the interesting part looking at Ten.Val.

Doug

yeah there were lots of missing files for those days..

It's that time again! April 22nd marks the next PDS release of MER data, this time sols 180-270.

places to look for the files once they get posted:
Planetary Image Atlas
Analyst's Notebook
PDS Top Level Search
PDS Geosciences FTP

I have a question regarding the rear/forward hazcam images:
In the early image-releases published at the JPL site
there have been always two images posted for each hazcam (front/rear):
one for the the left and one for the right side respectively.
Unfortunately, on the PDS/MER Analyst's notebook site there seems to
be only one Hazcam-Image (the left one) listed for each Sol.
Does anyone know how to obtain the other hazcam images ?
This is important, if for example the image of the left hazcam has
distortions/missing pixels etc. that are not present in the right hazcam image.

Take a closer look at the list of files available when you click on the listing for an image, either from a search or from the navigator interface. Since they were taken simultaneously, they only generate one menu listing under which both left and right and right frames can be downloaded.

Thanks a lot for the hint, slinted !