Curiosity Image Retrieval Tools, scripts and software Options

[fredk]

Hi Joe - curiosityrover hasn't updated for a couple of days - maybe it needs a kick?

[Gerald]

I'd guess, that Joe uses MSL Raw Images.
That site doesn't seem to work correctly at the moment; and I feel a little guilty. Probably they're just fixing an issue I reported.

[PaulH51]

I'd guess, that Joe uses MSL Raw Images.
That site doesn't seem to work correctly at the moment; and I feel a little guilty. Probably they're just fixing an issue I reported.

I concur Gerald, probably an issue with a minor change in the way the image data is reported that has stopped Joe's page scripts from running properly.

Does anyone know of any other MSL image libraries that provide the pointing information (heading and elevation etc) that are still working? I am really missing that data in order to fully understand aspects of certain images from sol 297 onward. Hope Joe gets his very useful library back to normal operation soon

[Gerald]

"The issue has been resolved." Thanks a lot!

[atomoid]

...the pointing information (heading and elevation etc)..

me too,... ahh, interesting rock is up again!

[PaulH51]

"The issue has been resolved." Thanks a lot!

Yes, now back to normal... Now back to work to catch up on the images......

[mhoward]

There is an issue on MSL Sol 351 that may affect those of us who maintain or use image retrieval tools. Several of the Right Mastcam images of the Phobos/Deimos transit (if that's what it is) have web raw id's that are 24 characters long instead of 23 characters long. The MMM (Mastcam MAHLI MARDI) web rawid format is not documented*, but still, 24 characters comes as a bit of a surprise, as every MMM web rawid in the previous 350 sols has been 23 characters long. Some computer code may not be able to deal with 24 characters. I can say my own code definitely is not able to deal with it; Midnight Loader simply crashed, and now is patched, but my web pages and app client will simply deny the existence of 24-character rawids for now. Other programs may see similar problems, or not.

Here's a typical 23-character rawid

And here's a 24-character example

* To anticipate a question: no, the MMM web rawid format is not documented in the MSL Software Interface Specification (MSL_CAMERA_SIS MSL_MMM_EDR_RDR_DPSIS.PDF) as far as I know; it appears to be a different format from the PDS image files, if perhaps related. I haven't had much time to look into it though; please correct me if I'm wrong.

[mcaplinger]

24 characters comes as a bit of a surprise, as every MMM web rawid in the previous 350 sols has been 23 characters long.

Huh. I think the extra character will appear if video is shot in "immediate JPEG" rather than "deferred JPEG" mode (perhaps the SIS explains what this means, I don't remember; in this mode there is only one thumbnail for each 16-frame "group of pictures") and I thought we had done this before (scoop video), but maybe not. There also may be future changes to the naming convention in general.

I had nothing to do with the naming convention and I also argued against incompatible changes to it, so I am not taking the blame for any inconvenience caused.

[mhoward]

I should probably just clarify before anyone wonders: the problem isn't the length of the filename as such, the problem is that programs may need to extract information from the filename; and if we can no longer even pretend to understand the contents of the filename, that doesn't work so well.

[jmknapp]

Thanks for the heads up--luckily my code wasn't looking at the part of the string past "MR" as I don't know what those numbers signify--ignorance is bliss!

[mcaplinger]

no, the MMM web rawid format is not documented in the MSL Software Interface Specification (MSL_CAMERA_SIS) as far as I know; it appears to be a different format from the PDS image files, if perhaps related.

I'm not sure myself how it's related, but I'll speculate.

The first four digits are the sol, I think that's without controversy

The two letters are the abbreviated instrument name, ditto.

The ten digits look like the sequence ID (elided from the six digits in the SIS to four digits) and the command number (three digits) in the sequence (together forming the product ID), per the SIS page 25, followed by three digits which is some kind of identifier for commands that yield more than one image (video and zstack). For normal image products these should always be 000.

The letter is the data type as defined on pages 23-24 of the SIS.

The _DXXX is as defined in the SIS page 24 and will always be _DXXX.

The digit after the data type letter is probably the version (SIS page 24), has it ever not been 1?

At this point, I would guess that we are seeing either a bug in the name generation that has been encountered for the Phobos/Deimos sequence, or a definition of the video identifier that causes it to have four digits in some cases rather than three; perhaps not a simple incrementing count. One would have to go back to previously acquired video to see.

For parsing at this point I would assume that the field between the product ID and the type could be 3 or 4 digits. As indicated previously I'm not sure how this might change in the future.

[jmknapp]

I installed a web front-end called DaDaBIK to my raw image mysql database:

http://curiosityrover.com/rawdb

log in as 'umsf' and password 'msl' & you can search the database by various fields.

[PaulH51]

I installed a web front-end called DaDaBik to my raw image mysql database:

Thanks Joe, very useful

[elakdawalla]

Not only useful, but also very fast. Thanks!

[jmknapp]

A new cool tool to view Curiosity images, or to get a better idea of what they're up to at a given location:

Curiosity Rover Synth



It's kind of beta so let me know about any issues or "enhancements".