Curiosity Image Retrieval Tools, scripts and software Options

[jmknapp]

To simply get a list of all the raw images for a given sol, I put up a web widget--access it with wget as follows, for example for sol 4:

wget -O sol4files.txt http://curiositymsl.com/cgi-bin/getfiles.cgi?sol=4

I.e., just set the sol=N parameter and you get a text file back with all the raw images currently on the MSL public web site

For a really basic web interface to same, see:

http://curiositymsl.com/rawlist

[elakdawalla]

Thanks, this is great! Note that with Navcam and Hazcam images it's indexing thumbnail versions of the raws (*-thm.jpg) whereas with the MSSS cameras the raws are linked directly.

[jmknapp]

Thanks, this is great! Note that with Navcam and Hazcam images it's indexing thumbnail versions of the raws (*-thm.jpg) whereas with the MSSS cameras the raws are linked directly.

You're right--thanks! It's fixed now.

[walfy]

Any chance that Midnight Mars Browser can be modified by the code wizards to do the trick for quick MSL downloads? I love that program! It sends photos of each camera to their own folders, even folders for separates sols. Very nice way to keep track. (Crossing fingers).

[elakdawalla]

Believe me, Mike knows very well we want that, and no amount of pestering will make it more likely to happen (or happen any faster), though compliments are probably always welcome. As with pretty much all of you guys, that isn't his day job. We just have to wait and hope he can find time somehow!

[elakdawalla]

To simply get a list of all the raw images for a given sol, I put up a web widget--access it with wget as follows...

Again, many thanks. I downloaded everything available so far, did some light filing (separated thumbs from fulls), and posted it here (30 MB):

https://planetary.s3.amazonaws.com/data/msl...ru_sol_0004.zip

[M_Welander]

It sends photos of each camera to their own folders, even folders for separates sols.

That is how Curiosity RAW Get works as well. It also keeps track of the images you've previously downloaded.

[rtphokie]

Anyone had any luck decoding the filenames on the raw images?

SSSSIICCCCCCCCCCT1_DXXX

SSSS = sol number (good to see 4 digits here)
II = is the instrument (ML = Mastcam left)
CCCCCCCCCC = ????, is this a serial number? It doen't appear to be a time (see below)
T = image size I or E
1_DXXXX = ????

Looking at a couple of examples:
0003ML0000139000I1_DXXX taken at 2012-08-09 06:01:57 UTC
0003ML0000134000I1_DXXX taken at 2012-08-09 05:59:34 UTC

Those timestamps are 2 min 23 seconds apart. the numbers in the filename do not seem to refer to time, Earth or Mars.


Also, some of the mastcam images ending in I1_DXXX also have higher resolution versions ending in E1_DXXX but aren't listed on the mars.jpl.nasa.gov site. Are they listed somewhere else?

[jmknapp]

Noticed a little bug on the JPL raw images web site--for full frame MAHLI images they show a thumbnail with a link to the full frame image, but it turns out the thumbnail IMG SRC is actually the full frame image. The html code sets the thumbnail width to 160 so it looks like a thumbnail, but it's a waste of bandwidth as all the full frame MAHLI images for a given sol will be downloaded when just looking at the thumbnail page.

Here's an example (MAHLI images with 'E' in the 17th character of the filename are full-frame, 'I' for thumbnail):

<td valign="top" align="center" width="200"><div style="margin:10px;"><a href="./?rawid=0001MH0000001000E1_DXXX&s=1"><img src="http://mars.jpl.nasa.gov/msl-raw-images/msss/00001/mhli/0001MH0000001000E1_DXXX.jpg" alt="Image taken by Mars Hand Lens Imager" width="160" border="0"></a><br><div class="RawImageCaption">Mars Hand Lens Imager<br>full<br><nobr><div class="RawImageUTC">2012-08-07 04:46:02&nbsp;UTC</div></nobr></div></div></td>

[mcaplinger]

Noticed a little bug on the JPL raw images web site...

That sounds like a useful bug report, but I have no idea who's in charge of the website. I'll ask around. Maybe Doug knows?

[jmknapp]

That sounds like a useful bug report, but I have no idea who's in charge of the website. I'll ask around. Maybe Doug knows?

I sent a feedback message under the category "site problem"--we'll see if there's a response. I notice that the same holds true for mastcam images. Most of the mastcam images so far have been thumbnails, but there are a few intermediate ("subframe" 1200x1200) versions which have the non-thumbnail as the IMG SRC. That could be a problem when the big mastcam images start rolling in.

[elakdawalla]

I reported the same to Veronica McGregor a few days ago (who is not in charge, but obviously can talk to whomever is) and she said that they officially do not care about bandwidth since everything is stored in the cloud. Now, that's fine for them but it'd be better for the user if they didn't have to download so much data. But I don't want to hassle them about this detail since it's the least of our problems with accessing data. If I'm going to bug them I want to bug them about things like getting metadata (filter information, pointing information etc) and faster distribution of the raws hopefully by automating things. As of the time I left JPL, as far as anybody knew it was a manual process. Priorities! Other people have been begging Mike Howard to make a version of MMB for Curiosity. Even if he wanted to, he couldn't right now, because they're not posting enough metadata.

[mcaplinger]

I reported the same to Veronica McGregor a few days ago (who is not in charge, but obviously can talk to whomever is) and she said that they officially do not care about bandwidth since everything is stored in the cloud.

That's not a useful response IMHO. The camera thumbnails are lovingly antialiased onboard to be as good as they can be at their minimal size, so just tossing them seems really lazy, and it's contributing to make the pages load really slowly even if you have a fast connection.

As for metadata, if I recall correctly this was provided in a separate file for MER. There are no technical barriers to doing the same thing for MSL...

Let me emphasize, just my own opinion and not an official position, etc.

[Astro0]

Let's remember that the mission is not obliged to give us access to these image at all. At least until the PDS release.
It's early days in the mission and the priority is the mission.

All of the so-called "issues" with getting quick and easy access to the images will sort itself out in due course.

Remember that we are privileged to get access to this data, but they are not obligated to provide it.
The fact that they do provide it at all is highly appreciated and while frustrating to some that it's not quicker and easier, we need to remember that their people and resources are not unlimited. Give them time.

UMSF is a great place to provide useful suggestions and ways to access the imagery.
It is of no value to complain about it.

[mcaplinger]

It is of no value to complain about it.

I'm on the camera team and I'll complain about it if I want to, thanks.

That said, this is kind of an interesting article.
http://www.informationweek.com/government/...web-s/240005286