Curiosity Image Retrieval Tools, scripts and software Options

[Reckless]

Brilliant
chemcam large is good too
and many thanks

Roy

[fredk]

This looks great, Joe! One suggestion: could you specify what the various sizes mean on the help page - maybe in a table?

[jmknapp]

This looks great, Joe! One suggestion: could you specify what the various sizes mean on the help page - maybe in a table?

OK, done.

chemcam large is good too

Yes, and in fact three from sol 45 just came down a few hours ago--evidently Jake is ready for its closeups.

[Ant103]

Wow ! Great great job there !

Will it be possible to display the result in grid form ?

[jmknapp]

Will it be possible to display the result in grid form ?

Maybe, although it introduces some complications like where to position the preview, and how much meta-info to show. Are you thinking of a grid with just the image thumbnails & no text until you mouse over or something like that?

[Ant103]

Hmm, maybe keeping some meta-info like the the filename and Sol. And by hovering, display additional informations. And one click to access the full version.

[elakdawalla]

Joe, this is really promising. I have a couple of requests if you'd be willing to entertain them:

- I agree with Ant103's request for a grid view of thumbnails. Right now the only site that gives you that is *shudder* JPL's. Only meta-info I'd need displayed with that are sol, camera abbreviation, and LMST (ideally using a 24-hr clock, thus avoiding "AM" and "PM"); I'd stick filename in the alt text so it'd be displayed on mouseover.

- Not very many results are displayed; in order to see all results I have to do a *lot* of clicking or wheel-scrolling. Too much of that kills my wrists :| I'm not sure how to fix this (though a grid display of lots of thumbnails would help).

- The homepage mirroring Google is cute, but I'm wondering if you could just put the help text on the homepage, or at least some suggested searches. I keep having to go to help to remind myself of syntax.

[jmknapp]

Joe, this is really promising. I have a couple of requests if you'd be willing to entertain them:

Agreed a grid view is the way to go--I won't have time for it until after a week or so (a fall vacation coming up--not too late to go to the beach I hope).

The AM/PM thing is kind of a personal preference--maybe the 24-hour clock is natural to some, but I'm continually doing the math. Also, for local solar time I was thinking that antemeridian and postmeridian is kind of natural.

As for mirroring Google, I wanted to include an "I'm Just Curious" button to just list all recent images as a kind of fallback, but was having problems getting that to work. I'll put a couple suggested searches up under the text box.

Meanwhile, I added image preview on hover and a sol clock to the http://curiositymsl.com page.

[EdTruthan]

I second the Wow's! Joe the search page is just stellar. Cannot thank you enough. MSL living just got much easier. If you're ever putzing around with adding improvements it'd be a nice feature too if the results for a given image could somehow include a "flag" of some kind indicating whether a corresponding left or right match were available (especially helpful with Navcam). Knowing what's obtainable (or not) for anaglyphing would then be a breeze. Thanks again for the great job.

[jmknapp]

"flag" of some kind indicating whether a corresponding left or right match were available (especially helpful with Navcam).

That would be nice, having spent time looking for such matches myself. The number one rule right now though is that any such tagging has to be automatic & I'm thinking that would be very tricky. Right now what I do is just search by a particular sol and change the sort order to "when taken." That way candidate pairs will be next to each other & there usually aren't that many to go through, particularly if the search is limited to 'large.'

An ultimate goal, and I do plan to do this, is to allow user tagging of images. That way the work of identifying stereo pairs, features, landmark names, etc. can be crowdsourced. That has potential downsides too, but it would be interesting to see how it would work out. It would be very handy to be able to search on terms like "jake" or "glenelg" or "phyllosilicate."

[atomoid]

it would be interesting to know how the firmware on MSL incorporates metadata for each image, and if so, it could be accessible for public use. im guessing for engineering purposes they do keep track of certain data per-image such as metrics on exactly where the camera is pointed and be able to sync it to all other metrics recorded by MSL. maybe its all recorded but not necessarily linked and has to be mined and refined if needed for anything.

Tapping into such a matrix of all that metadata could be used to generate movies by querying images that point in a particular direction (like a view of looking out the left window as driving along) or apply logic against a coordinate map to extrapolate images where the camera happened to get a certain coordinate in view (such as 'Jake' or some distant yardang).

[CosmicRocker]

I'm trying to learn which method is the most efficient for downloading MSL imagery. I know some of you are using wget for downloading, but that seems not to be a convenient option for me on a Windows system. I have to admit that, not being a programming type, writing and testing scripts of one sort or another is not very appealing to me. However, if the potential rewards are great enough I might be willing to give it a try.

Currently I am using the Windows application Curiosity RAW Get, which was made available at the beginning of this thread. I find it very convenient because:

1. It has an intuitive graphical user interface.
2. It allows the user to filter on camera type, left/right side, and various image characteristics.
3. It downloads the images selected according to the criteria above,
4. saves the files into a logical directory structure,
5. and it maintains some kind of image index so it knows which images you have already downloaded.

I'm impressed as well with Joe's web site, but so far I haven't figured out how, or if, one can use the site to mass download the images unless you use a 3rd party mass downloader application. Since so many people are talking only about Joe's site, I wonder if I am missing something important about how to use it.

[jmknapp]

True, my web page doesn't enable bulk downloading. With Ludo's app you can select images and export to a list for later use with wget.

As for MSL metadata, I gather that the rover tries to maintain a reasonably accurate idea of where it is in the "topo" frame and local level defined at the landing site (X north, Y east, Z up from the landing site), but with wheel slippage and other uncertainties, it's all subject to revision. I wonder if part of the daily drill is to tell the rover "oh, by the way, you're actually over here a little."

The camera frames (X horizontal, Y vertical, Z forward) have a calibrated relationship ultimately to the rover frame (X forward, Y right, Z down). Ideally, one should be able to take an XY position of a pixel in a camera frame and a time of observation and get an absolute position and direction vector in whatever coordinate system is convenient (J2000, MARS_IAU, MSL_TOPO, etc.).

Seems reasonable that the rover EXIF info so to speak would include an initial estimate, but who knows? I think they'll publish this kind of data on the NAIF web site eventually. In the meantime, for us it's by guess and by golly.

[ronald]

I'm getting used to Stellingwerffs http://msl-raw-images.appspot.com/lists.html.

Especially useful is the generated list of files you want to download. It is for wget only which is fine on my linux machine but at work we use OSX which comes with curl instead of wget. So here is the code line for using curl with a list of urls:

CODE

xargs -n 1 curl -O < urls.txt

Where urls.txt is a plain list of the image urls - you need to remove the wget stuff in the generated list ...

[RoverDriver]

....
It is for wget only which is fine on my linux machine but at work we use OSX which comes with curl instead of wget.
...

You do know you can have wget on OSX, right? I installed mac ports to have all the usual tools, including wget but if you don't want to install mac ports apparently you can download the source code from gnu.org and do the usual installation (tar, ./configure, make, make install). If you google "mac osx wget" the first link has the details. If you are used to work under linux, mac ports is your friend.

Paolo