IPB

Welcome Guest ( Log In | Register )

24 Pages V  < 1 2 3 4 > »   
Reply to this topicStart new topic
Curiosity Image Retrieval Tools, scripts and software
jmknapp
post Aug 11 2012, 08:41 PM
Post #16


Senior Member
****

Group: Members
Posts: 1289
Joined: 9-February 04
From: Columbus OH USA
Member No.: 13



To simply get a list of all the raw images for a given sol, I put up a web widget--access it with wget as follows, for example for sol 4:

wget -O sol4files.txt http://curiositymsl.com/cgi-bin/getfiles.cgi?sol=4

I.e., just set the sol=N parameter and you get a text file back with all the raw images currently on the MSL public web site

For a really basic web interface to same, see:

http://curiositymsl.com/rawlist


--------------------
Go to the top of the page
 
+Quote Post
elakdawalla
post Aug 11 2012, 10:28 PM
Post #17


Bloggette par Excellence
****

Group: Admin
Posts: 4364
Joined: 4-August 05
From: Pasadena, CA, USA, Earth
Member No.: 454



Thanks, this is great! Note that with Navcam and Hazcam images it's indexing thumbnail versions of the raws (*-thm.jpg) whereas with the MSSS cameras the raws are linked directly.


--------------------
My blog - @elakdawalla on Twitter - Please support unmannedspaceflight.com by donating here.
Go to the top of the page
 
+Quote Post
jmknapp
post Aug 12 2012, 12:51 AM
Post #18


Senior Member
****

Group: Members
Posts: 1289
Joined: 9-February 04
From: Columbus OH USA
Member No.: 13



QUOTE (elakdawalla @ Aug 11 2012, 05:28 PM) *
Thanks, this is great! Note that with Navcam and Hazcam images it's indexing thumbnail versions of the raws (*-thm.jpg) whereas with the MSSS cameras the raws are linked directly.


You're right--thanks! It's fixed now.


--------------------
Go to the top of the page
 
+Quote Post
walfy
post Aug 12 2012, 04:37 AM
Post #19


Member
***

Group: Members
Posts: 357
Joined: 5-January 10
Member No.: 5161



Any chance that Midnight Mars Browser can be modified by the code wizards to do the trick for quick MSL downloads? I love that program! It sends photos of each camera to their own folders, even folders for separates sols. Very nice way to keep track. (Crossing fingers).
Go to the top of the page
 
+Quote Post
elakdawalla
post Aug 12 2012, 04:47 AM
Post #20


Bloggette par Excellence
****

Group: Admin
Posts: 4364
Joined: 4-August 05
From: Pasadena, CA, USA, Earth
Member No.: 454



Believe me, Mike knows very well we want that, and no amount of pestering will make it more likely to happen (or happen any faster), though compliments are probably always welcome. As with pretty much all of you guys, that isn't his day job. We just have to wait and hope he can find time somehow!


--------------------
My blog - @elakdawalla on Twitter - Please support unmannedspaceflight.com by donating here.
Go to the top of the page
 
+Quote Post
elakdawalla
post Aug 12 2012, 04:49 AM
Post #21


Bloggette par Excellence
****

Group: Admin
Posts: 4364
Joined: 4-August 05
From: Pasadena, CA, USA, Earth
Member No.: 454



QUOTE (jmknapp @ Aug 11 2012, 01:41 PM) *
To simply get a list of all the raw images for a given sol, I put up a web widget--access it with wget as follows...
Again, many thanks. I downloaded everything available so far, did some light filing (separated thumbs from fulls), and posted it here (30 MB):

https://planetary.s3.amazonaws.com/data/msl...ru_sol_0004.zip


--------------------
My blog - @elakdawalla on Twitter - Please support unmannedspaceflight.com by donating here.
Go to the top of the page
 
+Quote Post
M_Welander
post Aug 12 2012, 05:37 AM
Post #22


Junior Member
**

Group: Members
Posts: 36
Joined: 7-November 04
From: Sweden
Member No.: 108



QUOTE (walfy @ Aug 12 2012, 06:37 AM) *
It sends photos of each camera to their own folders, even folders for separates sols.


That is how Curiosity RAW Get works as well. It also keeps track of the images you've previously downloaded.
Go to the top of the page
 
+Quote Post
rtphokie
post Aug 12 2012, 08:15 PM
Post #23


Newbie
*

Group: Members
Posts: 6
Joined: 26-September 11
Member No.: 6180



Anyone had any luck decoding the filenames on the raw images?

SSSSIICCCCCCCCCCT1_DXXX

SSSS = sol number (good to see 4 digits here)
II = is the instrument (ML = Mastcam left)
CCCCCCCCCC = ????, is this a serial number? It doen't appear to be a time (see below)
T = image size I or E
1_DXXXX = ????

Looking at a couple of examples:
0003ML0000139000I1_DXXX taken at 2012-08-09 06:01:57 UTC
0003ML0000134000I1_DXXX taken at 2012-08-09 05:59:34 UTC

Those timestamps are 2 min 23 seconds apart. the numbers in the filename do not seem to refer to time, Earth or Mars.


Also, some of the mastcam images ending in I1_DXXX also have higher resolution versions ending in E1_DXXX but aren't listed on the mars.jpl.nasa.gov site. Are they listed somewhere else?
Go to the top of the page
 
+Quote Post
jmknapp
post Aug 13 2012, 12:40 AM
Post #24


Senior Member
****

Group: Members
Posts: 1289
Joined: 9-February 04
From: Columbus OH USA
Member No.: 13



Noticed a little bug on the JPL raw images web site--for full frame MAHLI images they show a thumbnail with a link to the full frame image, but it turns out the thumbnail IMG SRC is actually the full frame image. The html code sets the thumbnail width to 160 so it looks like a thumbnail, but it's a waste of bandwidth as all the full frame MAHLI images for a given sol will be downloaded when just looking at the thumbnail page.

Here's an example (MAHLI images with 'E' in the 17th character of the filename are full-frame, 'I' for thumbnail):

QUOTE
<td valign="top" align="center" width="200"><div style="margin:10px;"><a href="./?rawid=0001MH0000001000E1_DXXX&s=1"><img src="http://mars.jpl.nasa.gov/msl-raw-images/msss/00001/mhli/0001MH0000001000E1_DXXX.jpg" alt="Image taken by Mars Hand Lens Imager" width="160" border="0"></a><br><div class="RawImageCaption">Mars Hand Lens Imager<br>full<br><nobr><div class="RawImageUTC">2012-08-07 04:46:02&nbsp;UTC</div></nobr></div></div></td>


--------------------
Go to the top of the page
 
+Quote Post
mcaplinger
post Aug 13 2012, 12:56 AM
Post #25


Senior Member
****

Group: Members
Posts: 1267
Joined: 13-September 05
Member No.: 497



QUOTE (jmknapp @ Aug 12 2012, 05:40 PM) *
Noticed a little bug on the JPL raw images web site...

That sounds like a useful bug report, but I have no idea who's in charge of the website. I'll ask around. Maybe Doug knows?


--------------------
Disclaimer: This post is based on public information only. Any opinions are my own.
Go to the top of the page
 
+Quote Post
jmknapp
post Aug 13 2012, 01:43 AM
Post #26


Senior Member
****

Group: Members
Posts: 1289
Joined: 9-February 04
From: Columbus OH USA
Member No.: 13



QUOTE (mcaplinger @ Aug 12 2012, 08:56 PM) *
That sounds like a useful bug report, but I have no idea who's in charge of the website. I'll ask around. Maybe Doug knows?


I sent a feedback message under the category "site problem"--we'll see if there's a response. I notice that the same holds true for mastcam images. Most of the mastcam images so far have been thumbnails, but there are a few intermediate ("subframe" 1200x1200) versions which have the non-thumbnail as the IMG SRC. That could be a problem when the big mastcam images start rolling in.


--------------------
Go to the top of the page
 
+Quote Post
elakdawalla
post Aug 13 2012, 01:45 AM
Post #27


Bloggette par Excellence
****

Group: Admin
Posts: 4364
Joined: 4-August 05
From: Pasadena, CA, USA, Earth
Member No.: 454



I reported the same to Veronica McGregor a few days ago (who is not in charge, but obviously can talk to whomever is) and she said that they officially do not care about bandwidth since everything is stored in the cloud. Now, that's fine for them but it'd be better for the user if they didn't have to download so much data. But I don't want to hassle them about this detail since it's the least of our problems with accessing data. If I'm going to bug them I want to bug them about things like getting metadata (filter information, pointing information etc) and faster distribution of the raws hopefully by automating things. As of the time I left JPL, as far as anybody knew it was a manual process. Priorities! Other people have been begging Mike Howard to make a version of MMB for Curiosity. Even if he wanted to, he couldn't right now, because they're not posting enough metadata.


--------------------
My blog - @elakdawalla on Twitter - Please support unmannedspaceflight.com by donating here.
Go to the top of the page
 
+Quote Post
mcaplinger
post Aug 13 2012, 02:18 AM
Post #28


Senior Member
****

Group: Members
Posts: 1267
Joined: 13-September 05
Member No.: 497



QUOTE (elakdawalla @ Aug 12 2012, 06:45 PM) *
I reported the same to Veronica McGregor a few days ago (who is not in charge, but obviously can talk to whomever is) and she said that they officially do not care about bandwidth since everything is stored in the cloud.

That's not a useful response IMHO. The camera thumbnails are lovingly antialiased onboard to be as good as they can be at their minimal size, so just tossing them seems really lazy, and it's contributing to make the pages load really slowly even if you have a fast connection.

As for metadata, if I recall correctly this was provided in a separate file for MER. There are no technical barriers to doing the same thing for MSL...

Let me emphasize, just my own opinion and not an official position, etc.


--------------------
Disclaimer: This post is based on public information only. Any opinions are my own.
Go to the top of the page
 
+Quote Post
Astro0
post Aug 13 2012, 03:02 AM
Post #29


Senior Member
****

Group: Admin
Posts: 2781
Joined: 21-December 05
From: Canberra, Australia
Member No.: 615



Let's remember that the mission is not obliged to give us access to these image at all. At least until the PDS release.
It's early days in the mission and the priority is the mission.

All of the so-called "issues" with getting quick and easy access to the images will sort itself out in due course.

Remember that we are privileged to get access to this data, but they are not obligated to provide it.
The fact that they do provide it at all is highly appreciated and while frustrating to some that it's not quicker and easier, we need to remember that their people and resources are not unlimited. Give them time.

UMSF is a great place to provide useful suggestions and ways to access the imagery.
It is of no value to complain about it.
Go to the top of the page
 
+Quote Post
mcaplinger
post Aug 13 2012, 03:54 AM
Post #30


Senior Member
****

Group: Members
Posts: 1267
Joined: 13-September 05
Member No.: 497



QUOTE (Astro0 @ Aug 12 2012, 08:02 PM) *
It is of no value to complain about it.

I'm on the camera team and I'll complain about it if I want to, thanks.

That said, this is kind of an interesting article.
http://www.informationweek.com/government/...web-s/240005286


--------------------
Disclaimer: This post is based on public information only. Any opinions are my own.
Go to the top of the page
 
+Quote Post

24 Pages V  < 1 2 3 4 > » 
Reply to this topicStart new topic

 



RSS Lo-Fi Version Time is now: 29th July 2014 - 06:52 PM
RULES AND GUIDELINES
Please read the Forum Rules and Guidelines before posting.

IMAGE COPYRIGHT
Images posted on UnmannedSpaceflight.com may be copyrighted. Do not reproduce without permission. Read here for further information on space images and copyright.

OPINIONS AND MODERATION
Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators.
SUPPORT THE FORUM
Unmannedspaceflight.com is a project of the Planetary Society and is funded by donations from visitors and members. Help keep this forum up and running by contributing here.