IPB

Welcome Guest ( Log In | Register )

Curiosity Image Retrieval Tools, scripts and software
DEChengst
post Aug 11 2012, 09:18 AM
Post #1


Member
***

Group: Members
Posts: 270
Joined: 29-December 04
From: NLA0:
Member No.: 133



I wrote a shell script that allows you to download all images from a given sol.:

http://paranoid.dechengst.nl/files/MSLget.sh

There are two rm commands in the script to clean up the tempdir. If you're afraid my script screws up your system replace them with rm -i commands. If you want any features added please let me know and I'll see what I can do.

EDIT1:

Added leading zeros in the directory names for easy sorting after a request for it on IRC.

EDIT2:

I see a lot of people downloading the script. To be clear: This is a script for *NIX systems and won't run on a standard Windows box. To get it running under Windows you need to install a *NIX environment like Cygwin. Your other option would be to install something like Ubuntu in VirtualBox.

EDIT3:

As RoverDriver pointed out you need to have wget on your system to use this script. The script now handles this gracefully.


--------------------
PDP, VAX and Alpha fanatic ; HP-Compaq is the Satan! ; Let us pray daily while facing Maynard! ; Life starts at 150 km/h ;
Go to the top of the page
 
+Quote Post
 
Start new topic
Replies
3d_mars
post Aug 17 2012, 03:59 AM
Post #2


Newbie
*

Group: Members
Posts: 11
Joined: 12-August 12
Member No.: 6550



First post. Here's a small script I wrote to automatically process left and right full frame Navcam image pairs into SBS stereoscopic images. Thanks go to jmknapp for his image list service, http://curiositymsl.com. I originally intended to automate the full process, from downloading the images to creating the final SBS output, but then I realized that I may want to process the images a bit before the final step. I use the free StereoPhoto Maker to create the final images. It's a windows app which is why I chose to write this in AutoIt (also free).

BTW, the images are PHENOMENAL when viewed with the Sony HMZ-T1. I purchased this hardware a few months ago in anticipation of this mission (and other 3D applications), and I've been extremely pleased with the quality of the high contrast dual OLED displays, which present a ghost-free stereo image.

CODE
#include <file.au3> ; for _filereadtoarray
#include <Array.au3> ; for _arraydisplay

; 20120816, version 0.1
; This script processes all full frame Navcam images using jmknapp's http://curiositymsl.com, downloads only full frame LR pairs
(if not downloaded already) and creates a StereoPhoto script file
; (todo) Also process Mastcam images
; 3d_mars


; prompt user for sol number to retrieve, $sol
Local $sol = InputBox("Sol Number", "Input Sol Number:")
If @error Then Exit

; create temp file for listing all URLs for a given sol
; example: c:\users\username\appdata\local\temp\URLs_sol_1.txt
$URL_list_file=@TempDir & "\URLs_sol_" & $sol & ".txt"

; download image URL list to text file
; InetGet option syntax: URL, filename, 1=force reload, 0=wait for download
InetGet("http://curiositymsl.com/cgi-bin/getfiles.cgi?sol=" & $sol, $URL_list_file,1,0)

; read URL list and import to array
; note: $allURLs[0] contains the number of records read into the array.
Local $allURLs
If Not _FileReadToArray($URL_list_file, $allURLs) Then
MsgBox(4096, "Error", " Error importing from file to Array error:" & @error)
Exit
EndIf

; search for Navcam full frames (*EDR_F*) insert into new array $allframes
; Navcam full frame example URLs:
; http://mars.jpl.nasa.gov/msl-raw-images/pr...AUT_04096M_.JPG
; http://mars.jpl.nasa.gov/msl-raw-images/pr...AUT_04096M_.JPG

; declare large array instead of resizing array multiple times, uses very little RAM
Local $allframes[9999]=[0]
$x = 1
For $URL in $allURLs
;check for Navcam full frame
If StringInStr($URL,"EDR_F") Then
$allframes[$x]=$URL
$x += 1
EndIf
Next

;record total number of frames
$allframes[0]=$x - 1

; read allframes to find matching left/right pairs
; insert into new arrays $leftframe_Nav, $rightframe_Nav

Local $leftframe_Nav[9999]=[0]
Local $rightframe_Nav[9999]=[0]
Local $leftframe_counter = 0
Local $rightframe_counter = 0

For $x = 1 to $allframes[0]
If StringInStr($allframes[$x],"NLA_") Then
;left frame found, store in $leftframe_Nav array
$leftframe_counter += 1
$leftframe_Nav[$leftframe_counter]=$allframes[$x]
$leftframe_Nav[0]=$leftframe_counter
ElseIf StringInStr($allframes[$x],"NRA_") Then
;right frame found, store in $rightframe_Nav array
$rightframe_counter += 1
$rightframe_Nav[$rightframe_counter]=$allframes[$x]
$rightframe_Nav[0]=$rightframe_counter
EndIf
Next

; if no left full frames then abort and exit
If $leftframe_counter == 0 Then
MsgBox(0, "", "No Navcam full frames found")
Exit
EndIf

;choose path and file name for images and generated StereoPhoto Maker script file
Local $MyDocsFolder = "::{450D8FBA-AD25-11D0-98A8-0800361B1103}"
Local $StereoPhoto_script_file = FileSaveDialog("Choose a path and filename for images and StereoPhoto Maker script file:", $MyDocsFolder,
"Text Files (*.txt)", 2, "navcam_sbs_sol_" & $sol & ".txt")
; option 2 = dialog remains until valid path/file selected

If @error Then
MsgBox(4096, "", "Save cancelled.")
EndIf

; using lists of left and right frames, find matches and save output
Local $right_matching_frame_number[9999]=[0]
Local $StereoPhoto_script_lines[9999]=[0]
Local $URL_left_temp, $left_image_file_name, $URL_right_temp, $right_image_file_name, $left_image_full_path, $right_image_full_path
For $x = 1 to $leftframe_Nav[0]
;extract frame number
$leftframe_image_number=StringMid($leftframe_Nav[$x],StringInStr($leftframe_Nav[$x],"NLA_")+4,9)
; returns -1 if no match found
$right_matching_frame_number[$x] = _ArraySearch($rightframe_Nav, $leftframe_image_number, 0, 0, 0, 1)

; download left and right images
If $right_matching_frame_number[$x] <> -1 Then

; find the file names
$URL_left_temp = StringSplit($leftframe_Nav[$x], "/")
$left_image_file_name = $URL_left_temp[UBound($URL_left_temp)-1]
$URL_right_temp = StringSplit($rightframe_Nav[$right_matching_frame_number[$x]], "/")
$right_image_file_name = $URL_right_temp[UBound($URL_right_temp)-1]

; create full path names
$left_image_full_path=@WorkingDir & "\" & $left_image_file_name
$right_image_full_path=@WorkingDir & "\" & $right_image_file_name

; build array for StereoPhoto Maker script file
$StereoPhoto_script_lines[$x]="21,0,0,0,0," & $left_image_full_path & "," & $right_image_full_path & ",,"

; download the images
; if the file already exists don't download it again
If Not FileExists($left_image_full_path) Then
InetGet($leftframe_Nav[$x],$left_image_full_path,1,0)
EndIf
If Not FileExists($right_image_full_path) Then
InetGet($rightframe_Nav[$right_matching_frame_number[$x]],$right_image_full_path,1,0)
EndIf

EndIf

Next


; write StereoPhoto Maker script file
Local $hFile = FileOpen($StereoPhoto_script_file, 2) ; 2 = erase previous
_FileWriteFromArray($StereoPhoto_script_file, $StereoPhoto_script_lines, 1) ; 1 = index to start
FileClose($hFile)


; open StereoPhoto Maker, select File menu, select multi conversion from list...
Go to the top of the page
 
+Quote Post
jmknapp
post Aug 17 2012, 01:36 PM
Post #3


Senior Member
****

Group: Members
Posts: 1465
Joined: 9-February 04
From: Columbus OH USA
Member No.: 13



QUOTE (3d_mars @ Aug 16 2012, 10:59 PM) *
First post. Here's a small script I wrote to automatically process left and right full frame Navcam image pairs into SBS stereoscopic images.


I had to look up what SBS stereo is (but hey, I can't even find my red-blue glasses to see some of the probably amazing anaglyph efforts here). So I take it that you need to have some expensive headgear like the Sony you mentioned to view them ($800) or at least shutter glasses and a special monitor? I wonder what the cheapest entry point is.


--------------------
Go to the top of the page
 
+Quote Post
fredk
post Aug 17 2012, 02:07 PM
Post #4


Senior Member
****

Group: Members
Posts: 4247
Joined: 17-January 05
Member No.: 152



QUOTE (jmknapp @ Aug 17 2012, 02:36 PM) *
So I take it that you need to have some expensive headgear
Not at all! Using StereoPhoto Maker that 3d_mars linked to, you can view side-by-side images without any special hardware or glasses at all. StereoPhoto Maker displays the L and R images side by side, but only displaying a vertical strip wide enough to merge by crossing (or diverging) your eyes. You can then easily pan to see the entire image.
Go to the top of the page
 
+Quote Post

Posts in this topic
- DEChengst   Curiosity Image Retrieval Tools   Aug 11 2012, 09:18 AM
- - RoverDriver   QUOTE (DEChengst @ Aug 11 2012, 02:18 AM)...   Aug 11 2012, 01:32 PM
- - RoverDriver   As an example of using wget to download the HiRISE...   Aug 11 2012, 01:34 PM
- - M_Welander   I've written a small too to help me download t...   Aug 11 2012, 01:52 PM
- - James Sorenson   For windows users, I've been using Star Downlo...   Aug 11 2012, 02:11 PM
|- - Pertinax   QUOTE (James Sorenson @ Aug 11 2012, 10:1...   Aug 11 2012, 02:49 PM
- - Marc   I'm also working in a phyton script that autom...   Aug 11 2012, 02:26 PM
- - scalbers   Also a shell script using wget initially mentioned...   Aug 11 2012, 02:42 PM
- - M_Welander   Hm... actually... The reason why I posted my tool ...   Aug 11 2012, 03:16 PM
|- - M_Welander   QUOTE Please don't start threads for each tool...   Aug 11 2012, 04:20 PM
- - CosmicRocker   QUOTE (M_Welander @ Aug 11 2012, 08:52 AM...   Aug 11 2012, 04:09 PM
- - CosmicRocker   It seems that it might be useful to create a subfo...   Aug 11 2012, 05:00 PM
- - Stellingwerff   I'm building a webportal for easier access to ...   Aug 11 2012, 07:40 PM
- - elakdawalla   On previous missions when it's been tough to p...   Aug 11 2012, 08:12 PM
|- - M_Welander   I think it'll be fairly easy to get this ...   Aug 11 2012, 08:31 PM
- - jmknapp   To simply get a list of all the raw images for a g...   Aug 11 2012, 08:41 PM
|- - elakdawalla   QUOTE (jmknapp @ Aug 11 2012, 01:41 PM) T...   Aug 12 2012, 04:49 AM
- - elakdawalla   Thanks, this is great! Note that with Navcam a...   Aug 11 2012, 10:28 PM
|- - jmknapp   QUOTE (elakdawalla @ Aug 11 2012, 05:28 P...   Aug 12 2012, 12:51 AM
|- - walfy   Any chance that Midnight Mars Browser can be modif...   Aug 12 2012, 04:37 AM
- - elakdawalla   Believe me, Mike knows very well we want that, and...   Aug 12 2012, 04:47 AM
- - M_Welander   QUOTE (walfy @ Aug 12 2012, 06:37 AM) It ...   Aug 12 2012, 05:37 AM
- - rtphokie   Anyone had any luck decoding the filenames on the ...   Aug 12 2012, 08:15 PM
- - jmknapp   Noticed a little bug on the JPL raw images web sit...   Aug 13 2012, 12:40 AM
|- - mcaplinger   QUOTE (jmknapp @ Aug 12 2012, 05:40 PM) N...   Aug 13 2012, 12:56 AM
|- - jmknapp   QUOTE (mcaplinger @ Aug 12 2012, 08:56 PM...   Aug 13 2012, 01:43 AM
|- - djellison   QUOTE (mcaplinger @ Aug 12 2012, 05:56 PM...   Aug 13 2012, 05:03 AM
- - elakdawalla   I reported the same to Veronica McGregor a few day...   Aug 13 2012, 01:45 AM
|- - mcaplinger   QUOTE (elakdawalla @ Aug 12 2012, 06:45 P...   Aug 13 2012, 02:18 AM
|- - M_Welander   QUOTE (elakdawalla @ Aug 13 2012, 03:45 A...   Aug 13 2012, 10:54 AM
- - Astro0   Let's remember that the mission is not obliged...   Aug 13 2012, 03:02 AM
|- - mcaplinger   QUOTE (Astro0 @ Aug 12 2012, 08:02 PM) It...   Aug 13 2012, 03:54 AM
|- - jmknapp   QUOTE (mcaplinger @ Aug 12 2012, 10:54 PM...   Aug 13 2012, 10:28 AM
- - CosmicRocker   QUOTE (M_Welander @ Aug 11 2012, 07:52 AM...   Aug 13 2012, 04:06 AM
- - algorimancer   I have my AlgorimancerPG rangefinder/photogrammetr...   Aug 13 2012, 02:28 PM
|- - mcaplinger   QUOTE (algorimancer @ Aug 13 2012, 07:28 ...   Aug 13 2012, 03:14 PM
|- - ugordan   QUOTE (mcaplinger @ Aug 13 2012, 05:14 PM...   Aug 13 2012, 03:41 PM
|- - jmknapp   QUOTE (mcaplinger @ Aug 13 2012, 10:14 AM...   Aug 13 2012, 03:44 PM
||- - mcaplinger   QUOTE (jmknapp @ Aug 13 2012, 07:44 AM) F...   Aug 13 2012, 04:18 PM
||- - jmknapp   QUOTE (mcaplinger @ Aug 13 2012, 11:18 AM...   Aug 13 2012, 06:07 PM
||- - mcaplinger   QUOTE (jmknapp @ Aug 13 2012, 10:07 AM) S...   Aug 13 2012, 06:28 PM
|- - algorimancer   QUOTE (mcaplinger @ Aug 13 2012, 09:14 AM...   Aug 13 2012, 04:44 PM
- - jmknapp   Trying to develop a handier web interface to the r...   Aug 14 2012, 12:17 AM
|- - RoverDriver   QUOTE (jmknapp @ Aug 13 2012, 05:17 PM) T...   Aug 14 2012, 01:18 AM
|- - jmknapp   QUOTE (RoverDriver @ Aug 13 2012, 08:18 P...   Aug 14 2012, 01:46 AM
|- - RoverDriver   QUOTE (jmknapp @ Aug 13 2012, 06:46 PM) ....   Aug 14 2012, 02:36 AM
- - elakdawalla   First: AWESOME! It would be great to filter b...   Aug 14 2012, 12:40 AM
|- - jmknapp   QUOTE (elakdawalla @ Aug 13 2012, 07:40 P...   Aug 14 2012, 01:20 AM
|- - mcaplinger   QUOTE (jmknapp @ Aug 13 2012, 05:20 PM) ....   Aug 14 2012, 02:33 AM
- - Oersted   Works great on a Samsung Galaxy Nexus smartphone, ...   Aug 14 2012, 12:44 AM
|- - volcanopele   QUOTE (Oersted @ Aug 13 2012, 05:44 PM) W...   Aug 14 2012, 02:29 AM
- - lyford   Allow me to say OMAGERD THANK YOU   Aug 14 2012, 12:58 AM
- - ChrisC   Fantastic, Joe! My one suggestion has to do w...   Aug 14 2012, 02:12 AM
- - Cargo Cult   Evening, all - It's definitely not a fantasti...   Aug 14 2012, 06:52 AM
|- - stevesliva   QUOTE (Cargo Cult @ Aug 14 2012, 01:52 AM...   Aug 14 2012, 03:53 PM
- - Explorer1   It works on my end; good job! It's always...   Aug 14 2012, 07:52 AM
- - algorimancer   The AlgorimancerPG (APG) rangefinder utility is up...   Aug 14 2012, 03:14 PM
|- - algorimancer   QUOTE (algorimancer @ Aug 14 2012, 10:14 ...   Aug 15 2012, 12:11 AM
|- - algorimancer   QUOTE (algorimancer @ Aug 14 2012, 07:11 ...   Aug 16 2012, 01:08 AM
- - fredk   Your site is already looking fantastic, Joe. I...   Aug 14 2012, 03:30 PM
|- - mhoward   QUOTE (fredk @ Aug 14 2012, 09:30 AM) I a...   Aug 14 2012, 04:31 PM
- - elakdawalla   MESSENGER has a feed of sorts -- through its iPhon...   Aug 14 2012, 04:44 PM
- - jmknapp   OK, lots of good suggestions, I'll get to most...   Aug 14 2012, 07:07 PM
- - Stellingwerff   Hi all, In the meantime my listing has also evolv...   Aug 14 2012, 07:41 PM
|- - walfy   QUOTE (Stellingwerff @ Aug 14 2012, 11:41...   Aug 14 2012, 09:16 PM
- - elakdawalla   Oh, cool. Hopefully you two guys' solutions wi...   Aug 14 2012, 08:44 PM
- - jmknapp   Hey, good news and bad news, the good outweighing ...   Aug 15 2012, 02:18 AM
|- - ElkGroveDan   QUOTE (jmknapp @ Aug 14 2012, 07:18 PM) I...   Aug 15 2012, 02:46 AM
|- - Stellingwerff   QUOTE (jmknapp @ Aug 15 2012, 04:18 AM) H...   Aug 15 2012, 03:46 AM
|- - jmknapp   QUOTE (Stellingwerff @ Aug 14 2012, 11:46...   Aug 15 2012, 11:14 AM
|- - Stellingwerff   QUOTE (jmknapp @ Aug 15 2012, 01:14 PM) N...   Aug 15 2012, 12:03 PM
|- - jmknapp   QUOTE (Stellingwerff @ Aug 15 2012, 08:03...   Aug 15 2012, 12:25 PM
- - djellison   Mars EPO - headed by Michelle Viotti   Aug 15 2012, 03:02 AM
- - jcims   Does anyone in here know if any metadata is availa...   Aug 15 2012, 05:39 AM
- - elakdawalla   It's a big improvement! Now we just need t...   Aug 15 2012, 05:41 AM
- - Stellingwerff   Just a quick update notice: http://msl-raw-images...   Aug 15 2012, 07:24 AM
- - vk3ukf   Hi all, this doc might help help those looking to...   Aug 15 2012, 11:39 AM
- - Stellingwerff   Hi all, Another update, starting to get there:) A...   Aug 15 2012, 08:14 PM
|- - algorimancer   QUOTE (Stellingwerff @ Aug 15 2012, 03:14...   Aug 16 2012, 12:58 AM
- - elakdawalla   This is awesome. Since you asked, the "killer...   Aug 15 2012, 10:05 PM
- - RoverDriver   If I may suggest something, I think it would be cr...   Aug 15 2012, 11:32 PM
|- - jmknapp   QUOTE (RoverDriver @ Aug 15 2012, 06:32 P...   Aug 16 2012, 09:21 AM
|- - MahFL   My impression was they gave the task to "scie...   Aug 16 2012, 10:04 AM
||- - RoverDriver   QUOTE (MahFL @ Aug 16 2012, 03:04 AM) My ...   Aug 16 2012, 10:56 AM
|- - elakdawalla   QUOTE (jmknapp @ Aug 16 2012, 01:21 AM) S...   Aug 16 2012, 04:03 PM
||- - mhoward   QUOTE (elakdawalla @ Aug 16 2012, 10:03 A...   Aug 16 2012, 08:05 PM
||- - elakdawalla   QUOTE (mhoward @ Aug 16 2012, 01:05 PM) P...   Aug 16 2012, 08:34 PM
|- - imipak   QUOTE (jmknapp @ Aug 16 2012, 10:21 AM) S...   Aug 16 2012, 08:17 PM
- - Reckless   This retrieval system is great but for me the link...   Aug 15 2012, 11:53 PM
- - jmknapp   I added an RSS feed for the latest released images...   Aug 16 2012, 12:31 AM
- - Stellingwerff   Somebody will have to talk with the MSL website pe...   Aug 16 2012, 02:41 PM
- - maschnitz   Emily, if you're in contact with Michelle, cou...   Aug 16 2012, 07:47 PM
- - Stellingwerff   Hi all, In part due to the huge sol=0 page (curre...   Aug 16 2012, 08:41 PM
- - Stellingwerff   Ok, definitely not happy with the performance (and...   Aug 16 2012, 11:22 PM
- - 3d_mars   First post. Here's a small script I wrote to ...   Aug 17 2012, 03:59 AM
|- - jmknapp   QUOTE (3d_mars @ Aug 16 2012, 10:59 PM) F...   Aug 17 2012, 01:36 PM
|- - fredk   QUOTE (jmknapp @ Aug 17 2012, 02:36 PM) S...   Aug 17 2012, 02:07 PM
|- - 3d_mars   QUOTE (jmknapp @ Aug 17 2012, 05:36 AM) I...   Aug 17 2012, 02:40 PM
- - jmknapp   Some updates to my raw image listing app: 1) resu...   Aug 17 2012, 01:41 PM
- - fredk   QUOTE (jmknapp @ Aug 17 2012, 02:41 PM) N...   Aug 17 2012, 02:41 PM
4 Pages V   1 2 3 > » 


Reply to this topicStart new topic

 



RSS Lo-Fi Version Time is now: 30th May 2024 - 01:32 AM
RULES AND GUIDELINES
Please read the Forum Rules and Guidelines before posting.

IMAGE COPYRIGHT
Images posted on UnmannedSpaceflight.com may be copyrighted. Do not reproduce without permission. Read here for further information on space images and copyright.

OPINIONS AND MODERATION
Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators.
SUPPORT THE FORUM
Unmannedspaceflight.com is funded by the Planetary Society. Please consider supporting our work and many other projects by donating to the Society or becoming a member.