Curiosity Image Retrieval Tools, scripts and software |
Curiosity Image Retrieval Tools, scripts and software |
Aug 26 2012, 07:20 PM
Post
#136
|
|
Junior Member Group: Members Posts: 43 Joined: 7-August 12 From: The Netherlands Member No.: 6493 |
They are not very consistent with the casing of the suffix. I actually ended up checking both upper and lower case while doing the head retrieval. That way I doublecheck (and repair) any incorrect filename.
|
|
|
Aug 27 2012, 12:11 AM
Post
#137
|
|
Senior Member Group: Members Posts: 1465 Joined: 9-February 04 From: Columbus OH USA Member No.: 13 |
I hope it wouldn't be too much for you to change this to lowercase .jpg ... Try it now Pando--I think it should handle either the JPG or jpg extension now, as long as it's consistent within each sol's directory for a given camera. -------------------- |
|
|
Aug 27 2012, 03:32 AM
Post
#138
|
|
Newbie Group: Members Posts: 11 Joined: 12-August 12 Member No.: 6550 |
Here's version 2.0 of my Navcam download tool, which simplifies the process of downloading multiple left/right stereo image pairs and creating stereoscopic images. This version includes a GUI and supports anaglyph as well as SBS stereoscopic 3D images, a horizontal image offset, a function to find the latest sol number, and a few other changes and additions. A windows executable is attached and the source is below. Here are the links to StereoPhoto Maker and AutoIt. Thanks again jmknapp for http://curiositymsl.com.
After downloading the image pairs and creating the script file, run StereoPhoto Maker, select File, Multi conversion from list, select the script file, the output image type, and then click the Convert Files button. Screenshot CODE ; This script processes all full frame MSL Navcam images using jmknapp's http://curiositymsl.com, downloads only full frame LR pairs (if not downloaded already) and creates a StereoPhoto Maker script file ; version 2.0, 20120826 ; UMSF: 3d_mars #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_Outfile=navcam_download.exe #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include <ButtonConstants.au3> #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <ProgressConstants.au3> #include <StaticConstants.au3> #include <WindowsConstants.au3> #include <file.au3> #include <array.au3> #Region ### START Koda GUI section ### Form=navcam_sbs.kxf $Form_Navcam = GUICreate("MSL Navcam Image Download and Script Generator", 569, 400, 368, 376) $SetFont = GUISetFont(9) $Label_Title = GUICtrlCreateLabel("This app downloads MSL Navcam full frame stereo pairs and exports a StereoPhoto Maker script for generating stereoscopic images. NOTE: Only matching left/right image pairs will be downloaded.", 24, 8, 521, 55) $Label_Input_Sol_Number_Text= GUICtrlCreateLabel("Input Sol", 24, 72, 86, 17, $SS_CENTERIMAGE) $Input_Sol = GUICtrlCreateInput("1", 90, 71, 105, 21) $Button_Prev_Sol = GUICtrlCreateButton("Prev Sol", 235, 69, 65, 25) $Button_Next_Sol = GUICtrlCreateButton("Next Sol", 300, 69, 65, 25) $Button_GetLatestSol = GUICtrlCreateButton("Get Latest Sol", 385, 69, 105, 25) $Checkbox_Create_Scriptfile = GUICtrlCreateCheckbox("Create StereoPhoto Maker script file", 24, 100, 270, 17) GUICtrlSetState($Checkbox_Create_Scriptfile, $GUI_CHECKED) $Checkbox_download_images = GUICtrlCreateCheckbox("Download Images", 24, 123, 130, 17) GUICtrlSetState($Checkbox_download_images, $GUI_CHECKED) $Input_Image_Path = GUICtrlCreateInput("Path to save images", 24, 150, 420, 21) GUICtrlSetTip($Input_Image_Path, "Input path to save images and script file") $Button_Browse = GUICtrlCreateButton("Browse...", 450, 148, 81, 25) $Label_hOffset = GUICtrlCreateLabel("Horizontal image offset", 24, 185, 160, 17) $Input_hOffset = GUICtrlCreateInput("-160", 180, 185, 86, 20) $Button_Start = GUICtrlCreateButton("Start", 24, 224, 65, 25) $Progressbar = GUICtrlCreateProgress(24, 264, 249, 17, $WS_BORDER) ;GUICtrlCreateLabel("text", left, top, width, height) $Label_Total_Image_Text = GUICtrlCreateLabel("Total Images:", 29, 304, 93, 17) $Label_Total_Image_Count = GUICtrlCreateLabel("0", 139, 304, 50, 17) $Label_Total_New_Image_Text = GUICtrlCreateLabel("Total New Images:", 29, 328, 107, 17) $Label_Total_New_Image_Count = GUICtrlCreateLabel("0", 139, 328, 50, 17) $Label_Status_Text = GUICtrlCreateLabel("Status:", 29, 368, 37, 17) $Label_Status = GUICtrlCreateLabel(" ", 71, 368, 400, 17, $SS_SUNKEN) GUISetState(@SW_SHOW) #EndRegion ### END Koda GUI section ### ; read settings from ini Global $sIniFile = @ScriptDir & "\navcam_sbs.ini" ;IniRead(filename,section,key,default) GUICtrlSetData($Input_Sol, IniRead($sIniFile, "Sol", 0, "1")) GUICtrlSetData($Input_Image_Path, IniRead($sIniFile, "Image_Path", 1, "Input save path")) GUICtrlSetData($Input_hOffset, IniRead($sIniFile, "hOffset", 2, "-160")) While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE _Shutdown() Exit Case $Button_GetLatestSol GetLatestSol() Case $Button_Next_Sol Next_Sol() Case $Button_Prev_Sol Prev_Sol() Case $Button_Browse _Browse() Case $Button_Start _Start() EndSwitch WEnd Func GetLatestSol() $jplsite_file = @TempDir & "\jplsite.txt" ;dump site into file $return = InetGet("http://mars.jpl.nasa.gov/msl/multimedia/raw/?s=", $jplsite_file, 1, 0) ;read file into array Local $jplsite If Not _FileReadToArray($jplsite_file, $jplsite) Then MsgBox(4096, "Error", " Error importing from file to Array error:" & @error) Exit EndIf While 1 ;search array for latest sol number $index = _ArraySearch($jplsite, "Total for Sol",0,0,0,1) If $index = -1 Then MsgBox(0,"","no match for sol number on JPL site") ExitLoop EndIf $jplsite_foundline = $jplsite[$index] ExitLoop WEnd $solstrt = StringInStr($jplsite_foundline,"Total for Sol") $solend = StringInStr($jplsite_foundline,": ") $sol_latest = StringMid($jplsite_foundline,$solstrt+14,$solend-$solstrt-14) GUICtrlSetData($Input_Sol,$sol_latest) EndFunc Func Next_Sol() $sol_num=GUICtrlRead($Input_Sol,1) $sol_num += 1 GUICtrlSetData($Input_Sol,$sol_num) EndFunc Func Prev_Sol() $sol_num=GUICtrlRead($Input_Sol,1) $sol_num -= 1 if $sol_num < 1 then $sol_num = 1 GUICtrlSetData($Input_Sol,$sol_num) EndFunc Func _Browse() $sol=GUICtrlRead($Input_Sol,1) ;choose path and file name for images and generated StereoPhoto Maker script file Local $MyDocsFolder = "::{450D8FBA-AD25-11D0-98A8-0800361B1103}" Local $StereoPhoto_script_file = FileSaveDialog("Choose a path and filename for images and image list:", $MyDocsFolder, "Text Files (*.txt)", 2, "navcam_sol_" & $sol & ".txt") ; option 2 = dialog remains until valid path/file selected GUICtrlSetData($Input_Image_Path,$StereoPhoto_script_file) EndFunc Func _Start() ; start a loop so the function can be aborted at any point Do GUICtrlSetData($Label_Total_Image_Count,0) GUICtrlSetData($Label_Total_New_Image_Count,0) GUICtrlSetData($progressbar, 0) $sol=GUICtrlRead($Input_Sol,1) $hOffset=GUICtrlRead($Input_hOffset,1) ; create temp filename for listing all URLs for a given sol ; example: c:\users\username\appdata\local\temp\URLs_sol_1.txt $URL_list_file=@TempDir & "\URLs_sol_" & $sol & ".txt" ; download image URL list to text file ; InetGet option syntax: URL, filename, 1=force reload, 0=wait for download GUICtrlSetData($Label_Status,"Download URLs") InetGet("http://curiositymsl.com/cgi-bin/getfiles.cgi?sol=" & $sol, $URL_list_file,1,0) ; if no URLs found then abort $URL_list_size=FileGetSize($URL_list_file) If $URL_list_size = 0 Then GUICtrlSetData($Label_Status,"No URLs found") ExitLoop EndIf ; read URL list and import to array ; note: $allURLs[0] contains the number of records read into the array. GUICtrlSetData($Label_Status,"Read URLs into array") Local $allURLs If Not _FileReadToArray($URL_list_file, $allURLs) Then MsgBox(4096, "Error", " Error importing from file to Array error:" & @error) ;Exit EndIf ; search for Navcam full frame (*EDR_F*) (exclude thumbnails) into new array $allframes which contains only image URLs we care about ; Navcam full frame example URLs: ; http://mars.jpl.nasa.gov/msl-raw-images/pr...AUT_04096M_.JPG ; http://mars.jpl.nasa.gov/msl-raw-images/pr...AUT_04096M_.JPG ; declare large array instead of resizing array multiple times, uses very little RAM Local $allframes[9999]=[0] GUICtrlSetData($Label_Status,"Search for Navcam full frame images") $x = 1 For $URL in $allURLs ;check for Navcam full frame If StringInStr($URL,"EDR_F") Then $allframes[$x]=$URL $x += 1 EndIf Next ;record total number of frames $allframes[0]=$x - 1 ; read allframes to find matching left/right pairs ; insert into new arrays $leftframe_Nav, $rightframe_Nav Local $leftframe_Nav[9999]=[0] Local $rightframe_Nav[9999]=[0] Local $leftframe_counter = 0 Local $rightframe_counter = 0 GUICtrlSetData($Label_Status,"Search for Left and Right frames") For $x = 1 to $allframes[0] If StringInStr($allframes[$x],"NLA_") Then ;left frame found, store in $leftframe_Nav array $leftframe_counter += 1 $leftframe_Nav[$leftframe_counter]=$allframes[$x] $leftframe_Nav[0]=$leftframe_counter ElseIf StringInStr($allframes[$x],"NRA_") Then ;right frame found, store in $rightframe_Nav array $rightframe_counter += 1 $rightframe_Nav[$rightframe_counter]=$allframes[$x] $rightframe_Nav[0]=$rightframe_counter EndIf Next ; if no left full frames then abort and exit If $leftframe_counter == 0 Then ;MsgBox(0, "", "No frames found") GUICtrlSetData($Label_Status,"No full frames found") EndIf ; using lists of left and right frames, find matches and save output Local $right_matching_frame_number[9999]=[0] Local $StereoPhoto_script_lines[9999]=[0] Local $URL_left_temp, $left_image_file_name, $URL_right_temp, $right_image_file_name, $left_image_full_path, $right_image_full_path GUICtrlSetData($Label_Status,"Find Left/Right frame matches") For $x = 1 to $leftframe_Nav[0] ;extract frame number $leftframe_image_number=StringMid($leftframe_Nav[$x],StringInStr($leftframe_Nav[$x],"NLA_")+4,9) ; returns -1 if no match found $right_matching_frame_number[$x] = _ArraySearch($rightframe_Nav, $leftframe_image_number, 0, 0, 0, 1) ; update progress bar Local $Percent = ($x / $leftframe_Nav[0] ) * 100 GUICtrlSetData($progressbar, $Percent) ; download left and right images If $right_matching_frame_number[$x] <> -1 Then ; increment total image counter (+2 because left and right frames found) GUICtrlSetData($Label_Total_Image_Count,GUICtrlRead($Label_Total_Image_Count,1) + 2) ; find the file names $URL_left_temp = StringSplit($leftframe_Nav[$x], "/") $left_image_file_name = $URL_left_temp[UBound($URL_left_temp)-1] $URL_right_temp = StringSplit($rightframe_Nav[$right_matching_frame_number[$x]], "/") $right_image_file_name = $URL_right_temp[UBound($URL_right_temp)-1] ; create full path names $left_image_full_path=@WorkingDir & "\" & $left_image_file_name $right_image_full_path=@WorkingDir & "\" & $right_image_file_name ; build array for StereoPhoto Maker script file $StereoPhoto_script_lines[$x]="21,0,0," & $hOffset & ",0," & $left_image_full_path & "," & $right_image_full_path & ",," ; download the images ; if the file already exists don't download it again If Not FileExists($left_image_full_path) Then ; increment total new image counter GUICtrlSetData($Label_Total_New_Image_Count,GUICtrlRead($Label_Total_New_Image_Count,1) + 1) If GUICtrlRead($Checkbox_download_images) = $GUI_CHECKED Then InetGet($leftframe_Nav[$x],$left_image_full_path,1,0) EndIf If Not FileExists($right_image_full_path) Then ; increment total new image counter GUICtrlSetData($Label_Total_New_Image_Count,GUICtrlRead($Label_Total_New_Image_Count,1) + 1) If GUICtrlRead($Checkbox_download_images) = $GUI_CHECKED Then InetGet($rightframe_Nav[$right_matching_frame_number[$x]],$right_image_full_path,1,0) EndIf EndIf Next GUICtrlSetData($Label_Status,"Processing complete") If GUICtrlRead($Checkbox_Create_Scriptfile) = $GUI_CHECKED Then GUICtrlSetData($Label_Status,"Save script file") $StereoPhoto_script_file=GUICtrlRead($Input_Image_Path,1) ; write StereoPhoto Maker script file Local $hFile = FileOpen($StereoPhoto_script_file, 2) ; 2 = erase previous _FileWriteFromArray($StereoPhoto_script_file, $StereoPhoto_script_lines, 1) ; 1 = index to start FileClose($hFile) GUICtrlSetData($Label_Status,"Script file saved") EndIf ExitLoop ; end function loop Until 1 ; open StereoPhoto Maker, File menu, multi conversion from list... EndFunc Func _Shutdown() ; save settings to ini IniWrite($sIniFile, "Sol", 0, GUICtrlRead($Input_Sol)) IniWrite($sIniFile, "Image_Path", 1, GUICtrlRead($Input_Image_Path)) IniWrite($sIniFile, "hOffset", 2, GUICtrlRead($Input_hOffset)) EndFunc
Attached File(s)
|
|
|
Aug 27 2012, 03:54 AM
Post
#139
|
|
The Insider Group: Members Posts: 669 Joined: 3-May 04 Member No.: 73 |
Try it now Pando--I think it should handle either the JPG or jpg extension now, as long as it's consistent within each sol's directory for a given camera. My gosh, thanks Joe! This works really well. I didn't realize they are mixing the uppercase/lowercase file extensions. |
|
|
Aug 28 2012, 09:48 PM
Post
#140
|
|
Junior Member Group: Members Posts: 43 Joined: 7-August 12 From: The Netherlands Member No.: 6493 |
Hi all,
I've done another update to http://msl-raw-images.appspot.com/lists.html
I'm also working on an email notification, besides the twitter one. I didn't provide the needed management functions yet, to subscribe, unsubscribe, etc. But if any of you guys or girls would like to get an email notification/report on new images, please drop me a message. I'll arrange the setup then. Greetings, Ludo. |
|
|
Sep 4 2012, 08:25 PM
Post
#141
|
|
Junior Member Group: Members Posts: 43 Joined: 7-August 12 From: The Netherlands Member No.: 6493 |
Hi all,
Remind me to never ever say anything about backwards compatibility again.... Big sorry for anybody who has made lists on my site in the last few weeks, I've repeatedly broken them:( I will really try to not do that again... To "force" me to not do so, let me share a list with you all as an example: http://msl-raw-images.appspot.com/lists.ht...5B-1C00EE731309 It subscribes you to a list I've made of most full images that show Curiosity herself. The list could for example be used to make a nice self portrait. I plan on keeping the list updated, so expect it to grow in the coming period when new images become available. Ok, the customary feature change report:
Still on my todo:
Greetings, Ludo. |
|
|
Sep 12 2012, 11:35 AM
Post
#142
|
|
Junior Member Group: Members Posts: 43 Joined: 7-August 12 From: The Netherlands Member No.: 6493 |
MAHLI is sending a set of new images with a slightly different filenames: I now see filetypes R,U,S & T, besides the regular E & I. Anyone knows what each letter stands for? I'll do an analysis myself later this afternoon if nobody has a list available somewhere. Quick Google didn't produce a result (yet).
Greetings, Ludo. |
|
|
Sep 12 2012, 02:24 PM
Post
#143
|
|
Senior Member Group: Members Posts: 2511 Joined: 13-September 05 Member No.: 497 |
MAHLI is sending a set of new images with a slightly different filenames... I believe these are associated with focus stacking products. See 7.6 Focus Stack Acquisition and Merge in the MAHLI paper. Not sure which is which but it should be fairly obvious. -------------------- Disclaimer: This post is based on public information only. Any opinions are my own.
|
|
|
Sep 20 2012, 05:23 PM
Post
#144
|
|
Senior Member Group: Members Posts: 4246 Joined: 17-January 05 Member No.: 152 |
Joe - I'm seeing all the latest images on your site listed as "sol 0" - is there a chance something's broken?
|
|
|
Sep 20 2012, 06:54 PM
Post
#145
|
|
Member Group: Members Posts: 178 Joined: 13-September 05 Member No.: 498 |
|
|
|
Sep 20 2012, 07:00 PM
Post
#146
|
|
Senior Member Group: Members Posts: 1465 Joined: 9-February 04 From: Columbus OH USA Member No.: 13 |
Thanks for the heads up--I'll check it out. I'm guessing that either the NASA website html changed or these recent photos don't have a timestamp for some reason.
Jan-1-2000 is when NASA ephemeris time currently starts (et=0) so indeed it's a null capture date. -------------------- |
|
|
Sep 20 2012, 07:07 PM
Post
#147
|
|
Senior Member Group: Members Posts: 2173 Joined: 28-December 04 From: Florida, USA Member No.: 132 |
Y2K strikes at last!
|
|
|
Sep 20 2012, 07:44 PM
Post
#148
|
|
Senior Member Group: Members Posts: 1465 Joined: 9-February 04 From: Columbus OH USA Member No.: 13 |
Y2K strikes at last! Ha!--in this case it was option [c] operator error. I've been working on an alternative interface to the database (more like a google search) and in the machinations managed to move a script file from where it was needed. Anyway, should be OK now. -------------------- |
|
|
Sep 21 2012, 05:04 AM
Post
#149
|
|
Senior Member Group: Members Posts: 2082 Joined: 13-February 10 From: Ontario Member No.: 5221 |
Will this include streamlining the page select at the bottom, Joe? The list of page numbers covers nearly the whole browser screen at the bottom, and that's in just over a month of imagery! I know I can set the items per page to 100, but that will just delay the inevitable as it grows and grows. For a mission that is going to last years, there must be a more permanent solution.
|
|
|
Sep 21 2012, 12:25 PM
Post
#150
|
|
Senior Member Group: Members Posts: 1465 Joined: 9-February 04 From: Columbus OH USA Member No.: 13 |
Here's another go at an alternative interface to the raw images:
http://curiositymsl.com/search Try searches like: mastcam -- all mastcam images mastcam sol39 -- all mastcam images on sol 39 mr large -- all large mastcam right images navcam -thumbnail -small -- all navcam images excluding thumbnails and smalls mardi -sol0 -- all mardi images excluding sol 0 sunrise -- all images taken at sunrise Mousing over the thumbnails brings up a preview. Scrolling to the bottom of the page loads more items if available. Click on 'help' for more info. -------------------- |
|
|
Lo-Fi Version | Time is now: 24th April 2024 - 04:09 AM |
RULES AND GUIDELINES Please read the Forum Rules and Guidelines before posting. IMAGE COPYRIGHT |
OPINIONS AND MODERATION Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators. |
SUPPORT THE FORUM Unmannedspaceflight.com is funded by the Planetary Society. Please consider supporting our work and many other projects by donating to the Society or becoming a member. |