What to do after the closure of Midnight Mars Browser? |
What to do after the closure of Midnight Mars Browser? |
Oct 15 2020, 03:54 PM
Post
#46
|
|
Member Group: Members Posts: 893 Joined: 15-June 09 From: Lisbon, Portugal Member No.: 4824 |
Those thumbnails are not part of the JSON feeds Another challenge! By the way, I couldn't reproduce the bug you've mentioned. It worked for all the days you pointed out and a few extra on either end of the interval. Perhaps it is related to the slowness of the loading? I'll make it more obvious in the tool when it is loading. ThanksFernando |
|
|
Oct 28 2020, 11:47 PM
Post
#47
|
|
Senior Member Group: Members Posts: 4246 Joined: 17-January 05 Member No.: 152 |
I've been parsing the MSL json files into html and thought I'd make the pages public in case anyone finds them useful. My pages are static html so there's no selecting cameras, changing sort orders, autoload scrolling, etc. It's inspired more by the linear style of the old curiosityrover.com than MMB, Fernando's page, or www.mars-browser.co.uk/curiosity.
The info and pics are straight from the json files and the mars.nasa.gov/msl-raw-images server with two exceptions. First I calculate the pointing according to mcaplinger's suggestion in this post. I haven't tested this much so let me know if you notice any problems. The second is that I deBayer the Bayered mastcam frames automatically as they arrive. Due to the chroma subsampling I do you shouldn't take the colours too literally, and there are lots of crosshatch artifacts, so these images are only meant to give a rough sense of the scene. On my site there's the "latest images" page located here: http://lcdm.ca/msl/. This checks for updates every half hour and adds any new images to the top, chronologically by time taken. After updates I use this by scrolling and paging down until I see image links change to the visited colour, which makes it very easy to keep up and not miss any new images, even if they're from old sols. There are also individual sol pages, eg: http://lcdm.ca/msl/2924/ (going back to sol 2714). These are always ordered by time taken. More details on this page. If anyone has any suggestions hopefully I'll have time to implement them (within the limitations of static html), so let me know. |
|
|
Oct 29 2020, 02:37 AM
Post
#48
|
|
Senior Member Group: Members Posts: 2428 Joined: 30-January 13 From: Penang, Malaysia. Member No.: 6853 |
Viewed on my Mobile phone and impressed Looking forward to seeing it in action on my home PC when my covid travel restriction order is supposed to be lifted in 11 days 😊
|
|
|
Oct 29 2020, 02:33 PM
Post
#49
|
|
Senior Member Group: Members Posts: 4246 Joined: 17-January 05 Member No.: 152 |
Thanks, Paul. Sorry my site's not small-screen friendly. Maybe now that we're moving again on Mars that'll relieve your earthbound status a bit?
|
|
|
Oct 30 2020, 12:42 AM
Post
#50
|
|
Member Group: Members Posts: 893 Joined: 15-June 09 From: Lisbon, Portugal Member No.: 4824 |
|
|
|
Oct 30 2020, 09:49 AM
Post
#51
|
|
Senior Member Group: Members Posts: 1619 Joined: 12-February 06 From: Bergerac - FR Member No.: 678 |
This.
Is. AWESOME ! It's fast, it's clear, filenames are immediately readable, and it's easy to navigate. And you even debayer the pictures o_o -------------------- |
|
|
Oct 30 2020, 07:01 PM
Post
#52
|
|
Solar System Cartographer Group: Members Posts: 10157 Joined: 5-April 05 From: Canada Member No.: 227 |
Yes, I really like this.
Phil -------------------- ... because the Solar System ain't gonna map itself.
Also to be found posting similar content on https://mastodon.social/@PhilStooke Maps for download (free PD: https://upload.wikimedia.org/wikipedia/comm...Cartography.pdf NOTE: everything created by me which I post on UMSF is considered to be in the public domain (NOT CC, public domain) |
|
|
Oct 30 2020, 11:35 PM
Post
#53
|
|
Senior Member Group: Members Posts: 4246 Joined: 17-January 05 Member No.: 152 |
Thanks, guys. I hope to improve the deBayering still - treat the edges better and maybe remove the glitches that appear in each frame.
|
|
|
Oct 31 2020, 12:54 AM
Post
#54
|
|
Member Group: Members Posts: 893 Joined: 15-June 09 From: Lisbon, Portugal Member No.: 4824 |
Here is an improved (I hope!) version of my MSL Image Viewer. Still work in progress so, for the time being, I'll continue to distribute it here. Having been on vacation last week I only had time to implement part of your suggestions - thank you all for those.
To install, download the zip file to your favourite location and unzip it. You should see the following 5 files:
Opening the MSLImageDisplay.html page automatically loads the images (in small format) of the most recent sol. Below the header, on the left, there is a bar with several sections: a sol navigator, sol statistics, image filters, and option selectors. To its right lies the images area. The sol navigator has:
Next comes the image filters panel. Those are grouped into two categories: cameras (mastcam, chemcam, hazard, etc.) and image types (full, subframe, etc.). Only the entries that exist on the sol are shown. It is now possible to select one or more entries in each category. Entries in the same category are ORed and the two categories and ANDed. For instance, select all Front Haz Right and Left cams but only for full frame images. "Show all" and "Hide all" buttons per category provide shortcuts. These filters are "sticky". When you move from sol to sol only the categories that were previously selected will be shown - until you change them. But note that all the images for the sol will still be loaded. Having selected (or deselected) a category that does exist on the new sol does not cause an error. Entries are selected by toggle buttons. A green or red rim indicates whether the filter is active or not. The option selectors panel currently contains just one button. The "List URLs (of visible images)" button opens a new page with the list of full URLs of the images that are currently visible. The page has a "Copy to the clipboard" button. The page's right side contains the sol's images. Place the mouse over an image to reveal the image's full URL. Clicking on an image opens it in a new page and shows the associated information. Images are shown at their natural size, if they fit the screen, otherwise they will be reduced to fit. From this page you can load the previous or the next image. Any number of such pages can be simultaneously open, allowing, for instance, to compare the images. In addition to the suggestions I did not have the time to implement I have other ideas. For instance, highlighting the recent images. But what is recent? 24h? 48h? 1 week? Please tell me your opinion. It is very easy to highlight recently added images for a specific sol. But finding all recently added images implies going through the json files for all sols. This takes time and uses network and should perhaps be implemented as an on-demand process (this is the drawback of a client-side, versus server-side, implementation). Your feedback is requested and most welcome. Sorry for the long post. Fernando EDIT: the attachment has been removed. A new version, supporting more functions, is available. See post #56 |
|
|
Nov 10 2020, 08:13 PM
Post
#55
|
|
Senior Member Group: Members Posts: 4246 Joined: 17-January 05 Member No.: 152 |
I've improved the deBayering on my site http://lcdm.ca/msl/. I now handle the edges of the frame better, so colours are good right to the edges. Also I use a simple method to remove the "hot column" in the ML images. This is a bright column (column 1070, a green-blue column, in the full-resolution frame) that runs down most of the ML images. I simply replace that column of pixels with the average of the neighbouring green-blue columns before deBayering. Due to the jpegging in the public Bayered images, the removal isn't perfect and you may notice some residual "scarring" within an 8-pixel jpeg block of the hot column. Both these changes apply to new images, not old ones still on my server.
I've also added lists of deBayered images at the top of each individual sol page. These can be used to download all at once, with a browser plugin or, for example, with CODE wget -Fi http://lcdm.ca/msl/2935/ML.html if you can use wget (or some similar command with curl). Again, these lists only include new deBayered images. |
|
|
Nov 11 2020, 05:37 PM
Post
#56
|
|
Member Group: Members Posts: 893 Joined: 15-June 09 From: Lisbon, Portugal Member No.: 4824 |
I am releasing a few additional functions for the MSL Image Browser. In a separate post I'll discuss some issues which have influenced some decisions and prevented implementing some of the suggestions - I'm very grateful for those.
Recent images are now highlighted by a red border. Right now - and this is very easy to change - "recent" means the image has been added in the last 72 hours. I had to choose a criteria. It could have been "added since my last visit" but this implies storing the date and time of "my last visit" in a special area of the browser (using a standard - and safe - HTML 5 API). Please let me know if you're interested in having this option. A row of buttons for each sol with "recent" images (according to the above criteria) appears just below the page banner. Click them to go the the respective sol. On the main page three new buttons (on the left panel) allow invoking three different views of a sol's image set:
To speed-up the download it is now done in groups of up to 250 images each. This number was chosen because, right now, 54.3% of all sols have 250 images or less, 86.9% have 500 images or less, 95.6% have 750 or less, and 96.2% have 1000 images or less. Thus, the vast majority of sols requires very few clicks. The use of -thm thumbnails is discussed in another post. While there are more images to download a "Show more" button appears at the bottom of the images set. I also added a "Go to Top" button at the bottom of the image set, for ease of use. One consequence is that if one is looking for all the images of a certain category then all the sol's images must be downloaded. The list and grid views are independently loaded and scrolled; it is possible to have downloaded all images in one view but not on the other but, since the images are downloaded, showing them in a view is very fast. There is a new counter in the sol's statistics showing how many images have been downloaded (which changes with the view, naturally). No changes were done to the URL list page. The image view page now has a "magnifier" capability. This is intended to help better see details of the bigger images. Click the button (it is a toggle) and a small lens is positioned on the upper left corner of the image. The enlarged contents of the lens are shown on an overlay at the page top left (below the banner). The enlarged area is a 6x6 of the lens. Please let me know if you are interested in other settings or being able to set it as required. Note: this does not increase the image's resolution, just enlarges the image, possibly blurring it. As such, the magnifier is not made available for small images. You can download the image, in the original size, by right-clicking on it (the same is valid for the main page views). Thank you for using the tool. Please let me know your thoughts, any requests, or bugs you may find. Fernando MSL_Image_Viewer_2020NOV11.zip ( 17.11K ) Number of downloads: 139 |
|
|
Nov 11 2020, 05:49 PM
Post
#57
|
|
Member Group: Members Posts: 893 Joined: 15-June 09 From: Lisbon, Portugal Member No.: 4824 |
What some people are beginning to figure out is how useful the Sequence name is.... ( in a file like NRB_655824982EDR_S0822188NCAM00595M_.JPG the SeqID is the NCAM00595 part ) - being able to filter by SeqID for a given Sol would also be really good Doug, I want to implement this next. But I found the json manifest also points to files with a different file name structure, for instance ... /msss/02938/mhli/2938MH0007060001004000I01_DXXX.jpg I found a document describing the structure of the file name in your example. Where can I find a description of these file name's structure (some parts are obvious)? Do image sequences also appear on these files? Any information is much appreciated. Fernando |
|
|
Nov 11 2020, 06:02 PM
Post
#58
|
|
Member Group: Members Posts: 893 Joined: 15-June 09 From: Lisbon, Portugal Member No.: 4824 |
On the other hand, I believe all images hosted at mars.nasa.gov/msl-raw-images are paired with thumbs generated on the ground - for that navcam the thumb is https://mars.nasa.gov/msl-raw-images/proj/m...00595M_-thm.jpg. It's these latter thumbs that should be used to speed loading of sites like this. I found that not all images in the json manifests have a -thm counter part (sol 0 gives thousands of errors, for instance). Because there is no way to find whether a file exists on the server without trying to download it, one is then left with the need to recover from the 403 error (when the -thm file does not exist) by loading the full size file... which may also not exist, which needs another recovery routine. This is more overhead. This is, in essence, why I opted for downloading images in relatively small groups. I also found the json manifests do not list the exact same files the official site shows. For instance, for sol 0 (my nemesis!) json lists 3270 images - of which fewer than 1000 exist on the given url - but the official site mentions 2198. Any information on this subject is very much appreciated. Fernando |
|
|
Nov 11 2020, 06:37 PM
Post
#59
|
|
Senior Member Group: Members Posts: 4246 Joined: 17-January 05 Member No.: 152 |
Where can I find a description of these file name's structure (some parts are obvious)? Do image sequences also appear on these files? I think MAHLI follows the file naming convention for mastcam, which is described in this paper. |
|
|
Nov 11 2020, 06:53 PM
Post
#60
|
|
Senior Member Group: Members Posts: 4246 Joined: 17-January 05 Member No.: 152 |
I found that not all images in the json manifests have a -thm counter part (sol 0 gives thousands of errors, for instance). I see what you mean. I only go back a couple of hundred sols and the -thm images always exist in these recent sols. But I can imagine that things are more sporadic in the older sols. About image counts, I find agreement between the public jpeg site and json, at least back a couple hundred sols. For sol 0, json v1 gives 3720 images, and the public jpeg site lists 2198 full-res images + 1520 thumbs for a total of 3718. That's very close, but the jpeg site seems to be missing two images! |
|
|
Lo-Fi Version | Time is now: 4th May 2024 - 09:38 AM |
RULES AND GUIDELINES Please read the Forum Rules and Guidelines before posting. IMAGE COPYRIGHT |
OPINIONS AND MODERATION Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators. |
SUPPORT THE FORUM Unmannedspaceflight.com is funded by the Planetary Society. Please consider supporting our work and many other projects by donating to the Society or becoming a member. |