I've been a heavy user of Midnight Mars Browser for several years and I'm incredibly grateful for the work that Michael has done.
Now I ask myself, should we create again an alternative way to look and search for images coming from Mars? In the last several months I developed a way to ingest InSight images in order to have a small database and a telegram bot that tells me when a new image is uploaded to the InSight web site and I can try to do the same with the MSL imagery.
But, I have several questions: What is the better way to show the data in the web page? Which is the best way to order the images?
I'd love to hear some suggestions for that.
Greetings.
I would suggest something that resembles the structure of MMB. I have found it very useful for years. Having access via a list of sols, and access to the previous and next sols, is great. And ordering images by time within a sol is very useful.
Phil
In addition to ordering by sols, ideally a page would also somehow allow you to know which images you haven't looked at yet, ie which are newly downlinked images. This can be as simple as including ordinary text links to the full size images (presumably on the jpl server) which the viewer's web browser will change the colour of once visited (as Joe's site does). Ie, you can let the browser's own history handle it.
The option to order by downlink time is very nice when very old images are finally downlinked, otherwise they may be many sols back and you might miss them.
Otherwise displaying the LMST and UTC of image capture, and UTC of downlink are important. Also displaying the full-res image size in pixels is nice, so you can tell if the image is just a subsampled thumb or not.
(And it goes without saying that the page should only load thumbs, not full-res that are rescaled by the browser, until you open the full-sized image! )
Thanks a lot for considering this, and best of luck!
Would it be very complicated to implement a function that would debayer the Mastcam images when we download them?
That would be a great feature to have. It would be easy to do a quck-and-dirty job (showing some green artifacts) if you could run something like gmic (runs on linux). But you would need server space to host the deBayered images.
Thanks for all the suggestions!,
I'm thinking on creating two versions: a simple similar to Midnight Browser that was perfect for looking at the last downlinked imagery and a more complex one that allows searching (by Sol, by time of the day...).
I'm already doing some automatic debayering for the downlinked images. The program checks if the image is bayered and runs a debayering algorithm (there is still room for improvement in that point
To reduce the need for server space, I was thinking that the debayering could be done on-the-fly during the downloading process, so that only the undebayered images would have to be archived.
It would still be nice to have at least deBayered thumbs stored on the server to ease browsing the images.
Thanks again for this, paraisosdelsistemasolar, and I'm sure you'd have a bunch of willing beta testers here if needed...
At this point I don't have any limitations regarding file space, so debayered files won't be any problem.
If more space is needed than my host allows, I can host it by myself at home. As soon as I have a beta version I'll show you all for feedback.
Greetings!
I was hoping Midnight Planets / MMB could be branched out to live into the future, so much work was perfected by Mike it would be great to continue that.
Per a recent http://www.unmannedspaceflight.com/index.php?s=&showtopic=682&view=findpost&p=246051, he sounds too busy to continue with it, but didn't mention what he plans to do with his code base, so I was hoping for an opensource dump, but hard to say if that will happen.. In any event, however the plans manifest going forward, i'd be happy help out anyway i can.
Sorry for the delay,
We have had some rough weeks at home, but I'm now working again on the code and I hope to have a first version in a few days (I hope so).
The code and the database will be open source as soon as I release a first version. The scraping is being done in Python, the web will be in PHP and the database will be MySQL.
Looking forward to this! Thanks for putting time into it 'Para-doodle'
Hi to everybody,
Sorry again for the delay, but I'm a simple geologist and not a good programmer. I'm about to finish the robot that reads the MSL image archive. It's not only for showing the latest images, I plan, also, to be useful for searching images. That said, some of the avaiable fields will be:
- MTC, LTST and LMST
- Leds on/off
- Sun height.
- open/closed LID in MAHLI images
- Focus merge products
- Distance/resolution to object (MAHLI)
I hope to learn about spice kernels for driving/position purposes, but that will be in a later phase.
Hey all,
I wrote a basic Navcam viewer that reads the official json files and shows Curiosity’s panorama’s in a web browser.
It currently comes without interface and is pretty featureless in comparison to the great MidnightPlanets but it performs really well.
Check it here: https://captainvideo.nl/rob/mslview.html
To view panorama’s from a specific sol just add ?sol=2633 (or any other sol number) to the web adress.
For example: https://captainvideo.nl/rob/mslview.html?sol=2633
Hope you enjoy!
Rob
Wow Rob, I really liked your panorama viewer!
I'm still finishing the image viewer, but I'm pretty confident at least all the big data ingestion will be done and finished by the end of the week.
Yes, many thanks for this, its very nice! and fast!! just a few hundred lines of code and already its the best available way to get a quick pano view of the latest MSL adventures (not to downplay, i recognize this certainly represents a heck of lot of work on your part, just saying all these years later NASA has yet to match what you've accomplished!)
It looks like it loads all image data straight from mars.jpl.nasa.gov, rather than self-hosting like MP did. I can just plug in sols at random and many happen to have a large sweep of navcams so provide a pretty good view, so its already quite useful if you know what you want to see. For now, sol2595 looks to be hardcoded in the javascript, and i'd suspect a sol browser having navcam counts per sol is likely a big effort in itself. makes me wish i had chosen a career in software dev so i could get in the game not just cheer from sidelines, great work!
Rob!
This.
Is.
Awesome!!!
Thanks for sharing this. If you have plans to develop further I would love to make some suggestions but it is already an instant bookmark for me.
So I went sol-searching for the intent of coding self-improvement...
And it seems as if the NASA index Rob is using is all jacked up: https://mars.jpl.nasa.gov/msl-raw-images/image/image_manifest.json
The data seems fine until you get to sol "191" which for some reason has "sol192" inside, then it just snowballs from there, by the time its gets to sol '2382' it has 'sol2639' inside! or maybe i just dont understand what NASA is doing..? anyways, i didn't look into it any further yet and so remain thoroughly flummoxed and perplexed! ..of course i had to check again, it goes nuts at 'sol' 200 implying no MSL downlink for 15 days -and yes thats what i see in http://www.midnightplanets.com/web/MSL/sol/00200.html "Memory anomaly", so Rob probably didnt use those numbers for anything and just looks at the catalog_url, so NASA seems to have incorrectly labeled their index 'sols' but its really just a plain old enumerated sequence unrelated to actual sols.. its nice to know there is this https://mars.jpl.nasa.gov/msl-raw-images/image/images_sol2639.json indexed for each recorded sol
Paraisos, if I may formulate another request for your MSL image browser: it would be very useful if we could select a range of images and download them all simultaneously, instead of having to do it one by one.
Hey everyone, you are welcome to visit the latest version of the Mars rover navcam viewer.
It is available at https://marslife.org and should work on every device that supports WebGL.
New features are:
- display of the rover traverse path.
- clickable sol/site positions for easy navigation.
- menu option to select and view image products from pds (when available).
- site-updates occur automatically when new mission data becomes available.
- initial support for viewing the MER missions.
Usage:
- Tab key (or upper right circle) for menu/sol list.
- Left/Right arrow keys (or the lower right buttons) to view the previous or next site.
- L key to show/hide site locations.
- P key to toggle path display.
- S key to toggle the simulated sky on/off.
- C key to toggle image brightness corrections.
The interface needs work and the path accuracy is flawed (especially with MER) but the
clickable sol/site buttons make it real easy to get acquainted to the Martian territory.
Feel free to post comments and suggestions. Have fun wandering on Mars!
Rob
Rob,
This is excellent! I haven't yet tried the MER paths but why do you say MSL's path is flawed?
Congratulations.
Fernando
Very nice work! your project is really coming along great, Congrats! i like the method of subtlety toggling the ui elements and options and how well the correction algorithm is working.
Since i'm sure you are not already nearly busy enough with all this haha, I will pile on extra features, the first being a separately toggleable pan/zoom overview map overlay with path and current location dot, and yes of course it has to be clickable so you can go to the sol nearest to the click location.
Fantastic work Rob, this is a super useful resource.
Thank you all!
@atomoid, There’s no algorithm... the brightness adjustments are done manually.
oh boy somebody's got their work cut out for them!! now that explains why if i go back to old sols that "correction algorithm" doesn't seem to function!
Hey all, two extra features are added to Marslife.org!
- The update script will now automatically generate an anaglyph set of images for each new panorama.
Set the 'image product' menu option to "STEREO" to view the anaglyph version of the panorama.
- The script will also post a tweet at the https://twitter.com/MarslifeOrg when a new panorama is added.
You can follow this twitter account to keep track of Curiosity’s latest movements on Mars.
It’s all highly experimental and bound to break at some point but I hope you will enjoy anyway!
Rob
Apologies, I'm fairly new to Twitter. This is the link https://twitter.com/MarslifeOrg.
Thank you!
This is a great way to view pictures from MSL & MER. I'm all for modern HTML5 tools.
But since the official Raw images webpage is crap, and since Midnight Planets don't work anymore, is it possible to have the pictures displayed also as a grid, just a simple HTML webpage, with a simple sol by sol classification ? Just like Midnight Planets did ? A simple access to the jpegs, without that "responsive design" thing, and without that JS messing around. A good old plain HTML page.
Agreed. Something with the look and feel of Midnight Planets, the same functionality, and capable of being extended to Perseverance...! However, I am getting used to working with the Raw image pages for InSight and Curiosity and they are actually quite good, better than I thought at first. You can select any sol and any instrument if you need to.
Phil
Like Phil I'm slowly getting used to the new page, but I'm far from happy with it. I just wish they left as it was until the end of mission, or at least had a revert option on it so you could use the old or new interfaces. Even the old interface wasn't perfect, but back then we had MP and Joe's page to use if we wanted the various whistles and bells they provided. Sadly no more, so we're stuck with the new interface which doesn't quite match the InSight raw image inferface for whatever reason. So let's see what we get for 2020's 23 cameras next year 😁
Hi everyone!
Here is a prototype for a Curiosity raw images viewer. It is not intended for smart phones.
Not so quick but still somewhat dirty: the code does not have much robustness. It consists of a single HTML file and does use CSS and JavaScript (how else?)
I'm releasing it in order to get some feedback: Am I going in the right direction? Does it have your favourite features? What is wrong and what else would you like to have it do? Just post here or PM me.
The crucial thing about it is the JSON file(s). If NASA changes those then everything will need reviewing.
About installing it:
Because I do not have a web site in which to host the file I am attaching it here. Simply download it to your favourite location. Click (or double click) on it to open it on your default browser.
However, this creates a problem as the page is local for the browsers and makes cross-domain requests (the famous https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS problem). The solution is to install a browser add-on. I am using Firefox so I installed https://addons.mozilla.org/en-US/firefox/addon/cors-everywhere/. There is one for Chrome as well https://chrome.google.com/webstore/detail/allow-cors-access-control/lhobafahddgcelffkeicbaginigeejlf. This is quite an easy process.
About using it:
Open the file. It will automatically be positioned on the most recent sol and display all its images, in small format. The images are displayed by sol. Any sol can be selected by typing its number on the appropriate entry field (top right) or using the scroller. Press the "GO !" button to load the images.The images can be filtered by instrument. A row of buttons, one for each instrument, is dynamically built when the images are loaded. Press a button to see the corresponding images. Clicking on an image opens it in large format on an independent (resizeable) window which also shows the available information for image. Any number of images can be opened (depending on your computer's resources).
Note: the code is fast but some sols have hundreds, even thousands of images. This takes time, and I do have a fast connection!
Future:
Nice, Fernando! I'm not sure what you meant by CORS - my browsers showed the images fine without any plugins.
One suggestion, to help those without a fast connection: load only the thumbnail images on the pages showing multiple images from one sol, and then load the full image when you click on the thumbnail. Chemcam images should be courtesy NASA/JPL-Caltech/LANL.
Moderators: Is there any chance this thread could be moved to MSL? It only talks about viewing MSL images and doesn't relate to "Image Processing Techniques". It's really a sequel to the old thread http://www.unmannedspaceflight.com/index.php?showtopic=7408.
Thank you so much fredk ! I like the suggestion, although I'll have to think about what to do when there are no thumbnails (not every image has one, I believe). Thanks also for the correction to ChemCam credits.
Fernando
Many thanks Doug, for your comments and encouragement.
I decided to do this after reading Damia's post (a few posts back) but I didn't work full time on it. I don't clock in (after all I'm retired) My estimate is it took more than 10 hours over two days. I know a bit of HTML, CSS and JavaScript ...
And now I have a bug to hunt down. Mnham!
Thanks again for the great feedback
Fernando
That IS really nice ! Already handier than the official raw page.
If I may suggest : having a "hard" link when hover a thumbnail, permitting the use of the link to be displayed at the bottom of any webbrowser, and maybe don't display the full frame in a pop-up window, but in the same window (user can control that behavior later, by "click+command/control").
MP also had something really nice and simple, a navigation "tool" like this (example) : "... > Curiosity > Sols > ←2538 / 2540→" on top of a page. And when in a specific image page (displaying the full frame), we could also navigate previous / next picture.
Just express my feeling on it But this is already a nice job !
PS : If you want some place to host it, I can do it. I can also decompose your page in source files (separate the CSS from the HTML and the JS).
Merci beaucoup Damia!
For your encouragement and great suggestions. I'm compiling a list and will work on them as time allows, now that I know I'm heading in the right direction (but I'll be mostly off next week, COVID allowing).
Thank you also for your offer to host the tool. Right now it is evolving, so it is not the right time, but I'll get back to you on that. Isolating the JS and CSS in separate files is good practice, this was just the prototype.
To all:I may need to create several html pages (in addition to the above mentioned JS and CSS files) so I may be distributing a small zip file in the future.
Thank you for the information, nice reviews and ideas. Keep'em coming!
I've been parsing the MSL json files into html and thought I'd make the pages public in case anyone finds them useful. My pages are static html so there's no selecting cameras, changing sort orders, autoload scrolling, etc. It's inspired more by the linear style of the old curiosityrover.com than MMB, Fernando's page, or http://www.mars-browser.co.uk/curiosity/.
The info and pics are straight from the json files and the mars.nasa.gov/msl-raw-images server with two exceptions. First I calculate the pointing according to mcaplinger's suggestion in http://www.unmannedspaceflight.com/index.php?showtopic=8460&st=345&p=245610&#entry245610. I haven't tested this much so let me know if you notice any problems.
The second is that I deBayer the Bayered mastcam frames automatically as they arrive. Due to the chroma subsampling I do you shouldn't take the colours too literally, and there are lots of crosshatch artifacts, so these images are only meant to give a rough sense of the scene.
On my site there's the "latest images" page located here: http://lcdm.ca/msl/. This checks for updates every half hour and adds any new images to the top, chronologically by time taken. After updates I use this by scrolling and paging down until I see image links change to the visited colour, which makes it very easy to keep up and not miss any new images, even if they're from old sols.
There are also individual sol pages, eg: http://lcdm.ca/msl/2924/ (going back to sol 2714). These are always ordered by time taken.
More details on http://lcdm.ca/msl/details.html
If anyone has any suggestions hopefully I'll have time to implement them (within the limitations of static html), so let me know.
Viewed on my Mobile phone and impressed Looking forward to seeing it in action on my home PC when my covid travel restriction order is supposed to be lifted in 11 days 😊
Thanks, Paul. Sorry my site's not small-screen friendly. Maybe now that we're moving again on Mars that'll relieve your earthbound status a bit?
This.
Is.
AWESOME !
It's fast, it's clear, filenames are immediately readable, and it's easy to navigate. And you even debayer the pictures o_o
Yes, I really like this.
Phil
Thanks, guys. I hope to improve the deBayering still - treat the edges better and maybe remove the glitches that appear in each frame.
Here is an improved (I hope!) version of my MSL Image Viewer. Still work in progress so, for the time being, I'll continue to distribute it here. Having been on vacation last week I only had time to implement part of your suggestions - thank you all for those.
To install, download the zip file to your favourite location and unzip it. You should see the following 5 files:
I've improved the deBayering on my site http://lcdm.ca/msl/. I now handle the edges of the frame better, so colours are good right to the edges. Also I use a simple method to remove the "hot column" in the ML images. This is a bright column (column 1070, a green-blue column, in the full-resolution frame) that runs down most of the ML images. I simply replace that column of pixels with the average of the neighbouring green-blue columns before deBayering. Due to the jpegging in the public Bayered images, the removal isn't perfect and you may notice some residual "scarring" within an 8-pixel jpeg block of the hot column. Both these changes apply to new images, not old ones still on my server.
I've also added lists of deBayered images at the top of each individual sol page. These can be used to download all at once, with a browser plugin or, for example, with
I am releasing a few additional functions for the MSL Image Browser. In a http://www.unmannedspaceflight.com/index.php?showtopic=8514&view=findpost&p=248559 I'll discuss some issues which have influenced some decisions and prevented implementing some of the suggestions - I'm very grateful for those.
Recent images are now highlighted by a red border. Right now - and this is very easy to change - "recent" means the image has been added in the last 72 hours. I had to choose a criteria. It could have been "added since my last visit" but this implies storing the date and time of "my last visit" in a special area of the browser (using a standard - and safe - HTML 5 API). Please let me know if you're interested in having this option. A row of buttons for each sol with "recent" images (according to the above criteria) appears just below the page banner. Click them to go the the respective sol.
On the main page three new buttons (on the left panel) allow invoking three different views of a sol's image set:
Small update to Marslife.org.
The sun, earth and both moons of Mars were already present in the Marslife simulated sky. Yesterday I added the Mars2020 spice data to the mix.
Sure, it's only a few fake pixels but to me it's a very exciting view to see Perseverance slowly approaching.
Just a thought - Are you using just the left-eye images?
You will find that when we take an 'upper tier' ( extra images to capture Mt Sharp and other topography above the horizon ) more often than not it is right-eye only....so you might be missing out on some images.
Great work though!!
Thank you Doug!
You are right, the website uses left-eye images only. Checking just these has proven to be a convenient and somewhat reliable indication whether the rover has travelled to a new location.
In fact, the whole update mechanism is based on a very complex left-eye algorithm ( if NumberOfNewLeftNavCamImagesPerDrive > 7: ) that triggers the website and twitter updates.
So yeah, while this approach has been adequate enough for its purpose, I see now that it's failing to include the pretty right-eye pictures.
I'll accept the challenge to come up with a solution. In the mean time I hope your team will be able produce a few extra 'upper tier' left-eye images as well so that we can have some juicy anaglyphs!
Rob
So - here's the trick. We will never take a left eye without a right eye. The right eye is the one being monitored for temperature - so if we do mono imaging ( like upper tiers ) it'll either be right eye mono - or - rarely - stereo ( you don't really get much range data on upper tiers so stereo is definitely an indulgence )
tldr You might find some mono-left-eye stuff when we were operating on RCE-A or before about Sol 1600. But since we discovered an idiosyncrasy about how the rover monitors camera temperatures - it's either stereo, or right eye mono. Never left eye mono. Our deck monitoring 3 frame mosaics and our upper tiers will usually be right eye mono
There's another good reason to use the right navcams - they are less noisy than the left. Compare these two frames, eg:
https://mars.nasa.gov/msl-raw-images/proj/msl/redops/ods/surface/sol/02967/opgs/edr/ncam/NLB_660905274EDR_F0841360NCAM00266M_.JPG
https://mars.nasa.gov/msl-raw-images/proj/msl/redops/ods/surface/sol/02967/opgs/edr/ncam/NRB_660905274EDR_F0841360NCAM00266M_.JPG
The extra noise in the lefts is most obvious in smooth areas such as the sky or the hazy distant rim of Gale.
A new option has been added to the Marslife website that allows for viewing the Right Navcam panorama's.
Currently this option has to be selected in the menu but it might become the default setting in the future.
Thank you Doug for the suggestion!
Not sure where this will lead to but I've had some succes adding unbayered Mastcam images to Marslife. (not yet available for public use)
Hey all,
I've build a first preview of the Marslife website that shows bayered Mastcam images on top of the Navcam panorama's.
It contains the left Mastcam images up to sol 3018, grouped in line with the original Marslife post-drive-imaging index.
I didn't expect the site to perform so well (showing 150+ images) but unfortunately it crashes on my mobile phone.
If you don't mind the long loading times you are welcome to give this preview a try: https://captainvideo.nl/marslife/preview.html
Toggle the M key to hide/show the Mastcam images, doubleclick an image to have the filename printed in the bottom-left text.
Clicking the filename will open a browsertab with the corresponding NASA MSL raw images page.
Enjoy!
It doesn't seem to work in Firefox 78.7.0esr (on Mac). All I get is a screen full of rubbish.
It does work though in Chrome 88.0.4324.96.
Fredk, when I click on the sol 3027 link in your MSL image browser, I get the following message: "Forbidden. You don't have permission to access this resource." Is this normal?
Edit: OK, it is working now.
Big mastcam update! Still a work in progress, lots of loose ends to fix and the code is an absolute mess. But here it is:
I will be running out of superlatives soon!
Everything works well here ( using Brave browser )
Clicking between placeholders & loading is just fine & intuitive. It would be great if there was a on/off toggle for each mastcam sequence. Tab function is a great feature that is getting a lot of use!
Can't wait to see where you take this... it was already awesome to begin with!
This new preview build is epic - I dropped a quick play by play of what it does into work slack to share it with the MSL team.
Bravo.....BRAVO.
Thank you both! Much obliged Doug for showing the site to the MSL team, I'm honoured!
While the new rover is drawing all the attention I'm happy to report that the update script for Marslife now includes support for Curiosity's L&R Mastcam images.
Marslife will automatically add new Mastcam images to the corresponding post-drive panorama as soon as the Nasa json file receives an update.
Check Marslife here: https://marslife.org
This is really nice. Just a couple of comments: Is there any ability to zoom in or out without a scroll wheel or touch screen? (Eg, shift-click.) I couldn't see how to do that with desktop firefox or chrome.
A fun feature might be to display the latest planned ("strategic") route, so we can get some idea of where we're headed. Of course that's only provisional, and someone would have to digitize that route.
I had noted images on the raw image page being assigned the wrong sol - like the upward views of the parachute on sol 0 being labelled sol 2 (the date needs to be when it's taken, not when received, or we will never know where we are). I hope that gets sorted out.
Thanks for doing this, it's a great tool and one which will be very useful.
Phil
Several posts moved to the new http://www.unmannedspaceflight.com/index.php?showtopic=8603 thread per recommendation of another Forum member.
Powered by Invision Power Board (http://www.invisionboard.com)
© Invision Power Services (http://www.invisionpower.com)