Printable Version of Topic

Posted by: Phil Stooke Apr 8 2021, 06:25 AM

Hi Everyone. I am starting a thread for discussions of the many ways people are interacting with Perseverance (and potentially other missions). These subjects are getting spread around other threads and maybe getting in the way of basic mission updates etc., so this is a place to bring this together. Feel free to contribute to it.
I am going to start by asking about panoramas - what's the best way to do this with Perseverance images, especially Navcam images from every end of drive?
Phil

Posted by: vikingmars Apr 9 2021, 04:34 AM

Thank you very much Phil for this new thread.
About the Navcam images, the best thing to do is obviously to build 'donut' panoramas at the end of drives... as you already did so well

Posted by: Ryan Kinnett Apr 9 2021, 06:07 AM

Hi all,

Wanted to introduce you to (yet another) rover and terrain visualization web app which I hope will help folks become familiar with Perseverance and the Jezero Delta.

My https://rkinnett.github.io/rover3d/ page mimics actual MER and MSL driving tools. The Perseverance rover model derives from the official JPL public release model, but has been heavily modified to finish incomplete texture and material mapping, and has been split into articulable parts which you can control via sliders, similar to the official rover driving tools. Or you can send the arm, RSM, and HGA to one of several selectable canonical poses, matching MSL's standard defined poses (note: the default arm pose in the public release model, with upper arm horizontal and forearm vertical, is non-standard and not generally used).

The Jezero Delta front-range terrain is composed of a HiRISE ortho-rectified luminance image (downscaled to 50cm/px for performance), blended with one of Justin Cowart's masterpiece CRISM color mosaics, and draped over a MOLA DTM.

Click "ABOUT" in the upper-left corner for general information, controls, and credits. The controls take a minute to get used to, but are optimized to help you scoot around the scene quickly and freely. Quick synopsis:

• Left-click and drag to rotate view around a center point
• Right-click (or hold shift and left-click) and drag to pan the view up/down/left/right
• Double click anywhere on the terrain to center the view at that point
• Press "r" key to center the view over the [r]over
• Press "d" key to [d]rop the rover onto the terrain behind your mouse cursor

Explore the northern Delta approach strategic route (cyan), southern approach route (blue), and the delta entry route (yellow)!

Planned future work:

• Some form of driving, wasd key-based and/or waypoint-based
• Sync to telemetry (after SPICE kernels are made available)
• Add named features and points of interest markers
• Render camera frustums
• Support local Navcam/Mastcam-Z terrain meshes

Screenshots:



I hope you find this useful!

Posted by: Andreas Plesch Apr 9 2021, 07:53 AM

Very nice 3D scene ! I think the larger scale and focus on the rover model complements very well what I have pieced together at https://bit.ly/PercyMAP which started out more as a zoomable interactive, detailed map and then expanded a bit to include 3d features.

I am developing that 3d map at https://github.com/andreasplesch/Mars2020/tree/main/docs/3dmap . The technology is quite different from three.js and the code needs refactoring but please feel free to adopt anything you may find useful in this repo.

May I ask if I can use your enhanced Perseverance glTF model on my page ? It would be pretty much a drop in replacement and would be a nice addition.

Posted by: Ryan Kinnett Apr 9 2021, 04:41 PM

Thanks Andreas. Nice job on that mapping page. It is definitely helping folks build a sense of scale around the landing area. I was hoping this thread could be a sort of showcase for tools that may be useful in various ways to mission followers. Please consider writing a synopsis about your mapping page here.

You’re welcome to use the rover model. Do you think you might want to articulate the rover? If so, then you’ll need some code to map and control the joints. If you don’t need to articulate then I can send you a gltf with the arm stowed and merged back into one solid mesh object, which helps with performance. Or something in between - we can make just the RSM articulable, for example.

Posted by: Art Martin Apr 9 2021, 04:56 PM

Sounds great but I just found out Firefox doesn't like it. Crashed my browser.

Posted by: ChrisC Apr 10 2021, 07:55 PM

Worked for me. Firefox 87 on Ubuntu Linux. No plugins / addons other than two cookie blockers and FB containment.

Posted by: Ryan Kinnett Apr 10 2021, 09:36 PM

Thanks guys for testing. I’ve got it crashing in Firefox as well. Looking into it.

Hey Phil, regarding your question about navcam panoramas, you’re familiar with the general process of manually downloading and stitching; are you asking for suggestions for how to automate the process, or maybe a web app for this?

If you’re looking for an automated local process, in linux it shouldn’t be too hard to write a python or shell script to look for new navcam images from a common sequence, and even verify in metadata that the images overlap, then download the images and stitch them using either hugin or python/opencv. You could then set this script up as a cron job to check once per hour, for example, and even send you some sort of alert when a new pano is ready. It would be neat to do this on a raspberry pi and set it up as a web server so you can view the panoramas from anywhere on your network or even over the internet.

If you’re looking for a webpage, marslife.org is probably your best bet. If you had something different in mind, I could potentially put something together.

Posted by: Andreas Plesch Apr 10 2021, 11:01 PM

Thanks for sharing the rover model. For now I am using the rover mostly for scale and because it is a nice model. So a mesh optimized for loading could be quite helpful. I managed to animate the helicopter model blades and could do something like that for the rover joints but do not have a good, idea how to make this a useful feature. Perhaps when arm joints pose data becomes available along with drive data. Mast cam pose is already available from the image json, I believe and it may be fun and useful to have an option to replay a given day. One could fetch (async) all the zcam json, organize it by day, develop daily animations from the poses, either sped up real time, or just for some fixed interval (5s), and play the animation for a requested day, until some other action is requested. Could be perhaps doable. The scene camera could optionally track although this may be disorienting.

An idea would be to develop a drive planning or future drive options app which could include arm planning and arm feasibility testing. But this really needs integration of navigation imagery and in particular integration of local surface reconstruction from the stereo images which is out of reach for me. Also overall (perhaps too) ambitious for me.

Given the limited although still high resolution of the HiRISE coverage, I am undecided if it will be worth keeping the 3d map page updated after we leave the helicopter flight zone. It may come down to some experiments or some need for me at a later point.

I have shared some info on my 3d map in other topics and can provide a summary and ideas for future features here.

Posted by: Phil Stooke Apr 11 2021, 01:35 AM

Hi Ryan - don't worry. What I was really doing was trying to encourage other people to make panoramas, as Jan van Driel does for Curiosity after every drive. I have enough to do without also making panoramas.
Phil

Posted by: Ryan Kinnett Apr 11 2021, 07:55 AM

I may have fixed the firefox loading issue. At least, it's working on both of my machines, including an older, budget laptop. I added a terrain resolution control and have it default to 2m resolution for mobile devices, otherwise 1m (downgraded from 50cm). Strangely, it seems Chrome can handle 16k texture maps whereas FF is limited to 8k.

In case it's useful, I posted an arm-stowed version of the rover model https://github.com/rkinnett/rkinnett.github.io/blob/master/rover3d/model/Perseverance_stowed.glb.


I'll start experimenting with automating navcam panoramas. Not sure I can be counted on to post after every drive though. That's very impressive dedication (as is yours!).

Posted by: vikingmars Apr 11 2021, 10:14 AM

Many people I know, including myself, encounter problems mosaicking Perseverance's NavCam images using regular softwares : AutoPano, PTGuy, Hugin, ICE, etc. (*).
When it comes to include the lowest part of the images, those are so much distorted that the results are ugly (sometimes the softwares show no result at all).
I've got so much work to do (apart from job) for our French national astronomical organization (Societe Astronomique de France), that I don't have enough time to enter into computer programming
But, I'd like feeding our educational website from time to time with lovely panoramic pictures with comments about the mission (see recent link on the SAF website as an example : https://saf-astronomie.fr/fonds-decran/ ).
Any solutions found to mosaic those NavCam images, please ?
Thanks so much in advance, this would really help the UMSF community (and the Planetary Society of which I am an active member)
(*) After devignetting and color correction

Posted by: Andreas Plesch Apr 11 2021, 05:34 PM

Thanks. I have updated https://bit.ly/PercyMAP with the arm-stowed rover model, still waiting for the traverse json to be hopefully corrected.

Posted by: Ant103 Apr 12 2021, 09:33 AM

For me it's not really about that.

For me, it's about those colours. I mean seriously, Curiosity color pictures are very homogeneous, there is little to correct between them (maybe sometime a little bit of R/B balance, exposure) but THAT'S ALL.

Why Perserverance imagery is so off ? All of it, both Nav/HazCams, Mastcam or WATSON. Because, stitching pictures, remove vignetting, I can do that. But when it comes to color weirdness like this, I can't. Even with a "single" picture like those full res Navcam cut in many pieces, there is this difference between the frames. I don't why we have that but it's annoying. And I think it's not encouraging, because I read people about the "real colors" of Mars saying that Percy imagery is deceiving, too yellow, or too blue, or too something, and why so different than Curiosity ?

Posted by: Andreas Plesch Apr 12 2021, 11:45 AM

Here is an overview of https://bit.ly/PercyMAP

https://bit.ly/PercyMAPis an interactive, 3d map https://github.com/andreasplesch/Mars2020/tree/main/docs/3dmap. The focus is on allowing exploration of the detailed surroundings of the landing area of the Perseverance rover and the flight zone of the Ingenuity helicopter using 10cm resolution EDL imagery, a 25cm HiRISE mosaic, and a 1m HiRISE DTM.

In addition, labelled way points the rover has reached during his stay in the area are shown. as well as a few landmarks such as the designated flight zone of the helicopter and the lookout location from which the rover observes the helicopter during its first flights. The blue way point symbols are linked to relevant posts and web pages. Animated rover and helicopter models at correct scales are shown as well.

Zooming in on a particular detail of interest is accomplished by mouse navigation. Double clicking on a spot will recenter the view, and subsequent zooming can then be achieved by wheeling or right mouse drag. There is a small help box which can be toggled on showing these mousing options.

It is possible to accurately determine coordinates and measure distances. Clicking on the map will add the location to a list of points which shows the coordinates of the point, and the distance to the point previously digitized. For example, it is possible to determine the distance from the lookout to the helicopter, or the distance between way points.

Other features include sliders for controlling the amount of vertical exaggeration, and image contrast. By default, topography is vertically exaggerated five times in order to emphasize the subtle relief in the area. Feel free to reduce to 1x or enhance as useful.

Finally, a row of buttons on the top provide 3d perspective views. NORTH looks towards north along the flight zone, LOOKOUT looks from the lookout location to the initial helicopter air field, and MAP returns to a top down map view. The RESET button resets the view to the initial configuration before any mouse navigation.

Future plans include improving the rover drive animation to adhere to ground level at any level of vertical exaggeration, recreating helicopter flights, or less restricted viewing, for example from the current position of the Mastcam. Viewing currently is guided towards a focus on the ground surface.

Posted by: phase4 Apr 12 2021, 12:32 PM

Hey all, i'm the person responsible for the elaborate hack that is called Marslife. Thank you very much for this thread!
I would like to take this opportunity to present the latest enhancement to the website; every Perseverance image that
came down as L & R stereopair can now be viewed as anaglyph!
It is as simple as pressing the 'A' key on the keyboard or select the option from the menu when you're on a mobile device,
Marslife will download the corresponding right-eye images for each visible sequence and switch to anaglyph viewmode.
Press the 'A' key again to return to normal viewmode.
Get your glasses ready and have fun!

Rob

https://flic.kr/p/2kS2ojG https://flic.kr/p/2kRWp1z

Posted by: Andreas Plesch Apr 12 2021, 01:23 PM

https://bit.ly/PercyMAP updated to rely less on correctness of the metadata in the traverse json. The path animation now uses the waypoints json to sort the segments into the correct order. The sorting now relies on segment length in the traverse json corresponding exactly to the dist_m property in the waypoints json which sofar is the case.

Posted by: phase4 Apr 12 2021, 11:50 PM

https://flic.kr/p/2kLhqqj

Yes, they are pretty hard to work with, especially the ECAM's. And way off when it comes to 'real' colors.
I guess that leaves them open for any artists interpretation...

In Marslife I also try to make the panorama's match, which is a breeze to do with the B&W Navcams from Curiosity.
The MSL Mastcam images look pretty good by themselves so I haven't made a color correction attempt yet.
But to make a presentable panorama with Percy's navcam images, that is indeed quite the challenge.
I wonder, is it possible that the color quality will improve during the mission?

For now I have settled to crank up the blue channel a bit, which looks, well, agreeable.
Other hints how get the colors more like they should be are most welcome.

Posted by: mcaplinger Apr 13 2021, 12:52 AM

I'm not sure what's going on with the JPL cameras, but WATSON raw color is exactly the same as MAHLI, and MCZ should be very similar to Mastcam. So if there are things you don't like about the color, they are common to MSL for those instruments. Unless the JPL raw site is doing something different -- I haven't followed what processing JPL is doing there. For sure they are doing a stretch. Honestly the MCZ color looks pretty decent to my eye -- better than the typical greenish hue of MSL Mastcam images.

Do we have to have yet another discussion about the subjective nature of true color?

"no best is quite so good you don’t conceive a better" -- ee cummings

Posted by: Ant103 Apr 13 2021, 08:09 AM

So, why I had much more difficulties with Percy's selfie than with Curiosity's ? Why I had to edit parameters that I never touched with MaHLI imagery, like color balance, levels ? Same thing with MCZ.

It's not about "true colors" here. It's just with MSL datas, I just have to touch exposure to blend them seamlessly. With Percy… It's a whole new game : exposure, yes. But also saturation, color balance, grey levels. Why ?

Posted by: vikingmars Apr 13 2021, 08:44 AM

Thank you Ant103 for your interesting comments. Yes you are right about the colors. They differ from a camera to another because they come from various providers : MastCam-Z from Arizona State University, engineering cameras from JPL, and WATSON from Malin Space Science Systems. Having worked on Martian colors since 1981, I would say (if I may) that the ones from MAHLI/WATSON are the closest to the real colors you may see on Mars. A clue (but not the only one) is that when pictures of the sky are taken with MAHLI/WATSON, the opacity changes in the atmosphere are well reflected with colors ranging from a gray pink (low atmospheric opacity) to a salmon pink (high atmospheric opacity).
About color picture process for all cameras, for me it's not really a subject : I found that old 'Viking' color processing techniques work well with Perseverance images.
Raw / Processed

Posted by: stevesliva Apr 13 2021, 02:20 PM

*before* that white-balance/true-color discussion ~8.5 years ago, there *was* a 'sol 42 / sep 18 2012' change in the "stretch" of navcam images to the MSL raw images webpage.
http://www.unmannedspaceflight.com/index.php?s=&showtopic=7418&view=findpost&p=191650

Just trying to jog memories. Hope it helps.

Posted by: Andreas Plesch Apr 14 2021, 05:19 AM

https://bit.ly/PercyMAP was updated since after the last drive the traverse json format was again slightly changed to report segment length with more decimals (down to atomic scale) than in the waypoint json (down to millimeters). Hopefully, the json formats will stabilize at some point.

Posted by: Andreas Plesch Apr 19 2021, 10:23 PM

https://bit.ly/PercyMAP was updated with an approximate animation of the first flight, a new location of the Van Zyl Overlook, and a label for the Wright Brother field.

Where would the engineering, pose data be made available ?

Posted by: Ryan Kinnett Apr 20 2021, 03:15 AM

SPICE kernels will be made available and maintained here:
https://naif.jpl.nasa.gov/pub/naif/MARS2020/kernels/

They’re not there yet. I’m guessing they’ll be made available before August to support public data releases and whitepapers around the six month mark. Look for tutorials and MSL examples to understand how the system works.

Posted by: Andreas Plesch Apr 20 2021, 04:03 AM

Thanks. There is a web API, for example https://wgc2.jpl.nasa.gov:8443/webgeocalc/api/kernel-sets, which should make kernel integration possible when available. Apparently, well defined location/orientation/time data are considered more sensitive in some way.

Posted by: Andreas Plesch Apr 20 2021, 06:44 PM

Update to https://bit.ly/PercyMAP to include a rotor spinup and spindown animation, as well as approximate hover turn. Also update of overlook view.

Posted by: Andreas Plesch Apr 26 2021, 04:00 AM

Further update to https://bit.ly/PercyMAP :

- approximate animation for flight 3, 5.2 m elevation to point 50m north and back.
- new view button for FLIGHT, perspective on flight path.
- enabled left mouse drag for free rotation around current center. It can be a bit confusing. Press RESET if you get lost.

Try reducing v.e. to 1.0 before zooming in.

Posted by: Andreas Plesch Apr 27 2021, 01:35 PM

Latest drive introduced null (?) geometries in traverse.json. Updated https://bit.ly/PercyMAP to accommodate and show last drive.

Posted by: Andreas Plesch Apr 28 2021, 02:12 AM

https://bit.ly/PercyMAP updated to latest drive, and to a more transparent flight zone curtain.

Here is a view from the way point at sol66:



The background is unfortunately completely off, and only provided for decoration. On the page, the background should be (more) correct now.

The field of view is about 30 degrees.

The rover is at the landing sport, and Ingenuity is flying 5.2m above ground towards the right just at the top edge of greenish flight zone curtain. The blue bubbles are earlier way points.

Posted by: ChrisC Apr 28 2021, 02:58 AM

Thank you so much for doing this! Anaglyphs are my main way of doing my (extremely modest) outreach efforts.

But you forgot to link to the website, perhaps assuming everyone already knew. Doh

----> http://marslife.org

Posted by: Andreas Plesch May 4 2021, 03:47 AM

https://bit.ly/PercyMAP updated to latest drive; the waypoint geojson changed a bit this time (no lat, lon properties anymore). I really hope the formats stabilize.

Posted by: Andreas Plesch May 8 2021, 04:37 AM

https://bit.ly/PercyMAP updated to include approximate animations of flights four and five. The spin up/down blade animation is now out of sync but the animations gave me a better appreciation of how long these flights really were. One has to zoom out quite a bit to see the complete flights, and little Ingenuity just keeps going and going. I will refine the timing and blade synchronization.

Further updated so Ingenuities altitude now scales with with the vertical exaggeration of the terrain for a better comparison. Scale down 1x v.e. to see the real altitude. Also improved the timing a bit.

Also, now rover (finally) uses and follows elevation data from traverse geojson or interpolated from waypoints geojson, for any v.e.

Posted by: Andreas Plesch May 15 2021, 03:14 AM

Minor updates to https://bit.ly/PercyMAP : improved help, wider initial view, additional waypoint links to relevant content.

Posted by: Andreas Plesch May 16 2021, 07:10 PM

https://bit.ly/PercyMAP updated to work with latest traverse json which has a small typo for the last drive: fromRMC is "2_2430" but should be "3_2430" as far as I can tell. I think the json generation may not be fully automated.

It appears that the elevation data in the traverse json for the last drive is slightly lower than the elevations in the HiRISE DEM. That is why the rover sinks a little bit into the ground in the drive animation.

It is interesting that the RMC counter reset after the last drive to a new first digit (project phase?), from 3 to 4. The current way point has now a RMC of "4_0". It could signify a new project phase starting from the current way point.

Posted by: Greenish May 16 2021, 09:08 PM

The RMC has a Site counter that updates each time the local working coordinate origin is reset. See "2.1.2. Site Frame" in https://pds-imaging.jpl.nasa.gov/data/msl/MSLPLC_1XXX/DOCUMENT/PLACES_PDS_SIS.PDF for details on how it relates to the Nav Frame, Local Level frame, etc..

Posted by: Andreas Plesch May 16 2021, 09:18 PM

Thanks, that's it:

We are in a new local work area.

Posted by: djellison May 16 2021, 11:54 PM

The resetting of the site index is usually triggered by an accumulation of time on the rover's IMU and thus an accumulation of inaccuracy in the rovers attitude knowledge - not some geographic or scientific reason. On MSL that IMU time comes in to something like a couple of hundred meters of driving.

You will see this site increment accompanied by a Navcam solar image used as part of that process - such as https://mars.nasa.gov/mars2020/multimedia/raw-images/NLM_0084_0674404328_348ECM_N0032560SAPP00601_00_2LLJ for M2020 or https://mars.nasa.gov/msl-raw-images/proj/msl/redops/ods/surface/sol/03113/opgs/edr/ncam/NRB_673856334EDR_F0872978SAPP07612M_.JPG on MSL

Posted by: Andreas Plesch May 17 2021, 01:53 AM

Makes sense. The "new local area in which to work" just refers to spatial referencing work.

Posted by: Andreas Plesch May 23 2021, 01:17 AM

In anticipation of flight 6, I updated https://bit.ly/PercyMAP with a slightly improved landing location in air field B, and a button to switch to the helicopter's view perspective as it is flying. Zooming in and out still works while flying.

The flight paths are pretty accurate but the changes in view orientations are just proof of concept to provide a sense what the helicopter sees.

With flight 6 it will be time to make a new base map since the trip will go beyond what is currently shown. It could be just an addition, or a shifted map to save resources since I want to maintain the DEM as high resolution as possible.

Posted by: Andreas Plesch May 29 2021, 03:46 AM

Here is a quick bookmarklet to extract the Urls of full images from whatever thumbnails are currently shown on the https://mars.nasa.gov/mars2020/multimedia/raw-images/:

Since the forum does not allow javascript urls, the easiest way to "install" is to navigate to:

https://andreasplesch.github.io/Mars2020/bookmarklets/extractRawUrls.html

and drag the link to the location bar or to the bookmarks folder. You can edit the bookmark to see the code or change the name.

When clicked a new window or tab opens with all the urls listed. The intended use is then to select all (ctrl-a) and paste as a list of urls after a wget command line to download all. So far no escaping of special characters was necessary.
Command lines can be pretty long, so this should work for a lot of urls. Otherwise paste into a file and use wget -i.

This allows using all the options to filter on the https://mars.nasa.gov/mars2020/multimedia/raw-images/, for example movie frames, and now helicopter images. Since there is still a bit of typing and work involved to actually download, hopefully this little tool is still acceptable for the Nasa servers.

Updated to open a window instead of an alert since alerts can only show a very limited amount of text.

Posted by: Andreas Plesch Jun 13 2021, 01:17 AM

https://bit.ly/PercyMAP was updated to accommodate changes in the rover json (no more 3d coordinates). This has now the effect that the rover drives off the map. Clearly. time to update the base map. Or perhaps a good opportunity to start over and refactor which would be very needed.

Posted by: Ryan Kinnett Jun 13 2021, 07:44 AM

If you have a VR headset (ideally Oculus Quest), check out Perseverance in virtual reality! You can teleport yourself and the rover anywhere around the delta region, drive the rover, and unstow the arm, all with audio recorded either on Mars or in JPL testbed.

https://rkinnett.github.io/marsroVR

Posted by: PaulH51 Jun 16 2021, 06:10 AM

Has anyone here found an App or utility for range finding and/or scaling targets close to the rover by using a stereo pair?

Posted by: neo56 Jun 25 2021, 01:50 PM

For those who would be interested, I have coded in Python (and in the good old IDL) the algorithms to compute Ls, LMST, LTST and elevation / azimuth of the Sun for a given time in UTC and latitude/longitude: https://github.com/thomasappere/MarsTime_PositionSun

This work is partly based on routines written by Aymeric Spiga, a planetary scientist in Paris with who I worked during my PhD.
I checked my outputs with the results given by Mars24 and there is a very good match.