Perseverance Imagery, technical discussion of processing, cameras, etc. Options

[Brian Swift]

Thanks for the response Mike.

1. Why would it be different?

IDK, maybe trying a new transfer function that doesn't decompand both 2 and 3 to 3. (Which obviously isn't a big deal)

2. No idea. I could look it up, but then I couldn't tell you. Did you look at the metadata?

As you noted, not exposure info in metadata.

3. I had nothing to do with the calibration target.

The question would be, was the calibration target imaged by Watson during testing, and if so did the AluWhite98 target show less blue (relative to green and red channels) than the other three greyscale targets?

4. Not that I know of, we had no responsibility for ACI calibration.

So far I haven't come across a detailed technical document on SHERLOC (the non-watson side.) If anyone has a link to one, please reply.

[fredk]

https://link.springer.com/article/10.1007/s11214-020-00765-9

Thanks, somehow I missed the mention of IR filters in that paper. I had been thinking that maybe a lack of IR filter was causing the wonky engineering cam colours (though in hindsight that might've shifted colours to the red). I guess we're simply seeing the effect of raw colour, with the responses of the RBG filters quite different from those of the eye.

[phase4]

I had been thinking that maybe a lack of IR filter was causing the wonky engineering cam colours

I assumed that next to auto-exposure the new color cams would also use auto-white balancing. But that's an uneducated guess.

[fredk]

I assumed that next to auto-exposure the new color cams would also use auto-white balancing.

Normally cameras don't apply white balancing to the raw (Bayered) images - that's done later, typically when the raw is converted to a jpeg. So white balancing can't be the cause of the yellowy engineering cam inages.

[MarkL]

How does the MET counter relate to real time on Earth please? Is there a zero point?
Thanks
Mark

[mcaplinger]

How does the MET counter relate to real time on Earth please? Is there a zero point?

Isn't this the same question that you asked and was answered in this thread back on post #102 and subsequent?

[ChrisC]

How does the MET counter relate to real time on Earth please? Is there a zero point?

Emily L did an epic writeup recently about the photo metadata that probably answers that question:

https://www.patreon.com/posts/orientation-to-48263650

(UMSF needs an "oh snap!" button )

[MarkL]

0 is 2000-001T11:58:55.816 ("epoch of J2000" in Coordinated Universal Time.)

This then drifts as a function of rover clock rate and is tracked by the NAIF file that Fred linked to, so you need to use the NAIF function scs2e or something like that to get a usable time. https://naif.jpl.nasa.gov/pub/naif/toolkit_...C/req/sclk.html

Thanks mate. Sorry I thought I forgot to ask. Too lazy! Much appreciated.

[MarkL]

I did a little playing around with sorting the raw images by different parameters (particularly by MET which is what interests me so I can browse images in sequence) on my Mac to get what I wanted and I will pass on a couple of Mac tips for accomplishing this. MacOS makes this process quite straightforward.

1. Obtain a regular expression (RegEx) filename change utility. I used Transnomino which worked very well.
2. Create aliases (ie. shortcuts) of the image files you want to sort (Command-a, Make Alias) and put them in a separate directory.
3. In the alias directory highlight all the aliases and rename them via the utility. Transnomino helpfully adds a context menu option so just right click the selected aliases and Rename with Transnomino.
4. Use the RegEx to rename as follows:

Find:
(\w)(\w\w)_(\d\d\d\d)_(\d{10})_(\d\d\d)(\w\w\w)_(\w)

Replace:
$7_$1_$4\.$5_$1$2_$3_$6_$7

This uses 7 RegEx capture groups (each delimited by parentheses) to rearrange the filename of the alias. It doesn't touch the original filename since the alias refers directly to the original file.

Example:

NLE_0045_0670932186_787ECM_N0031416NCAM00299_14_0LLJ01.png alias

is renamed to:

N_N_0670932186.787_NLE_0045_ECM_N0031416NCAM00299_14_0LLJ01.png alias

The sort order is, Image type (N/T), Camera subsystem, MET. This allows you to sort by name and keep all the thumbs separate from all the normal images. It sorts by MET within subsystems (Nav, EDL, Forward Hazard, Rear Hazard etc).

In the sol 45 and 46 images, there are a couple of cool "snake charmer" sequences of the arm and instrument head in motion in the thumbnails which I found easily after renaming the aliases as above. If you rename the way I suggest, the sequences are:

T_N_0670932316.774_NLF_0045_ECM_T0031416NCAM00298_01_600J01.png alias
T_N_0670932866.237_NLF_0045_ECM_T0031416NCAM00298_01_600J01.png alias

T_N_0671021417.992_NLF_0046_ECM_T0031416NCAM00298_01_600J01.png alias
T_N_0671024345.491_NLF_0046_ECM_T0031416NCAM00298_01_600J01.png alias

It is quite cool to watch the instrument head move around a fixed point in space. I suppose they are checking it out still.

[Ryan Kinnett]

For the less technically minded among us...Ryan Kinnet has put up a page that grabs a listing with links to the PNG files that you can then use any browser-plugin-batch-downloader with

https://twitter.com/rover_18/status/1364309922167488512

I tried Firefox with 'DownloadThemAll' and it worked perfectly.

Meanwhile THIS GUY has python code to also grab the data
https://twitter.com/kevinmgill/status/1364311336000258048

Hello UMSF!

I'm happy to learn my roverpics page is aiding some of the wonderful creations around here. That is precisely my intent with this and other things I'm working on. Please drop me a line if you have any ideas or suggestions to make that page more useful. One thing I hope to add in the near future is per-image local-level az/el, derived from metadata, to help us identify distant features in images. Longer term, I'm thinking about linking this to my rover3d page to render rover pose and camera view projection for any image.

Cheers

[phase4]

...white balancing can't be the cause of the yellowy engineering cam inages.

So that can't be it, thank you for the explanation.

[MarkL]

Hello UMSF!

Hi Ryan. I think lots of us use it daily. Its a terrific way to get everything at once. I have not done much with the json data or the URLs yet.

One suggestion - filter new images since last visit - otherwise I just download all of them if there have been some new ones added to a particular sol rather than manually look through them for new ones. Perhaps also a list of which sols have new images available or a log showing when new images have arrived by sol. Not sure the best way to implement that.

Really appreciate your site.
If anyone has done some python code to organize or selectively download that would be great.

Id also like to get peoples impressions of the various stitching software out there, particularly for Mac.

[Ryan Kinnett]

Hi Ryan. I think lots of us use it daily. Its a terrific way to get everything at once.

Thanks, I'm very please to hear that.

One suggestion - filter new images since last visit - otherwise I just download all of them if there have been some new ones added to a particular sol rather than manually look through them for new ones. Perhaps also a list of which sols have new images available or a log showing when new images have arrived by sol. Not sure the best way to implement that.

That a great idea, but unfortunately it's beyond my infrastructural capability. What you're describing would require tracking, either via cookies or a user management system, plus a completely different query system. The page is quite dumb. It's really just a graphical wrapper for the JSON interface, completely contained in a single html file with no libraries or back-end server support or anything complicated like that. You could save the index.html file to your desktop and run it from there, if you wanted to.

If anyone has done some python code to organize or selectively download that would be great.

Have you tried Kevin Gill's m2020 raw image query python tool?

[MahFL]

A new advanced filter option appeared on the raw page, "Movie Frames"...

[PaulH51]

The black margins on the Mastcam-Z frames are a bit of a pain to remove manually. I do this to permit stitching in MS-ICE without leaving shadows at some of the the seams.

Does anyone here know how I could use GIMP to process a batch at one session?

Grateful for any advice