Mars 2020 - Perseverance data in the PDS and the Analyst's Notebook, Working with the archived science & engineering data Options

[nogal]

PDS has released its first set of Mars 2020 Perseverance data.
Meanwhile, the Analyst's Nobebook for Perseverance is still "under construction".

Here is the full text of the release:

The NASA Planetary Data System announces the first release of data from the Mars 2020 Perseverance Rover mission to Mars. This release contains raw, calibrated, and derived data products covering the first 90 sols of the mission, February 18 to May 20, 2021. The data are archived at various PDS Nodes. Links to all the archives can be found at
https://pds-geosciences.wustl.edu/missions/mars2020/

Data from the following science investigations are included in this release.
Mastcam-Z – Mast-mounted Zoom Camera System
Engineering Hazard and Navigation Cameras
Helicopter Cameras
EDLCam – Entry, Descent, and Landing System Cameras
MEDA – Mars Environmental Dynamics Analyzer
MOXIE – Mars Oxygen ISRU Experiment
PIXL – Planetary Instrument for X-ray Lithochemistry
RIMFAX – Radar Imager for Mars Subsurface Exploration
SHERLOC – Scanning Habitable Environments with Raman and Luminescence for Organics and Chemicals
SuperCam – LIBS, Raman, Time-Resolved Fluorescence, VIS/IR spectrometers, Remote Micro-Imager and microphone
SPICE – Observation geometry and ancillary data

Future releases covering sols during which sampling events occurred will include 'sample dossiers' – collections of science and engineering data relevant to each returnable sample or blank. No sampling events occurred during the first 90 sols of the mission.

To subscribe to future announcements of Mars2020 data releases:
https://pds.nasa.gov/tools/subscription_service/top.cfm

To access all data archived in PDS:
https://pds.nasa.gov
The PDS Team
Mailto: pds_operator@jpl.nasa.gov
Phone: (818) 393-7165

[Brian Swift]

Great stuff.
Full resolution EDL raw videos look awesome - no YouTube/Twitter re-compression artifacts.
Thumbnail videos include some new footage (ddc_00000005_020_000000.mp4 has more movement of exterior lighting).
https://pds-imaging.jpl.nasa.gov/data/mars2...document_video/

[erentar]

Amazing. The documentation looks amazing, they really nailed it this time!

I am extremely happy they got around to publishing what the pipeline looked like https://pds-geosciences.wustl.edu/m2020/urn...peline_Flow.pdf

I wonder, was there ever a similar document released for MSL or MER missions?

[Andreas Plesch]

PDS now also has the first RIMFAX (radar) data, and I started to look at browse images (see map thread). The browse images have full horizontal resolution and somewhat limited vertical resolution (reduced by a factor of 8). They are very contrast stretched.

It turns out it is very possible to turn the calibrated sounding data in the csv file into images. I ended up using this script:

CODE

#!/bin/bash
# usage: csv2pgm.sh < rimfax_calibrated_IN.csv > rimfax_calibrated_OUT.pgm
export TEMP=csv2pgm.tmp
# extract Shallow mode and ignore calibration records
# field 91 is where the data start
grep Shallow, | grep XM | cut -d"," -f91- > $TEMP
export height=`wc $TEMP | awk '{print $1}'`
export width=`head -1 $TEMP | awk -F"," '{print NF}'`
#echo $width $height
# use numeric sort to find min. and max.
tr , \\n < $TEMP | awk '{printf "%f\n", $1}' | sort -h > S$TEMP
export min=`head -1 S$TEMP`
export max=`tail -1 S$TEMP`
#echo $min $max
# use awk to generate plain pgm format. Just normalize to one value per line, and normalize the values to 2 bytes.
tr , \\n < $TEMP | awk -F, -v width=$width -v height=$height -v min=$min -v max=$max '\
BEGIN {
print "P2"
print width, height
maxC = 65535
s = maxC / (max - min)
print maxC
}
{print int(s * ($1 - min) + 0.5)}
' | pnmflip -xy | pnmscale 1
rm $TEMP S$TEMP

It is centered around creating a plain pgm format image which is a very simple format and close to the csv data. The resulting file preserves 2 byte gray values which is nice. Gimp and probably Imagemagick understand the format, and there are many netpbm tools for it.

Using the script I could reproduce browse images, so I am pretty sure it is correct. I will convert all available recordings soon. Here is an example of sol 65, without stretching and double vertical resolution (lower section removed):

[tau]

The problem of inhomogeneous filter flatfields in raw images no longer exists in the radiation calibrated (RAD) images available in the Planetary Data System (PDS) Geosciences Node.
Here is an infrared multispectral false-color image, created from sol 69 Mastcam-Z right eye infrared filters 1 to 6 radiation calibrated images using principal components analysis.
The foreground in the lower halfth of the first image with bluish-gray rocks belongs to the basaltic unit. The rocks in Séítah show a yellow-orange false color. Kodiak hill stands out with a blue false color.
So we have at least three different rock types in this image.

The second image is the Mastcam-Z right eye filter 0 (visible RGB) RAD image (with increased brightness) for comparison.
Séítah and the basaltic foreground are almost indistinguishable in visible RGB colors.

[PaulH51]

PDS release 2 covering sols 90 - 179 (November 22, 2021) is now available link

[Phil Stooke]

Let me recommend this specific page:

https://pds-geosciences.wustl.edu/m2020/urn...ta_special_rmi/

All RMI mosaics with target names. If you are looking for names as I always am, this is gold. I am just in the process of getting them onto the maps. There will be other names associated with Mastcam-Z targets and WATSON targets but this is a good start. The Analyst's Notebook is not up yet but will be soon with limited functionality, and ultimately that will give all names.

Phil

[PaulH51]

Let me recommend this specific page:

https://pds-geosciences.wustl.edu/m2020/urn...ta_special_rmi/

All RMI mosaics with target names. If you are looking for names as I always am, this is gold. I am just in the process of getting them onto the maps. There will be other names associated with Mastcam-Z targets and WATSON targets but this is a good start. The Analyst's Notebook is not up yet but will be soon with limited functionality, and ultimately that will give all names.

Phil

Thanks Phil,

Can you kindly check the link?

I'm getting: "Page is down.... Service Unavailable HTTP Error 503. The service is unavailable."

I'll try again later in case it's temporary...

Cheers

Paul

[Phil Stooke]

It's fine now.

Phil

[Phil Stooke]

Just a heads up that the Analyst's Notebook for Perseverance is now up. No targets, but day to day activity summaries. I will be looking for names.

Phil