MSL data in the PDS and the Analyst's Notebook, Working with the archived science & engineering data Options

[Gerald]

This diagram shows qualitatively, how REMS UV RDR data are influenced by Masthead movements:

Data are rescaled to fit into one diagram.
Sol 89 Masthead movements can be found in this ADR file, explained in this FMT file.
UV data are contained in the same file as temperature data. They are explained in this FMT file.

So, when analyzing UV data, Masthead movements can be used as a filter to skip data invalidated by those movements.

[Gerald]

There are ways to further reduce the discrepancy between Sol 89 Brightness Temperature A and B by applying time-dependent calibrations:
Take the central part of the Sol 89 time series of the adjusted version of Brightness Temperature B, average it by a window of 100 records, and subtract Brightness Temperature A, averaged in the same way. The result shows the remaining discrepancy of the two temperature series. The goal is, to reduce this discrepancy to noise by applying simple adjustments.
The following two calibration steps lead to an improved consistency:
Step one: Subtract an appropriate biquadratic parabola for a time interval.
Step two: Subtract an appropriate exponential function for a time interval:

The applied functions aren't the best possible, but they may be sufficient to show the desired effect.

Discussion: A quadratic parabola instead of a biquadratic one didn't work. But using squared sine or cosine instead of the parabola may yield similar good results.
The exponential function is adjusted to the averaging window of 100 records; for other averaging windows either its parameters need to be adjusted to the way of averaging, or an appropriate exponential function itself has to be averaged over the window.

Interpretation: The Stefan–Boltzmann law is a temperature-related physical law that contains a 4th power. There might be a connection to the biquadratic parabola.
Newton's law of cooling follows an exponential law. There might take place a heat transfer, which explains the exponential function.

At the right wing of the central part of the Sol 89 Brightness B time series there are still discrepancies, which need to be explained in a different way. I didn't yet find an evident coincidence with other Sol 89 causes, like masthead movements applicable to UV.

[Gerald]

The REMS data don't comprise the same observation period each Sol.
So, here a comparison of 3000 seconds of Sol 82 with according 3000 seconds of Sol 89:


The Masthead wasn't in motion, and up to 0.01 degrees at the same pointing during both observation periods:
MASTHEAD_AZIMUTH: -179,00 degrees, MASTHEAD_ELEVATION: 43,00 (Sol 82) resp. 43,01 (Sol 89) degrees.
SOLAR_AZIMUTH and SOLAR_ELEVATION, relative to the rover coordinate system, were the only obviously changing parameters.
This is a hint, that the displaced peeks (Sol 82: 15:29 - 15:41 LMST, Sol 89: 15:43 - 15:52 LMST) should have been induced by the sun position relative to the rover.
The peak isn't there (14:00 - 16:00 LMST) in Sol 77 REMS RDR data. At that Sol the Masthead was at a different pointing. This indicates, that the peak was caused by the shadow of the Masthead.
At Sol 72 the peak occured (14:14 - 14:20 LMST), with Masthead pointing the same as Sol 82.

May be, Joe will be able to add more evidence by looking from Boom 1 towards the sun in a 3d simulation.

I didn't find any further obvious kind of systematic discrepancy between Sol 89 Brightness Temperatures A and B.

[Gerald]

A summarized list of ChemCam targets of the first 90 sols, calibration targets excluded:
 msl_ccam_obs_targets_noCal_Sols_10_88.txt ( 1K ) Number of downloads: 739

It consists of target/sol-pairs in a syntax usual for csv files; it's an excerpt of the file msl_ccam_obs.csv in this PDS subdirectory.
Images of targets and of spectra can be found in this PDS subdirectory.

[Gerald]

This is a "blink" gif between some raw Sol 88 ChemCam image and its processed PDS version, converted from a tif in this PDS subdirectory:

Link to the GIF.

[Gerald]

A new summarized list of ChemCam targets, first 180 sols; calibration target, sky and drill head observations have been removed:
 msl_ccam_observations_Sol_Target_TargetType_summary__sols_10_to_176.txt ( 3.52K ) Number of downloads: 711

This time it consists of target/target type/sol triples, again as comma separated textfile, ready for spreadsheet import.
It's again an excerpt of the file msl_ccam_obs.csv in this PDS directory, the syntax of which has been extended a bit. This makes it easier to filter for target types.

[Phil Stooke]

"MAHLI, MARDI, and Mastcam data are still undergoing PDS peer review by the PDS Imaging Node. The data are in lien resolution. When the major liens have been resolved the data will be posted on the Imaging Node web site (link below)."

You'll have to wait a wee bit longer for these goodies, however.

Phil

[mcaplinger]

You'll have to wait a wee bit longer for these goodies, however.

From http://pds-imaging.jpl.nasa.gov/ "The Imaging Node is pleased to announce the release of Mastcam, MAHLI, and MARDI EDRs and RDRs for Sols 0-179."

http://pds-imaging.jpl.nasa.gov/data/msl/M...R_RDR_DPSIS.PDF would be a good document to read first.

[mcaplinger]

http://pds-imaging.jpl.nasa.gov/data/msl/M...R_RDR_DPSIS.PDF would be a good document to read first.

BTW, the simplest way to use the archive products would be to stick with the EDRs, which for all JPEG compressed images (the vast majority) just consist of the JPEG data as we received it from Mars, with a simple header containing critical metadata at the beginning. I'd use only that metadata to get the SCLK, exposure time, filter, and other image parameters (ignoring the detached label), and continue to use the SPICE data for all the pointing information. IMHO getting the other metadata from the MMM archive products isn't going to be any better than using SPICE, but will likely be harder, especially if you've already written SPICE code.

Depending on what you're trying to do, doing your own processing might be as easy as using the RDRs, but that's a determination you have to make for yourself.
All of the information we use to make RDRs should be on the volumes.

[CosmicRocker]

.... The data are in lien resolution. ...

Phil: What does that mean?

[mcaplinger]

What does [the data are in lien resolution] mean?

It's legal jargon that PDS uses (inappropriately, IMHO) to indicate that there are issues associated with the dataset that they are awaiting correction of. In this instance I think these were all addressed in the volume errata files. Since they put the datasets online it doesn't really matter.

[Ant103]

Yay ! Finally the Mastcam and MaHLI images released in the PDS .

BUT, I have a big problem now. I'm using ImageJ and the PDS Reader plugin to open the IMG files in order to convert them into png files. This works for MER files, and Phoenix files. But with MSL, no. I have a message saying me : "This doesn't appear to be a PDS file". So, I do not know what else to do, because I'm on Mac OS 10.5, and there is no support from developpers for this -only 6 years old- OS. Many years ago, I tried a Gimp plugin to open IMG files, it did worked on OS 10.3 (and also Windows), but not on OS 10.5. For IMG2PNG (developped by Björn Jonsson), I can forget it, this is an executable, and I will not run any virtual machine in order to run it (by guessing it's even work). And dual boot, no, be serious! (I don't even have the space on my hard drive to do that, and the Windows installer).
Upgrade to OS 10.7 is not in my projects for now. I will certainly upgrade to OS 10.6 because I just want to run the latest versions of both Gimp, Blender, Inkscape and Hugin.

So, what can I do to open these precious MSL IMG files ?

[mcaplinger]

So, what can I do to open these precious MSL IMG files ?

The only program these were tested with, AFAIK, was NASAView: http://pds.jpl.nasa.gov/tools/release/software_download.cfm

I had nothing to do with the software development, but I'll try to look at this when I get a chance. But any solution I come up with will likely be based on the Python Imaging Library.

[mcaplinger]

But with MSL, no. I have a message saying me : "This doesn't appear to be a PDS file".

These files have detached labels; the IMG file has no label information. I don't know how the ImageJ plugin works, but possibly you need to open the LBL file rather than the image file?

For example, I just looked at http://pds-imaging.jpl.nasa.gov/data/msl/M...959E02_DRCL.IMG and LBL and this is a normal, 8-bit, band sequential RGB image with 1646 samples and 1198 rows (don't ask me where those image dimensions came from, something to do with removing some of the dark pixel columns or something? I had nothing to do with it

[Ant103]

This is weird. I can't open also Navcam IMG files with my plugin. But I thought it was the same Navcam as on MER. And so they could be open the same way. Can't tell what seems to be the problem…