MARDI images and videos Options

[Nix]

Do we know what Image data Curiosity will be attempting to upload on the next flyover pass? More hazcams only, or do weexpect to have the mast up and taking pics and ready to send?

in 1 or 2 days?

[Eluchil]

There is a low bandwidth ODY pass that will happen in about 10 minutes that may have a couple new HazCams. Does anyone know when the first MARDI images are expected to be downlinked? I saw that 18 had been assigned a (relatively) high priority, but the JPL Ustream was just saying that the current pass would have only a HazCam or two.

[Herobrine]

@yogi
Yesterday, when I went searching to see if better data existed, I first came across the real raw data on some University's site. When trying to find the site again later in the day, I ended up at PDS instead (I think I actually bounced off a UMSF post in my search results to get to it) and the data that's in PDS appeared to be the same as what I'd been working with from that university's site so I started using PDS instead.
Sorry if I got your hopes up that there might be a secret, better set of data out there. I didn't try to work with the EDR data for MARDI, though; I've used RDR data (the C00_DRXX.IMG files, specifically) for everything I've been doing so far. I think that's basically the original, uncompressed (8-bit?) data debayered and slightly processed (dark level correction or whatever) into separate R/G/B bands of 12-bit data stored in 16 bit samples. I could be totally wrong, but that's what it seems like to me. The C00_DRXX.IMG files seem to show the same "mosaicing" that is seen in released color JPG data products, which would seem to suggest the debayering is being done in a less-than-optimal way (but what do I know).

I've been reading those files in, bringing them down to 8-bit-per-channel using a sort of scaling aimed at distributing the (effectively 12-bit) range of values into the 0-255 range in a way that makes the resulting RGB histograms as flat as possible (there's probably a great name for that that smart people know), and dumping them to PNGs.
The way I've been doing the scaling has been more for me to easily see the content of the image than anything else. It often yields very crisp, aesthetically pleasing images, but also, depending on the content of the picture, often yields images with excessive color contrast.
If anyone's curious, here are a few different frames dumped from the process I just described, along with the corresponding NASA color JPG releases (referred to as "raw images" on the MSL site):
http://mc.herobrinesarmy.com/31.png | http://mars.jpl.nasa.gov/msl-raw-images/ms...0031E1_DXXX.jpg
http://mc.herobrinesarmy.com/350.png | http://mars.jpl.nasa.gov/msl-raw-images/ms...0350E1_DXXX.jpg
http://mc.herobrinesarmy.com/561.png | http://mars.jpl.nasa.gov/msl-raw-images/ms...0561E1_DXXX.jpg

@yogi You mentioned the EDR data products seeming to be like JPEGs with a header. I haven't even looked at the EDR stuff yet, but I can say that I don't notice compression artefacts in the C00_DRXX.IMG RDR data products. Here's a zoomed-in GIF switching between the color JPG from NASA and the PNG I generated from the C00_DRXX.IMG RDR data product. http://i.imgur.com/G7k3tQZ.gif There are severe, obvious compression artefacts in the JPG from NASA but none that I notice in the one derived from the C00_DRXX.IMG data (though it does still have that Bayer mosaic noticeable in some places).
To show the difference between the lossless C00 and the lossy E01 data products, I ran both of them (frame 357) through my little program that dumps a PNG version of them.
E01 C00
E01 C00

@James That's awesome. I'm so envious of those of you who have known for some time where to find the real raw data. It's a shame I only found UMSF about a year ago. I think there's a Pokemon meme that describes how I feel having only just found out about this mountain of data available for playing-with. NASA tricked me a long time ago with their "raw images" site for MSL, which has nothing but JPEGs on it. It led me to think that was the best image data publicly available, so all these years, I never bothered checking PDS for anything better.
I'm going to be very busy for the foreseeable future, just combing through all of the different types of data available for MSL and coming up with things to do with it all, for my own amusement.

That reminds me; that university site I mentioned before had a pretty nice interface for browsing all of the data products available. I was pleasantly surprised to see all sorts of great spatial data available with each HAZCAM data product. That site had handy visualizations for a lot of that data. I made a GIF of those yesterday to show someone how much data there was for each image. http://i.imgur.com/U7hGQsM.gif
Come to think of it, that university's site seemed to be a lot faster for downloading data products than PDS has been. Let me see if I can find it again...
( 2 minutes later... )
It was https://an.rsl.wustl.edu/msl/mslbrowser/br2.aspx, which I now see has been mentioned on UMSF multiple times in the past.
I'm like a frozen cave man, reanimated in modern times, discovering and being fascinated by things everyone else has been using for ages.

On that note, I think I'll pipe down for a while and continue figuring out where to begin with all of this fantastic data. That, and I apparently need to read through all of these MSL threads so I can stop being Encino Man.

[yogi]

@yogi
Yesterday, when I went searching to see if better data existed, I first came across the real raw data on some University's site. When trying to find the site again later in the day, I ended up at PDS instead (I think I actually bounced off a UMSF post in my search results to get to it) and the data that's in PDS appeared to be the same as what I'd been working with from that university's site so I started using PDS instead.
Sorry if I got your hopes up that there might be a secret, better set of data out there. I didn't try to work with the EDR data for MARDI, though; I've used RDR data (the C00_DRXX.IMG files, specifically) for everything I've been doing so far. I think that's basically the original, uncompressed (8-bit?) data debayered and slightly processed (dark level correction or whatever) into separate R/G/B bands of 12-bit data stored in 16 bit samples. I could be totally wrong, but that's what it seems like to me. The C00_DRXX.IMG files seem to show the same "mosaicing" that is seen in released color JPG data products, which would seem to suggest the debayering is being done in a less-than-optimal way (but what do I know).

Ah I see, thanks for the clarification. @Deimos already pointed out in comment #356 that the C* images were a lossless format; I admit I never ended up checking them out in detail myself (I only logged at the other EDR files which were JPG images that I "decoded" just by stripping off the 64byte header). From what I recall (I may be wrong), the C* EDR files actually were containers that decoded to multiple IMGs each.