MARDI images and videos |
MARDI images and videos |
Jul 9 2015, 04:59 PM
Post
#361
|
|||||
Member Group: Members Posts: 244 Joined: 2-March 15 Member No.: 7408 |
@yogi
Yesterday, when I went searching to see if better data existed, I first came across the real raw data on some University's site. When trying to find the site again later in the day, I ended up at PDS instead (I think I actually bounced off a UMSF post in my search results to get to it) and the data that's in PDS appeared to be the same as what I'd been working with from that university's site so I started using PDS instead. Sorry if I got your hopes up that there might be a secret, better set of data out there. I didn't try to work with the EDR data for MARDI, though; I've used RDR data (the C00_DRXX.IMG files, specifically) for everything I've been doing so far. I think that's basically the original, uncompressed (8-bit?) data debayered and slightly processed (dark level correction or whatever) into separate R/G/B bands of 12-bit data stored in 16 bit samples. I could be totally wrong, but that's what it seems like to me. The C00_DRXX.IMG files seem to show the same "mosaicing" that is seen in released color JPG data products, which would seem to suggest the debayering is being done in a less-than-optimal way (but what do I know). I've been reading those files in, bringing them down to 8-bit-per-channel using a sort of scaling aimed at distributing the (effectively 12-bit) range of values into the 0-255 range in a way that makes the resulting RGB histograms as flat as possible (there's probably a great name for that that smart people know), and dumping them to PNGs. The way I've been doing the scaling has been more for me to easily see the content of the image than anything else. It often yields very crisp, aesthetically pleasing images, but also, depending on the content of the picture, often yields images with excessive color contrast. If anyone's curious, here are a few different frames dumped from the process I just described, along with the corresponding NASA color JPG releases (referred to as "raw images" on the MSL site): http://mc.herobrinesarmy.com/31.png | http://mars.jpl.nasa.gov/msl-raw-images/ms...0031E1_DXXX.jpg http://mc.herobrinesarmy.com/350.png | http://mars.jpl.nasa.gov/msl-raw-images/ms...0350E1_DXXX.jpg http://mc.herobrinesarmy.com/561.png | http://mars.jpl.nasa.gov/msl-raw-images/ms...0561E1_DXXX.jpg @yogi You mentioned the EDR data products seeming to be like JPEGs with a header. I haven't even looked at the EDR stuff yet, but I can say that I don't notice compression artefacts in the C00_DRXX.IMG RDR data products. Here's a zoomed-in GIF switching between the color JPG from NASA and the PNG I generated from the C00_DRXX.IMG RDR data product. http://i.imgur.com/G7k3tQZ.gif There are severe, obvious compression artefacts in the JPG from NASA but none that I notice in the one derived from the C00_DRXX.IMG data (though it does still have that Bayer mosaic noticeable in some places). To show the difference between the lossless C00 and the lossy E01 data products, I ran both of them (frame 357) through my little program that dumps a PNG version of them. E01 E01 @James That's awesome. I'm so envious of those of you who have known for some time where to find the real raw data. It's a shame I only found UMSF about a year ago. I think there's a Pokemon meme that describes how I feel having only just found out about this mountain of data available for playing-with. NASA tricked me a long time ago with their "raw images" site for MSL, which has nothing but JPEGs on it. It led me to think that was the best image data publicly available, so all these years, I never bothered checking PDS for anything better. I'm going to be very busy for the foreseeable future, just combing through all of the different types of data available for MSL and coming up with things to do with it all, for my own amusement. That reminds me; that university site I mentioned before had a pretty nice interface for browsing all of the data products available. I was pleasantly surprised to see all sorts of great spatial data available with each HAZCAM data product. That site had handy visualizations for a lot of that data. I made a GIF of those yesterday to show someone how much data there was for each image. http://i.imgur.com/U7hGQsM.gif Come to think of it, that university's site seemed to be a lot faster for downloading data products than PDS has been. Let me see if I can find it again... ( 2 minutes later... ) It was https://an.rsl.wustl.edu/msl/mslbrowser/br2.aspx, which I now see has been mentioned on UMSF multiple times in the past. I'm like a frozen cave man, reanimated in modern times, discovering and being fascinated by things everyone else has been using for ages. On that note, I think I'll pipe down for a while and continue figuring out where to begin with all of this fantastic data. That, and I apparently need to read through all of these MSL threads so I can stop being Encino Man. |
||||
|
|||||
Jul 9 2015, 05:28 PM
Post
#362
|
|
Junior Member Group: Members Posts: 53 Joined: 15-July 09 Member No.: 4867 |
http://i.imgur.com/xIhWO7b.jpg (contrast exaggerated, obviously, though not intentionally; it's just a typical result of the scaling that's done to get the data back to 8-bit-per-channel in my processing) I'm a bit surprised that even with the "raw" lossless images, the Bayer pattern is still visible. In theory, this should not be the case isn't it ?
|
|
|
Jul 9 2015, 06:57 PM
Post
#363
|
|
Senior Member Group: Members Posts: 2542 Joined: 13-September 05 Member No.: 497 |
The MARDI RDR products were color-interpolated using a 5x5 interpolation kernel but there are some slight remaining artifacts that are more visible with a harsh contrast stretch and spatial filter like this appears to be. If you don't like it, go back to the original lossless EDRs, decompress them, and apply your own Bayer interpolation.
I think that the lossless versions of the MARDI descent images are all at http://pds-imaging.jpl.nasa.gov/data/msl/MSLMRD_0003/DATA/ but I'm not 100% certain. Look at the index files on the PDS volumes to see what's where. It's complicated, obviously, but we went to a lot of effort to document what the data format was and where to find it. -------------------- Disclaimer: This post is based on public information only. Any opinions are my own.
|
|
|
Jul 9 2015, 06:59 PM
Post
#364
|
|
Director of Galilean Photography Group: Members Posts: 896 Joined: 15-July 04 From: Austin, TX Member No.: 93 |
"Raw" means before the de-Bayering algorithms are applied - the raw data straight from the imaging sensor, warts and all.There are different versions, so having the raw data straight from the detectors allows for alternate processing.
http://www.stark-labs.com/craig/resources/...ayering_API.pdf has some details. Some of the PDS data might have minimal processing of darks/flats applied, I don't know. -------------------- Space Enthusiast Richard Hendricks
-- "The engineers, as usual, made a tremendous fuss. Again as usual, they did the job in half the time they had dismissed as being absolutely impossible." --Rescue Party, Arthur C Clarke Mother Nature is the final inspector of all quality. |
|
|
Jul 9 2015, 07:16 PM
Post
#365
|
|
Senior Member Group: Members Posts: 2542 Joined: 13-September 05 Member No.: 497 |
Some of the PDS data might have minimal processing of darks/flats applied, I don't know. The EDR is the only "raw" data for MARDI. All of the RDR products have had color interpolation performed. There are four flavors of RDR. All are decompressed and radiometrically calibrated. The four flavors are the permutations of color corrected and geometrically linearized. See http://an.rsl.wustl.edu/mer/help/Content/A...data%20sets.htm -------------------- Disclaimer: This post is based on public information only. Any opinions are my own.
|
|
|
Jul 9 2015, 07:36 PM
Post
#366
|
|
Member Group: Members Posts: 244 Joined: 2-March 15 Member No.: 7408 |
"Raw" means before the de-Bayering algorithms are applied - the raw data straight from the imaging sensor, warts and all. There are different versions, so having the raw data straight from the detectors allows for alternate processing. If we want to get real technical, there are no raw data products because, if I recall correctly, MARDI has a 12-bit analog-digital converter, but those 12 bits are (irreversibly) encoded non-linearly to 8 bit samples before the data is ever stored anywhere. |
|
|
Jul 10 2015, 11:28 AM
Post
#367
|
|
Junior Member Group: Members Posts: 30 Joined: 8-September 14 From: London, UK Member No.: 7254 |
@yogi Yesterday, when I went searching to see if better data existed, I first came across the real raw data on some University's site. When trying to find the site again later in the day, I ended up at PDS instead (I think I actually bounced off a UMSF post in my search results to get to it) and the data that's in PDS appeared to be the same as what I'd been working with from that university's site so I started using PDS instead. Sorry if I got your hopes up that there might be a secret, better set of data out there. I didn't try to work with the EDR data for MARDI, though; I've used RDR data (the C00_DRXX.IMG files, specifically) for everything I've been doing so far. I think that's basically the original, uncompressed (8-bit?) data debayered and slightly processed (dark level correction or whatever) into separate R/G/B bands of 12-bit data stored in 16 bit samples. I could be totally wrong, but that's what it seems like to me. The C00_DRXX.IMG files seem to show the same "mosaicing" that is seen in released color JPG data products, which would seem to suggest the debayering is being done in a less-than-optimal way (but what do I know). Ah I see, thanks for the clarification. @Deimos already pointed out in comment #356 that the C* images were a lossless format; I admit I never ended up checking them out in detail myself (I only logged at the other EDR files which were JPG images that I "decoded" just by stripping off the 64byte header). From what I recall (I may be wrong), the C* EDR files actually were containers that decoded to multiple IMGs each. |
|
|
Feb 17 2021, 07:37 PM
Post
#368
|
|
Newbie Group: Members Posts: 13 Joined: 21-December 19 Member No.: 8729 |
Hi guys,
after a long time trying to register and just reading your awsome stuff and learning, I am posting my own attempts to work with data. As we are all waiting for Perseverance to land, I was experimenting with the MARDI data. This is my try with the complete EDL MARDI footage. I upscaled it, made little color corrections and sharpened it. A lot of others have tried this. https://youtu.be/5j8t3aGzsgI This is the detail of the heatshield impact. I upscaled, reprojected, aligned the images, made some color corrections and sharpened them a little. https://youtu.be/i3LHppg48kI Of this I am a little proud. It took all three weeks to figure it out. This is a 8K VR 360 version from 8km above the surface (the beginning of the sequence without the first 4 frames). I sharpened the images, filtered out the color noise and stitched all the views together and after that I aligned around 390 images till the impact. On youtube, you can use the + key to zoom in on 360 videos. https://youtu.be/sXI6rQ7olEI I would love to hear the feedback from you guys. And thanks for everything here. It teached me a lot and I find it amazing, that such a comunity exists. |
|
|
Feb 17 2021, 11:53 PM
Post
#369
|
|
Member Group: Members Posts: 819 Joined: 3-June 04 From: Brittany, France Member No.: 79 |
I love your videos MarT! Particularly the last one which is really awesome, I feel like it gives a new perspective on the EDL.
-------------------- |
|
|
Feb 18 2021, 02:09 AM
Post
#370
|
|
Member Group: Members Posts: 447 Joined: 1-July 05 From: New York City Member No.: 424 |
Great work MarT. Looking forward to what you can do with the Mars2020 images.
|
|
|
Feb 18 2021, 02:41 AM
Post
#371
|
|
Senior Member Group: Admin Posts: 4763 Joined: 15-March 05 From: Glendale, AZ Member No.: 197 |
Amazing work Mart. That 8K video is something I've always thought about but been to lazy to try myself. Great job!
-------------------- If Occam had heard my theory, things would be very different now.
|
|
|
Lo-Fi Version | Time is now: 27th September 2024 - 08:32 AM |
RULES AND GUIDELINES Please read the Forum Rules and Guidelines before posting. IMAGE COPYRIGHT |
OPINIONS AND MODERATION Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators. |
SUPPORT THE FORUM Unmannedspaceflight.com is funded by the Planetary Society. Please consider supporting our work and many other projects by donating to the Society or becoming a member. |