MSL Descent and Landing - Remastering MARDI imagery Options

[AdrianC]

Hello everybody.
I have been reading this forum for quite a while now (8 months+) but never got the nerve and data to actually contribute.
Let me start off by asking everybody two questions.

"What do you think about a high quality remasterization of MSL's MARDI descent and landing imagery?"

"Is it worth the effort having a higher resolution, higher frame rate version of the descent?"

Obviously i'm talking about a high quality remasterization (details will follow). My background is in digital visual effects (VFX) to be more specific 10 years in matchmoving, this involves advanced photogrammetry knowledge, and digital compositing.

Final results will be:
-a minimum horizontal resolution of 2.6K
-25 to 60 fps (TBD)
-derived telemetry overplayed on imagery and current stage (ex. heat shield separation, engines)
-original radio comms as sound.

I am aware of the fact that such attempts already exist (here). This is just a mere optic flow interpolation which dose not do any justice to the mission.

The simplified roadmap is:
-I will use HI-Rise data to create a digital elevation map then convert to a polygonal mesh
-Compute absolute position of each MARDI frame
-Camera project on the ground geometry a series of MARDI frames. Each temporal sample will contribute to a higher resolution image by stacking it on the ground geometry.
-Final rendering will be based ONLY on MARDI data. Interpolation between camera positions will be as physically accurate as possible.

This is no easy task, my estimate is about 40-100 hours of work minimum and lots of data missing.There will be some programming, some small tool developement. But since next month I will be out of a job (corporate financial problems) I thought I might ask you guys about the possible interest of this project.

PS If i posted in the wrong section, I ask the moderators to kindly help me.

[sittingduck]

You've got me real excited! As long as you avoid the one mistake all previous "enhanced" MARDI videos have made: wildly stretching contrast and throwing the colors out the window. If you need any 3d modelling done - of the heat shield for example - don't hesitate to ask me, I've got some experience with spaceflight hardware modelling and rendering.

Good luck!

[mcaplinger]

This is no easy task, my estimate is about 40-100 hours of work minimum and lots of data missing.

At MSSS I've considered doing something similar, but I think this is an underestimate of the level of effort required, and IMHO perhaps past the point of diminishing returns. We have a better-interpolated version of the sequence than anything I've seen on youtube (but still done with optical flow), but I'm not sure when or if it will be released.

You will also find that the terminal descent part of the sequence, with its dust and flying rocks, is a completely different problem than the rest of the sequence (and managing the heat shield is also pretty different.)

[Zeke4ther]

Given your unique background, I would say, give it your best shot. You might surprise everyone.
At the very least you would have tried.

[xflare]

We have a better-interpolated version of the sequence than anything I've seen on youtube (but still done with optical flow), but I'm not sure when or if it will be released.

oh? Why's that?

[mcaplinger]

Many reasons, but mostly because it's old news at this point so there's little urgency, the approval process for us to release something is not completely straightforward, and there's some hope that the product could be improved -- for one thing, by inclusion of unreleased EDL telemetry as a graphic overlay or something.

Recall that the final Huygens descent movie wasn't released for something like 5 years after the event.

[MahFL]

Also to the general public it will look exactly the same...

[AdrianC]

Thanks for the feedback everyone.

About the estimated work time, yes it might be completely erroneous. There are lots of bumps I have to deal with, mainly with the data formats. I will get right back here with a complete set of questions regarding PDS IMG format (I'm reading the documentation ).
In the mean time if anybody has a favorite reader / converter for PDS IMG imges please let me know.

Also I started running a few tests on how to deal with the dust we see during the terminal phase. Nothing concise yet. As mcaplinger stated this is a troublesome portion of the imagery. My first take - I will try not to use the dusty image sets and and use data from previous frames. This violates the commitment of using MARDI imagery only, because the dust must be added back somehow.

This is an animation of 4 frames, on the left the difference between a clean frame minus the luminance of each dusty frame. Result an enhanced image of the dust blown by the lander.
(Please note this was done on a compressed data set (old set of Jpegs), no lens distortion removed, no color look up table, no vignetting removed.)

@MahFL you are right. I'am well used to this. In my world all alterations done to sequence of images must be invisible. There are a lot of movies out there that have hundreds of hours of work, invisible work, mostly on removing unwanted objects left there during production just because someone forgot. Nothing glamorous about that.

Attached thumbnail(s)

 

[Lightning]

Maybe you should wait for the uncompressed images to be downloaded to Earth ?

[mcaplinger]

That happened quite a while ago, though I guess I'm not certain they've been released to PDS (but I think so.)

[DeanM]

Adrian: "..image of the dust blown by the lander.."

MSL was the lander.

The dust was blown by the 'crasher'!

Dean

PS: And I very much like your left-panel sequence.

The two near-circular areas of excavated debris are interesting.
Could the one on the right be apparently larger than that on the left of screen because of camera parallax or because the SkyCrane was initially tilted to the right?!

[AdrianC]

Sorry. Obviously I meant SkyCrane.
The patterns in which the dust is blasted are very interesting. Yes the one on the left is smaller, either the engines where correcting attitude or the terrain is significantly sloped. Or both.
Once I am done registering all the images thing will get clearer.

[Lightning]

That happened quite a while ago, though I guess I'm not certain they've been released to PDS (but I think so.)

According to the latest data (up to my knowledge) visible here http://pds-imaging.jpl.nasa.gov/data/msl/M...001E01_DRCL.LBL release in may 2013, the parameter INST_CMPRS_NAME is still set to JPEG DISCRETE COSINE TRANSFORM (DCT).

So this still are not raw images.

[mcaplinger]

According to the latest data (up to my knowledge) ...
So this still are not raw images.

The latest is release 3 for sols 180-269: http://pds-imaging.jpl.nasa.gov/data/msl/M...R/SURFACE/0000/

Looks like lossless to me.

[Lightning]

Yes, indeed. Looks like the raw descent images are then still not available, maybe on Earth but at least in PDS.