Multiply overlapping Mars imagery, Looking for 5 or more images of same spot Options

[elakdawalla]

A reader (Bill Green, a retired JPL guy who helped develop MIPL) alerted me to a recent article in IEEE Computer, "Aerial Computer Vision for a 3D Virtual Habitat" (subscription only, I'm afraid) that describes a method that they refer to as the "high overlap paradigm" for developing extremely high-resolution digital elevation models by using many aerial photos from many look angles. I contacted the paper's first author, Franz Leberl, to see if he was interested in attempting to apply his method to images from Mars, and the answer was: Yes, he's very interested. So my question is, where on Mars do we have sets of highly overlapping images? Here's what Leberl said he needed:

For a fully automated process that associates a surface elevation with each pixel, we need 5 or so images of each terrain point. If it is more, that is even better. If you have a focal length and location of the principal point, that would be helpful also. We do not need any data about the camera positions; but if you had such data, we would want to compare our results to those you have. And if you also had previous results in the form of a digital surface model created by traditional two-image stereo, we could use this as well for an analysis and comparison.

Can the hivemind of UMSF help me connect Leberl with the data he would need to make some seriously detailed DTMs?

[nprev]

I bet that the Spirit & Oppy landing sites probably are by now the most imaged places on the planet @ hi-res. Not the most interesting locales from orbit to be sure, but they could serve as test/validation cases for the method.

[djellison]

That was going to be my response - Spirit's landing site has been images, I think, about 10 times.

[Bjorn Jonsson]

One possible problem though:

developing extremely high-resolution digital elevation models by using many aerial photos from many look angles

If most/all of the images were obtained with the spacecraft directly (or close to directly) above the landing site the viewing geometry might not be different enough from image to image. For traditional stereo I have found that as an absolute minimum the angles must differ by ~5 degrees with ~30 degrees being optimal for the software I'm using (example: subspacecraft longitude 0 degrees in one image and 30 degrees in another one). The lighting probably needs to be similar as well.

[PDP8E]

one of the best off-nadir shots from MRO has to Heimdall Crater. The Hirise site has 4 other images of Heimdall and I am sure MOC and CTX have coverage of Heimdall.

[nprev]

Hmm. MRO got just the one image of Phoenix' descent, right? Too bad there wasn't another taken a couple of seconds later. Lots of relative motion/look angle change happening there. I wonder if this method could generate a really detailed look at the chute, which would presumably be useful for future chute engineering.

[djellison]

Because the landing site HiRISE images are targeted, they're often off-nadir. End result - you get quite a range of viewing angles.

[algorimancer]

I would think that rover-level imagery would be ideal, especially for places like the Columbia Hills and Victoria crater -- lots of images from lots of perspectives.

[centsworth_II]

I would think that rover-level imagery would be ideal...

The the study linked in the opening post is about getting the information from aerial views. So the question here is how to use this method with orbital images of Mars rather than how to get the best elevation data for places the rovers have visited.

[tanjent]

Without having access to the article, couldn't one or more views from a high point be combined with the satellite views? IIRC this was the basis for revised estimates on the height of McCool hill and other topographic features from Spirit's vicinity. I guess the ability to factor in information from the horizon lines might be considered cheating in the problem they set out to solve.

[centsworth_II]

...I guess the ability to factor in information from the horizon lines might be considered cheating in the problem they set out to solve.

I wouldn't call it cheating, but I think the objective of using the method is to get 3D images of as much of the martian surface as possible. The rover-visited locals are a drop in that bucket, area-wise. May be good for ground truth testing of the method though.

[djellison]

It would make far more sense to use the orbital imagery to produce the best DEM possible, THEN use surface data to verify the accuracy of that DEM and thus hopefully the reliability of the technique.

[Pertinax]

A bit more cost efficient as well!

-- Pertinax

[tharrison]

CTX images the Spirit and Opportunity landing sites about once a month, baring unforeseen complications like safe mode, so there are about 30 overlapping images of each of them in the PDS at the moment. We also have about 800 sites that we monitor at a frequency from weekly to every few months for things like changes in surface features (i.e. dust devil tracks), gullies, dust-raising events, etc., so if a site that is more diverse topographically than the MER landing sites would be better for Leberl's method, I can look for one.

EDIT: Completely forgot about MSL—there are a bunch of overlapping CTX images of the candidate MSL landing sites, particularly of Gale.

If most/all of the images were obtained with the spacecraft directly (or close to directly) above the landing site the viewing geometry might not be different enough from image to image. For traditional stereo I have found that as an absolute minimum the angles must differ by ~5 degrees with ~30 degrees being optimal for the software I'm using (example: subspacecraft longitude 0 degrees in one image and 30 degrees in another one). The lighting probably needs to be similar as well.

For CTX, we've found that the minimum roll angle difference needs to be 9° for anaglyphs, but for DEMs, the larger the roll is, the better. HiRISE prefers angles >15°. When we acquire stereopairs, we try to get both images in the pair within ~4 months of each other (or at the same time the following Mars year) to make sure the illumination conditions are similar. Of course, both the illumination conditions and angle difference needed for optimal DEMs depends on the topography of that area.

Hmm. MRO got just the one image of Phoenix' descent, right? Too bad there wasn't another taken a couple of seconds later. Lots of relative motion/look angle change happening there. I wonder if this method could generate a really detailed look at the chute, which would presumably be useful for future chute engineering.

With CTX we can't command images less than 15 seconds apart (I'm not sure if this is the same for HiRISE, but I think they require more time between images due to the sheer file size). Taking an image of the same place on the same orbit would also require some special maneuvering of the spacecraft, because we typically only slew rather than pitch and yaw (although every so often MRO is pitched for some CRISM observations).

[Zack Moratto]

I wish this paper had more content. Yet that doesn't take away the multiview stereo coolness.

Does Leberl know that on Mars we don't use frame cameras? He's response seems to be that he expecting a simple pinhole model.