IPB

Welcome Guest ( Log In | Register )

High-Res DEMs from single HiRISE images, First results of new "Shape from Shading" algorithm
Nirgal
post Jan 16 2010, 03:30 PM
Post #1


Member
***

Group: Members
Posts: 713
Joined: 30-March 05
Member No.: 223



Hi all,

Here the long overdue continuation of the "Alien Landscapes" series. This time based on 3D
DEMs generated with "Shape from Shading" from single HiRISE images.
Enjoy smile.gif

Click on Images for larger version.

Detail views from PSP_002172_1410 (large gully system)





Detail view of Gullies from PSP_001376_1675





Detail of gully system in PSP_002022_1455



Dune Views from PSP_004339_1890





Detail from PSP_001834_1605





Here is some background info on the making of the images:

"Shape from Shading" (SFS) i.e. the possibility to extract shape information from a single image has always been a fascinating topic for me.
Now I found the time to implement a prototype for a new SFS algorithm based on some ideas that I've been thinking about for a long time.
The problem with existing SFS approaches (see here for a survey is that they either tend to over-smooth the details (due to the regularization constraint) or suffer from excessive noise in the high-frequency components of the reconstructed surface. Another problem is the large demand on CPU ressources which would make them very challenging to apply to large scale input data, such as HiRISE orbiter images.

So for a long time I was rather sceptical as to the potential of SFS and it was my impression that Methods based on multiple images (stereo) must be far superior to single-image SFS.

However, after a long time of experimenting, combining existing approaches with some new ideas, I got the following quite promising first results that I'd like to share:

All of the images were generated from a single HiRISE image (no depth information was used from stereo or laser altimeter data).
Also, no texturing or additional coloring/shading was applied when rendering the surface.
Every detail visible is real 3D down to the pixel-level...
For rendering I used a very simple model based on lambertian reflection with gouraud shading.

The resolution of the images is still moderate: that is downsampled details crops in the order of 0.5-1 Megapixels.
However, despite the heavy math machinery that drives the core of the algoritm (several systems of equations with millions of unknowns)
the processing time is still moderate (about 15 Minutes per med-res image, using about 2 Gigs main mem) such that the application to full-res HiRISE images should be possible smile.gif

The following image shows an example to illustrate the general principle (click to enlarge).


On the left hand side the 2D input image (simple noisy JPEG from the Web with unknwon light source direction). On the right hand side shows the recovered 3D surface re-lighted under a different light source direction.
Note that one problem of the current implementation of the algorithm is it's vulnerability to notable distortions in the low frequency components (i.e. large scale variations) of the generated surface. However I'm confident that this can be overcome by an improved version or by adding the large-scale depth information from stereo-based DEMs or altimeter data (MOLA) where available.
Go to the top of the page
 
+Quote Post
 
Start new topic
Replies
Bjorn Jonsson
post Jan 16 2010, 09:09 PM
Post #2


IMG to PNG GOD
****

Group: Moderator
Posts: 2256
Joined: 19-February 04
From: Near fire and ice
Member No.: 38



In a word:

WOW!!!

Are you draping the original image over the DEM or is the DEM itself this detailed?
Go to the top of the page
 
+Quote Post
Nirgal
post Jan 16 2010, 10:27 PM
Post #3


Member
***

Group: Members
Posts: 713
Joined: 30-March 05
Member No.: 223



QUOTE (Bjorn Jonsson @ Jan 16 2010, 10:09 PM) *
In a word:

WOW!!!

Are you draping the original image over the DEM or is the DEM itself this detailed?


There is no texture mapping or smoothing involved: every pixel is real 3D - The DEM itself is detailed down to the single pixel level.

That's the really interesting thing with this new techinque: The absence of the strong smoothing process that gives many conventional DEM-images that somewhat "washed", unnaturally soft appearance. smile.gif

(however: while it works perfectly for the higher frequency bands, there can be distortions in the lower frequency components ...
so the DEMs in their current form could probably not be used for exact inference about variations of absolute terrain heights over larger scales (that's where MOLA and conventional stereo based techniques have advantages: so it seems natural to combine the techniques) ...

P.S.: I also forgot to mention that in most images the vertical scale is exaggerated by a factor of about 1.5 to 2.
Go to the top of the page
 
+Quote Post
Bjorn Jonsson
post Jan 16 2010, 11:07 PM
Post #4


IMG to PNG GOD
****

Group: Moderator
Posts: 2256
Joined: 19-February 04
From: Near fire and ice
Member No.: 38



QUOTE (Nirgal @ Jan 16 2010, 10:27 PM) *
There is no texture mapping or smoothing involved: every pixel is real 3D - The DEM itself is detailed down to the single pixel level.

I suspected this from the first message but wanted to be sure. This makes this DEM even more amazing.

QUOTE (Nirgal @ Jan 16 2010, 10:27 PM) *
That's the really interesting thing with this new techinque: The absence of the strong smoothing process that gives many conventional DEM-images that somewhat "washed", unnaturally soft appearance. smile.gif

(however: while it works perfectly for the higher frequency bands, there can be distortions in the lower frequency components ...
so the DEMs in their current form could probably not be used for exact inference about variations of absolute terrain heights over larger scales (that's where MOLA and conventional stereo based techniques have advantages: so it seems natural to combine the techniques) ...

It would be really interesting to see DEMs for bodies like Europa and Enceladus. Also I have a global DEM of Rhea so a comparison to my results there would be interesting. As you mentioned a DEM from your software would be far more detailed than mine but low frequency variations would be less accurate (combining results from these two approaches might be interesting for big impact basins like Tirawa).
Go to the top of the page
 
+Quote Post
Nirgal
post Jan 16 2010, 11:16 PM
Post #5


Member
***

Group: Members
Posts: 713
Joined: 30-March 05
Member No.: 223



QUOTE (Bjorn Jonsson @ Jan 17 2010, 12:07 AM) *
I suspected this from the first message but wanted to be sure. This makes this DEM even more amazing.


It would be really interesting to see DEMs for bodies like Europa and Enceladus. Also I have a global DEM of Rhea so a comparison to my results there would be interesting. As you mentioned a DEM from your software would be far more detailed than mine but low frequency variations would be less accurate (combining results from these two approaches might be interesting for big impact basins like Tirawa).


Thanks Björn,

i too am interested in combing the various DEM approaches but at the moment, unfortunately lacking the time (still working on the improvement to the single-image SFS method)

here is another image (a close up of gully wall) that illustrates the high detail preservation (it's not even rendered at full resolution)

(click for larger version)


And for Stu and and ngunn: no vertical (i.e. profile) exaggeration this time :-)
Go to the top of the page
 
+Quote Post
ngunn
post Jan 17 2010, 10:18 AM
Post #6


Senior Member
****

Group: Members
Posts: 3516
Joined: 4-November 05
From: North Wales
Member No.: 542



QUOTE (Nirgal @ Jan 16 2010, 11:16 PM) *
no vertical exaggeration


Excellent! Now I can make an informed choice of footwear. smile.gif

Please excuse a very basic question. If these are made from single images how do you prevent albedo differences from masquerading as relief?
Go to the top of the page
 
+Quote Post

Posts in this topic
- Nirgal   High-Res DEMs from single HiRISE images   Jan 16 2010, 03:30 PM
- - lyford   WOAH! I want to climb those dunes..... am...   Jan 16 2010, 04:29 PM
- - mhoward   Mind-blowing.   Jan 16 2010, 05:03 PM
- - cbcnasa   The images are just amazing, fantastic job.   Jan 16 2010, 05:48 PM
- - jamescanvin   Those are just astonishing Bernhard, great work...   Jan 16 2010, 06:05 PM
- - ElkGroveDan   Amazing and beautiful stuff, as always. I don...   Jan 16 2010, 06:39 PM
- - eoincampbell   Another UMSF classic, it's truly a joy to come...   Jan 16 2010, 07:13 PM
- - PDP8E   Bernhard, This is fabulous progress report! I...   Jan 16 2010, 07:43 PM
- - djellison   O M F G   Jan 16 2010, 08:20 PM
- - Tesheiner   Unbelievable!!! I want to climb one...   Jan 16 2010, 08:36 PM
- - Bjorn Jonsson   In a word: WOW!!! Are you draping th...   Jan 16 2010, 09:09 PM
|- - Nirgal   QUOTE (Bjorn Jonsson @ Jan 16 2010, 10:09...   Jan 16 2010, 10:27 PM
|- - Stu   QUOTE (Nirgal @ Jan 16 2010, 10:27 PM) P....   Jan 16 2010, 10:29 PM
|- - Bjorn Jonsson   QUOTE (Nirgal @ Jan 16 2010, 10:27 PM) Th...   Jan 16 2010, 11:07 PM
|- - Nirgal   QUOTE (Bjorn Jonsson @ Jan 17 2010, 12:07...   Jan 16 2010, 11:16 PM
|- - ngunn   QUOTE (Nirgal @ Jan 16 2010, 11:16 PM) no...   Jan 17 2010, 10:18 AM
||- - Nirgal   QUOTE (ngunn @ Jan 17 2010, 11:18 AM) Exc...   Jan 17 2010, 12:25 PM
|- - ElkGroveDan   And for an Earthly analog here is the Emperor Face...   Jan 17 2010, 06:09 PM
|- - Nirgal   QUOTE (ElkGroveDan @ Jan 17 2010, 07:09 P...   Jan 18 2010, 01:15 PM
- - Reckless   Brilliant Bernhard As I was looking at those pic...   Jan 16 2010, 09:25 PM
- - nprev   Just freakin' amazing, is all! What immedi...   Jan 16 2010, 09:32 PM
- - ngunn   Me too. I always prefer to see landscapes without ...   Jan 16 2010, 10:49 PM
- - nprev   ...how sedimentary can you get?   Jan 16 2010, 11:26 PM
- - djellison   If you want a good test - try the Pathfinder obser...   Jan 17 2010, 03:14 PM
- - imipak   ...what everyone else has said! I know very l...   Jan 17 2010, 07:17 PM
|- - Nirgal   QUOTE (imipak @ Jan 17 2010, 08:17 PM) .....   Jan 18 2010, 10:26 AM
|- - sgendreau   QUOTE (Nirgal @ Jan 18 2010, 02:26 AM) I ...   Jan 18 2010, 04:27 PM
- - vikingmars   Bernhard, your work is "estupendo" ...   Jan 18 2010, 02:49 PM
- - climber   Bernhard, after Olivier's statement, knowing w...   Jan 18 2010, 05:20 PM
- - PDP8E   Bernhard, What could your technique do with a fair...   Jan 18 2010, 08:42 PM
- - Bjorn Jonsson   Earlier in this thread I mentioned Enceladus and E...   Jan 19 2010, 01:00 AM
|- - Nirgal   QUOTE (Bjorn Jonsson @ Jan 19 2010, 02:00...   Jan 19 2010, 03:43 PM
- - djellison   OK - yeah - do you think you'll be able to sha...   Jan 19 2010, 04:19 PM
|- - Bjorn Jonsson   QUOTE (Nirgal @ Jan 19 2010, 03:43 PM) ok...   Jan 19 2010, 05:37 PM
- - djellison   Things I want to try ( in no particular order ) T...   Jan 19 2010, 05:47 PM
|- - vikingmars   QUOTE (djellison @ Jan 19 2010, 06:47 PM)...   Jan 20 2010, 08:37 AM
- - mcaplinger   Not to be dismissive of this, but as Nirgal pointe...   Jan 19 2010, 06:23 PM
|- - Nirgal   QUOTE (mcaplinger @ Jan 19 2010, 07:23 PM...   Jan 19 2010, 09:27 PM
- - ElkGroveDan   Since we have a lot of examples of places with ver...   Jan 19 2010, 06:38 PM
- - Ant103   ASTOUNDING ! AMAZING ! You just did a WOND...   Jan 19 2010, 09:30 PM
- - Stu   You keep sharing your "pretty visualisations...   Jan 19 2010, 09:35 PM
- - Bjorn Jonsson   There are lots and lots of things I'd be inter...   Jan 19 2010, 11:05 PM
- - 4th rock from the sun   I'd VERY much like to produce a detailed DEM f...   Jan 20 2010, 10:09 AM
|- - mcaplinger   QUOTE (4th rock from the sun @ Jan 20 2010, 0...   Jan 20 2010, 02:19 PM
- - djellison   I'm not sure this would work on Radar.   Jan 20 2010, 01:56 PM
|- - 4th rock from the sun   It should work on radar, as there's the concep...   Jan 20 2010, 02:13 PM
- - imipak   *cough* Titan..? (Edit - sorry, I didn't spot ...   Jan 20 2010, 02:42 PM
- - elakdawalla   The software wouldn't work on radar images wit...   Jan 20 2010, 07:30 PM
|- - mcaplinger   QUOTE (elakdawalla @ Jan 20 2010, 11:30 A...   Jan 20 2010, 07:56 PM
- - vikingmars   Bernhard, just an idea... Because this 3D HI-Res D...   Jan 20 2010, 08:38 PM
|- - Nirgal   QUOTE (vikingmars @ Jan 20 2010, 09:38 PM...   Jan 20 2010, 09:30 PM
|- - Bjorn Jonsson   As should already be obvious I'm very interest...   Jan 21 2010, 12:56 AM
- - 4th rock from the sun   Thanks for sharing your insights into the photocli...   Jan 20 2010, 08:43 PM
- - climber   I'm wondering if you (Bernhard) can run your s...   Jan 21 2010, 10:07 PM
- - Phil Stooke   It can't be done on that image, it would have ...   Jan 22 2010, 12:32 PM
|- - Nirgal   QUOTE (Phil Stooke @ Jan 22 2010, 01:32 P...   Jan 22 2010, 03:40 PM
- - djellison   Two for the columbia hills are listed within this ...   Jan 22 2010, 03:59 PM
|- - Nirgal   QUOTE (djellison @ Jan 22 2010, 04:59 PM)...   Jan 23 2010, 04:00 PM
- - ngunn   Nirgal, I can't find your original thread so I...   Mar 1 2010, 10:08 PM
|- - Nirgal   QUOTE (ngunn @ Mar 1 2010, 11:08 PM) Nirg...   Mar 1 2010, 10:36 PM
- - ngunn   No I don't, unfortunately. Go to the 'Ende...   Mar 1 2010, 10:57 PM
- - Tesheiner   AFAIK, that's the non-map-projected version of...   Mar 2 2010, 06:52 AM
- - Nirgal   Upon a request posed over at the "Dust Avalan...   Mar 2 2010, 08:44 PM
|- - tdemko   This area has fascinated me for a while and Nirgal...   Mar 3 2010, 11:15 AM
|- - ngunn   QUOTE (tdemko @ Mar 3 2010, 11:15 AM) The...   Mar 3 2010, 12:57 PM
||- - Nirgal   QUOTE (ngunn @ Mar 3 2010, 01:57 PM) Fasc...   Mar 3 2010, 03:27 PM
|- - ElkGroveDan   QUOTE (tdemko @ Mar 3 2010, 03:15 AM) Thi...   Mar 3 2010, 02:32 PM
- - ngunn   YES!! Now that is everything I'd hoped...   Mar 2 2010, 09:03 PM
|- - Nirgal   QUOTE (ngunn @ Mar 2 2010, 10:03 PM) Can ...   Mar 2 2010, 09:33 PM
- - Tesheiner   Incredible! BTW, I think we'll have to sa...   Mar 2 2010, 09:06 PM
- - Floyd   A question for Tim and Nirgal: This is one of the...   Mar 3 2010, 02:33 PM
- - ngunn   Zoomable CTX of the area: http://global-data.mars....   Mar 3 2010, 02:58 PM
|- - Nirgal   QUOTE (ngunn @ Mar 3 2010, 03:58 PM) Look...   Mar 4 2010, 08:06 AM
- - Nirgal   For comparison, here is (a crop of) the original, ...   Mar 3 2010, 06:37 PM
- - Julius   To me it looks like an' island' which even...   Mar 3 2010, 07:11 PM
- - ngunn   Again, greatly appreciated Nirgal. Those ones show...   Mar 4 2010, 09:26 AM
- - ngunn   Nirgal, on closer inspection I'm having troubl...   Mar 4 2010, 02:19 PM
- - Nirgal   QUOTE (ngunn @ Mar 4 2010, 03:19 PM) Nirg...   Mar 4 2010, 03:57 PM


Reply to this topicStart new topic

 



RSS Lo-Fi Version Time is now: 31st October 2024 - 11:04 PM
RULES AND GUIDELINES
Please read the Forum Rules and Guidelines before posting.

IMAGE COPYRIGHT
Images posted on UnmannedSpaceflight.com may be copyrighted. Do not reproduce without permission. Read here for further information on space images and copyright.

OPINIONS AND MODERATION
Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators.
SUPPORT THE FORUM
Unmannedspaceflight.com is funded by the Planetary Society. Please consider supporting our work and many other projects by donating to the Society or becoming a member.