Printable Version of Topic

Click here to view this topic in its original format

Unmanned Spaceflight.com _ LRO & LCROSS _ It's June - Better LOLA?

Posted by: Rick Sternbach Jun 16 2010, 12:20 AM

Okay, It's June 15. Where's the updated LOLA data? I keep finding the stuff from March in all the usual places. Okay, I'm sure it may take a few days, but does anyone have a clue if the LDEMs are going to get cleaner and crisper this time around?

I've already seen a resin casting of a moon globe made from the LDEM_64 data, but I wanted to wait a bit until more blanks were filled in.

Rick

Posted by: DDAVIS Jun 16 2010, 12:40 AM

I will wait until the last data point is incorporated, what a great map that shall be!

Posted by: Phil Stooke Jun 16 2010, 01:48 AM

http://pds-geosciences.wustl.edu/missions/lro/lola.htm

OK?

Phil

Posted by: JohnVV Jun 16 2010, 02:13 AM

hay the gdr link finally works and is not gray.
ftp://pds-geosciences.wustl.edu/lro/lro-l-lola-3-rdr-v1/lrolol_1xxx/data/lola_gdr/
but still the old data 3-15-2010
now the rdr is updated 6-1-2010
ftp://pds-geosciences.wustl.edu/lro/lro-l-lola-3-rdr-v1/lrolol_1xxx/data/lola_rdr/lro_no_01/


Posted by: Rick Sternbach Jun 16 2010, 04:06 AM

QUOTE (Phil Stooke @ Jun 15 2010, 06:48 PM) *
http://pds-geosciences.wustl.edu/missions/lro/lola.htm

OK?

Phil


Well, if it doesn't have .img or .jp2 stuck to it, I can't use it. smile.gif It would be lovely to have, oh, a file with a name like LDEM_128.img . I am in no way smart enough to use anything else that I have been seeing with a June date on it. Yeah, I've checked with the PDS and Imbrium pages, and I'll wait until the image files "get good." Been spoiled by MOLA.

Rick

Posted by: JohnVV Jun 16 2010, 04:22 AM

QUOTE
Been spoiled by MOLA

well this is still a very,very early release
mola took a bit of time

Posted by: mhoward Jun 16 2010, 05:34 PM

Just to save everybody a little time: the LDEM_64.IMG on that page is byte-for-byte identical to the one released in March, despite the new file modified time.

Posted by: ValterVB Jun 16 2010, 06:42 PM

I see only the March release
here
ftp://pds-geosciences.wustl.edu/lro/lro-l-lola-3-rdr-v1/lrolol_1xxx/data/lola_gdr/
and here
http://imbrium.mit.edu/DATA/LOLA_GDR/
Where you found the update?

Ciao
VB

Posted by: Phil Stooke Jun 23 2010, 03:24 PM

Right... some of the data releases are delayed a bit, and maybe place-holder files are online while preparations continue. These are very large and complex datasets, and PDS documentation is time-consuming.

On another note - LRO has been in orbit for a year. In a few months the orbit is due to be raised as LRO enters its extended mission phase under the Science Mission Directorate.

Phil

Posted by: mhoward Jun 23 2010, 03:37 PM

I'm having a go at gridding the LOLA RDR data myself. It's going to take me several days to download all the data, though.

Posted by: James Fincannon Jun 23 2010, 06:38 PM

QUOTE (mhoward @ Jun 23 2010, 03:37 PM) *
I'm having a go at gridding the LOLA RDR data myself. It's going to take me several days to download all the data, though.



I have been using the RDR data (not the DEMs) for lunar illumination work. You have to be careful to use the right filters.

In RDRSIS.pdf it shows the shot flags for each shot.
http://imbrium.mit.edu/DOCUMENT/RDRSIS.pdf
http://imbrium.mit.edu/LABEL/LOLARDR.FMT

Use the ones with 0 as the least significant bit. It is interesting to understand the criteria for eliminating points. There are so many shots, that it is impossible to manually filter them all. The main automated filter they include is to compare the 5 shots against each other (not against other nearby shot sets). I think the slope limit is 37 deg between shots. This does a good job on most shots.

I have looked around both poles and found some that were erroenous comared against existing LOLA data from other passes (no shots have yet been manually editted/flagged in the RDR datasets so far). If you are interested in them, I can give you a list. It isn't that big and would likely not show up if you are using a tool like GMT to process the points into a DEM (unless you are shooting for high resolution) due to the averaging and surface fitting that tool uses.

Posted by: elakdawalla Aug 2 2010, 04:13 PM

An announcement from the PDS -- I don't know if this affects any of the work that you all are doing --

QUOTE
[NASA PDS] LRO LOLA INTERIM RELEASE ANNOUNCEMENT

To users of Lunar Orbiter Laser Altimeter (LOLA) data from the Lunar
Reconnaissance Orbiter (LRO) Mission:

The LOLA Team discovered and corrected a processing error that affected
time tags in RDR data products from day 2010-02-15 onwards. The Team
also corrected minor errors in EDR and RDR data product labels. This
interim release consists of revised versions of all previously released
RDR products and their PDS labels, and revised EDR labels only
(not data files).

The revised data are available from the LOLA Data Node via the Lunar
Orbital Data Explorer:

http://ode.rsl.wustl.edu/moon/

and from the PDS Geosciences Node:

http://pds-geosciences.wustl.edu

A revised product can be identified by the keyword
PRODUCT_VERSION_ID = "V1.03" in its label.

Posted by: James Fincannon Aug 2 2010, 04:24 PM

QUOTE (elakdawalla @ Aug 2 2010, 05:13 PM) *
An announcement from the PDS -- I don't know if this affects any of the work that you all are doing --



Yes, this will affect my work. Thanks.

I am baffled also by the lack of data density. I am waiting for LROC stereo imagery inferred heights correlated with the LOLA data. The problem is that LOLA, even at the poles, has <.3% of the surface area "painted" by LOLA light. Interpolations of height seem somewhat unreliable based on that. Normally, one could generate nice looking 3D models without worrying about this, but doing illumination analysis and being impacted by unknown nearby terrain or planning mobility paths can't be done with LOLA alone.

Posted by: JohnVV Aug 2 2010, 07:04 PM

QUOTE
I am baffled also by the lack of data density.

it is still early
the mars mola took a bit of time
the LOLA will also
QUOTE
An announcement from the PDS -- I don't know if this affects any of the work that you all are doing --

don't know yet
the last update was to fix the errors in the .lbl files we have been finding
" corrected minor errors in EDR and RDR data product labels "
those errors
the "time tags" ??? don't know but the derived image dost match up with Kaguya
will have to look
but people might want to read the errata.txt
http://pds-geosciences.wustl.edu/lro/lro-l-lola-3-rdr-v1/lrolol_1xxx/errata.txt
the relevant part
CODE
RELEASE NUMBER: n/a
RELEASE DATE: 2010-07-20
DATA COVERAGE: 2009-06-18 to 2010-03-22 (entire mission through 2010-03-22)
PRODUCT TYPES RELEASED FOR THIS ARCHIVE: No new products
PRODUCT TYPES UPDATED FOR THIS ARCHIVE: EDR, RDR
REASON FOR UPDATES: Data processing correction

The LOLA Team discovered and corrected a processing error that affected time
tags in RDR data products from day 2010-02-15 onwards. The Team also
corrected minor errors in EDR and RDR data product labels. This intermediate
release consists of revised versions of all previously released RDR products
and their PDS labels, and revised EDR labels only (not data files). A revised
product can be identified by the keyword PRODUCT_VERSION_ID = "V1.03" in its
label.

The following data files and corresponding labels, which were previously
included, have been excluded for this and future releases due to poor data
quality.

DATA/LOLA_EDR/LRO_CO/LOLAEDR091931559.DAT
DATA/LOLA_EDR/LRO_CO/LOLAEDR091931559.LBL
DATA/LOLA_EDR/LRO_CO/LOLAEDR091931759.DAT
DATA/LOLA_EDR/LRO_CO/LOLAEDR091931759.LBL

DATA/LOLA_RDR/LRO_CO/LOLARDR091931559.DAT
DATA/LOLA_RDR/LRO_CO/LOLARDR091931559.LBL
DATA/LOLA_RDR/LRO_CO/LOLARDR091931759.DAT
DATA/LOLA_RDR/LRO_CO/LOLARDR091931759.LBL

Gridded Data Record (GDR) and Spherical Harmonics Analysis Data Record
(SHADR) products have not been revised since Release 2.

Posted by: djellison Aug 2 2010, 09:37 PM

Humph. Still some major problems with this imho - the ends still don't match when you wrap it around or cut'n'paste in photoshop to drop the meridian in the middle.

Posted by: JohnVV Aug 3 2010, 01:23 AM

djellison seeing that is also true for most of the maps on pds and from nasa , that is nothing new

and are you referring to the gdr from march
http://pds-geosciences.wustl.edu/lro/lro-l-lola-3-rdr-v1/lrolol_1xxx/data/lola_gdr/

Posted by: mhoward Aug 3 2010, 02:38 AM

QUOTE (elakdawalla @ Aug 2 2010, 09:13 AM) *
An announcement from the PDS -- I don't know if this affects any of the work that you all are doing --


Thanks for the heads up. I'd identified long ranges of the data that clearly did not match their neighboring areas; it will be interesting to see if they are corrected now. (Of course, it sounds like that will require re-downloading all the RDR data again, which will probably take me over a week. Also I'm running low on hard drive space rolleyes.gif )

Posted by: djellison Aug 3 2010, 02:52 AM

QUOTE (JohnVV @ Aug 2 2010, 06:23 PM) *
djellison seeing that is also true for most of the maps on pds and from nasa


The recent swath of Cassini maps doesn't have this problem, nor the MOLA GDR's in my experience.

But yes - I'm talking about the LOLA GDR's

Posted by: mhoward Aug 12 2010, 04:16 PM

Interesting - unless I'm doing something wrong, it looks like the RDR data has been extensively corrected. This is good! I should have a better estimation of the improvement in a few days.

Posted by: Phil Stooke Aug 12 2010, 04:17 PM

It takes time to get it all fixed up... but people, if you want to see how unbelievable LOLA is going to be when it's all done, check out this amazing presentation from the NASA Lunar Science Forum, held at NASA Ames last month. This is by Maria Zuber, and - alas - it didn't survive the PDF-making process properly. I have asked if it can be fixed. But even so, it looks good. Check out the LOLA map of the floor of Shackleton on page 15. As I say I've asked for it to be fixed, so we'll see.


http://lunarscience2010.arc.nasa.gov/sites/default/files/Zuber.pdf

Other pressies here:

http://lunarscience2010.arc.nasa.gov/agenda

Lots of goodies.

Phil

Posted by: antipode Aug 13 2010, 10:39 PM

QUOTE
http://lunarscience2010.arc.nasa.gov/sites...files/Zuber.pdf


Hmmm, impressive! Lots of interesting looking mounds on the crater floor. More of that fluffy volatile rich material that LCROSS dived into?

Some of the other papers are very interesting too. Recommended!
P

Posted by: James Fincannon Aug 14 2010, 05:51 PM

"It takes time to get it all fixed up... but people, if you want to see how unbelievable LOLA is going to be when it's all done, check out this amazing presentation from the NASA Lunar Science Forum, held at NASA Ames last month. This is by Maria Zuber, and - alas - it didn't survive the PDF-making process properly. I have asked if it can be fixed. But even so, it looks good. Check out the LOLA map of the floor of Shackleton on page 15. "


Yes, it looks good, and is much better than what we have had and they are doing great work.

But there are some points that should be emphasized (and I do not think they are sufficiently)........

For a 25 m/pixel grid for within 25 km of the South Pole,
(1) Only 72% of the grid elements have at least one laser data point. This means 28% are empty, but the DEMs show them filled (interpolated). This is a concern to me because although it creates a nice continuous image/DEM, it needs an accompanying error map to help a user to understand the missing data, interpolation error, etc.
(2) The average number of laser data points in this grid is 1.5 +- 1.4. For a 25m by 25 m pixel, you would like around 25 laser data points to get good coverage with 5 m diameter spots. This means when the DEM is constructed, the height for that pixel is supposed to be an average height of the surface, but really it is the height average of from 0% to 12% of the surface area within the pixel.
(3) How does the laser data point treats the area it "paints"? Is this the average height within the 5 m spot or the highest spot or what?


With coarser grids (240 m by 240 m/pixel), the percentage of surface area with laser data spots is around 8%.

Thus the magic of creating the DEM (i.e. sausage making) has alot of aspects that people need to realize and see if it applies to their usage. I have been stymied from doing illumination analysis because of these concerns. Sure I can do it and have done it with my analysis tools and use either the DEMs or the actual laser points, but I cannot create an error bar, so I have to reassess this laser data DEM.

Posted by: zeBeamer Sep 1 2010, 10:57 PM

QUOTE (James Fincannon @ Aug 14 2010, 12:51 PM) *
Thus the magic of creating the DEM (i.e. sausage making) has alot of aspects that people need to realize and see if it applies to their usage. I have been stymied from doing illumination analysis because of these concerns. Sure I can do it and have done it with my analysis tools and use either the DEMs or the actual laser points, but I cannot create an error bar, so I have to reassess this laser data DEM.


James,
we discussed that offline, but I do not agree that we need to paint the whole Moon to have a realistic map at say 25m resolution. Indeed the current filling ratio of the 25x25m near the poles was 72% when we discussed that in June, but it will keep on improving.
The September release in a couple of weeks will have side products for each of the DEMs containing the counts of laser shots in each pixel. People can use that as a mask to see where you can be more or less confident in the measurement averaging (actually a median).
But I am not sure that is what will capture the interest of most people here. And having "gaps" in the DEMs to reflect the actual sampling would not necessarily make it better; I would expect most people want a full map, and do not want to do their own interpolation (they may not be familiar with the tools to do so) when they want to render a given region.
To reassure you, the data is not put through magic black boxes, and the workflow is actually pretty straightforward. It just gets messy to deal with when you have billions of points and those high resolutions.

QUOTE (James Fincannon @ Aug 14 2010, 12:51 PM) *
With coarser grids (240 m by 240 m/pixel), the percentage of surface area with laser data spots is around 8%.

Do you mean globally? In June, polewards of ~85deg, we had ~90% coverage at that resolution.

All,
The September release is coming very soon, and the DEMs will be updated this time, with more than 2 billion (good) points which went into them.
I updated the Celestia products, and they are already available here: http://imbrium.mit.edu/EXTRAS/CELESTIA/
They were made from the to-be-released 128ppd grid. Annoying seams should be gone (note to djellison and John). We are also releasing a 256ppd map (in four tiles), but I do not have the time currently to do it from that source (that would bring us to level6). And currently, it might be overkill. Others are welcome to try it out!

As for the new polar maps you saw in Maria Zuber's Ames presentation, this is not exactly what is going to be released. I'm not going into details here, but basically, the PDS release will still show some (reduced compared to before) orbit streaks near the poles. I will try to provide a better image of the South Pole, as it seems to be of interest here wink.gif

Erwan

Posted by: JohnVV Sep 2 2010, 12:40 AM

as has been stated before these things take time
at least it is not1960 to 1980's and the only way, outside a big university , was on a tape drive and in published print

Posted by: James Fincannon Sep 2 2010, 02:49 PM

"I do not agree that we need to paint the whole Moon to have a realistic map at say 25m resolution."

What is the meaning of the word "realistic"? It may be realistic for people who want to render topographic surfaces for illustrative purposes or animations or simulations that do not require interacting with the surface. But I think one would not consider it realistic if one is planning a rover path or doing certain types of illumination analysis.

I think you and your team are doing great work, but I think some accuracy/realism aspects go over most users heads and are ignored.

"Indeed the current filling ratio of the 25x25m near the poles was 72% when we discussed that in June, but it will keep on improving."

This is the number I quoted before and is what bothered me because the DEMs show continuous surfaces where they cannot really be definitive as such (other than interpolation). Yes, given enough time you should get "100%" (meaning, to other readers, at least one 5 m diameter laser spot within all of the polar 25m by 25 m areal elements). But I have a little trouble with the rationale of interpolating an entire pixel height based on maybe <4% of the area being painted by laser light (i.e. one laser spot for a surface that needs 25 laser spots to fully define). Maybe the LOLA data is meant just for a certain purpose and I am trying to use it for a purpose that was not intended and I should really use the stereo imagery derived terrain instead.

>The September release in a couple of weeks will have side products for each of the DEMs containing the counts
>of laser shots in each pixel. People can use that as a mask to see where you can be more or less confident in the
>measurement averaging (actually a median).

This will be helpful.

>And having "gaps" in the DEMs to reflect the actual sampling would not necessarily make it better; I would
>expect most people want a full map, and do not want to do their own interpolation (they may not be familiar
>with the tools to do so) when they want to render a given region.

Gaps would be no good in the DEM, but what I think some users would like is the number of shots per pixel (0 shots would tell them it is pure interpolation). It would also be nice to have an error estimate for each pixel (maybe based on the difference between the interpolation pixel height and the average of the heights of the laser spots within the pixel).

>To reassure you, the data is not put through magic black boxes, and the workflow is actually pretty straightforward.

Interpolation is kind of magical in that it is hard to intuitively know how the interpolation will work out all the time for every set of points. You are not simply drawing a straight line between points, it is much more complex.

>>With coarser grids (240 m by 240 m/pixel), the percentage of surface area with laser data spots is around 8%.
>Do you mean globally? In June, polewards of ~85deg, we had ~90% coverage at that resolution.

By this I mean, for a 240m by 240 m DEM, I created a grid of 5 m non-overlapping spots to fill it which gives you 48 by 48/ 5 m spots or 2304 laser spots needed to cover the whole pixel. Then using your average number of laser spots/pixel within 25 km of the south pole (136+-63), I get a maximum of 8.6% and an average of 6% of the area painted by laser light. Sure, the number of 240m by 240m pixels that have at least 1 laser spot is ~100% in the case, but what I am saying is the kind of interpolated height based on 6-8% of the surface area needs some sort of error bar associated with it, since it is very hard for any of us to figure it out just from the raw data.

>The September release is coming very soon, and the DEMs will be updated this time, with more than 2 billion (good) points which went into them.

This is great! Still, globally, this means you are covering ~.1% of the lunar surface with laser light. So will LRO be merging the stereo imaging with LOLA data for significant regions? Are you working with those guys?

Posted by: Rick Sternbach Sep 2 2010, 04:42 PM

QUOTE (zeBeamer @ Sep 1 2010, 03:57 PM) *
All,
The September release is coming very soon, and the DEMs will be updated this time, with more than 2 billion (good) points which went into them.
I updated the Celestia products, and they are already available here: http://imbrium.mit.edu/EXTRAS/CELESTIA/
They were made from the to-be-released 128ppd grid. Annoying seams should be gone (note to djellison and John). We are also releasing a 256ppd map (in four tiles), but I do not have the time currently to do it from that source (that would bring us to level6). And currently, it might be overkill. Others are welcome to try it out!

Erwan


Quick noob question; what's the difference between an LDEM file and a CDEM file? Just noticed all the new file names added.

Rick

Posted by: zeBeamer Sep 2 2010, 07:56 PM

QUOTE (Rick Sternbach @ Sep 2 2010, 11:42 AM) *
Quick noob question; what's the difference between an LDEM file and a CDEM file? Just noticed all the new file names added.


CDEM contains the counts. LDEM contains the altitude (discretized in half-meter levels).

Erwan


Posted by: zeBeamer Sep 3 2010, 09:20 PM

Here is one image of the South Pole region as shown by Maria Zuber in the Ames meeting about a month ago. Click for higher resolution.
http://web.mit.edu/mazarico/Public/UMSF/shackleton_25m_jpg.jpg
It's a 25-m DEM constructed from ~4000 profiles. The axis labels are in kilometers (stereographic projection around 0,-90).
Still a few blemishes, but impressive enough I hope wink.gif

Erwan

Posted by: Rick Sternbach Sep 9 2010, 11:31 PM

QUOTE (zeBeamer @ Sep 2 2010, 12:56 PM) *
CDEM contains the counts. LDEM contains the altitude (discretized in half-meter levels).

Erwan


Oh, and I just noticed a lot of file names *missing*, like all the JPG2000 files. With my limited Mac software here, those were the only files I could get to look at and use in Terragen. I can get a file like LDEM_64.img to open in MacDEM as I did with the MOLA files, but now MacDEM can't figure out the height range (I think). Image comes out where I can see the terrain with directional lighting turned on, but the shades are all super contrasty and black-white splotchy. Probably time to get a new machine and better terrain software. sad.gif

Posted by: Phil Stooke Sep 9 2010, 11:57 PM

Thanks Erwan - lovely map. I was sorry to se that Maria's presentation file archived at NLSF 2010 was messed up so we couldn't see the images properly, but this takes care of that one!

Phil

Posted by: JohnVV Sep 10 2010, 12:32 AM

just took a look at

http://imbrium.mit.edu/DATA/LOLA_GDR/
the img's have a sep3 date

PS. for the count files( CDEM_ ) pds2isis tosses en error about the " SAMPLE_BITS = 2" - isis wants 8 ,16 ,or 32
small edit in the LBL solves this

as per Rick Sternbach question there is a MAC isis3

and for just using pds2isis then > isis2raw most of the data dose not need to be installed just the BASE system ( small install ) and some of the data files

CODE
rsync -azv --delete isisdist.wr.usgs.gov::x86_darwin_OSX105/isis .
rsync -azv --delete isisdist.wr.usgs.gov::isis3data/data/base data/

but you can remove the very large maps if space is an issue /isis_3/data/base/dems ~4.8 gig

Posted by: Rick Sternbach Sep 10 2010, 06:30 AM

QUOTE (JohnVV @ Sep 9 2010, 05:32 PM) *
as per Rick Sternbach question there is a MAC isis3


Thanks; I looked into ISIS some time ago. Way, way, way over my head. - R.

Posted by: pch Sep 10 2010, 06:50 AM

QUOTE (JohnVV @ Sep 10 2010, 02:32 AM) *
just took a look at

http://imbrium.mit.edu/DATA/LOLA_GDR/
the img's have a sep3 date


Nice!
From the label LDEM_64.IMG is updated with observation up to June 27 and there is a new LDEM_128.IMG to play with smile.gif

Posted by: JohnVV Sep 10 2010, 07:08 AM

i just imported the 128 into Nip2 and did a simple "emboss " filter
two screenshots from nip
http://www.imagebam.com/image/2779fb97112386 http://www.imagebam.com/image/999f4f97112396
the second is zoomed in a bit
still need some work but better than the last set

give it a year and it will rival the MOLA

Posted by: JohnVV Sep 10 2010, 10:28 PM

Rick Sternbach and a few others i can post the 128 as a 2 gig raw image
500 meg north west,south west,north east , and south east
with 0 long and 0 lat in the center of the map ( -180 to 180)

--- rant ---
raw2isis has a bug, that i am about to report -- i would but the USGS is down for maintenance

16 bit unsigned ( my working format) gets saved as 32 bit REAL but the cubs say they are 16unsigned -- just a bit of an issue
and pds2isis is doing it too map2map is a MESS. telling isis it is 16 bit signed ( from the cube& img )when it is 32 bit float ( the data in the cube)

at least isis2raw IS WORKING the way it should
i have finally a 90north to 40 north ( polar), the same for the south pole and 45n to 45s ( simple cylindrical )
so i can clean them up a bit

Posted by: Rick Sternbach Sep 11 2010, 04:14 PM

QUOTE (JohnVV @ Sep 10 2010, 03:28 PM) *
Rick Sternbach and a few others i can post the 128 as a 2 gig raw image
500 meg north west,south west,north east , and south east
with 0 long and 0 lat in the center of the map ( -180 to 180)


John - That's a generous offer; thanks. Even the individual 500MB tiles would be fine in my case, as I'd be exploring smaller areas in Terragen or TG2. Eventually for rapid prototyping a physical globe, I'd be getting with the smart folks who have the computing horsepower to work with the .img files and grow the resin parts, but right now I'm hoping to "look around" the surface in 3D. - Rick

Posted by: charborob Sep 11 2010, 06:23 PM

QUOTE (Rick Sternbach @ Sep 11 2010, 11:14 AM) *
I'm hoping to "look around" the surface in 3D.

I'd like to see what results you obtain with Terragen. Post some of your picture when you have them.

Posted by: JohnVV Sep 11 2010, 08:16 PM

i will post links to them later on tonight ( i have to export them )


Posted by: zeBeamer Sep 12 2010, 01:03 AM

QUOTE (Rick Sternbach @ Sep 9 2010, 07:31 PM) *
Oh, and I just noticed a lot of file names *missing*, like all the JPG2000 files. With my limited Mac software here, those were the only files I could get to look at and use in Terragen. I can get a file like LDEM_64.img to open in MacDEM as I did with the MOLA files, but now MacDEM can't figure out the height range (I think). Image comes out where I can see the terrain with directional lighting turned on, but the shades are all super contrasty and black-white splotchy. Probably time to get a new machine and better terrain software. sad.gif


GeoJPEG2000 are forthcoming. We want to make sure they're correct, and we found some strange things happening with the programs used to make the conversion.
Anyway, release date has always been the 15th, if you can wait that long wink.gif

As for the 128ppd being too large, we did not plan on splitting those up, but that could potentially be done in the next release, if enough people think it's worthwhile. The problem is that when we eventually release a 1024ppd, we don't want to break them up 256 tiles (each equivalent to a 64ppd global). 64 "128ppd" tiles will be quite enough already!

Erwan


Posted by: JohnVV Sep 12 2010, 03:08 AM

As for the 128ppd being too large,
not for me but to upload a zip to a service like zshare , they need to be small
so there is a plan to release a 368640 x 184320 pixel image 68 megapixle map ( 1024 px per deg. )

Posted by: mhoward Sep 12 2010, 01:37 PM

Looking forward to the new RDRs. Thanks.

Posted by: Rick Sternbach Sep 16 2010, 04:54 AM

QUOTE (zeBeamer @ Sep 11 2010, 06:03 PM) *
GeoJPEG2000 are forthcoming. We want to make sure they're correct, and we found some strange things happening with the programs used to make the conversion.
Anyway, release date has always been the 15th, if you can wait that long wink.gif

As for the 128ppd being too large, we did not plan on splitting those up, but that could potentially be done in the next release, if enough people think it's worthwhile. The problem is that when we eventually release a 1024ppd, we don't want to break them up 256 tiles (each equivalent to a 64ppd global). 64 "128ppd" tiles will be quite enough already!

Erwan


I see the new LDEM files with the "PPD" stuck in the middle, and I can open the 4, 16, and 64 JP2000 versions, but no way can I get the 128 to open in Photoshop, even with almost 1GB RAM devoted to the application and a healthy secondary HD. I get a message talking about trouble with a "file-interface module," which I have no idea how to fix. As before, I suspect I'd need a Mac Pro with 8GB of RAM to play in the bigger sandbox. Anyhow, I'm attaching a view of Copernicus from the 64 file, quick render in Terragen. - Rick



 

Posted by: Mars3D Nov 25 2010, 08:26 PM

How are you guys processing the RDRs? I have found RDR2TAB and RDR2XYZ but they are for MAC only, has anyone found a win32 version? I have the fortran source code but no compiler.

Thanks for any help.

Posted by: JohnVV Nov 25 2010, 10:11 PM

QUOTE
How are you guys processing the RDRs? I have found RDR2TAB and RDR2XYZ

they take way to much work at this time
I use the GDR pds IMG files
http://imbrium.mit.edu/DATA/LOLA_GDR/

Posted by: mhoward Nov 26 2010, 04:45 AM

QUOTE (Mars3D @ Nov 25 2010, 02:26 PM) *
How are you guys processing the RDRs?


I'm using custom software to do it - on Mac, so can't really help there. Yes, most people are using the GDRs.

Posted by: Mars3D Nov 26 2010, 06:58 PM

Thanks for the replies.

I think I'll have a go at writing some code to process the rdrs. I have the code to do the gridding and bad data removal from the work I did with the MOLA data.

Posted by: James Fincannon Nov 26 2010, 07:06 PM

QUOTE (Mars3D @ Nov 25 2010, 09:26 PM) *
How are you guys processing the RDRs? I have found RDR2TAB and RDR2XYZ but they are for MAC only, has anyone found a win32 version? I have the fortran source code but no compiler.

Thanks for any help.



Sorry, I do not know of a win32 version. I wrote my own FORTRAN RDR reader based on the description of the RDR file. Erwan M. described to me a lot of conversions needed to create a DEM. I skip those because I need to use the raw data points for my work. Also, I found I needed to examine segments carefully with a viewer to make sure no erroneous data points were included (they are not flagged yet, perhaps in future versions).

Anyone heard when the WAC LRO stereo DEM is going to be published?

Posted by: mhoward Nov 26 2010, 07:18 PM

QUOTE (Mars3D @ Nov 26 2010, 12:58 PM) *
I think I'll have a go at writing some code to process the rdrs. I have the code to do the gridding and bad data removal from the work I did with the MOLA data.


Interesting. I've done the work and gotten results but haven't had time to release anything yet. At the moment I exclude bad LOLA data on a file-by-file basis. Fortunately they have cleaned up a lot of the data, just not quite all of it.

Posted by: JohnVV Nov 26 2010, 08:02 PM

it should build in MinGW on windows but you would need to spend a week or two setting up MinGW ( unless you have done that a lot -- then a day or two )
However i have not done this for the rdr FORTRAN files , so it is a guess .

MinGW runs much faster than CygWin

Posted by: Mars3D Nov 28 2010, 01:29 AM

I briefly tried MinGW but I was getting lots of compiler errors so I gave up. I have now written some c code to process the rdrs and it looks like its working ok. Now I just need to download all the rdrs which looks like its going to take a very long time.

Posted by: JohnVV Nov 28 2010, 04:05 AM

those errors are from mingw needing to be configured BY HAND for EVERY AND ALL text based files
that is why a new person to mingw will take 1 two 2 weeks to find all of the hard coded " c:\PROG~\????" hard links in it
every one needs to be reset to c:\Gnuwin32\Minjgw

the gnuwin32 project for some very odd and bad reason lies windows install everything to c\Program Files\ Program name
with ALL the spaces --- not good

then the paths c\GnuWin32\MinGW and c:\GnuWin32\Msys need to be added to the Microsoft system path
and Msys needs to be set up

a bunch of work

Posted by: mhoward Nov 28 2010, 02:55 PM

QUOTE (Mars3D @ Nov 27 2010, 06:29 PM) *
Now I just need to download all the rdrs which looks like its going to take a very long time.


Took me literally weeks to download it all over a typical DSL connection. And every time I finished, they would be updated laugh.gif

Posted by: Mars3D Nov 28 2010, 07:45 PM

QUOTE (mhoward @ Nov 28 2010, 02:55 PM) *
Took me literally weeks to download it all over a typical DSL connection. And every time I finished, they would be updated laugh.gif


I've worked out if it will take me 2 weeks of continuous downloading. So I should be finished about the time of the December update smile.gif

Posted by: JohnVV Nov 28 2010, 08:37 PM

seeing as the labels are the ones being updated
http://imbrium.mit.edu/DATA/LOLA_RDR/LRO_NO_10/
*.DAT Aug 17
*.LBL Nov 5

and the LBL's are 5.8 K each
very small to update

Posted by: mhoward Nov 28 2010, 08:41 PM

QUOTE (JohnVV @ Nov 28 2010, 01:37 PM) *
seeing as the labels are the ones being updated


Not correct - in the past most of the data has been updated, extensively.

Whether all the data will be updated in the next round, well - for the sake of our poor Internet connections it might be nice if it weren't, but it might also be good if it was. So far it's been improving over time.

Posted by: ValterVB Dec 31 2010, 10:50 AM

I think to have found an error on LDEM 256 download from http://imbrium.mit.edu/DATA/LOLA_GDR/CYLINDRICAL/
In the LBL file, MAP_RESOLUTION and MAP_SCALE are reversed, I Have checked on ftp://pds-geosciences.wustl.edu/lro/lro-l-lola-3-rdr-v1/lrolol_1xxx/data/lola_gdr/cylindrical/img/

Wrong
MAP_RESOLUTION = 118.451 <m/pix>
MAP_SCALE = 256 <pix/degree>

Correct
MAP_RESOLUTION = 256 <pix/degree>
MAP_SCALE = 118.451 <m/pix>

Someone can correct it?

Posted by: Phil Stooke Jan 2 2011, 07:57 AM

In the old days this was straightforward. Resolution was always given as so many meters (or kilometers, or your unit of choice) per pixel (or line pair... OK, maybe it wasn't so straightforward)... anyway, resolution = m/pixel or some equivalent. And map scale was given as 1:500,000, or 1:1 million, or "1 cm represents 10 km" or words to that effect.

Digital maps messed this up a bit because a digital file could be printed at any physical size. So the idea developed that digital map scales should be defined as the number of pixels across a standard distance unit - pixels per degree of latitude for instance. These things are not really universal and strictly speaking they are ambiguous, but that is common usage. So I have to say, actually, there is no mistake.

Phil

Posted by: ValterVB Jan 2 2011, 02:18 PM

Thanks for the answer Phil, but it is strange, because now I can't found the wrong file. Now all the LBL file have MAP_RESOLUTION with <pix/deg>. unsure.gif
I have an old file (PRODUCT_CREATION_TIME = 2010-09-15T00:00:00) with MAP_RESOLUTION in m/pix and MAP_SCALE in pix/degree, the file name is
LDEM_256_0_180_0N_90N.LBL.
Perhaps is changed from September

Ciao
VB

Posted by: JohnVV Jan 2 2011, 02:26 PM

QUOTE
Perhaps is changed from September

the date for the 256 dem is Nov 30 and for the 256 LBL is Dec 2
the labels were updated

Posted by: mhoward Jan 22 2011, 12:32 AM

Here's a little something to show how good the LOLA data is getting. This is a map generated from the LOLA data only, as if the moon were a uniformly-colored surface, with each point illuminated from a 45 degree angle to the left right. I use this kind of image to identify problem sections in the LOLA RDR data for what I'm doing, but I thought the result was kind of interesting in itself, so I put together a whole mosaic. Before pressing the link, please note that this is a big image - you might want to view it outside the browser. (And this is actually the half-size version.)

http://mmb.unmannedspaceflight.com/LOLA_diagnostic_illumination_whole_11K.jpg (11520x5760 pixel JPG image; 27MB)

Edit: and if you really want the full-size version, it's here: http://mmb.unmannedspaceflight.com/LOLA_diagnostic_illumination_whole_22K.jpg

Posted by: eoincampbell Jan 22 2011, 12:51 AM

QUOTE (mhoward @ Jan 21 2011, 04:32 PM) *
Here's a little something...

http://mmb.unmannedspaceflight.com/LOLA_diagnostic_illumination_whole_11K.jpg (11520x5760 pixel JPG image; 27MB)


Well worth the wait! Thanks!

Posted by: JohnVV Jan 22 2011, 01:18 AM

lighting from 45 deg up ( the basic "emboss" filter ) is the best way to view 16 bit height data
it was the only way i could find to easily show it
soon i am going to have to update my normal map

what did you use to destrip it

Posted by: Phil Stooke Jan 22 2011, 01:43 AM

Fantastic image! Thanks. This is going to be a fantastic dataset.

Phil

Posted by: mhoward Jan 22 2011, 02:11 PM

QUOTE (JohnVV @ Jan 21 2011, 06:18 PM) *
what did you use to destrip it


I'm not positive what you mean by destrip, but if you mean removing the vertical banding artifacts, that's why I went back to the source, the RDRs - to generate the map using a modified kriging technique with some different parameters. The image above is the resulting surface normal map cross-produced dot-producted with a 45 degree lighting vector, with no post-processing.

Also - and for me this is the "neato" part - I'm generating the surface normal map directly from the RDRs, which gives a more precise result than generating it from the gridded height map. I was pleased - and more than a little surprised - to see how precisely the resulting surface normal map matches up with the Clementine version 2 base map, in most areas. Computationally expensive to do - and took quite a long time to work out - but worth it.

Posted by: mhoward Jan 22 2011, 02:42 PM

QUOTE (Phil Stooke @ Jan 21 2011, 07:43 PM) *
This is going to be a fantastic dataset.


It's a real pleasure watching the map fill in over the months. I'm already looking forward to the next release.

Posted by: Phil Stooke Jan 22 2011, 05:35 PM

"I was pleased - and more than a little surprised - to see how precisely the resulting surface normal map matches up with the Clementine version 2 base map, in most areas."

And remember that it's the Clementine map that is wrong if there's a difference. Clementine control, based on a patchwork quilt of thousands of images, will be greatly inferior to this dataset. LOLA is going to be our ultimate lunar control system.

Phil

Posted by: mhoward Jan 22 2011, 06:07 PM

QUOTE (Phil Stooke @ Jan 22 2011, 10:35 AM) *
And remember that it's the Clementine map that is wrong if there's a difference.


Yes, I've kind of discovered that the hard way smile.gif Except: I suspect now that some of the surface normal maps that have been generated may have slight but problematic offsets, because of the way they're derived from the gridded height map. My previous ones certainly did.

Posted by: mhoward Jan 22 2011, 08:11 PM

For fun, here is the illumination map from above generated from LOLA data, combined with the Clementine map. So basically: the Clementine map with the lighting synthetically shifted to the right to bring out the topography more. The image comes in two three sizes: Preview-size, Large and Extra Large, all generously hosted by UMSF (your donation dollars at work).

http://mmb.unmannedspaceflight.com/LOLA_plus_Clementine_45r_5K.jpg (5760x2880 pixel JPG; 8.9MB)

http://mmb.unmannedspaceflight.com/LOLA_plus_Clementine_45r_11K.jpg (11520x5760 pixel JPG; 30.8MB)

http://mmb.unmannedspaceflight.com/LOLA_plus_Clementine_45r_22K.jpg (23040x11520 pixel JPG; 92.5MB)

Posted by: JohnVV Jan 22 2011, 09:49 PM

QUOTE
but if you mean removing the vertical banding artifacts

that is what i meant .
the processing of the rdr data

the gdr is very banded

QUOTE
And remember that it's the Clementine map that is wrong if there's a difference

i found that out with the kaguya data .Kaguya lines up well with LOLA. But most of the Clementine map is "predictably " off .An over all warp brings most of it
into alignment. However some small areas are just off by what looks like a random amount ( +- 5 to 20 pixels )

Posted by: mhoward Jan 22 2011, 11:43 PM

QUOTE (JohnVV @ Jan 22 2011, 02:49 PM) *
the gdr is very banded


I'll probably get around to posting my own version of the GDR, so you can see if it's any better. I'm not quite ready yet, as I usually bypass that step now. Also, it's a very big file.

Posted by: JohnVV Jan 23 2011, 08:04 AM

no kidding it is a big file
the *.dat's alone would be a bit much
then rdr2csv and filter that to the required data
I was looking in to doing that but the overall drive space required is more than i could do

Powered by Invision Power Board (http://www.invisionboard.com)
© Invision Power Services (http://www.invisionpower.com)