Printable Version of Topic

Posted by: elakdawalla Sep 28 2010, 09:23 PM

Because it has become a forum FAQ, I've created this sticky thread containing information on where to obtain new base images for Opportunity's traverse for Google Mars, and for discussion on creating new ones. I will continue to add links to new base image layers to this first post as they become available.

New users: http://planetary.s3.amazonaws.com/gmars_maps/Opportunity_base_maps.kml and open it in Google Earth
Then http://www.unmannedspaceflight.com/index.php?showtopic=681&view=getlastpost for the latest traverse map
You need to download the new KML file each time in order to follow Opportunity's peregrinations.

Google Mars comes with a color base image mosaic created from HRSC imagery. In addition, there is an inset full-resolution HiRISE image covering the area from the landing site at Eagle crater, through Victoria, up to the point between sol 2040 and 2041 (just west of Mackinac) where Opportunity drove off the map.

Unfortunately, this Victoria crater HiRISE layer included within Google Mars is not perfectly registered to the HRSC base map. As far as is known, there is nothing to be done about that. Both John Cody's image layers and Eduardo Tesheiner's traverse maps are aligned with the inset HiRISE layer included with Google Mars, NOT to the HRSC base map.

In June 2009 SFJCody http://www.unmannedspaceflight.com/index.php?showtopic=5476&view=findpost&p=141926 a reduced-resolution mosaic of HiRISE tiles that cover the entire future traverse area including Endeavour's rim. I have made some small modifications to that map and have http://planetary.s3.amazonaws.com/gmars_maps/MER-B_map_extension_low_res.kmz (17 MB). Download the file, run Google Earth, select the Mars view, and File>Open the KMZ to view it.

In September 2009 SFJCody posted another HiRISE base image, this one at full resolution, covering the Western Route and reaching not quite all the way to Santa Maria. http://planetary.s3.amazonaws.com/gmars_maps/ESP_011765_1780_RED.kml (4 MB). http://planetary.s3.amazonaws.com/misc/MER-B_Map_extension_v2.zip if you'd prefer to have it locally (256 MB) and I also wrote a http://www.planetary.org/blog/article/00002342/ about it.

During the discussion below, in late 2010, I created a small tile that covers just the immediate area around Santa Maria. http://planetary.s3.amazonaws.com/gmars_maps/PSP_005423_1780.kml.

In February 2011 Eduardo Tesheiner provided another set of base images covering the area from Santa Maria to Endeavour's rim. http://planetary.s3.amazonaws.com/gmars_maps/PSP_010341_1775_RED.kml. If you would like to work from a local copy, you can download http://planetary.s3.amazonaws.com/gmars_maps/PSP_010341_1775_RED.zip and unzip it to a folder on your drive, then open the file PSP_010341_1775_RED.kml within it.

All three base images can be loaded at once using http://planetary.s3.amazonaws.com/gmars_maps/Opportunity_base_maps.kml (the same one that is linked to at the very top of this post).

Posted by: elakdawalla Sep 28 2010, 09:33 PM

OK, now that I've posted the existing maps, here is a call for help with future ones. At her current pace, Opportunity will run off of SFJCody's high-resolution base map in about two months. He can't do the next map, and I tried but couldn't figure out how to make any region file generators work within the amount of time I allotted myself to spend on the project, so someone else needs to try to figure out how to do this!

The first step is to get the images. I can provide those, thanks to James Canvin, who processed the small piece of http://hirise.lpl.arizona.edu/PSP_005423_1780 that covers the area around Santa Maria plus the entirety of http://hirise.lpl.arizona.edu/PSP_010341_1775 which covers the rest of the way to Endeavour plus the entire western rim into 8192x8192 JPEG format subimages, each of which overlaps by 10% with the next. They can be downloaded here:

http://planetary.s3.amazonaws.com/gmars_maps/PSP_005423_1780_santa_maria_extension.zip (PSP_005423_1780, 33 MB)
http://planetary.s3.amazonaws.com/gmars_maps/PSP_010341_1775_Endeavour_1of3.zip (PSP_010341_1775, 183 MB)
http://planetary.s3.amazonaws.com/gmars_maps/PSP_010341_1775_Endeavour_2of3.zip (PSP_010341_1775, 199 MB)
http://planetary.s3.amazonaws.com/gmars_maps/PSP_010341_1775_Endeavour_3of3.zip (PSP_010341_1775, 148 MB)

Then, SFJCody had the following advice to me on how to get these into Google Mars:

Note that, for the base map to be useful to Eduardo and all of us who follow his maps of Opportunity's progress, the images should NOT be registered to the color HRSC base map included within Google Mars, but instead to the low resolution HiRISE base map linked to in the first post in this thread.

Posted by: eoincampbell Sep 29 2010, 02:51 AM

Which is so frustrating, I really appreciate your endeavors ,
I hope Google Mars catches up soon...I mean, shouldn't they ?...

Posted by: Tesheiner Sep 29 2010, 05:58 AM

I'm wondering if it is really necessary to overlap the subimages? When I had the idea of making my GE version of route map I spent some time looking on how to make additional background pictures, specially on the topic of "region files". IIRC, GE splits their CTX and HiRISE overlays on subimages with no overlay so, in principle, there's no need for the 10% overhead.

Example: Load the CTX picture P01_001414_1780_XI_02S005W on GE by selecting first "Spacecraft Imagery > CTX Image Browser" then click on the yellow square corresponding to that image. Then click on "Load this image" on the pop-up window and have a look to the respective KML file structure.


PS: I'll have to include a link to this thread on future map updates.

Posted by: SFJCody Sep 29 2010, 06:48 AM

The only reason I ever used the overlap was because of my crude method of matching the pieces by eye. There are bound to be better ways to do this.

Posted by: Tesheiner Sep 29 2010, 07:59 AM

Wow! I knew that was a *lot* of work put on creating the map layer but not that much!

Posted by: BrianL Sep 29 2010, 12:00 PM

Why can't Google do this work? Has anyone asked them?

Posted by: Stu Sep 29 2010, 12:51 PM

They're too busy quietly taking over the world, one website, one byte, one pixel at a time...

Good idea tho, we should definitely try encouraging them to join in the adventure.

Posted by: djellison Sep 29 2010, 02:11 PM

Is there any reason we should expect them to? Have YOU asked them?

Posted by: BrianL Sep 29 2010, 04:58 PM

Because it seems like an obvious benefit to all users of Google Mars. And I would not be so presumptuous as to be the lead voice on this when I have had nothing whatsoever to do with this imaging to this point. I'm just putting it out there as an idea for consideration, because I think Google has the resources to take this on.

Posted by: walfy Sep 29 2010, 07:30 PM

I just tried "stitching" in the first image. GE's image overlay tool is very clunky, very difficult to make a perfect match with the previous detailed overlay that we've been enjoying. I wish I had more time to monkey with this. Any reason why they have to be applied in these broken up tiles? Why can't the original large file be overlayed in one fell sweep? Probably an obvious answer somewhere...

Posted by: Tesheiner Sep 29 2010, 09:07 PM

You answered yourself.

It's several hundreds of MB.

And back to the main topic everyone, here's a good reference on how to work with "region files" in GE. It's the key point to have an extended and high resolution map coverage on GE. It's not too complicated to implement but the process must be automated somehow since we are talking about dozens or perhaps a few hundreds of regions.

http://code.google.com/intl/en/apis/kml/documentation/regions.html

Posted by: fredk Sep 29 2010, 09:24 PM

I'm not familiar witht the images in question, but presumably the full hirise frame is much larger than we'd need to cover the proposed future route to Endeavour. Could it be cropped down to a single image that covers, say, a kilometre or a half kilometre on either side of the proposed route to Cape York? Would that be small enough to handle, without needing to piece together smaller subframes?

It would be nice to have the west rim of Endeavour as well, but that's still some time away for Oppy, and by then perhaps the folks at google will update GM...

Posted by: Tesheiner Sep 29 2010, 10:01 PM

So, we are talking more or less about nine square kilometers which, at 25cm/pix, corresponds to a picture 36000 x 4000 pixels big. Ouch! Too much for a single picture, I would say.

Posted by: elakdawalla Sep 29 2010, 10:12 PM

The region file generators that make images for Google Maps take a large original image, like our 8192 by 8192, and chop it and/or downsample it into different versions that get displayed at different zoom levels. When you are zoomed way out, Google Mars might display the photo in a single tile whose source file is only 256 pixels square. When you zoom in some, it'll display tiles from it that were chopped into, say, 1024 by 1024 chunks, then downsampled to 256 pixels square. When you zoom all the way in, you'll see the screen tiled with 256 by 256 images at the original resolution. This vastly improves performance and is what allows you to sweep around and zoom in and out so quickly. It's a technique that GIS applications have done for a long time.

Posted by: alan Sep 30 2010, 01:33 AM

Are subimages of the path between Oppy's current location and Santa Maria available? They work much better for me than using the overlays SFJCody provided for GoogleMars which slows to a crawl when the high-res overlays are enabled.

Posted by: jamescanvin Sep 30 2010, 07:37 AM

I still have them on my disk, in fact they were still on my web server until last week when I deleted them to make way for the new ones, doh! When I get the chance I'll dig them out.

James

Posted by: walfy Sep 30 2010, 09:02 AM

I get it now, somewhat. I downloaded MapTiler on my Mac. It worked great, chopped up the large file with Santa Maria crater into the tiles-pyramid, but I can't figure out what coordinates to use in the "Georeference" box in MapTiler. It places the top image of the pyramid over the whole Earth, completely smothers it, when using the default coordinates. Anyone know the coordinates to use that would place the tiles near Santa Maria crater on Mars? If I could get the tiles to land near Santa Maria, I could then fine-tune their position, I'm assuming. Maybe it has to be done by tweaking the code in the KML file.

Posted by: Tesheiner Sep 30 2010, 12:43 PM

Wow, downloaded and tested MapTiler and I think that's *the* tool we need.
Regarding your question about coordinates: it uses a similar definition of bounding box as in KML. In this case you should input the north, south, east, and west coordinates separated by spaces. Just for testing I used the lat/long coordinates of a KML file I had here which corresponds more or less to the Meridiani landing site: -1.899331338 -2.44643146 -5.298233309 -5.552521575. You can do the same. The result is the picture of Santa Maria all over the landing site but, more important, it zooms in and out smoothly.

Now, about the real coordinates...
I would do the following (and will probably do later ondone):
1) Look for a feature near the right edge of SFJCody's map extension, annotate its lat / long coordinates, search for the same feature in the picture with Santa Maria and annotate its (x, y) coordinates.

2) Calculate the angular size of the picture and of a single pixel. It has 8192 x 8192 pixels so it covers 2048 x 2048 meters. Given the mars circumference size (TBD) the picture covers X degrees and a single pixel covers Y degrees.

R = 3396.2km (equatorial), 3376.2km (polar)
X = 2.048 / (2*PI*R) * 360deg.
Y = X / 8192

3) Calculate the lat / long coordinates of all four borders using the results of 1) and 2).

north -2.160015615
south -2.194771186
west -5.45866131
east -5.424110413

Now, let's try it...

Posted by: Tesheiner Sep 30 2010, 04:12 PM

Bingo!

Thanks Walfy for finding the right tool! http://www.maptiler.org/
Emily, I think the biggest problem to have the extended maps is solved but I still see an open point which is the transparency of the black areas outside the map-projected picture. The original 8k x 8k images should be available in PNG format with the transparency already set.

Posted by: walfy Sep 30 2010, 05:12 PM

Nice progress! I had a feeling you would propel this forward. I was able to set the black part of the images to transparency by first opening the image in Photoshop, then going to "Save for web & devices." In there you can choose the transparency to the black when you set it to save as a GIF. Not sure if it's the best way, but SFJCody's images were done similarly because they have the same very small transparency artifacts along their edges when you zoom way in.

To match the lighting of the new images with the old, maybe you could open the image of SFJCody's in photoshop, the one nearest Santa Maria, then increase the canvas size of the image large enough to place the new and slightly darker image with Santa Maria in it. Once they are in there side-by-side, adjust the curves settings on the Santa Maria image until it matches SFJCody's image. Then crop out SFJCody's image, then you have a color-matched Santa Maria image.

But personally, I don't think it's necessary to change the brightness levels of all the images. If you're going to do it for the first one, then you're going to have to do it for all of them! It's not a big deal for me if the end result has 2 large image overlays of different brightness side-by-side. It will save a lot of moil and toil to skip the step of matching the brightness levels.

Those are my two bits. OK, back to my job. Otherwise, I'd be tinkering with all this... maybe tonight.

Posted by: Tesheiner Sep 30 2010, 05:23 PM

I have Serif Photoplus instead of Photoshop but AFAIK they have similar capabilities. GIF is probably not the best option because if you set black = transparency all dark areas *inside* the real image are also made transparent. Another option is to edit the picture, set/fill all the contour as transparent and save as PNG. Simple but ... it's too much for my computer; the image is too big.

Posted by: walfy Sep 30 2010, 06:49 PM

It's not a problem, as there are no pixels in the actual image of the surface area that are as black as the background pixels outside the image. I tested this using the "Magic Wand" selection tool in Photoshop, which you can set to select only pixels of the same color of an image. It only chose those pure black pixels outside in the background, nothing on Mars' surface area. One drawback of GIF image format is that it reduces the number of pixel types to just 256. The GIF still look identical to the non-GIF image, at least to my eye, but the thought of the color gamut reducing to just 256 colors sounds bad to me!

Your PNG suggestion sounds great, if I have time tonight I'll try it . My Mac can handle the large files.

Posted by: fredk Sep 30 2010, 07:18 PM

Presumably these are grayscale images originally, so that means 8 bits per pixel, or 256 shades of grey. (Even if they had 12 or more bits/pixel originally, you won't be able to take advantage of those on GE.) So you won't loose any information with a gif (gifs are lossless). Even if the images you're starting with are colour 24 bits/pixel, or 256^3 colours, all those millions of extra colours are completely superfluous. Depending on what format you chose for the new images, you may actually save some on the filesize by converting the original image to 8 b/p greyscale first.

For the sake of those downloading on slower connections, it may be worth checking which format would give the smallest final filesize. Jpegs would be much smaller than gif or png, but you won't get transparency with the jpegs. Is the transparency really necessary? Does it just mean getting those black edge regions like in Tesheiner's pics above? Can you just crop away all of the black regions, at the expense of loosing a small part of the new map? Or just live with the black regions?

Posted by: Tesheiner Sep 30 2010, 08:04 PM

Good to know that because I saw an option on Map Tiler to choose a transparent color. In that case there's no need to have a GIF or PNG image, just select black as transparent and voila!



That's correct but I think there's a misunderstanding here. The image files I'm talking about are those 8k x 8k which are the *source* for the tool. It then chops the image and the result is a quite big number of very small 256x256 images in JPEG and (that's what I read) PNG files with their corresponding region files. Therefore, once we finally get these map extensions working, the "end users" should not be concerned about filesize or type but just download the resulting KML and JPEG files.

Posted by: fredk Sep 30 2010, 09:04 PM

My appologies if I'm missing something here! When I look at the files in the folder for the previous map extension, "MER-B Map extension v2.0" (which Emily linked to in her first post above), it contains close to 5000 individual image files. Almost all of them are png files (there are 6 jpegs). The big majority of those png files don't include any of the black regions near the edges, that you'd want to be transparent. So it seems to me that we could reduce the filesize and download times for the next extension by a large factor if these files were jpegs instead of pngs. (I see close to a factor of 4 reduction in filesize going from png to 80% quality jpeg.)

Presumably we could eliminate the need for pngs altogether if we cropped away the black as I mentioned above (or lived with it).

Again, sorry if there's some obvious reason those pngs couldn't be substituted with jpegs... Is it that people want a lossless format to avoid the compression artifacts of jpegs?

Posted by: Tesheiner Sep 30 2010, 09:35 PM

Oh, now I see our misunderstanding. You are talking about the previous map extension and that one was not made using Map Tiler.
The tool I'm talking about actually is quite optimized. Given a PGN input file it creates a big bunch of tiny JPEG files but only PNGs on those areas where some pixels are transparent. I did a little test with a 4k x 4k PGN file and the tool cuts it in 213 JPEGs and 54 PNGs spread on 36 folders. Not bad.

Another point would be to redo the previous map extension using Map Tiler. That would be great in terms of optimization but if it's not registered to the very same lat/long coordinates the route map would become completely displaced on top of the background map.

Posted by: elakdawalla Sep 30 2010, 09:54 PM

Google might not but Ross Beyer and his group at NASA Ames, who originally worked with Google to make Google Mars, is willing! http://www.unmannedspaceflight.com/index.php?showtopic=6750!!!

(Meantime, Tesheiner has contacted me about a shorter-term solution that should work for Santa Maria, based on the use of MapTiler that you all figured out in this discussion, hopefully we'll sort that tomorrow)

Posted by: SFJCody Sep 30 2010, 10:28 PM

Lots of activity in this thread. I guess my map is finally going to be supplanted with something better!

Posted by: walfy Oct 1 2010, 12:37 AM

Great news! If anyone should do it, some folks at NASA!

Posted by: elakdawalla Oct 1 2010, 04:39 AM

OK, we are almost there, but not quite. I downloaded MapTiler and ran Eduardo's selected of the Santa Maria image through it. In a matter of two minutes MapTiler generated 1000 tiles and associated KMZ files with a single KML file in the top directory. When I opened the KML file in Google Mars, bingo! It worked just as Eduardo said. The numbers for the corner coordinates aligned the tile nearly perfectly with the existing map (it's offset to the northish by about 3 meters, give or take a meter). So far so good. That is a solution that will work for people -- as long as they are willing to download 20 MB worth of data per 8000-pixel-square image tile.

But Eduardo suggested that I save everybody a lot of hard drive space and hassle by hosting the image tiles at the Amazon S3 server (where the Planetary Society serves large files) so that people don't have to download the entire enormous map. Using MapTiler I specified the location of the S3 server (http://planetary.s3.amazonaws.com), and it regenerated the tiles, KMZ files, and KML files. The top-level KML file looks like it points to the right place:

However, when I try to open the local copy of the KML file in Google Mars, it does not *quite* work. (Please try it for yourself: http://planetary.s3.amazonaws.com/santamariatest/santamariatest3.kml.) It's really puzzling, because it's not a matter of a broken link. It is clearly finding the georeferencing information within the KMZ files to correctly locate the corners of all the image tiles, but the images are not showing up:


What is really weird is that if I point my browser to one of the KMZ files, it finds the image data just fine. For instance, try http://planetary.s3.amazonaws.com/santamariatest/0/0/0.kmz (the very same link listed in the code block above, which is the lowest-res tile, the "apex" of the image pyramid) for yourself, and you should see this:

It works! So I don't understand why opening the KML file does not work.

Here is what I did with MapTiler. Help me figure out where I am going wrong and why the image tiles aren't showing up.
Tile Profile: Select Google Earth (KML SuperOverlay)
Source Data Files: Locate local file. I started with a PNG version of the image Eduardo selected, the tile that overlaps Santa Maria; I set the black pixels transparent using Photoshop. Click on its name and "Georeference" button. Put in 4 coordinates listed by Eduardo in green text in his post above, in 'north south east west' order as software specifies.
Spatial reference system: I selected WGS84. I wonder if we should be specifying some Mars-specific projection here -- but WGS84 seemed to work OK for the locally stored version of the map.
Tile Details: Accepted defaults (min zoom 0, max zoom 5; Hybrid JPEG + PNG)
Destination: Result directory: I specified the name of a local directory (the path to the image plus the folder name "santamariatest", so in the end it was "C:\Documents and Settings\Emily\My Documents\Google Mars\james canvin's images\santamariatest"); Destination URL: "http://planetary.s3.amazonaws.com"
Viewers: Google Earth (KML SuperOverlay)
Viewer Details: I specified a map Title (this is just metadata)

Posted by: Tesheiner Oct 1 2010, 09:36 AM

It's working perfectly for me.
It takes some time to download the first time but all LODs are working.

Posted by: walfy Oct 1 2010, 04:51 PM

It worked for me, too, but took awhile to download, though not a problem with patience. I would love to work on it too, but unfortunately, do not have the time now. Let's hope NASA gets on it posthaste!

Emily, maybe you need to refresh the KML file within GoogleEarth, as it might have a cached version of the KML file that's still linking to your own computer for the images. Just a guess, not really sure.

Posted by: elakdawalla Oct 1 2010, 05:08 PM

Holy cow, suddenly, this morning, it worked! I think we have a solution that will work out for us!

My question now is: should we regenerate the first map extension from scratch? It would require someone (*cough* Tesheiner) to relocate the corners of the beginning image tiles. I would also think it would be better to regenerate the base images so that they are non-overlapping (which probably needs James' help). And if we are going to do that, and are doing all the map segments at once, I think we should try to match the levels among the different base images to make them appear seamless. This last step I can do very easily (by determining what levels adjustment is needed using the browse versions of the HiRISE images, then using a Photoshop macro to run it on the folder full of 8192 by 8192 chunks).

--Emily

Posted by: Tesheiner Oct 1 2010, 05:45 PM

Better maybe, but we simply would not be able to follow Opportunity's trek on Google Mars without your help.
For an "end user" such map overlay might look just line another picture but those who have worked (or tried to work) with multi-resolution images on GE should understand how much time it takes to get such map working without the help of an automated software. Thanks a LOT for the big effort you put on that map!


I like this proposal.
Just a remark about the non-overlapping tiles. Better test it first with e.g. two or three tiles, just in case the gap between them is visible on GE. If that happens, some overlap should be kept.

Posted by: elakdawalla Oct 1 2010, 06:04 PM

Good point, we should test. I do think think that if you set corners to be slightly *inside* each other -- adjusting this at like the 6th or 7th decimal place -- then it should work at all levels of detail.

This is reminding me of something that was deeply maddening to me when I was first doing planetary GIS work in grad school, going between ISIS and ArcView. There were different conventions for what the corner of a raster image actually meant. For one piece of software, the convention was that the specified corner coordinates were at the extreme upper left, upper right, lower left, and lower right corners of the image. For the other piece of software, the convention was that the corner coordinates were at the centers of the corner pixels in the image. I do not remember for sure which was which, but I think it was ArcView that called the corner coordinates the centers of the corner pixels. This would make sense because it would prevent the possibilities of image-to-image gaps with extreme zoom that you raised.

Posted by: Tesheiner Oct 1 2010, 06:13 PM

Let's test it but IMO I think the corners should be calculated in a way that the whole north border of the new map matches with the base map around Victoria. For the example with Santa Maria I calculated the coordinates using Mars radius as I found in a web page with Google; now we should do it slightly different.


For our case, I think it doesn't matter the convention IF the definition is kept the same from tile to tile.

Posted by: elakdawalla Oct 1 2010, 06:34 PM

My point is that if the convention is for it to be the centers of the corner pixels, then if you specify the same lat/lon coordinate for the corners of two adjacent tiles, then the two tiles will *always* overlap by one pixel no matter what zoom level you use.

Posted by: elakdawalla Oct 1 2010, 10:50 PM

Well, Tesheiner, you were right. I ran MapTiler on the other Santa Maria segment, using the same east and west coordinates as for the first one, using the first one's north coordinate for the new one's south edge, and doing simple arithmetic to arrive at the north coordinate. Obviously this is not the exact correct location for the "Santa Maria North" tile, because it is really 10% overlapping, but I wanted to test to see if it would really line up adjacent to the first tile.

It did not. There is a gap -- a rather big one, actually, more than 11 meters. So it looks like we will have to stick with the overlapping images.

Posted by: Tesheiner Oct 1 2010, 11:37 PM

Strange...
I was expecting a gap of maybe a pixel or two but 11 meters is a lot!

I just did a different test to compare with your results. Opening the map with Santa Maria, I manually activated the option which enables opening the map folders to see the whole map structure. Then I deactivated the picture for LOD 0 (the top one) and all other LODs except 1. LOD 1 is composed of four individual pictures and there's no gap between them.

Then I checked the properties of those four images and the coordinates match each other exactly.

I'm puzzled.

Posted by: Phil Stooke Oct 2 2010, 01:42 AM

Am I missing something here? I downloaded those HiRISE images ages ago when they were first made available to us, and they go right up to the rim of Endeavour. And they are still available, are they not? Why are we adding more? Or did the old files evaporate?

Phil

(EDIT - oh yeah - I guess I did miss something. Sorry Emily!)

Posted by: elakdawalla Oct 2 2010, 04:00 AM

Are you talking about the reduced-resolution version? As I mentioned in the first post, SFJCody's first product was a reduced-resolution version that did go all the way to Endeavour. We're now working on full-res products.

Posted by: Tesheiner Oct 2 2010, 08:52 AM

I see what's happening here. The borders' coordinates I posted before for the tile including SM were calculated using two different values (polar and equatorial) for the mars radius. But after generating the map overlays giving those coordinates as "georeference" I see that the tool changes the south latitude from the given -2.194771186 to -2.19456651200000; check it on the top level KML file. Perhaps that's a consequence of using WGS84.
Then, if you generate the second map layer north of the first one calculating the north/south coordinates based on the ones I provided on that post and not the ones calculated by the tool, the gap appears. But if you generate the layer based on the coordinates from the KML file, there's no gap.

So, perhaps we won't need overlapping tiles but will have to have a close look on how to properly calculate the tiles coordinates.

Posted by: walfy Oct 3 2010, 02:07 AM

The northern border of the image of Santa Maria is at latitude 2° 9'36.06"S, according to GE. How do you turn that into the numerical convention you used above, such as the north -2.160015615 for 2° 9'36.06"S , as I'm attempting to fine-tune where the image is placed to match up more precisely with SJFCody's image.

Thanks for those numbers by the way! They are still very, very close!

Posted by: Tesheiner Oct 3 2010, 06:02 AM

You can select the way GE displays lat/long coordinates and among the options are: degrees with decimals or DDMMSS.
Or you can manually convert the minutes and seconds to the proper fraction value.

Wait a minute before matching the picture to the current extension map. If we finally redo SFJCody's map I think the new map may have slight differences in relation to the current one which would affect the placement of the tiles covering Santa Maria.

Posted by: Tesheiner Oct 3 2010, 10:39 PM

Back to this topic, I was playing working today on the task of finding the coordinates of the image tiles, and here's an Excel sheet with all of them and a little bit more.
 ESP_011765_Reg_to_PSP_001414.xls ( 118.5K ) : 768

The coordinates of the complete picture ESP_011765_1780 were calculated trying to register the image to the HiRISE picture embedded on GE. They don't match 100% but the difference is really small in the area I was concerned which is the point Opportunity's path moves from one image to the other one. From those coordinates to the ones corresponding to the individual non-overlapping 8k x 8k tiles the process was quite simple and straight forward.

I tested these data generating and processing the four tiles needed to cover the whole path and I'm quite happy with the result. GE zooms smoothly through all LODs.

Posted by: elakdawalla Oct 3 2010, 10:57 PM

Wow. Great work, Eduardo.

The next step that needs to be done is for James to regenerate the tiles in PNG format, but I know he can't get to this for a few days at least. Then I have to do levels, run them through MapTiler, and upload, which will take some time and I won't be able to start for two weeks -- I am covering DPS this week, and will be on vacation without access to my main computer for the week after that. So there may be a little break here. But I think we'll get to the new map before Opportunity gets to the edge of the current one

Just to have this posted in one place in the thread, the three images for the South of Victoria, Santa Maria, and Endeavour segments are, respectively:
http://hirise.lpl.arizona.edu/ESP_011765_1780
http://hirise.lpl.arizona.edu/PSP_005423_1780
http://hirise.lpl.arizona.edu/PSP_010341_1775
(correct me if I'm wrong!)

--Emily

Posted by: jamescanvin Oct 4 2010, 07:50 AM

I may (or may not) have a chance to set my scripts running to make the image tiles for Emily tonight. Has the decision been made if we want to stick with the current overlap, have no overlap or a smaller one? Any of these just as easy for me.

Posted by: Tesheiner Oct 4 2010, 08:22 AM

All tests I did with non-overlapping tiles resulted on no gap at all. So I would say 8192 x 8192 tiles without overlap and starting at the top left corner.

Posted by: elakdawalla Oct 23 2010, 10:06 PM

Sorry for the long break -- DPS happened, then some necessary travel. In the meantime James got me the two tiles of PSP_005423_1780_RED that we've been working on that cover Santa Maria in PNG format. Unfortunately, MapTiler is not handling these correctly, but I have a suspicion that it's because my old computer is running out of memory. I should have a brand-new computer in two weeks with more than enough memory to handle this problem, and then I'll revisit it. I *think* Opportunity will not get to the edge of the current map in two weeks...it would be awesome if I were wrong about that, though!

Posted by: tim53 Nov 2 2010, 04:00 PM

Here's mine:

It's a mosaic of CTX and HiRISE that I put together in Photoshop for traverse planning. I used Photoshop so I could take care of shading differences between overlapping images. I also used the lowest emission angle images to minimize topographic distortions to the map projection (which only corrects for the MOLA long-wavelength topography). At this scale, you can't see the HiRISE at full resolution, of course...

I made the mosaic so we'd have continuous HiRISE coverage of our route all the way to Endeavour.

When we're "done" with Endeavour, I'll have to make another map, I suppose!

-Tim.

Posted by: Tesheiner Feb 6 2011, 07:54 PM

I thought it might be time to resurrect this topic.
Once we're done with Santa Maria, Opportunity will be out of the current background picture in a matter of weeks.

I spent some time today reviewing the process required to have HiRISE coverage from Santa Maria until Cape York, which basically was: slice one of those big JPEG2000 pictures in 8k x 8k pieces, register the pieces to match with the portions already covered at high resolution, and render them with MapTiler. With a bit of patience and a couple of beers here's the result.


Voila! Ground coverage at 25cm/pix for the remaining path.


Now, what I need is some place to host the whole thing.

Posted by: elakdawalla Feb 6 2011, 10:24 PM

I'll be glad to host it. I can do it tomorrow -- Super Bowl today

Posted by: Robert S Feb 7 2011, 10:27 AM

Awesome!!! Can't wait for it to be uploaded, GREAT WORK !!!

Posted by: Tesheiner Feb 8 2011, 07:10 AM

Thanks, Emily, for the support and for hosting this big map.
Hello everyone! Follow this link to see the path to Cape York at HiRISE resolution.

http://planetary.s3.amazonaws.com/gmars_maps/PSP_010341_1775_RED.kml

Posted by: brellis Feb 8 2011, 08:01 AM

Thanks E.T. and Emily!

p.s. - happy birthday!

Posted by: fredk Feb 8 2011, 03:19 PM

This is fantastic! Thanks Tesheiner and Emily.

Posted by: Stu Feb 8 2011, 05:23 PM

Thanks, both, VERY much. I can't overestimate how useful that will be for my blog. Just fantastic.

Posted by: SFJCody Feb 8 2011, 10:50 PM

Fantastic! Much better than my earlier unfinished attempts!

Posted by: elakdawalla Feb 8 2011, 11:11 PM

Okay folks, I've now edited http://www.unmannedspaceflight.com/index.php?showtopic=6747&view=findpost&p=164644 to include all the base images.

Unless you have made edits to the base map layers in Google Mars, I recommend that you http://planetary.s3.amazonaws.com/gmars_maps/Opportunity_base_maps.kml, which contains all three of the base map extensions, neatly named and credited. If you use this version, all of the image files are being hosted remotely, on the Planetary Society's space on the Amazon S3 server.

Posted by: brellis Feb 8 2011, 11:34 PM

I've found my religion, and it is Google Mars, via Saints Emily and Eduardo! Thanks so much

Posted by: elakdawalla Feb 8 2011, 11:34 PM

Don't forget to thank SFJCody -- he did the lion's share of the work getting earlier map updates done.

Posted by: eoincampbell Feb 9 2011, 12:31 AM

Thank you one and all for mapping,hosting and posting this... Just Awesome

Posted by: Robert S Feb 9 2011, 09:02 AM

I am so pleased right now that words can't describe it! EXCELLENT WORK. I am also happy that you have it all covered on the first topic now!
You guys are the best, many thanks to all of you who have made this possible!

Posted by: vikingmars Feb 9 2011, 11:30 AM

It reminds me a guy from Lehman Brothers early 2008 saying "My religion is the Stock Market"...

Posted by: MoreInput Feb 24 2011, 09:14 PM

Also from me very much thanks for the maps: Let's set the sail to Cape York!

Posted by: elakdawalla Apr 14 2011, 04:06 AM

There's a lovely new color view of the northwest rim of Endeavour: http://www.uahirise.org/ESP_021892_1775

This makes me wonder what parts of Opportunity's path have been covered in color. It's a question I know I could answer myself if I had the time, but I don't. If I were going to answer it, I'd figure out which images crossed Opportunity's path and then identify the regions lacking color coverage.....

Posted by: ElkGroveDan Apr 14 2011, 04:18 AM

Posted by: eoincampbell Apr 14 2011, 04:51 AM

Thanks for posting that beautifully colored view...
I notice the related context swath was out-with Oppy's current travails...
Is HiRISE planned to continue on Oppy's trek, by any routine ?
Happy HiWISHES

Posted by: Astro0 Apr 14 2011, 08:02 AM

This is a start on the colour coverage with the latest images.

If anyone would like a http://astro0.files.wordpress.com/2011/04/route_hirisergb_overlay.jpg (17.7mb)

Posted by: Stu Apr 14 2011, 05:14 PM

Tweaked the new HiRISE image of Cape York to make it look a little less garish, hopefully a little more natural, and then, just for fun - so not claiming mega-accuracy - dropped some 'virtual Oppys' onto it at various places, to show the scale of the features at Cape York...

This might be useful for our lower-tech members who can't access HiRISE images via the IAS Viewer, or the Google Mars images.

Edit: more pix at: http://roadtoendeavour.wordpress.com/2011/04/14/a-new-view-of-cape-york

Posted by: Explorer1 Apr 14 2011, 05:36 PM

Oh wow!

Once we get on that color patch, that'd be as good a place as any to proclaim official 'arrival', right? For something this big, there's few well-defined borders....

Posted by: eoincampbell Apr 15 2011, 05:23 AM

Lovely view and I like those possible routes, Stu. Might the team forego those excursions in pursuit of the clays at Tribulation?

Posted by: KrisK May 19 2011, 10:30 PM

Lately I've been working on HiRISE GE extension of Endeavour rim, the area south of Cape York. As a result I uploaded this kmz example. The procedure was as follows:

1. cropped PSP_010341_1775_RED using JP2_PDS/PDS_JP2 ("Area" option)
2. converted JP2 to png using "convert" (ImageMagick)
3. loaded png to GE, tuned North, South etc. coordinates and finally saved as kmz

Surely this method is not optimal. As a consequence I have one big image being loaded so GE hangs a little bit while loading. But later it works ok At the beginning I wanted to use MapTiler (just as it was done in previous posts) but it produced strange results for this big png file. My tests with smaller example files looked more optimistic with that program.

Here are some screenshots:

http://www.flickr.com/photos/55907406@N08/5738088526/sizes/m/

http://www.flickr.com/photos/55907406@N08/5738088544/sizes/l/
http://www.flickr.com/photos/55907406@N08/5738088528/sizes/l/
http://www.flickr.com/photos/55907406@N08/5738088534/sizes/l/ (here it looks that I need to work more on that )


I decided to upload kmz maybe sbd would like to use it...
https://rapidshare.com/files/1768082589/test_endeavour_rim.kmz

Posted by: Tesheiner May 20 2011, 06:08 AM

That sounds great!
But, as you said, it's not optimal given the file size; actually, my laptop almost freeze during 1-2 minutes and finally finishes with a big "X" (can't display) instead of the real picture. I'll suggest you to keep working with MapTiler but splitting the big source file into smaller (e.g. 8000x8000) pieces first.

Could you detail it a little bit?


That's due to small differences in the lat/long coordinates you are using (in relation to the map I made). I can provide you with some data that should solve it, if you want.

Posted by: KrisK May 21 2011, 03:59 PM

Here you can see result: http://www.flickr.com/photos/55907406@N08/5743320902/sizes/m/

Finally I decided not to use MapTiler. Instead I wrote my own bash script to create super_overlays. The following link was very helpful:
http://code.google.com/intl/pl/apis/kml/documentation/kml_21tutorial.html#superoverlays
At this stage I can say that I almost there:

http://www.flickr.com/photos/55907406@N08/5742795813/sizes/m/
As you can see there is a brightness difference between tiles. Have to think what is the origin of the problem...

Below is the area (size 2^n) that I want to overlay in GE:
width = 16384
height = 3x16384
so I have three squares 16384x16384. Currently I work with the first square.



Yes please ...Previously I made the alignment mostly by hand now I have the following strategy:
1. take N,S,E,W out of LBL for full HiRISE product....DONE
2. recalculate N,S,E,W for my cropped area....DONE
3. add/subtract deltaLAT, deltaLON to perfectly match the existing overlay...TBD

So I suppose your data will allow me to finish step 3 right?

Posted by: Tesheiner May 21 2011, 09:39 PM

Mmm, more or less.
I'm providing some of the data I used to create the 8k x 8k tiles you already see on Google Mars and are hosted by TPS. So, if you use them and extrapolate the coordinates to the 16k x 16k tiles you want to create, they should match 100% with the current map. Note that I'm not saying that they are accurate (they aren't); they are only good for creating new maps matching with the current ones.

Lat/Long coordinates of PSP_010341 upper left corner: -2.07473360826446, -5.44490800928163
dLat/dLong of a pixel on the original picture: 4.21926255562616E-06

Posted by: KrisK May 22 2011, 04:55 PM

Thanks for the numbers. They were very helpful. It looks I have a perfect match:

http://www.flickr.com/photos/55907406@N08/5746683084/sizes/m/

Btw, asking out of curiosity...in LBL we have: MAP_RESOLUTION = 237098.79009322 <PIX/DEG> which gives:
1/237098.79009322=4.21765121452889e-6 but we are using 4.21926255562616e-6
Why is there a difference?


Oh and the problem with the brightness among tiles was caused by Google Earth. Somehow GE had problems using png's which I was creating using Image Magick "convert" program. Switching to jpg's resolved that issue.

Anyway...FINISHED!!!

http://www.flickr.com/photos/55907406@N08/5746292143/sizes/m/
http://www.flickr.com/photos/55907406@N08/5746842018/sizes/m/

Zipped kml's download link:
https://rapidshare.com/files/2039227902/endeavour_west_rim.zip
Edit: added another link just in case...
http://hotfile.com/dl/118744535/aaffb88/endeavour_west_rim.zip.html

The only missing thing is the black area which could be removed. I don't know but maybe I'll do it somewhere in the future.

Posted by: Zeke4ther May 23 2011, 04:50 AM

Great work! Well done!

Posted by: Tesheiner May 23 2011, 09:11 AM

As I said before, the data may not be accurate but is the one you have to use to match with the previous maps.
IIRC, I first calculated that value when attempting to match my version of ESP_011765 (the one covering the western detour) using MapTiler to SFJCody's version of the same map, which in other hand was already registered to the background covering Victoria embedded in GE. Later on, that value was used on subsequent maps to keep them matching together.

Posted by: Ross Beyer Jun 22 2011, 09:34 PM

As I had mentioned a long, long time ago, we (the Ames Planetary Content team) were working on constructing new HiRISE/CTX overlays in the Google Earth client for the Opportunity route. We're far enough along that I want to have you guys take them for a spin so that I can get your feedback.

The attached KML file has NetworkLinks that point to our servers at Ames.

As far as geo-position goes, we decided to leave Victoria Crater where it is, and adjust the surrounding images. At some point in the future (I've been told "after the mission is over" as a time frame), there will be a giant geolocation solution by members of the MER team, at which point we'd move everything to be consistent with that, but its far enough in the future that I'm not gonna worry about it right now.

Also, the HiRISE image that covers the Endeavor rim isn't just any HiRISE image, but an ortho-image made from a HiRISE terrain model, and if things work out the way that I want them to, by the time this basemap makes it into the client (end of summer? fall at the latest, I hope), the HiRISE terrain will be in there, too.

Posted by: hendric Jun 22 2011, 10:08 PM

Works very well from here.

Posted by: Tesheiner Jun 22 2011, 10:58 PM

From here too. It runs smoothly through the different levels of detail.

On the topic about geo-positioning there are, obviously and expected, differences with the maps we are using right now but it wouldn't take too much time for me to adjust my record of the route to this new HiRISE background; I only need to reposition some 15 "control points" I currently use on the map and all route points and ground features would be automatically aligned to the new background.

Posted by: brellis Jun 23 2011, 12:48 AM

It's always like Christmas morning surfing through the updated images on Google Mars. Thanks Ross, Tesh et al!

Posted by: CosmicRocker Jun 23 2011, 04:51 AM

That would be truly awesome. I can hardly wait.
Many thanks to you and your team for the excellent work.

Posted by: Ant103 Jun 23 2011, 07:43 AM

That's amazing ! Excellent work !

Posted by: Robert S Jul 17 2011, 03:11 PM

Fantastic job!!!!

Posted by: tim53 Aug 12 2011, 09:52 PM

Hi Ross:

I just completed a new basemap of CTX, MOC and HiRISE images in Arcmap that I'll be using for my localization work on Opportunity (I call this "HiThere!"). This map covers all of our traverse from Eagle Crater through Cape York, as well as all of Endeavour and Iazu Craters (since we can see so much of the interior of Endeavour, and Iazu is on our horizon). I first georeferenced the CTX low emission angle images to the HRSC and MOLA DEMs, then tied the HiRISE images to them (and each other, where they overlap). For Endeavour, there is still an annoying gap in the HiRISE coverage just east of center, so I used a couple of MOC images (3m/pixel) to fill that gap. Next step will be to correct the shading from one image to the next so the seams don't detract from the morphology. I've done this for the CTX background mosaic already, and I'm in the process of adjusting the HiRISE images. Next step will be to tile up and get the basemap into our planning software. But once that's done and the route is on it, exporting the path to applications like Google Mars ought to be straightforward. Much sooner than "after the mission is over", I'm hoping.

-Tim.

Posted by: elakdawalla Aug 13 2011, 02:32 AM

Apart from that, I applaud the rest of your post and hope to see the product before the mission is over!

Posted by: tim53 Oct 31 2011, 08:35 PM

This map is now completed. Stay tuned, as Trent Hare will be posting links to it on the USGS website.

While I consider the map "completed", I reserve the right to continue to update it as better/more images are acquired and I can georeference them to the base MOLA and HRSC topography maps. But I need to get cranking on the Gale basemap for localization and planning during Ops!

-Tim.

Posted by: Zeke4ther Oct 31 2011, 10:47 PM

cool!!

Posted by: tim53 Nov 17 2011, 10:08 PM

Hi all!

Sorry this took so long to finish (at least Opportunity is still going!), but Trent Hare of the USGS Flagstaff, has posted my new basemap on a couple websites for quick viewing. These sites are temporary, so I don't know how long they'll be up. There's also a downloadable kmz file for placing in Google Mars. I tried it, and it works, but a colleague here couldn't get it to display. There are some issues - like too hard a stretch - that will be dealt with in future postings.

From Trent:

There's also an FTP site for downloading a JP2 version of the file: ftp://pdsimage2.wr.usgs.gov/pub/pigpen/mars/hirise/MER_mosaics/

And the associated readme text file: ftp://pdsimage2.wr.usgs.gov/pub/pigpen/mars/hirise/MER_mosaics/MER_Meridianni_Endeavor_Basemap_25cm_README.txt .

I've only given this a quick look, so I apologize if there are issues that I've missed. The map itself needs a couple of corrections that I'll get back to after I return from the Curiosity launch week after next. I've got it into our planning software here for localization and traverse planning. Most annoying (to me) are some residual shading differences that I need to work on some more. The worst of these got that way when Fred Calef and I decided not to include a pair of MOC images over a gap in the HiRISE coverage, as they weren't a significant improvement in resolution over the CTX background mosaic and didn't have the dynamic range - I'd adjusted the shading of the HiRISE image to match the MOC, so it is darker than the CTX background.

In addition to making these kinds of corrections, I plan to make revisions as more data is acquired and released by HiRISE and as time allows.

I hope some here find the file useful. Enjoy!

-Tim.
P.S. Sample screen grab of mosaic with Opportunity path from Eagle to Sol 2777:

Posted by: Nirgal Nov 17 2011, 11:11 PM

That's 35 Gigapixels: Cool

Thanks especially for posting the full resolution version: much appreciated.

Posted by: Phil Stooke Mar 25 2012, 09:11 PM

These really are fantastic pictures. I can see I'll be looking at this a lot when I start illustrating the Opportunity route later this year. I suggested to Tim that he re-map the whole area at Microscopic Imager resolution so we could zoom all the way from CTX to HiRISE to reprojected Navcam to Pancam and finally to MI. But he thought that might overtax even his computer. Poo! Time for an upgrade...

Phil

Posted by: Tesheiner Aug 12 2012, 10:23 PM

Some people already mentioned that the background maps at Curiosity's landing site were updated. Actually, there was an update not only to that part of the planet but a much more global one, incluing changes here at Opportunity's site too. You might remember that HiRISE coverage on GE was only available around the landing site and Victoria, and there was and effort here at UMSF to extend that coverage up to Endeavour (see Emily's http://planetary.org/blogs/emily-lakdawalla/2011/2909.html). Later on, Ross Beyer of NASA/AMES mentioned about a working project and provided us access to newer and great background layers (see http://www.unmannedspaceflight.com/index.php?s=&showtopic=6747&view=findpost&p=174660). Now, it looks like those ones have been finally released and integrated on the "mainstream" Google Earth / Mars database.

Well, not only that.

Zoom in at the Opportunity site, double-clicking on the "MER Opportunity Rover" layer, and you will see that the route path embedded on GE which was cut short right after Victoria has been updated too and covers the ground up to the last driving sol. Also, the "ground features" along the route were also updated and are the same ones I use in my route map.

Coincidence? No. Have a look to the bubble which opens after clicking on the link at the "MER Opportunity Rover (USA)" layer.

Some time ago, I was asked permission to integrate my route map updates into Google Earth in a way that they could be viewed not only by the people here but also by *anyone* on Internet using GE. We agreed a mechanism to do that (a sort of pipeline) so if everything is working correctly (and it was until the last drive) the path "embedded" on GE should be updated too in a short time. I like it.

Posted by: Floyd Aug 12 2012, 10:36 PM

Congratulations Tesheiner. Thanks for all your great work here, and now the whole globe can follow Opportunity's moves.

Posted by: SFJCody Aug 13 2012, 03:05 AM

Awesome stuff! Thanks hugely for all your tireless efforts over the years.

Posted by: CosmicRocker Aug 13 2012, 04:10 AM

Excellent, Eduardo!

Posted by: Stu Aug 13 2012, 06:33 AM

That's fantastic Eduardo! After all these years of hard work following and recording Oppy's great trek you deserve recognition for your hard work.

Posted by: climber Aug 13 2012, 07:37 AM

Congratulations Eduard Tesheiner...

Posted by: Reckless Aug 13 2012, 07:54 AM

Well done Eduardo and thanks for all the work you've done.
Your updates of the MER routes have always been eagerly awaited buy me and other fans here at unmannedspaceflight and now the whole world of Mars fans can see your hard work.
Roy F

Posted by: polaris Aug 13 2012, 08:02 AM

A Gold medal for Eduardo !

Posted by: Ant103 Aug 13 2012, 12:35 PM

That's simply GREAT Eduardo ! Congratulations

Posted by: Zeke4ther Aug 13 2012, 07:26 PM

Congratulations Eduardo! You deserve the recognition

And the tech is pretty cool if updating here means an update to GE. Got to love it!

Posted by: Tesheiner Aug 13 2012, 09:32 PM

Not exactly like that. I have a single "database" from where I generate the KMLs posted here and the data feeded to GE, but both are independent paths.

Posted by: Pando Aug 14 2012, 02:26 AM

Hey Eduardo,

This is so very cool. I can't be more amazed at your dedication in providing timely maps for all of us.

As you know, I started tinkering around with Oppy maps here in February 2005, using just MS Paint, often relying on some best guesses on where the rover might be, and that was before Google Earth even existed. You have taken it to a whole another level and way beyond.

Again, congratulations! I sincerely hope that you have the means to do this for MSL as well.

Pando