Printable Version of Topic

Click here to view this topic in its original format

Unmanned Spaceflight.com _ Venus _ need help. how to uncompress .img files from magellan

Posted by: ncc1701d Jan 6 2011, 11:33 PM

Hoping someone can help.
I am trying to uncompress the FMAP .img files which are part of magellans images of venus so I can use a converter program to convert them from sinusodal to cylindrical mapping.

Things I do know is that I can just rename .img to .raw and open in photosho however the converter program newmap.exe which is an old dos program that is provided with the FMAP images requires I think the uncompressed .img files in order to convert sinusodal to cylindrical. I know the .raw didnt work. And I get error saying my .img files are compressed so it wont work.
I know .img files are not cd or dvd image file or so I think becuase I could not open them in MagicISO If they were CD images I would think MagicISO would work but I may be wrong though.
Anyone familiar with what I am trying to do. Any image processing guys out their do this with the magellan FMAPS?
I figure the experts would be here.
thanks

Posted by: djellison Jan 6 2011, 11:39 PM

URL to the files in question?

Posted by: Phil Stooke Jan 6 2011, 11:44 PM

.img files are not compressed, they are basically tiff-like files with a different header which can be opened in Photoshop as raw images if you know the pixel dimensions (which are given in the label). You probably knew that anyway.

But there are also .imq files which are compressed versions of the .img images. Those have to be treated differently - if you are getting the .img files from the old PDS cd-roms there might be a utility on the disk. I used to deal with those files all the time (and still have probably well over a hundred of the cd-roms collecting dust), but it's been so long I can't remember the details of it now.

It's possible your software just doesn't recognize the PDS .img format and simply thinks it is compressed when it isn't. If the software needs tiffs, for instance, you might have to open in Photoshop and save as a tiff.

We could really use a good clickable map interface to the various levels of Magellan products, too!

Phil

Posted by: elakdawalla Jan 7 2011, 12:31 AM

I can't remember if the FMAPs are stored the same way as the MIDRs, but the MIDRs were put in sinusoidal projection and then cut into 56 1024-by-1024 tiles per image; you have to open all 56 tiles as raw (which you can do in Photoshop as Phil said, just skip a couple of lines' worth of bytes to get past the header) to get the image before you can reproject and mosaic adjacent images together. This was a huge headache and I actuallly have a set of 27 CDs I made as a grad student containing Photoshop format (PSD) versions of all the C1-MIDRs where I assembled all 56 tiles for each image.

The map interface at the USGS website -- which was redesigned a couple of years ago -- provides clickable interface to the FMAPs. It is a bit slow but very good in that it can generate really enormous images at your choice of projection. http://www.mapaplanet.org/explorer/venus.html That interface is what I used to create the 3 Venus images I used in my Advent Calendar:
http://planetary.org/blog/article/00002801
http://planetary.org/blog/article/00002834
http://planetary.org/blog/article/00002845

Posted by: ncc1701d Jan 7 2011, 12:42 AM

here is the link
ftp://pdsimage2.wr.usgs.gov/cdroms/magellan/mg_1103/

for example grab a sinusodal image called "fl78n018.img" from folder fl78n018 and go into software folder and then pcdos folder and use newmap.exe

example: type this in newmap. (i think i made mistake in my message calling it newone)(this is 1994 program)

newmap fl73n003.img fl73n003new.img -p

("name of input file" "name of output file" "-p specfies cylindercal")

I get error saying my input file has not been decompressed
thats my unknown problem


I am in xp 32bit using dos prompt

thanks

Posted by: siravan Jan 7 2011, 12:46 AM

IMG files usually have either a text header or a separate text .lbl file, which you can read using a text editor and figure out what to do with them. A typical example is included here. It basically means that the file is organized into 20682 fixed-length records, each 30776 bytes in length (i.e. total file size is 30776*20682=636509232 bytes). ^IMAGE line means raw data begins at the second record (it is a 1 base system, so the first record is 1, not 0), hence, the image data starts at byte 30776. Here, each pixel is recorded as a 32 bit (SAMPLE_BITS) floating point number (SAMPLE_TYPE=PC_REAL, technically little-endian IEEE 754 floating-point number; in practice, this is the 'float' type in most programming languages). The image dimension is 20681 x 7694 pixels. The data is organized row first, so the first line of data is from byte 30776 to 30776+7694*4-1=61551. You can read it using a raw data import utility (Photoshop has one, I guess Gimp has one too), or if you are prone to write your own code, you can write a quick import program in the language of your choice.

PDS_VERSION_ID = PDS3

/*File format and length */
RECORD_TYPE = FIXED_LENGTH
RECORD_BYTES = 30776
FILE_RECORDS = 20682
^IMAGE = 2

OBJECT = IMAGE
LINES = 20681
LINE_SAMPLES = 7694
BANDS = 1
OFFSET = 0.0
SCALING_FACTOR = 1.0
SAMPLE_BITS = 32
SAMPLE_BIT_MASK = 2#11111111111111111111111111111111#
SAMPLE_TYPE = PC_REAL
MISSING_CONSTANT = 16#FF7FFFFB#
VALID_MINIMUM = -366.22
VALID_MAXIMUM = 1793.79
END_OBJECT = IMAGE

Posted by: ncc1701d Jan 7 2011, 12:49 AM

concerning the other post i just noticed.

I used to use
http://pdsmaps.wr.usgs.gov/maps.html
which I prefferer over the

http://www.mapaplanet.org/explorer/venus.html
but they disabled the content.

the first one I mentioned used lat and long box for entering info and I loved that.
The new website is harder to use and I get lost trying to find locations and input specific locations.
Why they dont have at least a lat center point to go with the long center point when trying to find your way around I dont know on the new site.
I requested copy of old program they used on pdsmaps but they have not responded. Thats why I am looking into this on my own

Posted by: siravan Jan 7 2011, 01:10 AM

Here is the first image (reduced to 25% of its original size) in the folder:



The header is included below. I used ImageJ to import the raw with parameters: height =2830 (LINES), width=3184 (LINE_SAMPLES), offset to the file=12736 (i.e. ^IMAGE*RECORD_BYTES), pixel type is byte (SAMPLE_BITS=8). Also note there is a histogram contained in the third record.


PDS_VERSION_ID = PDS3

/* FILE FORMAT AND LENGTH */

RECORD_TYPE = FIXED_LENGTH
RECORD_BYTES = 3184
FILE_RECORDS = 2833
LABEL_RECORDS = 2

/* POINTERS TO START RECORDS OF OBJECTS IN FILE */

^IMAGE_HISTOGRAM = 3
^IMAGE = 4

/* POINTER TO THE TABLE CONTAINING ORBIT LOCATIONS */

^TABLE = "73N003OR.TAB"

/* IMAGE DESCRIPTION */

DATA_SET_ID = "MGN-V-RDRS-5-DIM-V1.0"
PRODUCT_ID = "78N018"
SPACECRAFT_NAME = MAGELLAN
SPACECRAFT_ID = MGN
INSTRUMENT_NAME = "RADAR SYSTEM"
INSTRUMENT_ID = RDRS
TARGET_NAME = VENUS
START_TIME = "N/A"
STOP_TIME = "N/A"
SPACECRAFT_CLOCK_START_COUNT = "N/A"
SPACECRAFT_CLOCK_STOP_COUNT = "N/A"
PRODUCT_CREATION_TIME = 1993-09-28T15:55:50
IMAGE_ID = FL73N003
MISSION_PHASE_NAME = {"MAPPING CYCLE 1",
"MAPPING CYCLE 2",
"MAPPING CYCLE 3"}

/* DESCRIPTION OF OBJECTS CONTAINED IN FILE */

OBJECT = IMAGE_HISTOGRAM
ITEMS = 256
DATA_TYPE = LSB_UNSIGNED_INTEGER
ITEM_BYTES = 4
END_OBJECT = IMAGE_HISTOGRAM

OBJECT = IMAGE
LINES = 2830
LINE_SAMPLES = 3184
SAMPLE_TYPE = LSB_UNSIGNED_INTEGER
SAMPLE_BITS = 8
SAMPLE_BIT_MASK = 2#11111111#
CHECKSUM = 938107697

Posted by: ncc1701d Jan 7 2011, 01:37 AM

I appreciate all the input here so thank you all.
But I think your missing the problem.
I could easily just rename the .img files on the cds as .raw and open in photoshop and look at them. I did that.
However they are all in sinusodal projection. I cant use the sinusodal. They need to be converted to cylindrical.
The only program I know of to use to convert to cylindrical projection is the newmap.exe dos program they provide with the FMAPS set. It is finding a way to convert to cylindrical that is the biggest problem. Maybe their are other tools besides newmap.exe but thats all I know of and I am not programmer and that program is 16 years old.

Siravan mentions how most .img files have .lbl files with them.
Along time ago I thought I remember all the FMAPS having .lbls with each image but I dont see them with the .img files anymore on the link I provided.
Maybe that is why my newmap is saying they need to be uncompressed is just becuase they need that .lbl ? I dont know.



Posted by: siravan Jan 7 2011, 02:34 AM

I recompiled newmap for XP using the source code contained in the directory included (attached). I had to make some minor modifications to the source code, but it seems to be working, at least it generates reasonable looking output, however I have no way to test if it is the correct output.

 newmap.zip ( 37.04K ) : 837

Posted by: ncc1701d Jan 7 2011, 03:00 AM

for some reason I get my same message

*** ERROR *** The input file has not bee decompressed.

can you tell me what you typed to get your results and did you still do it under a dos prompt?

did you ever recreate the problem i was having above?

Any chance you could send me your modified .c file so I can see your changes?

Posted by: siravan Jan 7 2011, 03:13 AM

This is the file I got:



I'm not sure about the vertical line in the middle. Is this expected in reprojection?

The source code is  newmap.c.zip ( 10.66K ) : 770
. It still needs to be run in command prompt. Note that I could not reproduce the error.

Posted by: ncc1701d Jan 7 2011, 03:24 AM

I have to think on this now. Its not what I expected. I didnt expect a line in center but we may be getting into territory of blind leading the blind since i am still learing this program.
Can you tell me exactly what you typed at the dos prompt while using newmap to get this image?


Posted by: siravan Jan 7 2011, 03:58 AM

This is the correct image (I made a mistake in reading the file first time). It seems ok this time!



BTW, I think this image is part of Semuni Dorsa.

Posted by: ncc1701d Jan 7 2011, 06:36 AM

are you just typing

newmap fl73n003.img fl73n003new_1.img -p

is this what you typed?

what does that mean you made a mistake reading it the first time?
did you need to change the code again to make it work or just use different command line instructions?

the new one looks right for sure.

for some reason I keep getting my .....input file has not been decompressed ERROR and I am wondering if I am missing something simple.

Posted by: JohnVV Jan 7 2011, 07:16 AM

if you want it is cylindrical you could grab them from PDS map a planet
and they will BE in that format if you choose it
http://www.mapaplanet.org/explorer/venus.html
http://pdsmaps.wr.usgs.gov/PDS/public/explorer/html/gredrintm.htm
http://pdsmaps.wr.usgs.gov/PDS/public/explorer/html/gtdrintm.htm

there is a size restriction and it might take 24 hours for a 2 gig image to have a ink mailed to you.

but when i made the map for celestia i used the data listed on the pds geosciences page
http://pds-geosciences.wustl.edu/missions/magellan/gxdr/index.htm
the gredr
http://pds-geosciences.wustl.edu/geo/mgn-v-gxdr-v1/mg_3002/gredr/
mixed with the data from gedr
http://pds-geosciences.wustl.edu/geo/mgn-v-gxdr-v1/mg_3002/gedr/
and the topo map
http://pds-geosciences.wustl.edu/geo/mgn-v-gxdr-v1/mg_3002/gtdr/
mixed with some of
http://pds-geosciences.wustl.edu/geo/mgn-v-gxdr-v1/mg_3002/gsdr/

then a BUNCH of "inpainting"


but you are using Microsoft ???? hhhmmmm might be fun
i used ISIS3 on linux
but the north and south are in polar stereo projection and Gimp can remap that to cylinder
so can the old AND STILL GOOD tool MMPS
http://www.users.globalnet.co.uk/~arcus/mmps/
i do not know if there is a prebuilt windows
I DO know it dose build on windows xp using MinGW - i did that a few years back

Posted by: ncc1701d Jan 7 2011, 08:31 AM

thanks siravan
I got it to work. My problem was the original file got corrupted somehow so it was screwing up program.
From what I understand the final image should be 8466 wide by 2830 pixels height. After running newmap the output image information it generated was 2830 lines x 8466 samples.
Your image size of 2116x707 pixel. Where did you get that size?

Posted by: ncc1701d Jan 7 2011, 08:43 AM


JohnVV
I used to use pdsmaps and it was very helpfull but it looks to me now the content wont show up. Does it still work for you showing content and everything? I just see an icon where images should show up.

mapaplanet I dont like much. I miss lat and long boxes for finding your way around like pdsmaps had. Some small features are harder to find your way back to in mapaplanet

Posted by: JohnVV Jan 7 2011, 10:27 AM

some of it still works
http://www.mapaplanet.org/explorer/venus.html
leftlook
http://www.mapaplanet.org/explorer-bin/explorer.cgi?map=Venus&layers=venus_magellan_fmap_leftlook&west=-180&south=-90&east=180&north=90&center_lat=0&center=0&defaultcenter=on&grid=none&stretch=none&projection=SIMP&r=1&g=1&b=1&advoption=YES&info=NO&resolution=2

you need to use the advanced option on the left
10 deg bonding box
http://www.mapaplanet.org/explorer-bin/explorer.cgi?map=Venus&layers=venus_magellan_fmap_leftlook&info=NO&advoption=YES&lines=400&samples=400&sizeSelector=resolution&Resolution=40&projection=SIMP&grid=none&stretch=none&resamp_method=nearest_neighbor&north=10&west=350&east=10&south=-10&center=0&defaultcenter=on&center_lat=0
and a bigger
http://www.mapaplanet.org/explorer-bin/explorer.cgi?map=Venus&layers=venus_magellan_fmap_leftlook&info=NO&advoption=YES&lines=800&samples=800&sizeSelector=resolution&Resolution=80&projection=SIMP&grid=none&stretch=none&resamp_method=nearest_neighbor&north=10&west=350&east=10&south=-10&center=0&defaultcenter=on&center_lat=0


there is a windows build of a program i use " FWTools "
http://fwtools.maptools.org/
http://fwtools.maptools.org/windows-main.html
it DOSE run on XP might but on Windows 7 ? i do not know

now it will NOT open IMG files it DOSE open them as raw
and that is nice BECAUSE it not onlt will work with 8 bit ( 256 tone) but also
16 bit Unsinged AND singed AND in LSB AND msb
-- that is VERY VERY VERY important on windows some 16 bit gray images ( 65536 tones of gray )
are in LSB and some are in MSB - this IS A VERY big pain in the butt using MS Windows
then add in singed and unsinged and it is a BIGGER mess
it also opens up 32 bit floating point images

now above you posted the dimensions "8466 wide by 2830"
BUT the header states there are 2833 lines in the file ( this includes the text header ) .
So they need to be "skipped" in the raw import

this one is 3 full lines so it can just be cropped after , at some latter time .But not always , it "could be 1/3 of 1 row -- then the image will look very odd.


i use Ghex ( linux)
http://live.gnome.org/Ghex
hexedit might do for windows
http://www.hexedit.com/download.htm
or
http://mh-nexus.de/en/hxd/
and read the text part of the file

now OpenEV ( one the main tools in fwtools) using "Proj4" can remap them to simple cyc
and do that in the GUI

Posted by: elakdawalla Jan 7 2011, 05:03 PM

QUOTE (ncc1701d @ Jan 7 2011, 12:43 AM) *
mapaplanet I dont like much. I miss lat and long boxes for finding your way around like pdsmaps had. Some small features are harder to find your way back to in mapaplanet

It took me a little while to get used to the new mapaplanet, but as JohnVV said you just need to open the advanced options on lower left and you can input lat/lon and even change the longitude of projection, which is nice. My only complaint about mapaplanet is that it is slow.

Posted by: Phil Stooke Jan 7 2011, 08:44 PM

I agree, I use Map-a-Planet all the time and really like it, though it is slow and offline a bit too often.

Phil

Posted by: ncc1701d Jan 7 2011, 08:54 PM

ok here is little more clarification about what bugs me on mapaplanet. I have played with advanced before and wish their was center of latitude field as well. Maybe your guys expertise will help.
Given the information provided here concerning pankake volcanoes

http://en.wikipedia.org/wiki/File:PIA00084_Eistla_region_pancake_volcanoes.jpg

it says
This Magellan full resolution mosaic, centered at 12.3 north latitude, 8.3 degrees east longitude,

In this new advanced tool area I can enter in a 8.3 in the "center of longitude of projection field" at the bottom and I suppose I can do some guessing to enter in the right number in the north,west,east south fields and may find my way.
I am not successuful at it.
I would be curious how fast the experts can find out this volcano location in mapaplanet given the information above

verses pdsmaps where you just enter in fields a center of long and lat and then just set the long lat boxes of degrees to view.


and...

Also then if i enter in the wrong numbers when guessing the nortt,west,east,south my center of long of projection changes and other numbers automatically change and then I start getting lost.

Maybe this should be easier and I am missing something simple? I just dont remember having all these issues in pdsmaps






Posted by: elakdawalla Jan 7 2011, 09:08 PM

OK, I see your problem. The "Center of Projection" longitude has nothing to do with where the image is centered. It has to do with how the image is projected -- sinusoidal projection is a projection where only one line of longitude is vertical, and all other longitude lines are sinusoidal curves that converge toward the pole. Typically you want the center of projection to be the center longitude of the image, which is the mapaplanet default; there's only rare circumstances where you'd want to specify a different longitude (mostly having to do with building large mosaics out of several chunks).

Here's how I find your volcanoes. Go to http://www.mapaplanet.org/explorer/venus.html. Pick Left-Look FMAPs. Click Advanced Options. You want a map centered at about 12 north, 8 east. So I'd put in 22, 2 for north and south and -2, 18 for east and west and click "Submit Changes." Oops, I get an image only 40 pixels square. No problem, I just bump the resolution up from 2 pix/deg to 100 pix/deg. Go away for a few minutes while it thinks. Curse when I come back and it tells me the "connection was reset." Bump the resolution to 50 pix/deg, and in 5 seconds I have a nice picture and I see those pancake volcanoes right in the middle. Use the mouse to draw a bounding box around the area I'm interested in and it automatically zooms, using the pixel extent of my previous map. I ask it for a gaussian stretch and bump the resolution up a bit more, to half the full resolution (704 pix/deg) and attached is what I get (warning, it's 4 MB). There is a link at lower right where you can view it as a JPEG; using the "shopping cart" you can download it in a lossless format. For illustration purposes here's one at 10% the full resolution.


Because the area you picked in this example sits on the equator, mapaplanet defaulted to showing it in simple cylindrical projection rather than sinusoidal, so the "center of projection" thing is completely meaningless, by the way.

The whole process (including writing this post) took me about 7 min.

 

Posted by: ncc1701d Jan 7 2011, 11:54 PM

Thank you. I found it.
Just out of curiosity did you think the pdsmaps was easier to use or does it make a difference in your mind given your expertise in the business?

Posted by: JohnVV Jan 8 2011, 12:27 AM

early on i liked pds-map-a-planet
for the simple reason that i had NO CLUE what to do with a .img, or .imq or .pds
and on Windows at the time ( pre 2004 ) the software was very limited , still is.
at the time i as just switching from the photo darkroom to the computer so i had a bit of knowledge of digital images

now years later ( and still "learning" ISIS3 /IDL / other programs )
-- it is like the game "go" 10 min to learn and a lifetime to master . ---

now it is much easier to import ( say fl73n003new.img) into isis3 and make a map file and remap



Powered by Invision Power Board (http://www.invisionboard.com)
© Invision Power Services (http://www.invisionpower.com)