PDS data format? |
PDS data format? |
Jan 21 2015, 02:49 AM
Post
#1
|
|
Newbie Group: Members Posts: 14 Joined: 6-April 11 From: Cologne, Germany Member No.: 5951 |
Hi(gh)!
I would like to generate POV-Ray meshes from PDS altimetry data files (for example the global Vesta file from Dawn Public Data) - but I have no clue what data format they contain... obviously, its binary, simple cylindrical projection, based on a 255 x 255 x 255 kms sphere (Vesta), 32 bits per data point. But what data format - int or float? Signed or unsigned? Big or little endian? Where do I get detailed information on the format of these PDF files? See you in Khyberspace! Yadgar |
|
|
Jan 21 2015, 02:56 AM
Post
#2
|
|
Founder Group: Chairman Posts: 14431 Joined: 8-February 04 Member No.: 1 |
Probably better to link to a web-page rather than directly to a 500+,mb .pds file.
http://dawndata.igpp.ucla.edu/tw.jsp?secti...Ms/DLR_HAMO_DTM Simply opening the file in a text editor reveals..... CODE PDS_VERSION_ID = PDS3 /* FILE DATA ELEMENTS */ RECORD_TYPE = FIXED_LENGTH RECORD_BYTES = 69124 FILE_RECORDS = 8644 LABEL_RECORDS = 2 /* POINTERS TO DATA OBJECTS */ ^IMAGE_HEADER = 3 ^IMAGE = 4 /* DATA OBJECT DEFINITIONS */ OBJECT = IMAGE INTERCHANGE_FORMAT = BINARY LINES = 8641 LINE_SAMPLES = 17281 SAMPLE_TYPE = IEEE_REAL SAMPLE_BITS = 32 BANDS = 1 BAND_STORAGE_TYPE = BAND_SEQUENTIAL MINIMUM = -21527 MAXIMUM = 19005 END_OBJECT = IMAGE /* MAP OBJECT DEFINITIONS */ OBJECT = IMAGE_MAP_PROJECTION A_AXIS_RADIUS = 255.0 B_AXIS_RADIUS = 255.0 C_AXIS_RADIUS = 255.0 CENTER_LATITUDE = 0.0 CENTER_LONGITUDE = 180.0 COORDINATE_SYSTEM_NAME = PLANETOCENTRIC EASTERNMOST_LONGITUDE = 359.9999991462263 LINE_PROJECTION_OFFSET = 4320.0 MAP_PROJECTION_TYPE = SIMPLE_CYLINDRICAL MAP_RESOLUTION = 48.00000026515721 MAP_SCALE = 0.09272061600000001 <km/pixel> MAXIMUM_LATITUDE = 90.0 MINIMUM_LATITUDE = -90.0 POSITIVE_LONGITUDE_DIRECTION = EAST SAMPLE_PROJECTION_OFFSET = 8640.0 WESTERNMOST_LONGITUDE = 0.0 END_OBJECT = IMAGE_MAP_PROJECTION /* MISCELLANEOUS */ TARGET_NAME = VESTA PRODUCER_INSTITUTION_NAME = "German Aerospace Center (DLR)" PRODUCT_CREATION_TIME = 2013-05-24T13:27:30.000 MISSING_CONSTANT = -32768 DESCRIPTION = "DTM: Height [m] equals DN. Vertical reference: Height above sphere/ellipsoid (as defined by map axis)." /* IMAGE HEADER DATA ELEMENTS */ OBJECT = IMAGE_HEADER HEADER_TYPE = VICAR2 INTERCHANGE_FORMAT = ASCII BYTES = 69124 ^DESCRIPTION = "VICAR2.TXT" END_OBJECT = IMAGE_HEADER END That data is enough to open it as a raw file in Photoshop, Gimp or ImageJ etc. and then use it as a displacement map on a mesh. |
|
|
Jan 21 2015, 03:54 AM
Post
#3
|
|
Newbie Group: Members Posts: 14 Joined: 6-April 11 From: Cologne, Germany Member No.: 5951 |
INTERCHANGE_FORMAT = BINARY O.k., it's binary... LINES = 8641 LINE_SAMPLES = 17281 ...17281 by 8641 sample points... SAMPLE_BITS = 32 ...4 bytes per sample, which would be either standard-precision float or int... MINIMUM = -21527 MAXIMUM = 19005 Obviously, the sample values range from -21527 to 19005 metres relative to the rotation ellipsoid - which means that they are in fact integer! But why do they use signed int rather than signed short int? And which bit order - big oder small endian? A_AXIS_RADIUS = 255.0 B_AXIS_RADIUS = 255.0 C_AXIS_RADIUS = 255.0 ...the underlying ellipsoid, here a perfect sphere... MAP_PROJECTION_TYPE = SIMPLE_CYLINDRICAL ...so spacing of points can be calculated from the x and y dimensions of the data array... SAMPLE_PROJECTION_OFFSET = 8640.0 Is this a value to be added to or subtracted from the sample values? MISSING_CONSTANT = -32768 Missing data code... That data is enough to open it as a raw file in Photoshop, Gimp or ImageJ etc. and then use it as a displacement map on a mesh. When I try to open it, I just get a strange noisy thing which, when zoomed in, appears like vague rastered interference shapes... perhaps I better should try writing a C++ program which extracts the sample data and stores them as an uncompressed TGA image file! See you in Khyberspace! Yadgar |
|
|
Jan 21 2015, 05:40 AM
Post
#4
|
|
Founder Group: Chairman Posts: 14431 Joined: 8-February 04 Member No.: 1 |
You're probably not culling the header before reading the image. I just tried it and it loads fine in Photoshop as a 32 bit file when you tell it the header is 69124 bytes. One needs to apply a stretch, obviously, to get an image with visible contrast. Moreover you may be confusing the line and the sample. Lines = height. Samples = width.
|
|
|
Jan 21 2015, 05:59 AM
Post
#5
|
|
Senior Member Group: Members Posts: 2504 Joined: 13-September 05 Member No.: 497 |
...4 bytes per sample, which would be either standard-precision float or int... Obviously, the sample values range from -21527 to 19005 metres relative to the rotation ellipsoid - which means that they are in fact integer! Nope. It says SAMPLE_TYPE = IEEE_REAL, that's a single-precision float. -------------------- Disclaimer: This post is based on public information only. Any opinions are my own.
|
|
|
Jan 21 2015, 11:05 AM
Post
#6
|
|
IMG to PNG GOD Group: Moderator Posts: 2250 Joined: 19-February 04 From: Near fire and ice Member No.: 38 |
It should be possible to use IMG2PNG to convert this to a 16 bit PNG image if that helps. See http://www.mmedia.is/bjj/utils/img2png/index.html
|
|
|
Jan 27 2015, 03:45 AM
Post
#7
|
|
Newbie Group: Members Posts: 14 Joined: 6-April 11 From: Cologne, Germany Member No.: 5951 |
Hi(gh)!
Nope. It says SAMPLE_TYPE = IEEE_REAL, that's a single-precision float. I started with a small C++ program (using g++ under Linux) to at least get a sense of the data stored in global.pds - but the data I got were far beyond the limits given in the header! Here is the code: #include <cstdlib> #include <fstream> #include <iostream> using namespace std; int main() { ifstream Rawdata; Rawdata.open("global.pds", ios::binary); if (!Rawdata) { cerr << "File cannot be read!\n"; exit(-1); } float h; int i=0; while (Rawdata.read((char*)&h, sizeof(h))) { i++; if (i > 2585) // skipping the ASCII header cout << i << "\t" << h << endl; } Rawdata.close(); } and this is an example of what it read from global.pds: 592609 1.08726e-38 592610 1.08726e-38 592611 -1.07807e-38 592612 -1.07807e-38 592613 -1.07807e-38 592614 -1.07807e-38 592615 1.07807e-38 592616 1.07807e-38 592617 -1.06889e-38 592618 -1.06889e-38 592619 1.06889e-38 592620 1.06889e-38 592621 -1.05971e-38 592622 -1.05971e-38 592623 1.05971e-38 592624 1.05971e-38 592625 -1.05052e-38 592626 -1.05052e-38 592627 1.05052e-38 592628 1.05052e-38 592629 -1.04134e-38 592630 -1.04134e-38 592631 1.04134e-38 592632 1.04134e-38 Clueless, Yadgar |
|
|
Jan 27 2015, 07:09 AM
Post
#8
|
|
Senior Member Group: Members Posts: 2504 Joined: 13-September 05 Member No.: 497 |
Most likely a byte-ordering problem. PDS files are usually in network order. I'd take Bjorn's advice and not reinvent the wheel.
-------------------- Disclaimer: This post is based on public information only. Any opinions are my own.
|
|
|
Jan 27 2015, 09:03 PM
Post
#9
|
|
Newbie Group: Members Posts: 14 Joined: 6-April 11 From: Cologne, Germany Member No.: 5951 |
Most likely a byte-ordering problem. PDS files are usually in network order. I'd take Bjorn's advice and not reinvent the wheel. I tried a small routine programmed by Gregor Brandt (http://stackoverflow.com/questions/2782725/converting-float-values-from-big-endian-to-little-endian): float ReverseFloat( const float inFloat ) { float retVal; char *floatToConvert = ( char* ) & inFloat; char *returnFloat = ( char* ) & retVal; // swap the bytes into a temporary buffer returnFloat[0] = floatToConvert[3]; returnFloat[1] = floatToConvert[2]; returnFloat[2] = floatToConvert[1]; returnFloat[3] = floatToConvert[0]; return retVal; } - and I got believable values, such as: 9681062 229914 9681063 229918 9681064 229924 9681065 229928 9681066 229934 9681067 229938 Obviously, the underlying basic unit is metres rather than kilometres... See you in Khyberspace! Yadgar |
|
|
Jan 27 2015, 09:32 PM
Post
#10
|
|
Senior Member Group: Members Posts: 2504 Joined: 13-September 05 Member No.: 497 |
Obviously, the underlying basic unit is metres rather than kilometres... Like it says in the header: "DTM: Height [m] equals DN." -------------------- Disclaimer: This post is based on public information only. Any opinions are my own.
|
|
|
Jan 28 2015, 12:42 AM
Post
#11
|
|
Member Group: Members Posts: 128 Joined: 10-December 06 From: Atlanta Member No.: 1472 |
If you really need to do it in C/C++, I can send you the source of a converter I wrote near 10 years ago that can handle it and converts it to FITS format. But this is generally not the optimal solution and it is better to use img2png (although as far as I know, Bjorn has yet to port it to Linux) or just read the raw data using GIMP or ImageJ as pointed above.
|
|
|
Jan 29 2015, 08:40 AM
Post
#12
|
|
Newbie Group: Members Posts: 14 Joined: 6-April 11 From: Cologne, Germany Member No.: 5951 |
Hi(gh)!
If you really need to do it in C/C++, I can send you the source of a converter I wrote near 10 years ago that can handle it and converts it to FITS format. But this is generally not the optimal solution and it is better to use img2png (although as far as I know, Bjorn has yet to port it to Linux) or just read the raw data using GIMP or ImageJ as pointed above. I intend to generate an ASCII POV-Ray mesh2 script from the PDS data - with my current amount of RAM (16 GiB) I will need to drastically scale down the data resolution, such as picking only each tenth data point from each tenth row, to get 1.5 million rather than 150 million vertices... up to now, it seems to work well, but nevertheless, thanks for your readiness to help! See you in Khyberspace! Yadgar |
|
|
Apr 7 2015, 07:39 AM
Post
#13
|
|
Member Group: Members Posts: 890 Joined: 18-November 08 Member No.: 4489 |
Are you still having issues ?
for the DEMS like the mola and lola the pds img files are 100% raw no attached header they have a detached lbl for the header the dawn vesta data is on the older dawn site see the post from a few days back . http://www.unmannedspaceflight.com/index.p...st&p=219315 for NON dem files the header can be static or dynamic as in it might be the SAME length for most in a mission or change with EACH image most img 's in pds hav a vicar header and a detached pds lable but not all some have a pds attached header if you are not using say isis3 or some other program that reads the pds format Use a HEX editor to read the plain text header on the binary image some text editors can even read the plain text gedit and kate can so i would expect on windows that "SciTE " should for future reference the link in post 2 has a isis3 cub file "gdal_translate" has no problem converting it to a GEO tiff ( Photoshop should be able to open that ) |
|
|
Lo-Fi Version | Time is now: 29th March 2024 - 02:23 PM |
RULES AND GUIDELINES Please read the Forum Rules and Guidelines before posting. IMAGE COPYRIGHT |
OPINIONS AND MODERATION Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators. |
SUPPORT THE FORUM Unmannedspaceflight.com is funded by the Planetary Society. Please consider supporting our work and many other projects by donating to the Society or becoming a member. |