HiRISE Expert Question: EDR/RDR with more than 8 bit dynamic range, Is the 14-to-8 bit LUT always applied on board or is LUT optional |
HiRISE Expert Question: EDR/RDR with more than 8 bit dynamic range, Is the 14-to-8 bit LUT always applied on board or is LUT optional |
Jan 21 2009, 07:15 PM
Post
#1
|
|
Member Group: Members Posts: 713 Joined: 30-March 05 Member No.: 223 |
Is there any way to obtain the original 14-bit dynamic range resolution of the HiRISE instruments from the raw data, at least for some images ?
or is this always converted to 8-bit already on the spacecraft ? After reading the RDR/EDR specs it seems that the on-board application of 14-to-8 bits reduction Lookup-Tables (LUT) is not obligatory but may be an optional intrument setting that could be commanded individually for each image (just as other settings like the "pixel binning"). (see PDS parameter "MRO:LOOKUP_TABLE_TYPE" in the "INSTRUMENT_SETTING_PARAMETERS" parameter group.) So the question is: are there some images taken either without the LUT, or a LUT that preserves more than 8 bits ? This would be very interesting especially with the large spatial range of the camera that covers Mars scenes with an extremely large dynamic range from deep shadows over black basalt to sun facing icy slopes. This is not to say that the availiable DRA adjustments (e.g. IAS-viewer's auto-DRA feature) already do a very good job even with only 8-bits But nevertheless I'm curios if higher-than 8 bits per-pixel data could be obtained in any way ... |
|
|
Jan 21 2009, 07:32 PM
Post
#2
|
|
Senior Member Group: Moderator Posts: 2262 Joined: 9-February 04 From: Melbourne - Oz Member No.: 16 |
The HiRISE JP2's are 16 bit images, at least the ones I've looked at, and they definately seem to have more than 8 bits of depth.
Look at the accompanying label files. (.LBL) -------------------- |
|
|
Jan 21 2009, 07:36 PM
Post
#3
|
|
Senior Member Group: Members Posts: 3648 Joined: 1-October 05 From: Croatia Member No.: 523 |
The HiRISE JP2's are 16 bit images, at least the ones I've looked at. They definately seemed to have more than 8 bits of depth. They're not related to one another. An image can still have 256 (8 bits) of discrete brightness levels, but they can be allocated along the range of 14 bit DNs (16384 values). This is what LUT encoding does. Flatfields/dark frames will tend to soften that histogram up, but the original RAW data is still 8 bit and even when calibrated you cannot reproduce a smooth DN gradient original data could. It's lost forever. -------------------- |
|
|
Jan 21 2009, 07:57 PM
Post
#4
|
|
Member Group: Members Posts: 713 Joined: 30-March 05 Member No.: 223 |
but the original RAW data is still 8 bit and even when calibrated you cannot reproduce a smooth DN gradient original data could. It's lost forever. Yes, and as I understood, this is because the original 14-to-8 bit LUT ist performed on board the spacecraft (as a means of reducing the amont of data to be sent to earth). But my question was, if this on-board LUT is an optional instrument setting as other compression methods are, like the pixel binning. And if so, if there are any images taken (or planned to be taken) with on-board-LUT turned off. This would be interesting because, as pointed out by jamscanvin above, the resulting JP2s of the end-products do support full 16 bits per pixel. EDIT: @jamescanvin: within the 16 bit JP2s: did you also observe (color count) more than 256 distinct grey levels per image ? |
|
|
Jan 21 2009, 08:10 PM
Post
#5
|
|
Junior Member Group: Members Posts: 28 Joined: 27-October 06 From: Tucson, AZ Member No.: 1292 |
It looks like these are unLUTted.
PSP_001350_2710 MARS PSP_001365_2720 MARS PSP_001371_2720 MARS PSP_001379_2680 MARS PSP_001385_2680 MARS PSP_001442_2715 MARS PSP_001936_1875 MARS PSP_001892_1715 MARS PSP_002532_0935 MARS PSP_005558_9040 MOON PSP_005558_9045 MOON PSP_005770_1745 MARS PSP_005715_2065 MARS PSP_005778_2140 MARS PSP_006847_1780 MARS PSP_007769_9010 PHOBOS PSP_007769_9015 PHOBOS PSP_008644_2485 MARS (Phoenix) |
|
|
Jan 21 2009, 09:57 PM
Post
#6
|
|
Senior Member Group: Moderator Posts: 2262 Joined: 9-February 04 From: Melbourne - Oz Member No.: 16 |
EDIT: @jamescanvin: within the 16 bit JP2s: did you also observe (color count) more than 256 distinct grey levels per image ? Yes, look with the IAS viewer there are around 1024 levels in a random image I just looked at (10 bit). However as ugordan points out that is likely to be 8 bit raw data that has been calibrated. In any case (for my one test case) there are not anywhere near the number of levels that would be expected for true 14bit data. -------------------- |
|
|
Jan 22 2009, 12:19 AM
Post
#7
|
|
Member Group: Members Posts: 713 Joined: 30-March 05 Member No.: 223 |
Yes, look with the IAS viewer there are around 1024 levels in a random image I just looked at (10 bit). Thanks for the info. But how did you do that with IAS viewer ? It looks to me that IAS viewer operates only with 8 bit per pixel. Which software tool do you use to process/convert the JP2 to a 16 bit-per-channel image ? I used to do this with Photosjop (which can read JP2) and then processing/saving it as 16-Bit PNG. However, due to main memory restrictions this does not work for the larger JP2s. |
|
|
Jan 22 2009, 02:17 AM
Post
#8
|
|
Senior Member Group: Members Posts: 2511 Joined: 13-September 05 Member No.: 497 |
Is there any way to obtain the original 14-bit dynamic range resolution of the HiRISE instruments from the raw data... Just because the ADC is 14 bits doesn't mean that any given image will use all 14 bits; there is headroom at the bright end and shadows aren't perfectly black because of atmospheric backscatter, etc. I don't know how HiRISE works but CTX images can be taken at half spatial resolution in 12-bit mode as long as no compression is commanded. I don't think this has ever been used operationally, though (I used it in ground calibration, occasionally.) Since you are just encoding shot noise without square-root encoding, there is in theory no value to using the mode. -------------------- Disclaimer: This post is based on public information only. Any opinions are my own.
|
|
|
Jan 22 2009, 08:42 AM
Post
#9
|
|
Senior Member Group: Moderator Posts: 2262 Joined: 9-February 04 From: Melbourne - Oz Member No.: 16 |
But how did you do that with IAS viewer ? It looks to me that IAS viewer operates only with 8 bit per pixel. I don't have the viewer at work so I can't explain things accurately here but on the left near where the viewer tells you the coordinates that the mouse is pointing it also tells you the pixel value (this goes up to 1000 or so). Also some of the buttons at the top-middle will tell you the pixel range and adjust the black/white mapping of the portion of the image you are viewing. Which software tool do you use to process/convert the JP2 to a 16 bit-per-channel image ? I read the images using the FreeImage C++ library for further processing. There is a special PDS_JP2 C++ class library, but it looked too complicated for me to work out at short notice when I started working on the 'ripple mapping' project. (I was already familiar with FreeImage as I use this when making the MER panoramas.) I have however used the JP2_to_PDS and PDS_to_JP2 standalone applications to do such things as crop out sections of the image I want. You may find it easier to convert to the PDS version depending on your experience/assessable software. See http://hirise.lpl.arizona.edu/tools/pds_jp2.php -------------------- |
|
|
Jan 22 2009, 03:44 PM
Post
#10
|
|
Member Group: Members Posts: 713 Joined: 30-March 05 Member No.: 223 |
Thanks a lot, James ! this is exactly what I'm looking for. I wasn't aware of those tools yet, because I have been using plain Photoshop for the HiRISE-Images and a self written calibration C-program for the MRO-CTX images converting directly from PDS/IMG to PNG. (With 16-Bit-PNG being my over-all preferred format for all further image processing purposes, for which I'm using a IMG-to-PNG16-bit converter) I assume that the PDS_JP2-tools & library do retain the 16-bits-per-pixel data (when, for example, extracting sub-images with JP2_to_PDS) , right ? |
|
|
Jan 22 2009, 04:08 PM
Post
#11
|
|
Senior Member Group: Moderator Posts: 2262 Joined: 9-February 04 From: Melbourne - Oz Member No.: 16 |
Yes, as far as I can see, all the information is retained when using the JP2_to_PDS and PDS_to_JP2 applications.
-------------------- |
|
|
Jan 22 2009, 08:52 PM
Post
#12
|
|
Member Group: Members Posts: 713 Joined: 30-March 05 Member No.: 223 |
I checked the JP2-Header information and found that apparently only the non-map-projected JP2s ("NOMAP") contain more than 8 bits pixel depth (actually 10 bits, which is consistent with the RDR documentation that describes the calibration process results as 10-bit (after the inverse lookup table from the original 8 bits transmitted from the space craft)
The map projected JP2 (quicklooks) however are stored with only 8 bits/pixel. |
|
|
Jan 23 2009, 08:14 AM
Post
#13
|
|
Senior Member Group: Moderator Posts: 2262 Joined: 9-February 04 From: Melbourne - Oz Member No.: 16 |
The map projected JP2 (quicklooks) however are stored with only 8 bits/pixel. However the map projected downloadable products are also 10bit -------------------- |
|
|
Jan 23 2009, 09:08 AM
Post
#14
|
|
The Poet Dude Group: Moderator Posts: 5551 Joined: 15-March 04 From: Kendal, Cumbria, UK Member No.: 60 |
You all sound like you're having so much fun talking about this! This is one of those "I really wish I had a Babel fish to stick in my ear..." threads...
-------------------- |
|
|
Jan 23 2009, 09:11 AM
Post
#15
|
|
Member Group: Members Posts: 713 Joined: 30-March 05 Member No.: 223 |
However the map projected downloadable products are also 10bit It seems that the bit-depth vary from image to image: I looked at the JP2-Metadata of seveal HiRISE images using the "kdu_expand" tool from the Kakadu JPEG-2000 tool suite. For example: > kdu_expand -i PSP_010589_1510.NOMAP.JP2 -record info.txt > cat info.txt yields: ... Ssize={15000,10024} Sprecision=10 Ssampling={1,1} ... whereas > kdu_expand -i PSP_009271_1485_RED.JP2 -record info.txt > cat info.txt yields: ... Ssize={50000,10024} Ssigned=no Sprecision=8 Ssampling={1,1} ... Also when using the IAS viewer's current-pixel-data/value display it seems that for some images the max. value is 256 whereas for others its up to 1000. Maybe some of the HiRISE experts (GuyMac ?) could shed some light on how exactly the bit-depth (8 or 10 bits) of the final JP2-products is determined. P.S.: Although the difference between 8 bits and 10 bits may not seem too significant for most viewing purposes (and with the additional 2 bits of precision introduced "only" by the calibration process but not from original instrument precision (which is lost during LUT) ) However, It does IMO make a difference when it comes to using the raw data as basic input for further processing chains to build upon (like I do, for example for b/w colorizations). In this case every additional bit of precision can be useful |
|
|
Lo-Fi Version | Time is now: 25th April 2024 - 06:20 AM |
RULES AND GUIDELINES Please read the Forum Rules and Guidelines before posting. IMAGE COPYRIGHT |
OPINIONS AND MODERATION Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators. |
SUPPORT THE FORUM Unmannedspaceflight.com is funded by the Planetary Society. Please consider supporting our work and many other projects by donating to the Society or becoming a member. |