Printable Version of Topic

Posted by: djellison Jan 7 2009, 01:10 PM

I've got a few .tab's from the PHX Lidar

Basically - after playing with it for a while, I'll end up with a spreadsheet of values. Each column being one sample 30s apart - each row being a different altitude.

Does anyone know a means of taking a CSV or similar, and turning it into a small image with pixel brightness representing the values from the CSV?

I've thought of one technique using conditional formatting in Excel - but it probably wont work very well

Posted by: jamescanvin Jan 7 2009, 01:23 PM

My terrain mapping (and in fact my MER image processing as well) does quite a bit of converting of number arrays to image pixel values using the FreeImage C++ library. If you sent me the file I could probably do the conversion to without too much trouble.

IDL can do it as well (not sure about the free GDL, never used it) using 'plotgrid'. Not sure if you could get 16bit images using this method though (if you needed that).

Posted by: djellison Jan 7 2009, 01:29 PM

This is what I've managed so far

Using LS003RLA_00896474226_10DCM0.TAB
It's basically one very tall tab - that repeats every 500 rows - the 500 rows being 500 10m altitude samples.

I cut and pasted each 500 cell chunk, so they were in a grid, not a line.

Then used conditional formatting with Excel.

And I got this - it worked WAY better than I was expecting.

261 seconds left to right. 5000 metres bottom to top.

I have to use Excel 07 on a PC to do it though - Excel 08 doesnt do conditional formatting like that. A bit annoying.

Posted by: ElkGroveDan Jan 7 2009, 02:39 PM

It sounds like the kind of thing I was learning to do in FORTRAN shortly after you were born Doug. But I've long ago forgotten all that.

Posted by: djellison Jan 7 2009, 02:58 PM

Next step - doing it greyscale - for the two different frequencies sampled. Then merge as R and G in a colour image to highlight differences.

Actually http://www.nasa.gov/mission_pages/phoenix/images/press/MET_Lidar_plotTV_001.html - using THAT data set is probably a sensible next step.

Posted by: lyford Jan 7 2009, 04:36 PM

Brilliant!

Posted by: RoverDriver Jan 7 2009, 05:39 PM

I'm lazy and cheap. I use the PGM format and then convert it to whatever format I need to distribute. Here's a crude way of generating an image from your data:

1) save the data table as text, no commas, just spaces between the data values.

2) using your favourite editor, edit the file and add at the top of the page:

P2
<sizex> <sizey>
<maxval>

where the <sizex> and <sizey> indicate the number of columns and rows in your table and <maxval> should be the maximum value in your table (needs to be less than 65535).

3) use your favourite image conversion tool to convert the image to whatever format you prefer. I use the netpbm set of tools or xv (hey I know, I'm old!).

Note that all the numbers need to be integers. Here's an example of a PGM image:

P2
3 4
11
0 1 2 3 4 5 6 7 8 9 10 11

Hope this helps

Posted by: stewjack Jan 7 2009, 06:12 PM

Are you familiar with Topcat? A Java Tool for OPerations on Catalogues and Tables, a viewer/editor for astronomical tabular data, including FITS table extensions and VOTable documents. It imports CSV data

First a warning - I really have little understanding of the program. I just downloaded it to create spherical plots from data created by my Einstein@home WU's. I just followed some written instructions. Also the latest version may be corrupted. I could only get version 3.1 to launch on my WIN XP system.

http://www.star.bristol.ac.uk/~mbt/topcat/#standalone

ftp://andromeda.star.bris.ac.uk/pub/star/topcat/

Posted by: sci44 Jan 7 2009, 07:37 PM

Me too! Thats probably the simplest way - but if you want to put a little maths in between input numbers & plotted colors, I often use a simple BASIC script. This is for QBASIC.EXE, which everyone on the planet will either have (or can download free from MS)
Something similar would work for Linux smallbasic/freebasic.
--------------
REM Simple CSV plotter
INPUT "X,Y,Filename"; mx, my, fil$
SCREEN 13
OPEN fil$ + ".csv" FOR INPUT AS 1
FOR y = 1 TO my: FOR x = 1 TO mx
INPUT #1, v
PSET (x, y), v
NEXT x: NEXT y
CLOSE 1
---------------
Needs a PALETTE statement loop to fill the palette up before hand - as is you get the default 256 VGA mode 13 palette.

Posted by: imipak Jan 7 2009, 07:55 PM

Doug, could you post the CSV data? I was half-way through composing a post about how it'd be trivial in Perl, with the right image manipulation modules, when I realised I might have to provide some evidence for a wild statement like that (Disclaimer - I'm incredibly lazy, a terribly hack coder, and it's years since I was paid to actually program, so if I fail to come up with anything it reflects only on me, not Perl. Still, if it's good enough for http://marsandme.blogspot.com/... )

(Edit - thanks sci44, that's really thrown down the gauntlet..! Does your snippet need additional code to actually write a file? What would it produce, a bitmap? )

Posted by: djellison Jan 7 2009, 09:34 PM

Kudos to Card-Carying lurker Pete He pointed me toward Surfer.

Posted by: sci44 Jan 7 2009, 10:17 PM

As is, it just plots straight to a DOS-mode screen in glorious clunky DOS VGA graphics, but if you hit shift-PrtSc (printscreen) then Alt-Tab to Windows, you can paste the screenshot into Paint & save a Bmp. Or do it more properly with VisualBasic, VB script, FreeBasic, or for that matter Perl, C (my choice), etc etc. Nice thing about Qbasic is everyone has it/can get it, and very small short progs with no "overhead" code can do useful things - its nice for posting short examples up on forums, etc.

To write to a PGM file, from line 5 (FOR y=....) onwards replace with:-

OPEN fil$+".pgm" FOR OUTPUT AS 2
PRINT #2,"P2" : PRINT #2,mx;my : PRINT #2, 256 : REM put max no colours for that 256..
FOR y = 1 TO my: FOR x = 1 TO mx
INPUT #1, v
REM Do calculation/manipulation of (v) here..
PRINT #2, v;
NEXT x: NEXT y
CLOSE 1 : CLOSE 2
---------------
Although thats untested.. Edit: seems to work ok..
Just put the manipulation you want in the middle, ie:
IF v > 7 THEN v = 1 ELSE v = 0
etc etc..

Posted by: lyford Jan 8 2009, 02:08 AM

*sigh*

http://grin.hq.nasa.gov/ABSTRACTS/GPN-2003-00060.html

Posted by: vk3ukf May 24 2009, 09:13 AM

Love the pic of the guy with the crayons, I found some others of this just recently.

http://gizmodo.com/5266151

They include a close up of the sheet so you can see the numbers.

Posted by: 4th rock from the sun May 25 2009, 02:33 PM

Interestingly, the relation between the visible numbers and the painted colors is not very accurate. Did they painted according to some sort of value average?
I'd really like to have those data print outs and type all the values!

Posted by: djellison May 25 2009, 02:44 PM

I think they had only a few colours - so they bracketed wide ranges of values with the same colour. You can see the pencil lines where they drew the areas that they then coloured in...... i think.

Posted by: 4th rock from the sun May 26 2009, 10:37 AM

I looked again to the close up image where the numbers are visible. Here's what I could make out of it. I had to align some columns vertically because the printouts seem off by one or two rows at places. So here you have numbers to numbers !

 Livro1.txt ( 928bytes ) : 497