Printable Version of Topic

Posted by: Drkskywxlt Aug 18 2010, 07:51 PM

As part of my research, I'm starting to learn ISIS and display MARCI imagery. I've quickly learned that MARCI imagery is pretty hard to work with since it's a push-frame camera, which a few people claim is the reason there's a relative lack of papers published using its data. I tried to run through the process for an entire pole-to-pole MARCI swath and after processing cam2map for nearly 48hrs, the Linux box booted me off. That's on 8GB of RAM, BTW. After that, I started cropping the images to much smaller chunks. Below is my first recognizably Martian image that I've been able to produce (I've only been at this for a few days). This is from the southern mid-latitude highlands. Some things I don't understand...why is the left part of the frame somewhat washed out/bright? The red/yellow bands around the edge are areas where there apparently is no data in the green or blue channels...not sure why. My processing steps are:

marci2isis
spiceinit
marcical
crop
cam2map
automos

I need to figure out how to really speed this up. I want to look at complete MARCI global mosaics over at least 1 martian year if not the entire MRO mission. At this rate, that would take years . Apparently an ISIS update is coming out in the near future which will help work with LROC WAC images. LROC WAC is also pushframe, so maybe that will help with MARCI imagery as well?

Thanks in advance!

Posted by: djellison Aug 18 2010, 08:20 PM

I feel your pain - I tried the exact same process and gave up far earlier than you did.

Posted by: Drkskywxlt Aug 20 2010, 04:28 PM

After several more failures, I've got another viewable image. This is on the edge of the southern polar cap. Talked to someone who has done work with LROC WAC (very similar to MARCI). She did it on a 4000-core cluster!! So, basically, you need a supercomputer or darn near to really get the full use of these cameras. I'm looking at using MOC WA images instead...starting to play with those soon.

Posted by: stevesliva Aug 20 2010, 04:52 PM

What requires all those cores? You were talking RAM before.

Just curious... memory hogging is far more common than core hogging.

Posted by: Drkskywxlt Aug 20 2010, 05:08 PM

To make that last image, took about 1.5 hours from start to finish and I used two different 8GB of RAM processors to complete. And that's only a tiny fraction of 1 full MARCI pole-to-pole swath! So, doing 2000 or 4000 of those images at once would let me do several days of MARCI images all at once in the same time.

Apparently, there's no systems where I am that have much more RAM than 8GB. So, you just need lots of cores doing the same thing to speed it up. ISIS is apparently not set up for true parallel-computing yet, although I guess there's been talk to make it so.

Posted by: Drkskywxlt Aug 20 2010, 05:10 PM

One more on the other side of the south pole.

Posted by: stevesliva Aug 20 2010, 06:38 PM

Oh, batch processing with the batch parallelized. My company's perfectly set up for that sort of thing, which is why I was curious. I can get a few hundred cores on boxes with 256GB of memory, but I'd probably get fired if I did it for images. But like you said, truly SMP or well-threaded applications are more rare. So we just pool lots of machines are throw individual jobs at them.

Interestingly enough, the World Community Grid is set up to do the same sort of thing, albeit with much smaller boxes:
http://www.worldcommunitygrid.org/about_us/viewAboutUs.do

Not sure an RFP for planetary science image processing would fly there... but I would gladly donate you my laptop's 2 cores and 2 GB RAM each night if there were an easy way to set up a client.

Posted by: Drkskywxlt Aug 20 2010, 07:01 PM

Very kind, thanks!

Posted by: Drkskywxlt Aug 20 2010, 08:25 PM

The large crater in these two images is Schiaparelli Crater. If you know your Martian geography, you'll know this is just a bit (relatively speaking) east of Opportunity's landing site. The first is what directly comes out of ISIS. It seems the dark albedo features come up as deep blue, not really sure why. I've stretched the colors a bit in the 2nd. Does that look more "Mars-ish"?

Posted by: elakdawalla Aug 20 2010, 08:42 PM

It seems the blue channel really brightens to both east and west edges of the mosaic -- is that a photometric effect? MARCI does see a wide longitude strip, and I know photometric effects are what have made it take so long to create the LRO WAC mosaic, but I would have expected it only to brighten on one edge (west), not both. Is it atmospheric scattering?

Posted by: Drkskywxlt Aug 20 2010, 09:00 PM

Emily...I've been trying to figure that out. Cartrite on the ISIS Support forum notices the same effect, but it never really gets answered by the ISIS staff. I cropped the sides off the image off, and that helps. When I keep the full image width, there's just bright blurs on each side. I suspect it might be an atmospheric effect, but I don't know enough about imagery and the cameras to make an educated guess.

Posted by: volcanopele Aug 20 2010, 09:15 PM

Could it be due to scattered light in the optics of the camera system? Have to deal with that a lot with the WAC on Cassini.

Posted by: djellison Aug 20 2010, 11:13 PM

Rememebr the FOV of MARCI is HUGE. The edges you're looking really out left and right from the spacecraft so whilst it's nicely map projects, you're looking through a lot more atmosphere, and I don't think that's something you can easily calibrate out and is probably what's causing the problem.

Posted by: ugordan Aug 20 2010, 11:45 PM

Yup, probably a combination of Rayleigh scattering and just plain dusty atmosphere at highly off-nadir look angles.

Posted by: mwolff Aug 21 2010, 01:33 AM

Finally, a question that I can answer with some knowledge! The brightening on the west side is due primarily to atmospheric effects, but it is a combination of simple atmospheric pathlength (i.e., larger emergence angles) and decreasing scattering angle (i.e., climbing the the phase function towards the forward scattering peak). On the east side, it can also involve surface photometry as the low phase angle range (where opposition effects are more prevalent) can range quite a portion of the image in terms of north south. In addition, on the east side, you have the double air mass (large incidence and emergence angles) effect to counteract the decreased amplitude of the scattering phase function at lower phase angles.

The presence of even a small amount of water ice haze can dramatically increase these effects...dust tends to be more neutral on the eastern side of the strips.

Posted by: mwolff Aug 21 2010, 01:40 AM

What one can do, and what was done for the MOC WA images to make the nice Daily Global Maps (DGM), is to create a "flatfield" using a large number of images coadded together, say for most of a month. This produces a nice function which can remove *uniform* dust loading (i.e., longitude independent) and the opposition surge. I haven't tried it myself to remove the opposition surge or atmospheric edge effects (I try to use these edge effects to study the atmosphere), but I believe it is being done internally at MSSS to produce DGMs for internal project use.

A similar effort was done using many months of images (actually more than a year of images) to provide an improvement to the ground-calibration flat fields. I read about this being applied to Clementine images, and it seemed to work well enough for the flat field improvements.

Posted by: Drkskywxlt Aug 24 2010, 02:44 PM

Using a MARCI swath from a later orbit (6124), I made this image of Lunae Planum. The dark albedo terrain seems to be closer to "true color" in this image vice the ones I posted last week with the blue tinge. I didn't change anything, so perhaps the MARCI team just got better with the color matching as time went along?

Posted by: Drkskywxlt Jul 27 2012, 05:43 PM

I never posted some of the other (slightly more interesting) MARCI images I did manage to produce. I abandoned this awhile ago, but am hoping to team with someone who has been paid to process MARCI imagery for more research. Anyway, here's a few more:

Posted by: Drkskywxlt Jul 27 2012, 05:45 PM

.

.

Posted by: mhoward Jul 27 2012, 05:52 PM

Very nice. First one in that pair shows Tempe Terra and Kasei Valles; second image is just above the first one. They overlap nicely.

Posted by: Drkskywxlt Jul 27 2012, 06:19 PM

Yes, I split a larger image into two pieces. Combined, they represent about 1/5th (if memory serves) of a single pole-to-pole swath of the 12 MARCI takes every day.

Posted by: jccwrt Dec 26 2020, 08:35 PM

I've decided to take up working with MARCI images, given that it's a pretty sizeable dataset with a lot of goodies in it. I wanted to use it to render some overflight videos of the Martian surface, maybe one day with the goal of adding in some narration or running an ISS HDEV style stream showing the Martian surface passing underneath at real-time orbital speed. For now, here's an overflight of the North Pole / Nili Fossae / Tyrrhena Terra region on March 7, 2020 at appx. 10x real-time speed.

https://youtu.be/DcVRRpqdzeo

Posted by: jccwrt Mar 6 2021, 03:47 AM

Continuing to refine my process and sharpen my skills with Blender.
This particular video simulates a view out the spacecraft window so to speak, starting over the South Pole, flying over Terra Sirenum, Daedalia Planum, and Olympus Mons.

https://youtu.be/Sc5eQTorIiA

Nearly impossible to derive a consistent haze profile from MARCI images, considering how small it is in the original data. So I cribbed a guesstimate from some color processings of HRSC limb scans I did a while back. Not really sure how to approach adding motion to the haze profile, and the current still-image mask seems kinda static.

Posted by: jccwrt Jun 1 2021, 12:51 AM

Continuing to work with MARCI data, now rendering videos at real-time orbital speed. This orbit captures some dust storms along the southern edge of Claritas Fossae, nice albedo markings in Syria Planum, a handful of the Tharsis Montes, and the northern polar cap lit by the midnight sun.

https://www.youtube.com/watch?v=koNXF6myc5Q

Posted by: vikingmars Jun 1 2021, 05:09 PM

This is an ASTOUNDING movie !
Thanks so much Jccwrt for sharing this work with us

Posted by: jccwrt Oct 5 2021, 03:32 PM

I rendered out a new video this weekend! This one overflies eastern Hellas, Tyrrhena Terra, Syrtis Major, and the Nili Fossae regions. As a bonus it flies over Jezero the day after Perseverance's landing.

https://youtu.be/PjwY9S0uEQc

By the way, if anyone else is interested in processing MARCI images I have a shell script for use with ISIS3 that helps speed things up significantly.

https://github.com/AsterAphelion/marci-parallel

Posted by: vikingmars Oct 6 2021, 07:21 AM

What a nice video ! Thanks so much Jccwrt