MSL image metadata sources & documentation? |
MSL image metadata sources & documentation? |
Oct 10 2015, 11:38 AM
Post
#1
|
|
Junior Member Group: Members Posts: 30 Joined: 8-September 14 From: London, UK Member No.: 7254 |
Hi -
if this is not the right place to ask, sorry! Please show me the right place. I'm looking through JPL's JSON endpoint (http://json.jpl.nasa.gov/data.json) for rover imagery metadata, especially the one for MSL. I'd love:
Specific things I'm struggling to figure out from files such as http://msl-raws.s3.amazonaws.com/images/images_sol713.json:
Any insights or pointers to documentation? The reason I'm looking into this is just out of curiosity (I'm a layperson). I discovered mhoward's wonderful Midnight Planets recently and would love to build a part of something similar as a learning experience. I've checked out NAIF which Michael mentions on his website as a source of metadata but I'm not sure where exactly to look (I've dug through their collection of command line tools written in C but it all seemed a lot more complex than should be required to just find out where, in what attitude and what camera model an image got taken). Thanks heaps for any help/pointers! Tobias |
|
|
Oct 10 2015, 09:14 PM
Post
#2
|
|
Member Group: Members Posts: 890 Joined: 18-November 08 Member No.: 4489 |
normally this type of information is in the pds archive along with the imaging data
for example ( random picked MSL folder ) http://pds-imaging.jpl.nasa.gov/data/msl/MSLHAZ_0XXX/ the images are in "data" http://pds-imaging.jpl.nasa.gov/data/msl/MSLHAZ_0XXX/DATA/ the documentation is in the documents folder http://pds-imaging.jpl.nasa.gov/data/msl/M..._0XXX/DOCUMENT/ and the README file describes what is in the archive http://pds-imaging.jpl.nasa.gov/data/msl/M...XX/AAREADME.TXT as to vectors it is tangent to the barycenter ( for mars this is only 20 Centimeters different than the geographical center so that is used) ( the naif spice kernels are used for that ) the cameras then are all RELATIVE to the rover body orientation for that information see the naif tutorials on using spice http://naif.jpl.nasa.gov/naif/tutorials.html for MSL that is ftp://naif.jpl.nasa.gov/pub/naif/MSL/ ftp://naif.jpl.nasa.gov/pub/naif/MSL/kernels/ for working with this kind of data i use ISIS3 -- but MSL really is not supported well as i recall IDL is used for the msl imaging data you might want to read through "MSL data in the PDS and the Analyst's Notebook, Working with the archived science & engineering data" http://www.unmannedspaceflight.com/index.php?showtopic=7613 |
|
|
Oct 11 2015, 01:32 PM
Post
#3
|
|
Junior Member Group: Members Posts: 30 Joined: 8-September 14 From: London, UK Member No.: 7254 |
*facepalm* Thanks so much, I have no idea why it didn't occur to me to cross-references the data available on json.jpl.nasa.gov against the PDS and the Analyst's Notebook which have much better documentation for that some data!
For example the documentation on the coordinate frames on Analyst's Notebook and on PDS is super helpful. I think I even saw that document a year ago, no idea why I forgot about it. Based on that document, it seems that the rover_xyz coordinates are (probably in meters) relative to a particular site's frame; I hadn't noticed before that the rovery_xyz values reset to 0 from time to time (e.g. on sol 14 which was a new site). It looks like the position of those sites themselves may be missing from the JSON data that I've seen. I'll have a look through Analyst's Notebook to get the locations of those sites, or if that doesn't work then I'll fall back to the NAIF data too which I'd rather avoid (last time I spent a day looking through the NAIF documentation, it seemed really complex / more powerful than what I needed). Thanks again for those pointers! That should give me enough to read for next weekend normally this type of information is in the pds archive along with the imaging data
for example ( random picked MSL folder ) http://pds-imaging.jpl.nasa.gov/data/msl/MSLHAZ_0XXX/ the images are in "data" http://pds-imaging.jpl.nasa.gov/data/msl/MSLHAZ_0XXX/DATA/ the documentation is in the documents folder http://pds-imaging.jpl.nasa.gov/data/msl/M..._0XXX/DOCUMENT/ and the README file describes what is in the archive http://pds-imaging.jpl.nasa.gov/data/msl/M...XX/AAREADME.TXT as to vectors it is tangent to the barycenter ( for mars this is only 20 Centimeters different than the geographical center so that is used) ( the naif spice kernels are used for that ) the cameras then are all RELATIVE to the rover body orientation for that information see the naif tutorials on using spice http://naif.jpl.nasa.gov/naif/tutorials.html for MSL that is ftp://naif.jpl.nasa.gov/pub/naif/MSL/ ftp://naif.jpl.nasa.gov/pub/naif/MSL/kernels/ for working with this kind of data i use ISIS3 -- but MSL really is not supported well as i recall IDL is used for the msl imaging data you might want to read through "MSL data in the PDS and the Analyst's Notebook, Working with the archived science & engineering data" http://www.unmannedspaceflight.com/index.php?showtopic=7613 |
|
|
Oct 11 2015, 04:12 PM
Post
#4
|
|
Senior Member Group: Members Posts: 2542 Joined: 13-September 05 Member No.: 497 |
last time I spent a day looking through the NAIF documentation, it seemed really complex / more powerful than what I needed I'm not sure exactly what you're trying to accomplish, but the NAIF stuff is not a lot more complex than it needs to be IMHO. "There is no royal road to geometry" as Euclid said -------------------- Disclaimer: This post is based on public information only. Any opinions are my own.
|
|
|
Oct 13 2015, 09:38 AM
Post
#5
|
|
Junior Member Group: Members Posts: 30 Joined: 8-September 14 From: London, UK Member No.: 7254 |
I'm not sure exactly what you're trying to accomplish, but the NAIF stuff is not a lot more complex than it needs to be IMHO. "There is no royal road to geometry" as Euclid said The geometry is not what I'm afraid of; or at least I first need the data before I can start worrying about geometry (it seems like it should just be some matrix multiplications). The data I specifically want to get out of SPICE is the rover and camera reference frames for each image taken by MSL. I was hoping I'd be able to download e.g. a bunch of .csv files that listed these coordinates for each site, and for each image relative to that site. Instead, I'm too stupid to even find this data in NAIF/SPICE. The required reading list for the toolkit is 24 documents long. After reading through many of the 46 tutorial documents I figured out that the data should be in an "FK kernel" so I downloaded the FK kernel file for MSL. Turns out that file, which is in plain text, maps site names (e.g. MSL_SITE_144) to integer codes (e.g. -76644) but doesn't contain the corresponding coordinates (there is no FRAME_-76644 definition). I'd be very grateful for any pointers to where I can download that data (MSL site coordinates)! |
|
|
Oct 13 2015, 01:12 PM
Post
#6
|
|
Member Group: Members Posts: 244 Joined: 2-March 15 Member No.: 7408 |
I could be wrong since I've only ever touched SPICE kernels for spacecraft, not rovers, and I'm definitely the wrong person to be answering this, but until someone who knows what they're talking about comes along, I'll say that I think the data you want might be in the SPK kernels. They're .BSP files and are not plain text. I'm pretty sure there are existing open source libraries you can use. I decided to parse them myself, as I always seem to be trying to reinvent the wheel, and eventually got them figured out, but I remember feeling that the documentation was confusing and seemed to assume I would just know what would be where.
Edit: I just re-read your post and now I think I may have misunderstood what you were looking for and that my answer won't help you. Edit 2: Actually, if I now understand what you're looking for, the SPK kernels might be where you want to look after all. In http://naif.jpl.nasa.gov/pub/naif/MSL/kern...pk/aareadme.txt it describes http://naif.jpl.nasa.gov/pub/naif/MSL/kern...f_rover_tlm.bsp as QUOTE an ever-growing SPK file containing the position of the rover relative to the sites along the path and position of the sites relative to the landing site, created using telemetry view data from OPGS PLACES server, covering from SOL 0000 through the current SOL, updated twice each SOL at 06:30 and 18:30 LMST. Additionally: http://naif.jpl.nasa.gov/pub/naif/MSL/kern..._iau2000_v1.bsp is an "SPK file containing the position of the actual landing site". http://naif.jpl.nasa.gov/pub/naif/MSL/kern..._struct_v02.bsp is an "SPK [file] containing fixed offsets between various rover structures and instruments". See http://naif.jpl.nasa.gov/pub/naif/MSL/kern..._struct_v02.cmt. |
|
|
Oct 13 2015, 03:32 PM
Post
#7
|
|
Junior Member Group: Members Posts: 30 Joined: 8-September 14 From: London, UK Member No.: 7254 |
Oh, awesome! Thanks so much, Herobrine! It sounds indeed like the SPK kernel is what I need. In my mind I had classed this data as falling into the category "Definitions of and specification of relationships between reference frames (coordinate systems)" (FK kernel) when apparently the correct SPICE category is "Space vehicle or target body trajectory (ephemeris)" (SPK kernel).
Sweet! Now I can focus on the next step, namely looking through the C or MATLAB based programming examples to figure out how to extract the coordinates from that kernel file into an "easy" format (eg. csv). I'll try that as soon as I can (probably on the weekend because this is just my hobby). Thanks again for your help! Update: running utilities/toxfr.exe MSL/kernels/spk/msl_surf_rover_tlm.bsp gave me a sort-of-text format that contains lines like [...] 'LS-to-4 -43.856 67.197 1.735 ' [...] 'LS-to-18 -1364.773 -1202.160 -10.022 ' I bet those are the coordinates of the site 4 / site 18 frames relative to landing (the x/y coordinates keep growing for higher numbers, and the third coordinate is probably elevation so it's smaller). Awesome! Thanks again! |
|
|
Oct 13 2015, 05:04 PM
Post
#8
|
|
Senior Member Group: Members Posts: 2542 Joined: 13-September 05 Member No.: 497 |
SPICELIB is set up so that the SPK and CK files can be defined in any frame, or potentially different frames between files. If you load all of the kernels (including static kernels like FK), and you ask for the position of the rover in, say, Mars-center-relative coordinates, by using the frames kernel the software will automatically do all of the needed transforms to get you to the desired coordinate frame.
-------------------- Disclaimer: This post is based on public information only. Any opinions are my own.
|
|
|
Dec 28 2015, 11:43 AM
Post
#9
|
|
Junior Member Group: Members Posts: 30 Joined: 8-September 14 From: London, UK Member No.: 7254 |
Thanks for everyone's help! That was very helpful for the sort of Midnight Planets clone for Android that I've been working on over Christmas as my holiday project (see screenshot). It's far behind Midnight Planets' feature set, e.g. there is no stereo view yet. Not good enough to publish yet.
I'm relying on json.jpl.nasa.gov for most metadata (everything other than the relative location of the sites, which I got out of SPICE), but there are many gaps in the data; is the data in SPICE more complete? json.jpl.nasa.gov has also started returning a lot of HTTP 403 Access Denied, e.g. the fact that the screenshot doesn't have any color images is because they 403; I'm also getting 403 here: http://msl-raws.s3.amazonaws.com/images/images_sol1000.json |
|
|
Lo-Fi Version | Time is now: 21st September 2024 - 06:43 PM |
RULES AND GUIDELINES Please read the Forum Rules and Guidelines before posting. IMAGE COPYRIGHT |
OPINIONS AND MODERATION Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators. |
SUPPORT THE FORUM Unmannedspaceflight.com is funded by the Planetary Society. Please consider supporting our work and many other projects by donating to the Society or becoming a member. |