Curiosity Image Retrieval Tools, scripts and software |
Curiosity Image Retrieval Tools, scripts and software |
Oct 25 2012, 05:57 PM
Post
#196
|
|
Senior Member Group: Members Posts: 1465 Joined: 9-February 04 From: Columbus OH USA Member No.: 13 |
I suggest you establish a controlled or at least strongly suggested vocabulary, or at the very least a way to list tags currently in use. Certain things that get imaged a lot -- the chemin inlet port, marsdial, etc -- should somehow get tagged identically. That's a good idea--right now it's pretty much unconstrained, leaving to question whether the genius or madness of the crowd would prevail. Most sites like youtube, flickr, etc. suggest tags as you say. Not sure if it's my pay grade to come up with a list a priori though, so maybe a list of current tags is the way to go for a start. Maybe if people used different tags once in a while for the same thing it wouldn't be too onerous to manually go in occasionally and merge synonymous tags and that would build up an 'approved' list more organically. EDIT: I made a change that hopefully should make it less likely the activation email ends up in a spam folder. Hey, that's why it's called beta. -------------------- |
|
|
Oct 26 2012, 03:49 AM
Post
#197
|
|
Senior Member Group: Members Posts: 2228 Joined: 1-December 04 From: Marble Falls, Texas, USA Member No.: 116 |
Your site keeps getting better and better. I love it. My registration proceeded very quickly and I had no problem with spam issues.
-------------------- ...Tom
I'm not a Space Fan, I'm a Space Exploration Enthusiast. |
|
|
Oct 26 2012, 06:14 AM
Post
#198
|
|
Junior Member Group: Members Posts: 60 Joined: 3-January 09 Member No.: 4520 |
Just an FYI, Joe - the software lingo that magically opens up results on Google for that tag feature is "tag autocomplete".
JQuery has plugins that handle it decently on the browser side. I imagine the PHP server side has some how-tos out there, too. But it can be "work", and I can definitely understand being hesitant. And thanks again for the site, I use it every day. |
|
|
Oct 26 2012, 02:29 PM
Post
#199
|
|
Senior Member Group: Members Posts: 1465 Joined: 9-February 04 From: Columbus OH USA Member No.: 13 |
Alright, I changed the tag entry page so that it uses a jquery plugin for tag suggestion and auto-completion. Pretty slick!
I put in a list of terms to start, mostly gleaned from the literature, but I'm not too familiar with geological terms, so any suggestions welcome. I'm kind of in awe of the lingo and analysis tossed around here by the geology types. Here's the initial list: http://marstag.com/suggestedtags.txt -------------------- |
|
|
Oct 27 2012, 04:01 PM
Post
#200
|
|
Senior Member Group: Members Posts: 4256 Joined: 17-January 05 Member No.: 152 |
Hi Joe - a question about your site http://curiositymsl.com/table/view/. I noticed that the sol 25 MR sequence that was returned recently didn't show up on your site (yet). Do I remember right that you ignore older released images? How far back do you go? If it means too many http requests (or whatever) to check all of the sols every 15 minutes, can you check the older sols less frequently (even once per day)?
|
|
|
Oct 27 2012, 06:35 PM
Post
#201
|
|
Senior Member Group: Members Posts: 1465 Joined: 9-February 04 From: Columbus OH USA Member No.: 13 |
I go back 60 sols so it should have been in the bunch. The sol 25 MR images seem to be there now,, yes? Not sure why there might have been a delay.
Good idea about occasionally going further back, maybe once a day. It does take a while to scrape and process all those pages though, and, knock on wood, this mission might last a thousand sols. There has to be a statute of limitations somewhere! Kinda funny thinking about a file sitting there on the rover for months waiting for the "call." -------------------- |
|
|
Oct 27 2012, 08:22 PM
Post
#202
|
||
Senior Member Group: Members Posts: 4256 Joined: 17-January 05 Member No.: 152 |
OK, I see the problem. My browser isn't handling the auto loading properly when I scroll to the bottom of the page. You can see in this screenshot a discontinuity of three days, between images released Oct 27th and 24th:
That discontinuity is at the bottom of the original full page, and that explains why I wasn't seeing the sol 25 images. I tried this on a current version of firefox and it works fine. So unfortunately I'm stuck with this (until I upgrade from firefox 2.0.0.20!). |
|
|
||
Oct 29 2012, 10:09 PM
Post
#203
|
|
Senior Member Group: Members Posts: 4256 Joined: 17-January 05 Member No.: 152 |
Good idea about occasionally going further back, maybe once a day. It does take a while to scrape and process all those pages though, and, knock on wood, this mission might last a thousand sols. Which page do you scrape? Do you search each camera and sol page (such as this one) for asterisks (which mark newly received images)? Or do you search this camera list page for orange ("OrangeRawMark") sol numbers, and then search only those sols for asterisks?
|
|
|
Oct 30 2012, 09:48 AM
Post
#204
|
|
Senior Member Group: Members Posts: 1465 Joined: 9-February 04 From: Columbus OH USA Member No.: 13 |
I scrape every camera/sol page regardless of the "new" flags--I think Ludo determined at one point that the flags are not reliable.
-------------------- |
|
|
Nov 17 2012, 08:10 PM
Post
#205
|
|
Member Group: Members Posts: 154 Joined: 19-September 12 Member No.: 6658 |
Neither Joes nor Ludos sites do work since they changed something over at the Nasa raw images web page. Browsing for images there is really unpleasing and sometimes confusing. HELP! |
|
|
Nov 17 2012, 09:53 PM
Post
#206
|
|
Senior Member Group: Moderator Posts: 3431 Joined: 11-August 04 From: USA Member No.: 98 |
Yes, the view-by-sol functionality on the raw images site is basically broken as of today. If you view images by sol and save the page data, the saved file won't include any image data. I think it probably has something to do with the sol selector functionality (maybe that's new?).
Also, the page for sol 100 currently isn't even showing all sol 100 images. Oddly, http://mars.jpl.nasa.gov/msl/multimedia/raw/?s=100 doesn't show all the images, but adding http://mars.jpl.nasa.gov/msl/multimedia/ra...100&camera= seems to work correctly. At the moment. So, the simplest workaround for those of us trying to get data from these pages seems to be to view by instrument and sol, or add "&camera=" to your query. |
|
|
Nov 17 2012, 10:49 PM
Post
#207
|
|
Senior Member Group: Members Posts: 1465 Joined: 9-February 04 From: Columbus OH USA Member No.: 13 |
Yep, looks like they've made some changes, jquerying things up quite a bit, and the usual scrape urls by sol don't work. E.g., it used to be that scraping this url:
wget -O xyzzy http://mars.jpl.nasa.gov/msl/multimedia/raw/?s=100 ...would get you a file xyzzy with all the sol 100 images in the html, but now it doesn't, you just see the jquery command. Anyway, going through the code that is retrieved using the above, I see that this is the magic jquery incantation to get the files: $("#listImagesContent").load("../../admin/modules/multimedia/module/inc_ListImages_Raw.cfm?s=100"); Accordingly, this wget command works as a replacement: wget -O xyzzy http://mars.jpl.nasa.gov/msl/admin/modules...s_Raw.cfm?s=100 Bottom line, I think it's working again. The detail above is for the benefit of Ludo or others trying to scrape the raw image pages. -------------------- |
|
|
Nov 17 2012, 11:06 PM
Post
#208
|
|
Member Group: Members Posts: 154 Joined: 19-September 12 Member No.: 6658 |
BIG thanks!
|
|
|
Nov 17 2012, 11:19 PM
Post
#209
|
|
Senior Member Group: Moderator Posts: 3431 Joined: 11-August 04 From: USA Member No.: 98 |
Again: ?s=100 isn't returning all the images. ?s=100&camera= is.
Just FYI. |
|
|
Nov 18 2012, 03:27 AM
Post
#210
|
|
Senior Member Group: Members Posts: 4256 Joined: 17-January 05 Member No.: 152 |
Thanks a lot for quickly getting your site working again, Joe! It's such a nice way to browse the new images.
|
|
|
Lo-Fi Version | Time is now: 22nd September 2024 - 06:17 AM |
RULES AND GUIDELINES Please read the Forum Rules and Guidelines before posting. IMAGE COPYRIGHT |
OPINIONS AND MODERATION Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators. |
SUPPORT THE FORUM Unmannedspaceflight.com is funded by the Planetary Society. Please consider supporting our work and many other projects by donating to the Society or becoming a member. |