IPB

Welcome Guest ( Log In | Register )

25 Pages V  « < 12 13 14 15 16 > »   
Reply to this topicStart new topic
Curiosity Image Retrieval Tools, scripts and software
jmknapp
post Oct 25 2012, 05:57 PM
Post #196


Senior Member
****

Group: Members
Posts: 1465
Joined: 9-February 04
From: Columbus OH USA
Member No.: 13



QUOTE (elakdawalla @ Oct 25 2012, 10:52 AM) *
I suggest you establish a controlled or at least strongly suggested vocabulary, or at the very least a way to list tags currently in use. Certain things that get imaged a lot -- the chemin inlet port, marsdial, etc -- should somehow get tagged identically.


That's a good idea--right now it's pretty much unconstrained, leaving to question whether the genius or madness of the crowd would prevail. Most sites like youtube, flickr, etc. suggest tags as you say. Not sure if it's my pay grade to come up with a list a priori though, so maybe a list of current tags is the way to go for a start.

Maybe if people used different tags once in a while for the same thing it wouldn't be too onerous to manually go in occasionally and merge synonymous tags and that would build up an 'approved' list more organically.

EDIT: I made a change that hopefully should make it less likely the activation email ends up in a spam folder. Hey, that's why it's called beta.


--------------------
Go to the top of the page
 
+Quote Post
CosmicRocker
post Oct 26 2012, 03:49 AM
Post #197


Senior Member
****

Group: Members
Posts: 2228
Joined: 1-December 04
From: Marble Falls, Texas, USA
Member No.: 116



Your site keeps getting better and better. I love it. My registration proceeded very quickly and I had no problem with spam issues.


--------------------
...Tom

I'm not a Space Fan, I'm a Space Exploration Enthusiast.
Go to the top of the page
 
+Quote Post
maschnitz
post Oct 26 2012, 06:14 AM
Post #198


Junior Member
**

Group: Members
Posts: 60
Joined: 3-January 09
Member No.: 4520



Just an FYI, Joe - the software lingo that magically opens up results on Google for that tag feature is "tag autocomplete".

JQuery has plugins that handle it decently on the browser side. I imagine the PHP server side has some how-tos out there, too. But it can be "work", and I can definitely understand being hesitant. And thanks again for the site, I use it every day.
Go to the top of the page
 
+Quote Post
jmknapp
post Oct 26 2012, 02:29 PM
Post #199


Senior Member
****

Group: Members
Posts: 1465
Joined: 9-February 04
From: Columbus OH USA
Member No.: 13



Alright, I changed the tag entry page so that it uses a jquery plugin for tag suggestion and auto-completion. Pretty slick!

I put in a list of terms to start, mostly gleaned from the literature, but I'm not too familiar with geological terms, so any suggestions welcome. I'm kind of in awe of the lingo and analysis tossed around here by the geology types.

Here's the initial list:

http://marstag.com/suggestedtags.txt


--------------------
Go to the top of the page
 
+Quote Post
fredk
post Oct 27 2012, 04:01 PM
Post #200


Senior Member
****

Group: Members
Posts: 4246
Joined: 17-January 05
Member No.: 152



Hi Joe - a question about your site http://curiositymsl.com/table/view/. I noticed that the sol 25 MR sequence that was returned recently didn't show up on your site (yet). Do I remember right that you ignore older released images? How far back do you go? If it means too many http requests (or whatever) to check all of the sols every 15 minutes, can you check the older sols less frequently (even once per day)?
Go to the top of the page
 
+Quote Post
jmknapp
post Oct 27 2012, 06:35 PM
Post #201


Senior Member
****

Group: Members
Posts: 1465
Joined: 9-February 04
From: Columbus OH USA
Member No.: 13



I go back 60 sols so it should have been in the bunch. The sol 25 MR images seem to be there now,, yes? Not sure why there might have been a delay.

Good idea about occasionally going further back, maybe once a day. It does take a while to scrape and process all those pages though, and, knock on wood, this mission might last a thousand sols. There has to be a statute of limitations somewhere!

Kinda funny thinking about a file sitting there on the rover for months waiting for the "call."


--------------------
Go to the top of the page
 
+Quote Post
fredk
post Oct 27 2012, 08:22 PM
Post #202


Senior Member
****

Group: Members
Posts: 4246
Joined: 17-January 05
Member No.: 152



OK, I see the problem. My browser isn't handling the auto loading properly when I scroll to the bottom of the page. You can see in this screenshot a discontinuity of three days, between images released Oct 27th and 24th:
Attached Image

That discontinuity is at the bottom of the original full page, and that explains why I wasn't seeing the sol 25 images.

I tried this on a current version of firefox and it works fine. So unfortunately I'm stuck with this (until I upgrade from firefox 2.0.0.20!).
Go to the top of the page
 
+Quote Post
fredk
post Oct 29 2012, 10:09 PM
Post #203


Senior Member
****

Group: Members
Posts: 4246
Joined: 17-January 05
Member No.: 152



QUOTE (jmknapp @ Oct 27 2012, 07:35 PM) *
Good idea about occasionally going further back, maybe once a day. It does take a while to scrape and process all those pages though, and, knock on wood, this mission might last a thousand sols.
Which page do you scrape? Do you search each camera and sol page (such as this one) for asterisks (which mark newly received images)? Or do you search this camera list page for orange ("OrangeRawMark") sol numbers, and then search only those sols for asterisks?
Go to the top of the page
 
+Quote Post
jmknapp
post Oct 30 2012, 09:48 AM
Post #204


Senior Member
****

Group: Members
Posts: 1465
Joined: 9-February 04
From: Columbus OH USA
Member No.: 13



I scrape every camera/sol page regardless of the "new" flags--I think Ludo determined at one point that the flags are not reliable.


--------------------
Go to the top of the page
 
+Quote Post
ronald
post Nov 17 2012, 08:10 PM
Post #205


Member
***

Group: Members
Posts: 154
Joined: 19-September 12
Member No.: 6658



huh.gif

Neither Joes nor Ludos sites do work since they changed something over at the Nasa raw images web page. Browsing for images there is really unpleasing and sometimes confusing.

HELP!
Go to the top of the page
 
+Quote Post
mhoward
post Nov 17 2012, 09:53 PM
Post #206


Senior Member
****

Group: Moderator
Posts: 3431
Joined: 11-August 04
From: USA
Member No.: 98



Yes, the view-by-sol functionality on the raw images site is basically broken as of today. If you view images by sol and save the page data, the saved file won't include any image data. I think it probably has something to do with the sol selector functionality (maybe that's new?).

Also, the page for sol 100 currently isn't even showing all sol 100 images. Oddly, http://mars.jpl.nasa.gov/msl/multimedia/raw/?s=100 doesn't show all the images, but adding http://mars.jpl.nasa.gov/msl/multimedia/ra...100&camera= seems to work correctly. At the moment.

So, the simplest workaround for those of us trying to get data from these pages seems to be to view by instrument and sol, or add "&camera=" to your query.
Go to the top of the page
 
+Quote Post
jmknapp
post Nov 17 2012, 10:49 PM
Post #207


Senior Member
****

Group: Members
Posts: 1465
Joined: 9-February 04
From: Columbus OH USA
Member No.: 13



Yep, looks like they've made some changes, jquerying things up quite a bit, and the usual scrape urls by sol don't work. E.g., it used to be that scraping this url:

wget -O xyzzy http://mars.jpl.nasa.gov/msl/multimedia/raw/?s=100

...would get you a file xyzzy with all the sol 100 images in the html, but now it doesn't, you just see the jquery command.

Anyway, going through the code that is retrieved using the above, I see that this is the magic jquery incantation to get the files:

$("#listImagesContent").load("../../admin/modules/multimedia/module/inc_ListImages_Raw.cfm?s=100");

Accordingly, this wget command works as a replacement:

wget -O xyzzy http://mars.jpl.nasa.gov/msl/admin/modules...s_Raw.cfm?s=100

Bottom line, I think it's working again. The detail above is for the benefit of Ludo or others trying to scrape the raw image pages.


--------------------
Go to the top of the page
 
+Quote Post
ronald
post Nov 17 2012, 11:06 PM
Post #208


Member
***

Group: Members
Posts: 154
Joined: 19-September 12
Member No.: 6658



BIG thanks!
Go to the top of the page
 
+Quote Post
mhoward
post Nov 17 2012, 11:19 PM
Post #209


Senior Member
****

Group: Moderator
Posts: 3431
Joined: 11-August 04
From: USA
Member No.: 98



Again: ?s=100 isn't returning all the images. ?s=100&camera= is.

Just FYI.
Go to the top of the page
 
+Quote Post
fredk
post Nov 18 2012, 03:27 AM
Post #210


Senior Member
****

Group: Members
Posts: 4246
Joined: 17-January 05
Member No.: 152



Thanks a lot for quickly getting your site working again, Joe! It's such a nice way to browse the new images.
Go to the top of the page
 
+Quote Post

25 Pages V  « < 12 13 14 15 16 > » 
Reply to this topicStart new topic

 



RSS Lo-Fi Version Time is now: 30th April 2024 - 11:08 PM
RULES AND GUIDELINES
Please read the Forum Rules and Guidelines before posting.

IMAGE COPYRIGHT
Images posted on UnmannedSpaceflight.com may be copyrighted. Do not reproduce without permission. Read here for further information on space images and copyright.

OPINIONS AND MODERATION
Opinions expressed on UnmannedSpaceflight.com are those of the individual posters and do not necessarily reflect the opinions of UnmannedSpaceflight.com or The Planetary Society. The all-volunteer UnmannedSpaceflight.com moderation team is wholly independent of The Planetary Society. The Planetary Society has no influence over decisions made by the UnmannedSpaceflight.com moderators.
SUPPORT THE FORUM
Unmannedspaceflight.com is funded by the Planetary Society. Please consider supporting our work and many other projects by donating to the Society or becoming a member.