Out of the noise level..., are we seeing what we want to see? Options

[dvandorn]

One concern I've been worrying, like a bone, over the last several years, is the very tiny bits of data, barely above the noise level, upon which so many new "discoveries" seem to be based.

We're characterizing planetary systems based on the wiggles of stars -- when in some cases those wiggles are so far down near the level of chaotic noise vs. solid, unambiguous data that there could be other reasons (including sensor-induced noise) to account for them.

And as we build fairy castles out of data gleaned just barely above the noise level, we base further conclusions upon "understandings" that themselves are based on far too many single-point data points pulled from nearly-unreadable noise.

I'm not arguing that all of these discoveries are bogus. I'm just wondering how far we ought to go in trying to pull information out of noise. And I'm wondering just how much some of these detectors are designed to display what the PI wants to see -- not what's really out there.

-the other Doug

[djellison]

I don't think the exoplanet example is particularlly appropriate

http://brucegary.net/tutorial_exoplanet/All_BLG_Data.png
http://www.eso.org/outreach/press-rel/pr-2...-05-preview.jpg
http://www.ucolick.org/~laugh/tres1.vanmunster1.jpg
http://www.ucolick.org/~laugh/Tres1_gary1.jpg

A long long long way from a single data point out of unreasable noise - I don't think that criticism is valid.

These are all very small changes - yes - but these are also very very sensitive instruments that are specifically designed to find small changes - and found them they have - particularly when you see how spot on with predictions these changes are once a discovery has been made.

I don't think Exoplanets are pulling information from noise at all.

I think there are cases where people are pulling something from nothing - typically the fringe theory or artificiality nuts (you know who I mean ) but a real scientists who's going to end up with his work being peer reviewed before published just couldn't get away with pulling something from nothing.

[dvandorn]

Yes, that could have been a poor example. Actually, I was moved to post this train of thought when I read a post on how some people are planning on trying to chart Pluto's winds based on a momentary "flash" during the midst of a stellar occultation. It seemed to me that we're trying to pull WAY too much information out of such a transient phenomenon. That led me to other examples.

And, hey, just because something passes peer review doesn't mean the detectors themselves aren't designed to show what the PI wants to see... especially if the peers reviewing the work also want to see the same kind of result.

-the other Doug

[Guest_Richard Trigaux_*]

I worked a bit on this (extracting useful data from noise).

Basically working with noise is statistics. And, when we see a departure from a statistical average, we have methods to qualify the odds of this departure being a random variation or a significant one. For instance a 90% confidence means that the departure has 90% chances of being a real effect. 99% confidence is that it has 99% etc. We have accurate tests, like Khi test, linear regretion, cross corellations, etc... which allow to detect effects which are impossible to guess with the naked eye simply looking at a curve.


There are many examples:

-political polls require about a 1000 person samples to, theoretically, give a 1% accuracy on predictions. Often larger errors are made, but never more than 3-4%.

-SETI qualifies a candidate signal by its probability to arise from random. If a given random pattern can arise once every 1000 hours, we expect to find 1000 of them in 1000 000 hours.

-In medecine and biology, much lower samples are requested, for example 150 persons and 150 witness group to test the efficiency of a medecine for emergency heart attacks. With a difference of 20% they were happy to say that the cure was effective.

-the most impressive is extracting a radio signal from noise. FFTs and other are able to clean signals which are other wise completelly buried in noise.



So your concern is not technically relevant. But I would not swear that what you describes never happens...

[climber]

Talking about star occultation, there's a post on Emily's blog : http://planetary.org/blog/article/00000579/
Want to be the fist one to see new Pluto's sattellites with your own eyes dvandorn?
Hurry up to New Zealand ***

*** Tasmania, there's the Tasmania's devil, Reunion's full of Mosquitos with Chicungunia, Australia's full of Australians (Astro0's fine anyway)

[ljk4-1]

Talking about star occultation, there's a post on Emily's blog : http://planetary.org/blog/article/00000579/
Want to be the fist one to see new Pluto's sattellites with your own eyes dvandorn?
Hurry up to New Zealand ***

*** Tasmania, there's the Tasmania's devil, Reunion's full of Mosquitos with Chicungunia, Australia's full of Australians (Astro0's fine anyway)

I have already started a thread on this very topic:

http://www.unmannedspaceflight.com/index.p...indpost&p=54453

As the Web site states, I hope the occultations will be useful to the New Horizons team,
as moon P2 will also occult the star, a first for such observations.

[Bob Shaw]

I really don't see why there's any need to spend any more money on these grand planet-forming enquiries - we know perfectly well how, and why, the planets were formed - indeed, we have done since the late 1960s.

Bob Shaw

Attached thumbnail(s)

 

[dvandorn]

Bob, I am simply ashamed of myself that I didn't think of bringing in "It's turtles all the way down" before you did. I must be getting senile...

-the other Doug

[edstrick]

"...how some people are planning on trying to chart Pluto's winds based on a momentary "flash" during the midst of a stellar occultation. ..."

They are probably looking for the central flash during an occultation that passes near the center of the shadow. This *HAS* been observed in the Titan occultation of a rather bright star and was not small, it was a very strong, intense, well observed signal seen at more than one occultation station. What is happening is that when the line of sight is at <or close to> the center of the shadow, the refracted image of the star forms a ring entirely, or partially around the disk of the planet or moon. If the atmosphere is rotating or has large regional winds, the rotational "centrifugal" force flattens the atmosphere at some latitudes and increases it's curvature at others, changing the shape of the refracting lens and the image of starlight in the shadow.

For the case of a spherical planet and non-rotating atmosphere (spherical symmetry) the starlight is focused to a radially symmetric "cusp" at the center of the shadow. For the case of a flattened atmosphere, the lens has astigmatism and the cusp is distorted. Occultation brightness profiles across different parts of the cusp can be used to precisely test predictions for different atmosphere rotation and structure models.

[Bob Shaw]

Bob, I am simply ashamed of myself that I didn't think of bringing in "It's turtles all the way down" before you did. I must be getting senile...

-the other Doug

oDoug:

No, better to simply say that good taste prevailed!

Bob Shaw