How to Spot Pseudo-Science

Most of the really interesting stuff in this blog happens in the comments section. Not only do I get to read extremely insightful thoughts from people who are wiser than I, but I also get to find out about people’s reactions to what I write, as well as a lot of other completely unrelated stuff.

Something that has come up repeatedly is the question: “I have just seen this guy who claims X. Is this reliable?” That is: how can I tell science from pseudo-science?

By Shang

By Shang

This is a short guide to help you determine exactly this. Instead of a long discussion, I’ll try to make this into bullet points, so that it stays entertaining and easy to browse.

  1. Can they spell? Seriously. When the authors cannot spell their own name, their take on whatever scientific matter they’re tackling is suspect. This also goes for the name of famous scientists: “Feynmann” or “Eintsein” may be real people, but they have nothing to do with physics.
  2. Where did you read this? I understand it is unrealistic to expect non-scientists to read scientific magazines. But, for example, anything from “The Epoch Times” can almost immediately be dismissed as bogus. Did the New York Times say anything about this too? That’s a good sign. No major newspaper mentioned the news? Well, there’s probably a reason for that.
  3. Is it referenced? Always look for a link to an article: that usually means something (though it can be faked, as many “scientific” magazines will publish pretty much anything.) Anyway, a reference to a research paper should count as points up.
  4. How huge is the finding? If it’s huge, you can be 99.9% sure it’s not true. Huge discoveries happen rarely: that’s why they are huge. And, when they do, scientists are usually cautious about it and present it in a way that sounds less than what it is. For example, we now have several indications that we’ve found Dark Matter, but you still haven’t seen “Dark Matter found” anywhere. That’s because we need more data and no serious scientist is going to make a proclamation until the evidence has mounted up.
  5. Does it talk about “media suppression” or “conspiracies”? Dismiss it immediately. I don’t have to explain why.
  6. Does it prove some widely-accepted theory to be wrong? Dismiss it immediately. This does not mean widely-accepted theories cannot be wrong. But, if we were to discard relativity, we’d need a huge amount of evidence and doubts would be slowly mounting up. This would never happen in a day.
  7. Does it claim to have solved the world’s energy problems? Bogus. When this happens, you’ll know. It will be everywhere.
  8. Does it talk about energy and forces in a vague way? Dismiss. Energy and force are precisely defined scientific concepts that have absolutely nothing to do with, well, other stuff.
  9. Does it make value judgments? Dismiss. Scientific articles do not judge: they just present data and conclusions in the most neutral way possible. There may be a few exceptions, but better safe than sorry.
  10. Does it rely on ancient or esoteric knowledge? Remember the bubonic plague? Its treatment was based on ancient knowledge. Knowledge gets replaced as we learn: that is the whole point of the scientific method. Something being ancient does not equate to it being good. This doesn’t mean ancient stuff cannot work, but if it does it’s not because it’s ancient, just because it’s effective. “Ancientness” is not a valid argument. Ancient wisdom also says that the Earth is at the center of the universe and that everything is made of 5 elements.
  11. Is it “all natural”? Funny how that works. Smoking a joint: natural or artificial? One would say quite a lot of human labor has gone into it. In fact, anything produced by us is by definition artificial. That said, crap is natural and still we don’t eat it. Not everything natural is good; not everything artificial is bad. “Naturalness” should not be equated to health. That said, this doesn’t mean some products currently in the market do not carry a huge amount of substances the benefits of which for your health are anything but clear. But that is another discussion and I’m unfortunately not an expert.
  12. Are the claimants making any money if people believe them? That, of course, could apply to scientists that makes any discovery (they may get a better position or tenure) but we’re talking about different money scales. If there is a substantial economic interest behind a claim, you should be at least suspicious.
  13. Wikipedia is your friend. Every time I’m not sure about something, I go to Wikipedia and straight to the “criticisms” section. It is readily apparent whether the criticisms make sense.
  14. Ask Google. Google “Claim X reliable” and you’ll find thousands of posts in forums and the like. For example, some months ago a read something in the Epoch Times about crystals in the desert that could only be produced by atomic bombs but were thousands of years old (therefore aliens). It took me 20 seconds to find alternative explanations that did not involve little green men. Since then I’ve been paying attention every time a piece of news from that site shows in my Flipboard and I’ve learnt to quarantine anything they say.

And, to end up, a couple of articles that expand on the subject:

http://skeptoid.com/episodes/4037

http://imgur.com/gallery/EWI1zPT (you may not agree with everything, but it’s still funny)

And here’s a link to a site that allows you to get a doctor’s degree in science in about 20 seconds! http://thunderwoodcollege.com/

Enjoy.

Enhanced by Zemanta

20 thoughts on “How to Spot Pseudo-Science

  1. Johannes Nelson

    Hahahaha. I might print this list out and hand it to a few people I know. Some family members of my friend just sold their beautiful ranch in Oregon because of a youtube video they saw about nuclear fallout from Japan riding the winds over and killing human beings. If I used your list as a check-list, just about every box would be ticked.

    Great having you back! I have been a shitty blogger lately. A post here, a post there. Not really setting aside time to read or write. Maybe I’ll get my act together now.

    Also, I really love the new look.

    Reply
  2. elkement

    Thanks a lot – this is a very concise and useful summary!
    (I would have liked to like this post, but the Like button is greyed out… seems there are some issues on WP in general today, related to popups, invalid certificates etc…)

    I get such questions, too. Most recently I did a cross-check of some therapeutic devices utilizing ‘quantum effects’. Mattresses with weaved in wires, AKA known as magnetic field therapy 🙂
    But the most baffling thing was: Those guys really misused references of scientific papers. Their ‘leader’ / ‘founder’ was a person with a quite common name and he was ungoogleable (probably a fake). As his initial+surname was so common they attributed all kinds of papers to him at the website – in diverse fields. Connecting the dots (alleged / faked CV…) the guy was a child prodigy – having submitted papers when he was only a few years old 🙂

    It’s a funny coincidence: Having started to blog about quantum theory I consider to reblog an older post of mine on fringe science or write a new version. Unfortunately pop-sci QM attracts people interested in ‘quantum healing’, ‘quantum consciousness’ etc.- I want to add a disclaimer.

    Reply
    1. David Yerle Post author

      Well, of all the people who need my blog, you’re probably the one who needs this the least!
      Funny what you say: they now give references and make it sound as if they’ve actually written papers. Maybe I should have included something like “if it starts with quantum and ends with healing, therapy or anything related to your body or mind, dismiss.”
      And yes, writing about quantum mechanics attracts that kind of crowd sometimes. It’s the same kind of crowd that loves it when you publish a blog article attacking science but then goes on to justify their absurd beliefs with scientific-sounding words. I can still remember the backlash when I wrote against science skepticism… that was truly something to behold.

      Reply
  3. Marco Parigi

    Given these guidelines which are entirely sensible, I am interested in your view on the article in the following piece. journalofcosmology.com/JOC22/milton_diatom.pdf

    It would appear 4 & 6 rules of thumb to be violated at the least. I am not prepared to dismiss it out of hand – I am interested to know the evidentiary or scientific grounds to reject it rather than the sensible rules of thumb. The main grounds appear to be Ockham’s razor.

    Reply
    1. David Yerle Post author

      Funny you mention this one. I was actually thinking about this article in particular when I mentioned that some scientific magazines will publish almost anything. I also didn’t know what to make of this until I saw an article by a very reliable Spanish science popularization blog (http://francisthemulenews.wordpress.com/2013/09/20/otra-vez-el-dr-wickramasinghe-encuentra-vida-extraterrestre-en-la-tierra/) which talked about it. Apparently, the main author is Nalin Chandra Wickramasinghe, who seems to find proof of panspermia every year. Also, apparently the “Journal of Cosmology” where he publishes his research is just a webpage that he himself controls and not an actual, serious journal.
      So I would give this source no credibility whatsoever, though I must admit this one had me fooled until I dug a little deeper. Some of these people can be extremely crafty.

      Reply
      1. Marco

        Yes – I had read that Slate article and the one that particularly referred to the Milton Diatom discovery. http://www.slate.com/blogs/bad_astronomy/2013/09/20/et_claims_of_alien_life_in_earth_s_atmosphere_are_unfounded.html
        Using the same guidelines on whether this bad astronomy blog makes dubious claims, I have come up with my own reasons to doubt Phil Plait’s claims (That turbulence, wind, etc. has kept the diatom in the stratosphere)
        Phil Plait’s rejection of the alternative assertion basically hinges on Ockham’s razor rather than observations that demonstrate how it got into the stratosphere from Earth. The fact that Phil spends so much of the article attacking ad hominem rather than sticking to the scientific reasons increases my suspicions. I clicked through to most of the other references against Wickramasinghe’s studies including the one you mentioned, and the same reliance on Ockham’s razor is the main counter (ie – if you find any life forms on what you think is a meteorite, it either isn’t a meteorite, or it is obviously contamination) At any rate, I am not convinced just based on the authority of a consensus of scientists – I would like to be convinced by the science, and for the moment it does not. I am not convinced that living things are not common in things like comets.

        Reply
        1. David Yerle Post author

          Me neither! I was actually convinced of the Panspermia hypothesis by an article I wrote about some months ago. However, I do think that the criticisms that the author did not consult with specialists in meteors or, well, life, are sound. Also, I tend to distrust someone who will only get published in his own magazine. Not allowing independent authors to check your results is a bad, bad sign.
          Summarizing: I have something against this author and this particular article. I don’t have anything against Panspermia. I do believe, though, that if life came from space it most certainly did not resemble diatoms.

          Reply
          1. Marco

            I think that there is an element of peer pressure, as in peer review, that other journals reject his articles. It’s like there is a completely separate peer group, and his is smaller. I feel I can find out everything that is pertinent about meteors from reliable web sources, rather than from an expert who may or may not have a personal agenda. Not sure why you think diatoms could NOT be space-farers – They have a protective silica outer, can survive from a dormant frozen state, can thrive in a large variety of watery environments.
            In all, the criticisms from “experts” doesn’t seem to gel with all the freely available and reliable information on the web. “Independent” authors in this case are from an opposing majority peer group. I have read all the opposing views I can find. What does it matter that they don’t officially review them?

          2. David Yerle Post author

            Hi Marco,
            Sorry for the late reply. I understand what you mean, but I do believe peer review is our only way of being reasonably certain that certain research holds water. You could call his research “peer-reviewed” by his small peer group, but the odds of a certain peer group being biased in some way are much larger than the odds of the whole scientific community being so, simply because of the law of large numbers. There is no bias in the scientific community against panspermia, not that I’m aware of. Therefore, there should be no pressure to not publish his research because of its topic or claims: the only reason his research is not being published has to be, then, that his data is insufficient and ill-analysed. The chances of there being a widespread bias against him versus the chances of him being a lousy scientist are overwhelmingly against him. It is possible, however, that this is what’s happening. I just think it’s highly unlikely: that’s what peer review is for. To minimise the chances of sub-par research being published because of certain interests or biases. That said, if you found it convincing (and you obviously spent a great deal more time than I looking at this) I wouldn’t discourage you from taking it seriously. What he did may be bad science, but it’s hardly pseudo-science.
            On to the other point. I don’t think diatoms are a good candidate for panspermia for two reasons: first, they have too many DNA bases. An analysis of the number of DNA bases shows a rough exponential increase: taking that regression towards the past shows that, if life came from space, it should have had a lot less bases than diatoms, which are already highly developed. Second, if life did come from space, shouldn’t it be equipped to not only survive in space, but to thrive in it? I admit, though, that the second point is highly debatable and corresponds to a personal preference with no other basis.

          3. Marco

            Especially after having had a look at the responses to this kind of research over a number of years, I do detect a clear but pernicious bias against panspermia. Usually it is masquerading as Ockham’s razor. The frustrating thing is that there is constant denial of bias in the same articles that I perceive the bias. For a random scientist reading an article, the bias is perceived to be in the smaller peer group. What it really hinges on is whether panspermia is extremely rare in space, or whether it is constantly raining down from space. If it is extremely rare, all these “discoveries” of Wickramasinghe’s are obviously false positives, while if the opposite is true, NASA scientists’ lack of evidence of panspermia (in meteorites or the stratosphere) is obviously a false negative. Circumstancial evidence has to be leaned on a little bit in the same way as we try to approximate the prevalence of a disease in society before we trust a screener to be a reliable probability value of having a disease. Also, our calculations have to take into account latest evidence. Scientists assume comets to have frozen cores, when the Stardust mission found minerals that only form in liquid water. Another false positive? or an admission that there could be millions of comets in space with the perfect conditions for a diatom to thrive?

          4. David Yerle Post author

            Hi Marco, again sorry for the late reply. It’s been a pretty horrible start of the year work-wise.
            Anyway, now I see what you mean. I have to admit I had never seen it this way: I always thought about panspermia as a one-off (or several-off) event in the past, not as something that could be constantly happening as we speak. As always, you have given me food for thought and I will have to think about this for quite a while longer and possibly look at more articles until I make up my mind one way or another.
            You really have a knack for making me change my mind or, at least, for sowing doubt into it…

  4. Steve Armstrong

    These are rules to live by, but speaking of John Zande’s aliens, I find the presentation to the National Press Club below, a bit unsettling. It violates guidelines 5, 6, and 7, but these appear to be responsible individuals who are not knowingly telling falsehoods. The witnesses start around the 14 minute mark.

    http://www.youtube.com/watch?v=7vyVe-6YdUk

    I find this one a tough call, and I want to make a reasonable judgement on the subject. In other words, neither fear of ridicule, nor wishful thinking should cloud the issue.

    Reply
    1. David Yerle Post author

      Started looking at it, but it’s 2 hours long! I don’t think I’ll get this kind of time in a while… that said, I have problems with UFOs simply because I believe that a civilization that gets technology powerful enough to travel through stars will have modified themselves so much they will barely be recognizable as living beings. In fact, we could be surrounded by aliens right now and never know. I will give this a chance when I get a quiet moment…

      Reply
  5. Matthew Rave

    Nice post. I’m intrigued by your dark matter comment. I am a theoretical physicist, but not an astrophysicist, so it’s not surprising I haven’t kept up with the latest discoveries in the great beyond. (So much physics these days is ultra-specialized, and I suspect Betelgeuse could nova and I wouldn’t find out till they talked about it on the Daily Show.) Anyway, do you have any links to technical articles about possible dark matter candidates? Thanks!

    Reply
    1. David Yerle Post author

      It’s been here and there. I’ve mainly heard about it through Lubos Motl’s blog (yes, he’s quite a quirky character, but his stories on physics are interesting) who has published quite a lot of articles about this. Here are a couple:
      http://motls.blogspot.de/2013/04/dark-matter-direct-search-cdms-suggests.html
      http://motls.blogspot.de/2011/09/cresst-ii-joins-dark-matters-war-on.html
      He goes as far as calling it the “Dark matter war!” Apparently, there are now several experiments trying to detect dark matter and it seems like some of them have detected signs of it (CDMS, I believe) while others have results that rule that out. So, for now, all we can do is wait for more data while they jump at each others’ throats.

      Reply

Leave a Reply