Reliable Research - Island of Sanity

Island of Sanity


Reliable Research

I'm writing this during the covid epidemic. I've got articles on this site from decades ago so maybe you're reading this after covid is long forgotten. But that's another subject.

As I write this, there's an ongoing debate about vaccines. Some believe that they should be mandatory. Others believe they were rushed into production and are dangerous. Still others distrust all vaccines. People on all sides claim to have research to back up their position.

Something that I find fascinating about this debate is that the pro-vaccine folks routinely ridicule the anti-vaccine folks as scientifically illiterate and generally stupid. They'll say things like, "All your 'research' comes from conspiracy sites on the Internet. I go by real science."

I recently had an amusing conversation where I pointed out to one such person that he, too, was getting all his information from the Internet. So what was the difference between him and the anti-vaxxers? Well, he explained, the difference was that he was relying on reputable sources, like CNN and the CDC. But, I pointed out, both these groups have been found to have made numerous mis-statements over the course of the epidemic, in some cases statements that they knew were false at the time they made them. He didn't really respond to that, but went on to say that he was relying on "articles from peer-reviewed scientific journals". Oh, I said, that's a good point. I then asked him to give me citations to a copule of these peer-reviewed articles. He never replied.

I'll hazard the guess here that he has never actually read an article from a peer-reviewed scientific journal. He couldn't even give the name or author of such an article. No, he relied on some web site he trusts making a vague statement that "science has proven that ...", and he just blindly accepts that as 100% true.

For the record, I am not an anti-vaxxer. I got the covid vaccine as soon as I could get it. But I am anti-pseudoscience. And a lot of what is called "science" today is pseudo-science and anti-science. Science used to mean "knowledge gained by experiment and observation". But today it has apparently been redefined to mean "something said in a press release by an organization that has the word 'science' in its name."

When someone says, "science has proven that ...", what they usually mean is "I heard a story in the news that says that ..." When someone says, "All scientists agree that ...", this is almost always false. 100% of scientists rarely agree on anything. It is rarely a question of scientists say X and nutcase fanatics say Y, but usually more like some scientists say "mostly X" and others say "sort of Y", and some extremists fanatics say "always X" and others say "exactly Y".

And just by the way, people who make these sort of statements often have a touchingly profound faith in peer-reviewed journals.

What is "peer review"? It simply means that before a scientific article is published, other scientists in the field read it and critique it. Sometimes their critiques are minor, like "in the table you say that 62% of test subjects responded this way but in the text you say it's 64%". Other times they're more serious, like "you haven't considered the effects of temperature on this process, and Dr Jones recently published a paper in which he showed that temperature can alter the rate of the reaction by over 80%". Sometimes the objections can be addressed with minor editing. If the objections are serious enough, the journal may insist the author make major revisions or do further research. They may refuse to publish the article at all.

Is peer review a good idea? Sure. It's almost always good to have a competent person check your work. Does peer review guarantee that everything said in a scientific paper is 100% true? Of course not. The reviewers may make the same mistakes the original author did. They may be sloppy in their review. Etc.

Indeed, one could argue that in some cases peer review could prevent ground-breaking new research from being published. If everyone in the field just knows that X is true, and a maverick comes along and says that X is false, maybe reviewers would be reluctant to approve his paper for fear that they would look foolish to their co-workers and friends. Note that I'm speaking in general here, not necessarily anything about vaccines in particular. And I'm not accusing anyone of participating in some vast conspiracy to hide the truth. Just saying that, when something is common knowledge, it can be difficult for people to go against that. Oh, and when I said "look foolish to their co-workers and friends", perhaps I should have said "co-workers, friends, and people who approve research grants".

© 2021 by Jay Johansen


No comments yet.

Add Comment