The Real Lesson of the Vaccines-Cause-Autism Debacle

The doctor who claimed to have proven a link between autism and routine childhood vaccinations has been stripped of his license to practice medicine, the latest in a long string of humiliations. It turned out that his now-discredited study was carried out while he was quietly being well-paid by lawyers who were suing vaccine makers, and was based on blood taken from kids at a birthday party who were each slipped eight bucks for their trouble.

Let's review: A corrupt charlatan tried to foist a sloppy and possibly fraudulent piece of research onto a vulnerable community--many parents of autistic children are haunted by the lack of good explanation for the disorder--but the persistent work of mainstream scientists laid bare the scheme. And that's what we expect from science, isn't it? There may be the very occasional peddlers of junk science who sneak into reputable journals, but science surely if slowly brings their egregious transgressions to light, and expunges them and their work like so much gristle.

But here's an alternate view to consider: Even prestigious medical journals routinely run dubious studies that are tainted by bias, corruption or incompetence, and are rarely called out on it--but because this particular study got so much attention the scientific community for once actually went through the trouble of exposing it.

That probably seems far-fetched, but it's much closer to the truth. The evidence supporting this point of view is so plentiful and consistent that it's daunting to contemplate having to narrow it down to a mere handful of quick points so as not to make this an absurdly long post, but let me try, drawing on my research in Wrong (where these factoids are elaborated on and sourced): About two-thirds of the findings published in top medical journals turn out to be wrong or significantly exaggerated. Nearly all studies that find a particular type of food or a vitamin lowers the risk of disease fail to hold up. In spite of all the hype for gene discoveries that promise to deliver insights into our personal disease risks and customized treatment, studies have found that less than 10 percent of the published studies that identify gene links to disease hold up. Studies suggest that only about one percent of the scientific studies based on fraudulent data are identified and reported. On average roughly one-third of medical researchers surveyed in various studies admit to having committed or become personally aware of at least one act of research misconduct within the previous three years. Two-thirds of the drug-study findings that indicate a drug may cause harm are not fully reported by researchers. A third of the studies published in top medical journals contain statistical errors. And here, at least in this context, is the kicker: An estimated ninety-five percent of medical findings are never retested.

Ironically, the occasion of the rare public outing of one out of an ocean of bad studies actually serves to make medicine look good, because the story comes off as the community successfully rooting out the tiny percentage of bad work that sneaks in.  Rarely does anyone openly point out in public that bad studies are the rule rather than the exception, let alone actually identify many of the vast number of studies that are not trustworthy, let further alone actually correct the record with better research.

Yes, in this case the system worked, because the findings were so inflammatory in terms of the politics of the large autism community, and because they so blatantly contradicted what scientists were already sure they knew about the subject. But it's a real exception to how the system fares, not a general validation. And that's the insight we really ought to take away from this sorry episode.

Why this blog?

I should probably kick off this blog with some general comments about its motivation and goals. I've spent the last three years working on a book about why experts usually end up misleading us. The book, called WRONG, is just coming out now. (You can learn more about it at www.freedman.com.) Although WRONG looks at all sorts of experts and their highly flawed advice, I think the problem that has the most impact on the most people is the way that published medical research runs off the rails. We're fairly bombarded on a daily basis with the latest medical findings, typically ripped from the pages of prestigious medical journals, and echoed in our most respected mass media publications and shows, telling us which foods and habits will lower our disease risk, what new pills show promise, which genes predict our health fates, and so on. There's just one little problem: According to scientists themselves, most of these published findings will turn out to be wrong or highly exaggerated. Eating those foods probably won't help you much, the pills probably won't work well, the genes probably won't tell you much. And this turns out to be true not just of what you read in the newspapers or see on TV or on news websites, but also of the work published in our top medical journals. (Journalists tend to make things worse, but we don't cause the basic problem.) There are many reasons why most published findings turn out to be flawed, which I explore in detail in WRONG, and will be touching on as the need arises in this blog. But for now, let me just state that there is a wealth of consistent and mostly uncontested evidence from highly respected scientists that the published medical research you hear about is on average not trustworthy. (And if like many people you've been led to believe that large, randomized controlled trials can generally be trusted, you've been misled.)

You generally won't hear about problems with the credibility of medical research when you're alerted to new findings in the mass media, because almost no one has an incentive to focus on the problems or tell you about them. Why would most researchers take the trouble in their papers or in their interviews to highlight why their work is probably wrong? Why would most journalists take up space in their articles and time in their on-air reports to discuss why you probably shouldn't take the reports very seriously? You might think you could count on your doctor to figure out which published findings really ought to be put into practice, but you probably can't--most doctors don't have the training, time or inclination to deconstruct the studies they read about in journals in order to spot the potential problems with them.

I'm not a doctor, I'm not a scientist, I'm not an academic researcher. I'm a journalist. I've spent a lot of time speaking to, and reading the work of, a lot of highly regarded scientists who study the problems with medical research, and I think I now know a thing or two about these issues. I'd like to share the insights I've gained (and the insights of the many scientists I'll be interviewing for this blog), and help people apply them to the current news about medical findings, so that we can all make better guesses about which of these findings are really likely to improve our health and possibly save our lives, and which of them ought best be regarded with great skepticism. (Sadly, most of them will fall into the latter category.) You, of course, will use your judgment to decide how much you should trust me and this blog and the people I quote in it, as we all should with all the findings and advice and pronouncements that stream at us from so many sources.

I want to make it very clear that I'm not interested in turning anyone away from science and mainstream medicine--far from it. I'm as appalled as anyone by junk science, and I'm not an advocate of alternative medicine (though I respect its placebo effect, as do many mainstream clinicians). I trust my doctor, and believe many if not most scientists to be among society's heroes, if highly imperfect ones. I believe that science is exactly the right way to get at truth. But we all need to understand just how messy and biased and flawed a process science is, how slowly and rarely it actually manages to close in on the truth, and how it becomes even further distorted when it's communicated to us through journals and the media.