Saturday, October 16, 2010

Medical Wrongness in The Atlantic (and a few notes)

I have a big feature article in the new (November) issue of The Atlantic, which just came out. It's the magazine's annual look at "Brave Thinkers," and it leads off with my longish profile of Dr. John Ioannidis, who has spent most of his career studying and exposing the many problems with published medical research. My book Wrong opens with Ioannidis, and comes back to him a few times. But in the article I go much deeper into his work, his background, his personality and his thinking--I spent several days with him in Greece for the article--and say more about what his discoveries mean for medicine and doctors' ability to treat us effectively. It's not a pretty picture. Think of the article as an extremely long, important and (I'd like to think) colorful post for this blog.

By the way, sorry for the long gap since the last post, I got caught in an extended perfect storm of article deadlines, and travel for research and speaking engagements. (And it's not over yet.)

Meanwhile, on another semi-personal note that affects this blog, I've been transitioning to some new projects. I'll say more about these projects in the coming months, but for now let me just say I have a new special interest in obesity and weight loss, which I (and many others) think have become the biggest single health threat to the length and quality of our lives that is also potentially fixable. The blog's basic theme isn't really changing, but a lot of the posts will be about issues relating to obesity and weight loss. Note that I've parenthetically added the new special interest in obesity to the blog's title. I hope it goes without saying I'd love to get feedback from MSOMed readers on this partial change in focus.

Tuesday, August 24, 2010

Diet Secret Revealed by Scientific Study! (Gulp)

A new study getting some very prominent and excited press, including a big feature spot on the Today show this morning, has found that drinking two glasses of water before a meal is the secret to losing weight. The Today show and the articles have noted that the study was a randomized controlled trial, which as we all know is the gold standard of medical research. So get ready to guzzle the H2O, and finally lose those pounds!

Or maybe not. Diet studies are terrible, and for a number of reasons. They're of course subject to all the same problems that all medical studies suffer from: poor design, bad measurement, researcher and subject bias, patient selection problems, poor analysis, and more. Randomized controlled trials--in which, in the simplest case, subjects are randomly divided into two groups, one of which gets a special treatment and the other doesn't--can help with some of these problems, because in theory whatever mistakes are made or biases take hold in the study with one group, the same should apply to the other group, so any differences in the results with the two groups should be the real deal. Well, it doesn't always work out that way, even when an "RCT" is done right. But in this case, as with most diet studies that claim to be RCTs, it wasn't done even close to right. The really big advantage to RCTs, as I wrote in my last post, comes from their being "blind," meaning that both researchers and subjects ("double blind" is the technical term) don't know which subjects are in which group. That's what really has at least a shot at eliminating biases. But you can't blind diet studies, because, for example, you can't fool people into thinking they've just drunk two glasses of water when they haven't, or vice-versa. And that means that the researchers and subjects are free to be very biased toward the treatment under study. They all want the research to pan out, and they tend to find ways to make sure it does. It's also questionable whether non-blind randomized studies are truly randomized. The problem here is that once people in the study are subjected to the different treatments, you can start getting very different drop-out rates in the two groups, sometimes right up front. If you agreed to be in a study of some great new diet technique, and then were immediately told you're going to be put on a plain old diet without getting to even try the technique, wouldn't you be more tempted to just blow it off? After all, you've been on diets, they didn't work--why bother trying again with the same old thing? The researchers are probably hoping you'll fail on it so that their technique will prove effective, and they may communicate discouragement. It's a mess of a study set-up.

It gets worse. Diet studies rarely go on long enough to determine whether people actually keep the weight off, which as we all know is the real question. People tend to lose weight on almost any kind of diet, typically over a period of a few months--but then they gain it all back, and often more, over the next year or so. This water-drinking study as originally published some months ago went on for 12 weeks, a classic set-up for misleadingly concluding that the diet was effective. But here's where things get interesting. Though the original study only lasted 12 weeks, the newly released findings have the researchers claiming in public statements that the water-drinkers in the study on average kept the weight off for a year, and even lost a bit more. This is an extraordinary claim that sets off all sorts of alarm bells for me. Careful diet studies almost never report that sort of longer-term weight-loss success. And while the water-drinkers reportedly lost and kept off an average of 17 pounds after a year, the non-water-drinkers in the study lost and kept off an average of nine pounds after a year. Frankly, I'm more impressed by the non-water-drinking results--it's even more unusual for the non-gimmicked dieters in a study to keep weight off that long. In looking at the original 12-week study, I didn't see any specifics about what, if anything, the researchers did (or reasonably could have done, for that matter) to ensure that the subjects in the groups dieted in a way resembling the way people typically diet, that they didn't receive some special encouragement or coaching, that the two groups were treated identically by researchers, that the groups accurately reported what they were eating and drinking, that the water-drinking group really drank the right amount of water before eating and that the non-water-drinking group didn't drink a lot before or during meals. It also wasn't clear what special pains, if any, were taken to make sure that bias wasn't introduced by drop-outs, or by people who didn't comply with the instructions or accurately report data. (It has been shown again and again that inaccurate self-reporting, which is a problem in many health studies, is particularly egregious in diet studies.) Whatever problems crept into the 12-week study, I can't imagine it was anywhere but downhill in extending it to a year. The researchers certainly sound in the original report like they were trying to be careful, but there's really only so much researchers can do, and the result in most diet studies is that there are potential holes big enough to drive a car through. And that certainly seems like the case with this study. I can't help noting that the authors are releasing these newer findings before they have been peer-reviewed. Now, I've written at some length elsewhere that the peer review process isn't nearly the guarantee of quality that it is often taken to be by the public and by journalists. But non-peer-reviewed data is generally considered one step up from fairy tales in the research community, and I wonder if the claims of this study might change after peer review.

But you don't have to accept any of my reasoning about why this, or any other diet study, is likely to be fairly useless in telling you what's going to help you or anyone else lose weight. Here's a better reason to reject these studies: Scientific studies have found that anything is the secret to losing weight, including low carbs, high carbs, low fat, high fat, big breakfasts, small breakfasts, lots of veggies, lots of fruits, lots of nuts, snacking, not snacking, and on and on--and scientific studies have also found that every one of these gimmicks doesn't work, including drinking a lot of water. There's only one reasonable explanation for these conflicting findings: the scientific studies just aren't very reliable. They fail to shine any light on the problem. We don't have to agree on why that's the case, but it's hard to argue it isn't the case.

It's easy to explain why the water-drinking trick doesn't work in the real world. It's a type of diet I call a "sickening" diet. These diet gimmicks work by making you physically uncomfortable in some way, with the result that your appetite is dulled, as usually happens when you don't feel great. Any diet that forces you to eat or drink a large amount of some type of not-especially-appealing food or beverage will have this effect. Some people don't feel well even when eating low carb or low fat, which helps to explain why these diets may help many lose weight initially. The way the water throws your appetite off is obvious: it fills your stomach, and then when you add food on top of it you get an uncomfortably full sensation. None of these sickening diets are especially effective in the real world for what should be an obvious reason: people get sick of sickening themselves. They don't like it. They don't want to cram in another grapefruit, or celery stalk, or glass of olive oil, or chunk of meat. Go ahead, try the two glasses of water trick. Let me know how long you go before you find yourself staring at those two glasses of pre-dinner water and feeling, perhaps even with a touch of nausea, that the last thing you want to do is force them both down your gullet. Even worse, as the water or other sickening agent becomes more and more repulsive over time, your desire to go back to normal levels of eating (or normal varieties of food, if you've been restricting yourself in some way), will grow, especially as you lose weight, and you'll become hungrier and will experience intense cravings. No wonder people gain it all back and more.

Folks, people have been trying to lose weight by drinking a lot more water for at least four decades. I remember my mother forcing down eight glasses a day in the 1970s under the Stillman diet. She didn't keep the weight off, and neither did most people who have struggled with dieting and tried the water trick. If this silly gimmick worked, you'd have known it a long time ago.

Now having said that, let me say that drinking while dieting is a pretty good idea, as long as we're talking about modest, non-repulsive amounts of water or any low-calorie beverage. It's easy to mistake thirst for hunger, and it's certainly true that having at least a little something in your stomach can temporarily help take the edge your appetite. So sure, step up your water- or tea- or diet-coke-drinking, within reason. Just about every diet expert has been saying that forever, and it very well might help your diet efforts a bit. I'm not surprised the water drinkers in this new study lost and kept off more weight than the non-drinkers. I'm just highly skeptical of the claim that it makes a big difference, and especially of the claim that it can make the difference between your dieting successfully or not over the long term. The researchers (and especially the mass-media reports) seem to believe that it will, but I don't think that belief is anywhere close to being supported by the study. If you're tempted to believe it, then I sincerely wish you the best of luck in getting yourself to stuff down two full glasses of water before sitting down to every meal you take in for the rest of your life, and in restricting how much you eat at all of these meals because of it.

Sunday, August 15, 2010

Should You Bypass Obesity-Related Stomach Surgery?

The latest option for losing weight via surgically reducing stomach size: Having most of your stomach cut away and taken out by a procedure performed mostly through your mouth and down your throat. (A bit of the work is done through small slits in your abdomen.) What little is left of the stomach is then stitched up to form a mini-stomach, with the obvious result that it only takes a relatively small amount of food to fill it up. Other, more-established options for stomach surgery include the gastric bypass, a more extensive procedure in which in addition to the stomach being downsized it is also reconnected to a shorter span of intestine, cutting down on calorie absorption, and the lap band, in which much of your stomach is squeezed off by an adjustable band.

These procedures are often discussed as if they have been proven safe and effective in studies. But keep some points in mind when considering how "proven" a surgical technique is. There are (with a very few exceptions) no good, ethical ways to conduct randomized, blind trials of surgical procedures. It would be hard enough to do the randomization, in which patients are (to oversimplify a bit) randomly assigned to either a group that will receive the surgical procedure or a group that will not, because not many patients are willing to have surgical decisions made for them by the toss of a coin. But even if that issue were overcome, the real problem would be with blinding, which means patients don't know which group they're in. It's not easy to get people to not know whether they've had surgery or not--and even if you could pull it off it's even tougher to do it in a way that doesn't run afoul of ethical considerations. (It's been attempted, believe it or not, but hell was raised.) The result is that studies of newer surgical procedures are mostly observational studies, which means doctors just note how patients seem to be doing. (I'm not counting animal studies. Neither should you. No matter what researchers say, on average these studies don't translate well to humans.) Some procedures that have been widely performed for many years get "case control" (epidemiological) studies, which look back at groups of patients who have received the treatment and compare them to groups who have not. While you can't fully tell how trustworthy a study is just by noting what type of study it is--it's wise to consider all medical studies at least potentially somewhat untrustworthy--as a rough generalization randomized, blind trials tend to be more trustworthy than case-control studies, and observational studies are the bottom of the barrel, though to be sure there are lots of exceptions and qualifications to this pecking order.

In short, the fact that gastric bypass, and, to a lesser extent, lap band procedures, have been found in case-control studies and many observational studies to be fairly safe and effective should not be taken very seriously. These types of studies can be subject to intense biases on both the parts of doctors and patients--they generally want the procedures to succeed, and may distort their observations and do a poor job of recording all relevant data. Researchers also typically don't have access to all the data they need to fully analyze the situation. (For example, in the New England Journal of Medicine article I reference just above, the authors to their credit note that the study ignores the possibilities--likelihoods, I'd reckon, though the authors beg to differ--that people who had the surgery got much more or better medical attention afterward than other obese people get, and that they may have been healthier to begin with.) The procedures are a huge money-maker for surgeons, and the patients are often desperate to lose the weight and may be eager to convince themselves and others that they've done the right thing and that things are going to work out well. These biases, which can be and often are devastating to the reliability of medical studies, are exactly what randomized, blind studies attempt (if not always successfully) to eliminate. That's why many surgical procedures are widely accepted as safe and effective, only to eventually prove themselves not so safe and/or less effective than other, less invasive and less risky treatments--the list includes many types of heart-related operations, back-pain-related operations, brain surgeries, and on and on.

This newest stomach-reduction procedure has been performed on one person so far. And yet surgeons just about everywhere will soon be able to perform it on anyone they deem suitable, providing nothing obviously terrible happens immediately to this one patient as a result. True, these sorts of "laparoscopic" (performed through small incisions and/or through the mouth, using a tiny camera and light to allow the surgeon to see what's going on in there) procedures tend on average to be less risky than conventional, large-incision versions of the same surgery, but there can be reasons why they are sometimes less safe and/or less effective in some ways--for example, the restricted access and vision may result in a sloppier job of cutting or stitching, or in missing problems with bleeding.

And there are reasons to question whether the people who are getting these procedures really need them enough to justify the risks. (Though the twice-aforementioned NEJM study found that overall death rates went down for obese people who had gastric bypass surgery due to fewer deaths from heart disease, diabetes and cancer--findings that for a number of reasons I suspect may exaggerate the benefits of the procedure--around one percent died from the surgery itself shortly after the operation, and many had complications.) The people who sign up for these operations are typically obese people who have tried to lose the weight through diet and exercise but have failed, and whose health appears to be at some risk. But trying to compare the potential payoff from trying to diet and exercise to the likely benefits of surgery is a bit of a fixed contest. I've been studying the question of why people fail with diets and exercise, and it's clear that the success rates would likely be much, much higher if overweight people weren't getting such almost uniformly terrible advice about how to diet and exercise, and if their motivation to lose weight through dieting and exercise weren't on average so low. Why is their motivation often so low? For one thing, they keep hearing from many researchers and obesity experts that they'll probably fail with diet and exercise, and for another they've been led to believe that surgery provides a much easier and surer option that will provide all the same benefits of a successful diet and exercise program without any of the work--these procedures essentially are intensely marketed much the way cosmetic surgery is hawked, because they're similarly ridiculously profitable. (Another issue is whether the success rate for dieting and exercising is really anywhere near as low as we've been told--but that's a more complicated story.) Meanwhile, the success and benefits of surgical weight-loss keep getting pumped up through studies that are likely somewhat biased and otherwise flawed.

I don't doubt there are obese people who, when the whole picture is soberly assessed, might be making a reasonable decision to undergo one of the procedures. But I suspect it's a fraction of the number of people who are actually getting the surgeries. I was especially appalled to see a big and very well-researched and well-written cover story in the Atlantic earlier this year on the subject of obesity essentially advocate for, or at least seriously suggest the idea of, the government putting its anti-obesity money into handing out gastric bypass surgery for the obese, presumably instead of into research, education, good-nutrition programs, more-exercise initiatives and other anti-obesity public-health initiatives. The article, which for sure in many ways was terrific, more or less declared dieting and exercise to be a waste of time--a point of view which is pushed by many published medical studies relying on highly biased and flawed set-ups (as evidenced, for example, by the fact that findings from these studies widely and sometimes sharply conflict), but which flies in the face of the significant documented success rates that hundreds of real-world programs have achieved with obese people (and which has also been backed up by other published studies, for what it's worth).

I think this sort of give-up-on-dieting-and-exercise-and-hold-out-for-surgery(-or-a-pill) thinking is dangerous, and is in fact a contributing factor to the obesity epidemic. If you're thinking of getting some form of stomach-reduction surgery, I'd urge you to do some more research into the possible complications and into dieting and exercising options, and to discuss what you find with your doctor. And if your doctor is pushing you toward the surgery and especially if he seems biased toward it, try a second opinion from a doctor you have good reason to believe has an objective view about both diet-and-exercise programs and surgical alternatives. Many, many obese people have shed the weight and kept it off through dieting and exercise. Shouldn't you be absolutely sure you've given dieting and exercise your very best shot under the right program before you have one or even two of your major organs hacked up and extensively rejiggered?

Wednesday, July 14, 2010

Can Pregnant Moms Program Babies for Obesity?

A new study claims it shows that the children of pregnant mothers who put on lots of weight are much more likely to be obese years later than the children of mothers who don't put on as much weight in pregnancy. The announcement of the findings has triggered comments from other researchers and experts suggesting that this study is simply the latest in a series of studies supporting the notion that women who eat too much during pregnancy are placing their children at greater risk of obesity.

Nope. This new study tells us very little, and none of the others tells us much, either. The only sort of study that would have at least a small chance of giving us some real insight into the effect of eating during pregnancy on children's lifelong tendency toward obesity would be one that randomly assigned many, many pregnant women to either a group that was somehow made to put on a relatively large amount of weight during the pregnancy, or a group that was somehow made to put on a relatively small amount of weight. Then the study would have to follow the weight gain of the children in the two groups for a few decades. Such a study would still be plagued with potential confounders, since, for example, it wouldn't be blinded (the women and the researchers would likely know which group each woman was in), and the two groups of mothers might start behaving very differently from one another because of the particular way in which their weight had been controlled. But it doesn't matter, because no one is likely to do that study--I can't imagine researchers being able to line up enough pregnant women willing to participate, and even if they did that they'd be able to effectively control the two group's weight gains during pregnancy, or even if they could that the whole project would make it through ethics review boards. So all researchers can do is track the weight gain of a bunch of pregnant women, wait to see what happens to their children, and then go back and look for links. This approach almost always leads to utterly unreliable results, because we have no way of knowing what causes what in the resulting observed links, and indeed the links themselves are often merely artifacts of bad data and/or bad analysis.

All these pregnant-weight-gain studies really show at best is that mothers who tend to put on too much weight are more likely to have children who put on too much weight. Not only is that painfully unsurprising, but it's tremendously useless information. It doesn't tell us whether that's happening because of a genetic predisposition for weight loss that gets handed down from parent to child, or because mothers tend to pass healthy or unhealthy eating habits onto their children in some way, or because exposing very young children or possibly even fetuses to excessive intakes of food or to less-healthy foods somehow changes their metabolisms or alters their appetite mechanisms. (That first study's conclusions seem to vaguely imply the latter, for what that's worth, which I think is very little.) The studies give even fewer clues as to what ought to be done about the situation. If it's genetic, there's probably little point in addressing what pregnant mothers eat, which seems wrong. If the mother's eating is programming the child in the womb in some way, then what the mother eats during pregnancy becomes critical--and what and how the child eats after birth is not so important, a conclusion that defies everyday observation and common sense. If the mother's weight gain is "transferred" to the child via the food attitudes and habits that the child picks up from a mother who simply doesn't do a good job of controlling her weight, then genes or what happens during pregnancy is largely irrelevant. I think there's a pretty good chance that what we're seeing here is some combination of the three--genes, changes that take place during pregnancy, and (probably especially) family eating habits passed on to the child later on. But the studies don't help pin any of this down in the least.

So what should pregnant mothers take away from all this? Come on, you know the answer to that. They should eat sensibly (and as per doctor's advice), avoid either excessive or insufficient weight gain during pregnancy (unless the doctor orders otherwise), and most important (in my opinion and in the opinion of most of the many experts I've interviewed on this subject) work hard to transmit generally accepted good eating habits to their children, starting from birth and continuing on through all of childhood. (If your doctor disagrees with that last bit, get a new doctor.) I doubt we'll ever see a study that provides convincing evidence that doing anything else makes sense.

Wednesday, July 7, 2010

Is Long Life Mostly In the Genes?

Update: A problem with the study that I describe in this post has surfaced within the scientific community, and has even received mass-media attention. It's a technical problem of exactly the sort I warn about in the post. The only thing that's unusual here is that the problem has been called out prominently.

A new study has shown that living into your nineties and beyond is a matter of having the right genes, according to recent headlines. The technique that the researchers employed to identify the genes of long life is much the same one behind many of the discoveries of links between genes and traits or genes and disorders that we hear about frequently. To oversimplify a bit, the game involves looking at many genes in a large group of people who have some trait in common, and then trying to spot which particular genes turn up more frequently in these folks than in a large group people who don't have the trait.

I recommend not paying much attention to this study, or to most studies that purport to have identified key genes behind some important trait or disorder, and for a few reasons. First of all, this basic gene-linking technique, although an impressive one and getting better, is still beset with a variety of inherent limitations and potential flaws that often lead to simply misidentifying genes as being closely linked to a trait in most people when they are not, and misses many and possibly most of the genes that actually influence the trait. Part of the problem is that the technique is a highly statistical one based on probabilities involving all sorts of manipulations that can leave the picture convoluted and murky. In addition, the very notion that a relatively small number of genes is likely to play a huge role in determining a trait or disease risk is for the most part a mistaken one, or at best a gross oversimplification. Genes act not alone or even as a small band, but rather as a vast orchestra. Even more confusing, genes can be "turned off" so that they might as well not be there, or partly turned on so that they contribute only weakly to a trait or disease risk. What turns genes on and off? Each other, the environment, other molecules in our cells, and strips of DNA that aren't even parts of genes. It gets worse: Genes can appear in single form, or in multiple copies that increase their influence in ways most gene tests don't detect. They can pop out of their slot in the genome, replicate themselves, and then reinsert themselves somewhere else in the genome. They can flip around backwards, which also isn't detected by gene tests, even though a backwards gene can behave differently. Throw in the fact that genes can be changed by viruses, and that any protein made by a particular gene can do different things in different people, or different things in the same person at different times depending on what else is going on in the body, and you've got complexity of such staggering breadth and depth that scientists who are being honest about the situation admit they'll be lucky to make a small dent in the task of sorting it all out in their lifetimes.

Consider: A 2009 study of about 6,000 people came up with a technique for predicting the height of a person based on looking at the 54 genes most closely linked to height. The results turned out to be one-tenth as accurate as averaging the heights of both parents and adjusting for sex, a technique introduced in 1886 by statistician Sir Francis Galton. University College Dublin biomolecular researcher Helen Colhoun and colleagues have estimated that 95 percent of studies that find gene links are just plain wrong. Harvard genetics researcher Joel Hirschhorn and colleagues surveyed the medical literature on 166 published genetic links to illness that were each examined in at least two other published studies, and found that only six of the links--less than four percent--held up in all the studies. John Ioannidis of Tufts University Medical School has calculated the average odds of a gene-link study being right at one out of hundreds, or worse, depending on the disease. In the case of schizophrenia, for example, Ioannidis reckons a one out of 2,000 chance of a gene-link study being right. His estimates suggest you would have almost as good a chance of identifying some types of genuine gene links by throwing darts at a diagram of the relevant sections of the human genome as you would by reading research journals. Researchers claim their hit rate has improved quite a bit in the past few years, but we won't know the extent to which that's true for a while to come. Needless to say, all these substantial uncertainties and vagaries often get swept under the carpet in the reported findings of an exciting new study, and are almost always ignored in the mass-media reports.

None of this is to say that the study in question here didn't correctly identify many genes that may in fact in some cases be potentially important contributors to living a very long life. But even if the claim is largely or at least partly right, it's still a misleading one. When we hear it, we assume that we'll need many or most of these genes to have a good shot at a long, healthy life--and that's an absolutely unwarranted conclusion. People who live extraordinarily long lives need everything to go in their favor, including many things that may be largely irrelevant to those of us who hope to keep decent health "merely" into our 80s. Assuming we need the genes common in people who live to be 100 may be like assuming a good-performing car needs to have the components common to race cars.

Even if it does indeed help quite a bit to have the right genes in order to live a long and healthy life, it also helps to get a lot of other things right. What has come through from all the genetic studies more clearly than anything else is that for most of us genes really don't influence your destiny as much as, or at least any more than, your behavior does. Folks, if you want to live relatively long lives, and perhaps more importantly stay relatively healthy late into that life, there is powerful evidence that eating sensibly, exercising, breathing clean air, avoiding undue stress, and refraining from obviously bad habits such as smoking, heavy drinking and not wearing seatbelts, can for many and possibly most of us trump genes. In fact, for all we know, some of the genes that the very-long-lived had in common in this study were genes that influence healthy behaviors--in which case anyone engaging in the behaviors would get the same benefits, whether they had the genes or not.

Your fate is probably not genetically sealed--it is much more likely that your health is to a large extent under your control. Don't let announcements of marvelous gene-link breakthroughs lead you to abdicate that responsibility.