Bad Science

Let me start with a confession: I like science. There's little so elegant as a scientific theory, well-tested, refined, especially while it's undergoing the refinement. There's beauty in the ability of good science to push against expectations and recognize when those expectations are flawed, and then to re-evaluate the expectations and data to try to come up with a better explanation and understanding.

When scientists won't push against their expectations or recognize when their experiments put those expectations in doubt, then they're no longer doing science, and they might as well go play with the ID idiots.

So it strikes me as important when the big news splash of the day is that a research project reveals that in the randomized population of women between 50 and 79 studied, a low fat diet didn't reduce breast cancer, colorectal cancer, or cardiovascular disease risks. Here's the full text article for the Breast Cancer Study, free, from JAMA. And here's the colorectal cancer study abstract. And here's the abstract for the cardiovascular disease study.

This is big news, and you'd think honest scientists everywhere would jump up and start rethinking their assumptions.

Except, not. Because it seems that their assumptions control their readings of the studies. ABC news published a bunch of little critical blurbs from experts here.

The main critiques:

It was a flawed study, and didn't pass scientific review. Then why the hell did JAMA publish the results? And even more to the point, why did MY tax dollars go to support it when they could have supported something worthwhile, like, you know Headstart, or the Arts. This study could have supported a LOT of humanities work.

It followed women between 50 and 79, and thus wouldn't show results which might show if they started with 20 year olds, or ten year olds, and followed them for 50 years. No doubt, it's important to do long term studies. But here's a cluepon: 10 year olds don't worry about breast cancer or heart disease, much less colorectal cancer. Most people who care about these diseases are grown up, middle-aged. So the research tells us something about what dietary changes MIGHT or MIGHT NOT do. Here, it turns out, going low fat doesn't change outcomes for these three areas of interest among people who might think that would help improve their outcomes.

The study wasn't long enough. Again, let's be realistic. Say you take your average Bardiac, not quite 50, but heading that direction, and I say, "hey, what can I do to reduce my risks or these bad health problems?" If my risk isn't reduced in 8 years, who gives a flying f***? I'm not going to live forever, and if changing something doesn't actually reduce the risk while I'm still young enough to want to avoid dying, then it's not worth doing. (The death I dread is the long wait of being largely incapacitated by age or Alzheimers and unable to escape a miserable existence. At that point, if I get one of these horrid diseases, dope me up and let me die. Please.)

This complaint sounds like the kind of BS that tells women what our risk of getting breast cancer by the time we're 85 is. Except that since most of us won't live to 85 anyway, it's a misleading statistical number. (CDC page on life expectancy from 2002.)

The women didn't reduce their fat intake to the 20% of calories the study had as its goal; instead, they averated 24% of their calories as fats, while the women not on the low fat diet consumed on average 35% of their calories as fats. NewsSource.
The basic question is to ask if the difference in fat consumption is statistically significant given the overall numbers, and I'm guessing JAMA would have tossed the study fast if that weren't statistically significant.

But, I think this result is VERY interesting, because while the averages I quoted are for the study period as a whole, each group increased its fat consumption so that by year six, the low fat subjects were consuming 29% of their calories through fats, and the control group was consuming 37% of their calories as fats. (Same source) Why does this interest me? Because these women were participating willingly in a study, and all those early studies on work production show that when workers THINK they're being studied, they produce more goods. So let me assume that these women were probably pretty darned motivated.

And even motivated women weren't able to reduce average consumption of fats to the study's target level, and instead, increased their consumption over time.

Doesn't that suggest that a low fat diet is DIFFICULT for even really motivated people? If we want to think in terms of overall population habits, then finding a way of eating that's healthy AND NOT difficult would be a good goal. And low fat isn't that.

Now, naturally, were I an honest scientist, at this point I'd have to raise my hand and acknowledge that I have a stake in the results. Here's my stake: I find low fat food substitutes pretty damned boring and distasteful. I don't know that I own any stocks (via my retirement mutual funds) that promote high fat OR low fat foods.

***

Since the alternative was to grade quizzes or short papers, I decided to look around the blogosphere to see what the medical types are saying. The few responses I've found have been interesting, and not terribly sympathetic with the experts cited by the news sources.

Retired Doc says that he thinks the problem with the study is that it was already outmoded by the time it was done because,

The current party line is reduce saturated fat and trans fat to reduce heart disease risk and monounsaturated fat may actually be good for you. (emphasis mine)

He may be right, but look at the way he says it. Science isn't supposed to have a "party line." It's supposed to engage in open, honest, self-reflective, self-critical study of it's assumptions. Do the responses of the experts reflect the party line rather than real scientific inquiry?

(To be fair, retired doc says he hasn't pushed fat reduction diets.)

Sydney of MedPundit goes with the argument that the age factor makes the study problematic, and says,

its chief limitation is that it was performed in women ages 50-79. Here's a news flash for everyone - the number one cause of heart disease is aging. Changing your diet after you've already completed most of the aging process is not going to improve your chances of avoiding heart disease. It isn't even clear that following a special diet in your youth will decrease your risk of heart disease in old age (although it might decrease your chances of having heart disease at a young age.)

Things that increase the risk for heart disease the most also happen to be things that increase the rate of aging of the arteries , diabetes and smoking being chief among them. Diet (and cholesterol) make miniscule contributions.

So I get a sense that while Sidney sees flaws in the study, she says that diet isn't a big factor anyway. In other words, she's skeptical of the experts' responses, too.

Like Sidney, DB of DB's Medical Rants has some doubts about the experts' responses. What does he suggest, then, for those who want to avoid problems? He asks, and then answers

So what is a health conscious person to do? Make your best guess.

Now that's helpful, eh? Very scientific.

I guess I'll turn on some John Lee Hooker and enjoy a dinner of something Dean Ornish would run from like a scared rabbit.

(Yes, weight and health problems are bad. I recognize this. People in the US have unhealthy practices. We also don't read nearly enough Shakespeare or Chaucer, or poetry in general. I fight the good fight on my end pretty much every day.)