Science vs. scientism, keto vs. low-fat diets, and why you shouldn’t drink a teenager’s blood.
11 minute read
To paraphrase both Albert Einstein and Don Henley, the more data we collect, the less we seem to understand it.
Case in point: Do we really need the U.S. FDA to warn older people not to infuse themselves with youthful blood plasma in some vain attempt to slow the aging process? Really?
Apparently, yes. Yes, we do.
Let’s imagine the conversation between FDA Commissioner Scott Gottlieb and Head of Biologics Peter Marks ahead of issuing their public statement about a Silicon Valley startup doing just such a thing:
“You have to be fucking kidding me, Scott! I thought the sparkly vampire trend was over.”
“It’s not like this was the first time someone’s tried this, Peter. We can go back hundreds of years – thousands, even. Trying to cure aging with youthful ‘serum’ has a long history of quackery. Same quacks. Different day.”
“But seriously. $8,000 for one liter of plasma! And the blood comes from 16 to 25-year-old people. What parent in their right mind is giving consent for the 16 and 17-year-old children?”
“Not any parent I’d like to meet.”
“And I can’t imagine these kids are earning anywhere close to $8,000 for their efforts.”
Scott opens his web browser and clicks on the Ambrosia website.
“Hey, Peter. I’m looking at the website, it’s $12,000 for two liters. That’s 50 percent off the second IV bag! What a deal.”
“This isn’t funny, Scott.”
“I know, I know. But you have to admit, warning people about things like this is sort of like a sad game of Mad Libs. Swap out the word ‘plasma’ and we could just as well be talking about any number of quack treatments.”
“Ugh. Okay, let’s write the press release.”
Of course, I’m just imagining what that conversation might have sounded like, but I’ll bet I’m close. Here’s an excerpt from the actual statement:
“There is no proven clinical benefit of infusion of plasma from young donors to cure, mitigate, treat or prevent these conditions, and there are risks associated with the use of any plasma product.”
Go ahead and read the full statement if you like; it’s one of the most entertaining (and depressing) government publications you’ll ever read.
One more thing: Unfortunately, I am not fictionalizing the price. The FDA pushed Ambrosia out of business last week, so you won’t see much on their website if you visit them today … but that’s why we have the Wayback Machine. Sadly, $8,000 and $12,000 for one and two liters of youthful blood plasma were indeed the published rates.
In Ambrosia’s case, it seems that experiments in mice inspired the therapy, as well as the resulting company to commercialize that therapy. Make note of the word choice: Inspiration. Inspiration, in and of itself, is not the problem. The problem is that inspiration is not a clinical trial. An idea, no matter how compelling, is not a fact.
The real question we need to ask ourselves is: Why do we keep falling for things like this?
Our first instinct is to blame the easy targets – investors such as billionaire Peter Thiel – for enabling these startups with access to capital, media attention, and perceived legitimacy. But that’s a bit of a red herring. I can’t speak for Thiel’s motivations, but it’s not hard to understand the appeal: Investors are interested in the $8,000 per liter price tag. Even if there are only 10,000 people in the world who can afford a “treatment” regimen of, say, 12 infusions, that’s a billion-dollar market.
An investor is like a dog with a bone. Don’t blame the dog. We need to quit throwing him bones.
No, the issue is deeper than that. At the heart of the matter is our misunderstanding of data, evidence, and the scientific method. Or put simply: We confuse science with scientism.
“Do whatever it takes to avoid fooling yourself into thinking something is true that is not, or that something is not true that is.”
– Neil deGrasse Tyson
Before we tease apart the difference between science and scientism, we need a refresher on the scientific method. Don’t worry, we won’t go back to your 10th-grade biology class, and you won’t need to dissect a fetal pig. (#nightmares)
At its core, the scientific method is a thought process – a logical progression from idea to repeatable process – in five steps.
- Start with a question or an observation. For example, younger people seem healthier than older people. Why is that? Why do people become less healthy as they age? Do they need to? You’ll notice this is not a conclusion, but rather it is a good question. And yes, sometimes we know things work before the scientific method tells us why they work. We’ll get to that with Ketogenic diets in the next section. Stay tuned.
- Form a hypothesis. That’s just a fancy name for an educated guess. What makes it different from a guess is that it can be falsified with evidence. Astrology isn’t science because you cannot disprove that it wasn’t an “alignment of Jupiter” that healed your heartburn. To follow our example, researchers could notice that older mice respond to infusions of plasma from younger mice and that the same benefit might be measured in humans.
- Make a prediction. There’s an important reason you write this down – quantitatively and specifically – before you move on to step 4. Not only does this help you design a proper experiment, but it also helps to eliminate the human tendency to reframe your initial hypothesis to fit your future observations (aka hindsight bias). In our example of blood plasma infusions, we need to define precisely what effect we will measure. Is it better for cardiovascular performance? Some measure of blood chemistry? Reduction of observable wrinkles in the skin? What, exactly, does “better” mean?
- Test your prediction with controlled experiments. The gold standard in science is the double-blind, randomized controlled trial (DB-RCT). That means you have a test group and a control group. Participants are randomly assigned to the groups and not even the researchers know who is in which group (only a database number identifies them). This high standard isn’t always possible, and that’s often where we run into problems. Are there enough participants? Do they change their behavior because they are being observed? Are they truly following the protocol? Health, diet, and exercise studies in humans are rife with these sorts of issues. In our example, one group could get blood plasma from a “young” donor, another group from an “age-matched” donor, a third group could get a dyed saline solution, and a fourth group (a control), would get nothing. It’s expensive, time-consuming, and difficult – in other words, far more complex than I’m letting on.
- Analyze the data. Only statistical analysis can tell us if the results of a well-designed experiment are truly the result of the treatment and if those results are more than you would expect from random chance. This analysis still may not tell you if there was no other possible explanation for the findings if the effects will hold up over time, or for different races, cultures, genders, or lifestyles of people. I think you can see for yourself how complex a study of “youthful blood serum” might be.
In addition to those five basic steps, serious scientists carefully document their work so that others can attempt to replicate it. If they can’t, they may have simply gotten lucky. Non-corporate (and even some corporate) researchers submit their research findings for external peer review – a rigorous if imperfect, process of allowing others to critique their work.
Yes, this is how the scientific method is supposed to work. And no, it doesn’t always work that way. But it is one of the best ways we have to understand our world at a deeper level.
By now, it should be obvious why the FDA took the action it did. Ambrosia skipped over the grueling work in the middle of the scientific method, jumping from “idea” to “marketing” that idea.
I like to call this shortened process “scientism.”
I hope you’ve noticed I am not a scientist. I learned the scientific method in high school and college as many of you did.
I am an advertiser. I learned the scientism method because it is a more effective method of mass persuasion. I am a fan of the former, but I am an expert in recognizing the latter. Let me tell you something you already know: Compared with scientism, true science doesn’t stand a chance in the court of public opinion.
I think it’s about time that changed.
I’m going to teach you the secrets to distinguish between the two, immunizing yourself from the persuasive effects of advertising and marketing.
The Top 10 Differences Between Science and Scientism
- Science uses many data points and hundreds of subjects – the more, the better. Scientism uses a handful of data points and limited numbers of subjects – whatever proves to be the minimum necessary to persuade you.
- Science draws narrow and specific conclusions based on large volumes of data. Scientism draws broad and general conclusions based on limited data.
- Science examines trends in that data over long periods before it accepts a conclusion as “true.” Scientism concludes (and uses its findings) as quickly as possible.
- Science attempts to determine causation (treatment X caused effect Y). Scientism is quick to accept correlation alone (treatment X happened at the same time as effect Y, or X happened just before Y).
- Science draws tight lines of evidence between links in a logical chain. Scientism is comfortable with “reasonable-sounding” rhetorical leaps.
- Science always wants to use the most rigorous and controlled study designs. Scientism is comfortable drawing conclusions solely based on “observational” study designs.
- Science is comfortable with incomplete explanations and telling you what it doesn’t (yet) know. Scientism works best with tidy narratives.
- Data, as seen by science, is often messy and incomplete. In Scientism, data are either “smooth curves” or discarded altogether in favor of compelling personal anecdotes.
- The quality of science is based on its ability to predict future events. Scientism is always “right” because it focuses on retrospective examinations of past events.
- True science is often complicated and boring. Scientism gets people excited.
Perhaps the biggest difference is that scientism is so much easier, faster, and cheaper … and that it looks like science at first glance. But scientism is a façade, a mirage, and an illusion. Now that you know the difference, you can begin to spot scientism everywhere you look.
Second case in point: Ketogenic diets.
There’s one problem with the scientific method – a flaw that practitioners of scientism use to their advantage: Knowledge doesn’t always flow from idea to repeatable process. In fact, most of the time, we notice something works before we realize why it works. Humans are a practical species. If it works, we don’t wait for a scientist to tell us why it works. We use it now.
A hundred years ago, physicians began to notice that Ketogenic diets seemed to help patients suffering from brain seizures. (To oversimplify, a Ketogenic diet dramatically reduces carbohydrates in favor of fats and proteins, forcing the body into the biochemical process of ketosis, where it burns fats instead of sugars for energy.)
Clinicians didn’t have a good idea why they worked (although they had some educated guesses), but in the face of uncontrollable suffering, anything that seemed to demonstrate relief became something to try.
More recently, dietitians and clinicians frustrated with the lack of success of so-called “low fat” diets began to experiment with other alternatives. They knew about Ketogenic diets, began to try them with patients, and saw positive results.
Those successes formed the inspiration to start the process of the scientific method.
What have researchers found so far?
In the early days of research, many studies focused on small groups of people, for short periods, with poor experimental controls. (Humans are notoriously fussy research subjects.) But over time, a wide array of studies confirmed statistically-significant weight loss benefits as a result of Ketogenic diets.
You can read consumer-friendly summaries of this research at the National Institutes of Health, and the Harvard Medical School.
Here’s the issue: These studies also find that other diets also can be effective – for example, vegan diets, Mediterranean diets, and simple calorie restrictions. They also are careful to point out that we do not know how these diets compare over long periods (most of the studies track patients for about one year), how they compare with people from different races or cultures, or how they compare with individual genetic/gut-biome profiles. Other well-designed studies point to a “Goldilocks zone” of appropriate carbohydrate, fat, and protein intake – too much of any nutrient in proportion to another seems to be linked with early mortality. Also, while seizure benefits of Ketogenic diets have been shown in mice, human trials are as yet inconclusive. (Mice are less difficult research subjects. They do what they’re told and eat what you feed them. Humans don’t.)
It is in this uncertainty that scientism flourishes.
A related case in point: The media coverage of Ketogenic diets.
The University of California San Francisco’s article “explaining” what we know and don’t know about Ketogenic diets is an example of what you might find as you try to research this diet for yourself. There are no links to actual research. The entire article spans a mere 697 words, about 10% of those words devoted to this prime example of scientism:
“For instance, Weiss himself has been on a low-carb high-fat (though not strictly ketogenic) diet for more than six months and claims he does feel much better. But he’s clear about what he knows and what he doesn’t. He’s lost weight and his borderline pre-diabetes is gone.”
“I think I feel great,” he said. But that might be because he’s eating less processed food, sleeping better, or enjoying compliments on his new physique.”
I don’t blame the scientists for the sloppy article; the UCSF news center is simply giving the public what it wants. Most publication editors simply don’t allow for complexity, because nuanced arguments don’t get clicks. But think about it for a moment: This is an article from a university written by actual scientists – what is the average person supposed to conclude?
I also know several people who claim success with Ketogenic diets. Their successes are inspirational and deeply moving. But personal anecdotes are not science. Their successes may not translate to my success or your success. Worse, following their experience could be dangerous depending on your specific health situation.
I also have heard the story about how dairy farmers pissed off President Johnson in the 1960s by refusing to back him. He politically retaliated by encouraging a spurious link between eggs and cholesterol – as well as the first so-called “food pyramid” (with his grain farmer supporter products at the base). This episode helps explain the “high-carb, low-fat” diet trend that captivated public opinion for 50 years. But conspiracy and political revenge don’t prove Ketogenic diets work. They simply mean Johnson was a liar and an asshole.
If there is that much confusion coming from universities, your friends and neighbors, and even the Surgeon General of the United States, what do you think an advertiser is going to tell you?
You already know the answer.
I’ll bet you didn’t need a formal market research study to see the rapid growth in Ketogenic products. A new product (or an existing relabeled product) seems to reach the market almost every day, many promoted by so-called “influencers” who stand to benefit when you add the product to your shopping cart.
It was the same pattern in product marketing we saw with low-fat, Mediterranean, Atkins, and dozens of other diets.
They’re not interested in your health. They’re interested in cashing in.
To this point, we’ve focused on the culture of scientism rampant in health and diet advice. However, I suspect you’ve been screaming into your keyboard about all of the other places you see the same thing:
- The positive (or negative) impacts of corporate mergers and acquisitions
- Financial and stock market advice
- Student graduation rates and education policy
- The causes (and “solutions”) to poverty
- Discussions about “Big Data” (home to scientism’s cousin “data-ism”)
- Similar discussions about “Artificial Intelligence,” “Blockchain,” and the “Internet of Things”
- New cryptocurrencies and ICOs (Initial Coin Offerings)
- Just about anything your politicians are debating this week
- Most social media
Anyplace you find complexity, you will find scientism.
I am not an expert in drinking blood, Ketogenic diets, or all the other places complexity pops up. In all those things, I’m just as foolish as everyone else. But I am an expert in persuasion, which makes me an expert in scientism.
That’s the one area I’m qualified to give advice. Here goes:
- When you read a popular article talking about a “scientific” issue, check to see how well it matches the definitions of “science” or “scientism” based on the handy reference guide in this article.
- Better yet, don’t read popular articles about science at all – instead, use Google Scholar or some other search engine to find the actual study written by the actual scientist. Read the “abstract” of the article. You’ll likely discover the “headline” from the popular media doesn’t match the research.
- Those search engines are also a handy place to see how many other scholars have cited this author – an imperfect, but telling, way to judge credibility in academic circles.
- Read anything you find imagining yourself as a detective solving a crime. One of the first questions they ask is Cui bono? (Who benefits?) In other words, who paid for the study? What group stands to gain? Who stands to lose? What is someone trying to sell you?
- Related to that last one: Be suspicious of any product claim tied to a major trend (Keto or otherwise). The advertiser is using the “halo effect” to trick your subconscious brain into positive associations between their product and a hot trend.
If all else fails, skip the popular press and read Science Daily, a surprisingly readable summary of actual scientific research. You also can find glimmers of excellent reporting in the smallest of news outlets: Susan Perry is one to watch.
In the end, should you try a Ketogenic diet? It seems like it’s safe, and it’s likely to be good for you, but it’s not the only choice. You may do just as well on many other diets … or simply be more conscious of what you eat. But remember, non-expert talking here, you should check with an actual doctor.
And should you drink the blood of teenagers? I feel safe in saying no. Just, no.