Personal Science Week 231005

Home remedies vs. The Science™

If you’re reading this, you probably think you’re reasonably well-informed. But we personal scientists always wonder if we’re missing something.

This week we look at some home remedies and the effectiveness of FDA approved drugs. We’ll also speculate about an objective test of open-mindedness.

Home Remedies

The site Earth Clinic has a ton of home remedies for just about every ailment you can think of. Many of the “cures” look like uninformed amateur speculation, but who are we personal scientists to judge? For example, here are their most popular suggestions for migraines:

There are hundreds of other suggestions, all from user reviews. There’s also an “Editor’s Choice” with a few curated remedies (magnesium, nattokinase, avoid MSG, etc.)

We discussed other crowd-sourced treatment sites in PS Week 4 Aug 2022

Effectiveness of Common Medications

Before you dismiss these home remedies as unscientific, look at this large meta study that summarized dozens of common prescription medications by how effective they had actually proven in rigorous studies.

Only the medications with results on the right hand side are effective at achieving quality of life results, as opposed to surrogate measures like blood test results.

Note that these studies look at the ultimate results of the medication, not just what researchers call a “surrogate endpoint”. Often (usually?) a professional study of a medication will pick as their definition of success or failure something that can be easily and objectively measured quickly and cheaply. But you don’t usually care about whether those metrics themselves change, you care about whether you actually got better — or at least, that you feel better. So this paper is interesting because it looks at the actual effectiveness on endpoints, not on one of those indirect measures.

Statin drugs, for example, are well-proven to lower cholesterol but do they actually reduce heart attacks? The answer appears to be no. There are plenty of good reasons to take a statin drug — and who knows, maybe this study is incorrect or biased — but if nothing else, the personal scientist in you should want to ask more questions.

Spurious Correlations

Another reminder to be careful of correlations. Tyler Spirgen’s site, Spurious Correlations will generate charts like this all day:

Ideological Turing Test

How open-minded are you? One good test is how well you can articulate both sides of an argument. The gold standard is the ability to describe an argument is terms that its proponents find indistinguishable from their own. This so-called “Ideological Turing Test” is similar to the famous Turing Test thought experiment of AI researchers, who argue that a machine is intelligent if a human observer can’t tell the difference between its responses and those of a human.

University of Sheffield researchers found 100 people on different sides of arguments about COVID vaccination, Brexit, and veganism, asking them to rate responses to questions about their choice. Without knowing the opinion of the person giving the answer, the respondents rated the answers for the likelihood that the person was on the same side. Conclusion:

those who could pass the ITT were more likely to rate their ideological opponents have having ‘good reasons’ for their position, were less likely to rate their opponents as ‘ignorant / unintelligent / irrational’, and less likely to rate their opponents as ‘immoral /of bad moral character / unethical’.

In other words, it really is true that the best way to prove you understand something is to be able to argue the opposing side as convincingly as its strongest proponents.

The authors made an interactive web App that lets you play with the underlying data.

About Personal Science

The late UC Berkeley professor Seth Roberts used the term “personal science” to describe his passion for applying the techniques of science — data-driven, always skeptical — to personal, rather than professional questions.

Sometimes those questions, unfortunately, can get you into trouble, especially if you question the cherished beliefs of people whose confidence exceeds their accuracy. To weed out the unprepared, we also offer a paid subscriber option to our series on Unpopular Science. In this week’s post we look at how it might be true that Republicans die of COVID more than Democrats, but that the real reason probably has nothing to do with politics.

As always, please let us know about your own personal science experiences.