Personal Science

Share this post

Personal Science Week - 10 Aug 2022

www.personalscience.com

Personal Science Week - 10 Aug 2022

Most research findings aren't even wrong, prebiotics, and more.

Richard Sprague
Aug 11, 2022
Share this post

Personal Science Week - 10 Aug 2022

www.personalscience.com

Personal scientists are well aware of the “replicability crisis” in the social sciences. Surprisingly few peer-reviewed and widely-discussed research results can be reproduced when another group tries to prove the exact same thing. But when attempting to compile a database of studies for her PhD, Anne Scheel found that the problem is even worse than that. For most studies it was nearly impossible to state formally what the researchers were trying to prove in the first place, much less describe whether they succeeded or not.

Just one example: Does screen time affect a person’s well-being? It depends on what you mean by “screen time” and what you mean by “well-being”. Most studies, she notes, don’t bother to define these terms in ways that other, independent researchers could use for replication. Among her suggestions, she concludes that these studies need to move toward more machine-generated hypothesis tests.

For the Personal Scientist, the takeaway is to trust scientific papers less and treat all claims with skepticism.

Prebiotics’ effect depends on the initial configuration

Even when you have a precise target for research, the final result can be less than actionable.

Duke University researchers tested 28 people with prebiotics over a 6 week period. The prebiotics, specific foods known to feed healthy gut microbes (as opposed to probiotics, which are live microbes taken in pill form) were given out at regular intervals and the people were tested to see the effects. The results? The prebiotics had an effect all right: overall “healthy” microbiome composition changed as expected when taking the prebiotics. But when the researchers looked more carefully at the results, they noticed that the changes depended more on the person’s initial microbiome than it did on the specific prebiotics they happened to take. Some people respond well to prebiotics and some people respond barely at all.

How specific prebiotic compounds affected each test subject. Higher is “better”. Note the wide variance, despite averages being “statistically significant” (p<0.05)

What should you take away from this study? That prebiotics affect the microbiome? Not if you’re one of the people below that yellow line.


About Personal Science

In a world of impossible complexity, we often have no choice but to rely on “experts” for medical and other advice. But if you want the truth, or if there’s a specific problem you must solve for health or other reasons, experts are no substitute for your own critical inquiry.

What are the skills necessary to navigate through this overabundance of conflicting information? We believe the techniques of science can help. Personal Science is the practice of applying the scientific method for personal reasons, instead of professional ones.

Share

Share this post

Personal Science Week - 10 Aug 2022

www.personalscience.com
Comments
TopNewCommunity

No posts

Ready for more?

© 2023 Richard Sprague
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing