It’s easy to criticize professional scientists—maybe a little too easy. Sometimes in our zeal to encourage more personal science, we appear so negative on Peer Review and Credentials that we can be accused of throwing the baby out with the bath water. Now that LLMs are making it easy to dig through the historical record, is nobody safe?
There’s an old SNL skit where Steve Martin plays the medieval barber Theodoric of York who confidently dismisses the previous generation of doctors for their superstitious beliefs in demonic possession, but “nowadays we know” that the real problem is an “imbalance of bodily humors, perhaps caused by a toad or a small dwarf living in her stomach”. When the treatment fails, he allows himself a moment of self-reflection:
Joan: You charlatan! You killed my daughter, just like you killed most of my other children! Why don’t you admit it! You don’t know what you’re doing!
Theodoric of York: [ steps toward the camera ] Wait a minute. Perhaps she’s right. Perhaps I’ve been wrong to blindly follow the medical traditions and superstitions of past centuries. Maybe we barbers should test these assumptions analytically, through experimentation and a “scientific method”. Maybe this scientific method could be extended to other fields of learning: the natural sciences, art, architecture, navigation. Perhaps I could lead the way to a new age, an age of rebirth, a Renaissance! [ thinks for a minute ] Naaaaaahhh!
The skit is funny because it reminds of the occasional overconfidence of our modern doctors and scientists. Previous generations were wrong but “nowadays we know”.
But of course previous generations were wrong—it’s called progress! And now as we head into an era of ubiquitous AI-generated fact-checking we’re about to see even more clearly how wrong today’s most respected scientists can be.
That said, some big chunks of science don’t get overturned so easily. Generations of replications have proven that some ideas are more right than others. But how “right” were they at the time the ideas were first presented? And is ours the first generation that seems plagued by scientific fraudfudging?
I wanted to find out. Inspired by Hypercritical, a once-popular podcast by two Apple aficionados 1whose motto was “Nothing is so perfect that it can’t be complained about”, I asked Claude to help me nitpick some of the greatest scientists in history.
If we applied the today’s standards of science—accurate note-taking, no plagiarism, appropriate citations and accreditation—what would happen to the OGs?
Ptolemy got credit for something he probably didn’t actually measure himself. Galen didn’t actually dissect humans—he lived in an age that forbade those experiments. The real crime in both cases was that it took 1400 years for somebody to look for themselves rather than obey what was in the book.
When biographer Richard Westfall double-checked the math between the various editions of Newton’s work, he found inputs (speed of sound, precession of the equinoxes) that had been tuned to implausible precision. On Newton’s speed-of-sound “crassitude of air particles” adjustment, Westfall calls it “nothing short of deliberate fraud.”
Simple chi-square analysis of Mendel’s published results from his pea-plant experiments show a probability of a fit that good ≈ 7 in 100,000.
Pasteur was accused at the time of aggressive self-promotion, so perhaps it’s no surprise that his own private notebooks show he quietly cribbed a rival’s method.
And so on with the other Greats…apply enough scrutiny and you’ll find anomalies.
The point here isn’t to bring these giants down to size so much as to recognize that science is done by humans, and humans can make mistakes.
So as the inevitable march of LLM fact-checking makes its way through the canon, we’d do well to remember that science is ultimately about what’s true, not who made the discovery or how they did it.
Personal Science Weekly Readings
Speaking of great scientists, if you haven’t already subscribed to the Seeds of Science Substack (you should!) you’ll have missed their rundown on The State of Substack Science, a careful breakdown of the science-related Substacks (almost 3,000 of them). They even turned it into an interactive lookup tool that can show, among other things, that Isaac Newton has been mentioned substantively in 134 newsletters.
If you’re wondering how much of what your teacher taught you is true, check out Retrocodex: it’s a simple way to learn about corrections to facts that were commonly taught in school. Enter your graduation date and it summarizes stuff where the facts have changed.
For example, I hadn’t heard that the National Geographic Society in 2021 added “Southern Ocean” as an official ocean to our familiar list of Atlantic-Pacific-Indian-Arctic.
Personal Scientist of the Week
Ben Tossell used Codex to pull years of his reading history into a folder and asked Opus to organize it into themes, by year. Perhaps not surprisingly for a guy who makes his living writing and investing in startups, almost half of his reading revolved around how companies get built, grown, and funded.
About Personal Science
Professional scientists get paid to do experiments and churn out papers, much of which will go unread and unused. But if you do science for personal reasons, you know at least one person will benefit—you!
We publish new thoughts every Thursday, so if you have comments or other ideas you’d like to discuss, please let us know.
The two hosts, John Siracusa & Dan Benjamin would take any Apple-related product and point out all the shortcomings.