Personal Science Week - 260813 Fauci

Should we have handled COVID differently?

The COVID pandemic was the largest test of public science in my lifetime, and six years on we still have no 9/11-Commission-style accounting — no single document that says here is what worked, here is what didn’t, here is what we’d do differently. The major institutions and media publications that were extremely eager to give us “the latest science” at the time seem uninterested in objective truth-seeking that might give us the answer.

I’m reminded of this upon seeing Dr. Anthony Fauci called before Congress, only to invoke the Fifth Amendment amid legal questions about a preemptive presidential pardon. A week later the committee voted 8–5, on party lines, to hold him in contempt.

The odd part, and the thing that raises my personal science eyebrows, is how little difference there is between your opinions about Fauci and how you voted in the last election. Fine—he’s a political figure now, in front of a political commission. But I can also predict more generally what you think about six-foot distancing, masks, school closures, vaccine effectiveness, and many other empirical questions. These are scientific questions, unrelated to politics. Shouldn’t we want to know the answers?

The cure vs. the disease

Much of the news focuses on the origins of the pandemic, but regardless of how that turns out, Dr. Fauci was the public face of nearly all the COVID era responses. Whether or not he directly ordered counter-measures like school closures, mask mandates, or restaurant restrictions, he publicly supported and encouraged them.

Many of the consequences will linger for generations. Face masks that were never required anywhere before the pandemic will be mandatory whenever “there’s something going around”. New digital infrastructure makes remote work and food delivery far more ubiquitous than it would have been otherwise. A precedent has been set for lockdowns and other restrictions that were unthinkable before 2020.

Even if our political system can’t answer society-level questions about the effectiveness of these unprecedented measures, every personal scientist should be curious to know what worked and what didn’t—and what we would do differently next time.

What I actually observed

If you’re a professional scientist in need of research grants or publishable results in a politically-charged environment, I don’t blame you for keeping your experimental proposals to yourself. But personal scientists are free from those restrictions. I don’t have expensive lab equipment or a big research staff, but I do have my own observations and powerful LLMs to help me sort facts from opinion. Here is my own personal experience:

1. I know several people hospitalized with COVID, some seriously. Most or all were unvaccinated — either infected before vaccines existed, or they declined.

2. I don’t personally know anyone who died unexpectedly. Not one. Among the hundreds of people I’ve asked, I’ve never gotten a single example of a genuinely out-of-the-blue death. The death of a grandmother is a very sad, of course, but not really unexpected.

3. All of my 80+ year old family members who were infected recovered just fine, even those who caught it in 2020 before vaccines. Of course I know many other families who weren’t so lucky, but I don’t think my situation is terribly unusual.

4. I personally know dozens of people with long COVID, some severely debilitated. COVID was certainly a very serious issue for them.

Now fine, these are personal anecdotes that are true for me and my direct acquaintances. Can we be a little more rigorous?

source: Wikipedia

Do the math

Many of the numbers that were unknown or disputed at the time are now known with much greater certainty: the total number of deaths, the infection fatality rate (IFR) and its ranges broken down by geographical area and other demographics like age, sex, and race. The median US COVID death age was 78, and three-quarters were over 65.

How typical was my personal experience?

The average American knows about 600 people by name. US COVID deaths ran about 1.1 million, or 0.33% of the population. So a 600-person network should contain roughly two COVID deaths total, across four years. And yeah, that kind of tracks for me—we have a family friend who succumbed early in the pandemic, and maybe one or two others if I think hard enough.

But what about unexpected deaths — healthy, under 55? That’s maybe 2% of COVID deaths, or ~22,000 nationally: 0.007% of the population. The simple math:

22,000 / 330,000,000 = 0.007% then 600 × 0.00007 ≈ 0.04 unexpected COVID deaths among people I know well.

You’d need to know about 15,000 people to expect one unexpected COVID death. So for me, the fact I know nobody is not at all unusual. In fact, it’s true for 96% of Americans, including you.

Another way to think about this: a 600-person network sees between three to eight deaths a year from all causes in a normal year. COVID added two, nearly always an older person; call it a 10% increase in funerals. Too small to be detectable.

These are real deaths, and every one is a tragedy of course. But context is everything.

More context and a control

Now let’s think about another group of people I “know”: There are 100 governors and lieutenant governors, 535 members of Congress, and some number of actors, athletes, and executives whose names everyone knows. We hear about famous deaths all the time. If this was a stop-the-world event, where were they?

First, look at the ordinary background rate of unexpected death in healthy people under 55 — cancer, cardiac, accident, overdose. I count four unexpected non-COVID deaths among the “famous” : Basketball star Kobe Bryant, an actress from Glee, an actor from Black Panther, and a rapper.

For COVID deaths, I first looked in Wikipedia, where to have a page you must be a “notable” person—a pool of 150,000 Americans. I found just 3 COVID deaths that met the criteria (”under 55, unexpected”): Nick Cordero, Luke Letlow, and Adam Schlesinger. But I’ve never heard of them. Have you?

In other words, by the “famous people” standard, we get COVID death numbers that track my own personal experience: an uptick you’d only notice if you track very carefully.

Woodstock

For more context, I looked at another serious pandemic. The 1968–69 Hong Kong flu killed ~100,000 Americans out of 200 million, against COVID’s 1.1 million out of 330 million. Age-standardize — the 1968 US had a median age of 28 and 9.6% over 65, versus 38 and 16.5% in 2020 — and the overall menace looks similar.

I looked up “famous deaths” for that period and, as with COVID, I found just a few unexpected deaths from the pandemic (a CIA director, a broadway star), versus many non flu-related deaths (a Soviet Cosmonaut, a boxer in his prime, a Formula One racer).

Except. There were no nationwide restrictions, no daily infection dashboards, no masking, no vaccine ID cards required to enter a restaurant.

In fact, the famous Woodstock concert was held in August 1969, in the trough between the waves. Nobody remembers 1968 as an event, because nobody was counting.

So I wonder, based on my own personal experience, what would have happened if we hadn’t been watching Dr. Fauci, but instead just relied on our own personal observations that “something is going around”.

No masks here (source: Wikipedia)

But the vaccines

Of course, without the public attention to the real dangers of SARS-COV-2, it’s hard to imagine the political will to do anything like Operation Warp Speed, that gave us a vaccine in record time. It’s impossible to tell how many lives were saved, including perhaps many famous ones. It’s a counter-factual that we’ll never answer to everyone’s satisfaction.

Here I was surprised at the range of estimates: various groups using different methodologies claim anywhere from a few hundred thousand on the low end, to 3 million+ on the high end. But all of them agree that 90%+ of the lives saved are in people over 60.

Another undisputed fact: before vaccines the average age at death was in the high 70s and 80% were over age 65.

So what would I have done differently? Honestly, I think it’s hard to defend Dr. Fauci’s draconian recommendations.

Personal Science Weekly Readings

I realize that my skeptical conclusions are not particularly welcome among people who would like to believe all those masking/hygiene efforts were worthwhile. To see many more details for how I derived the above figures, see my lengthy Claude transcript. And please leave a comment to tell me what I got wrong!

Speaking of errors in science, Paul Litvak did his own informal test of AI Peer review in order to see how well LLMs can catch errors in scientific papers. tldr; current AI models are pretty good, especially when you double-check with multiple LLMs. Although his testing isn’t super-rigorous, it’s pretty good for a first pass. He had Claude help him take the original papers and subtly modify them with substantive errors that undermine the paper’s conclusions. You can read his entire methodology

Finally, I can’t wait to read Gary Wolf’s new book; preorder a copy here: quantifiedself.com/book.

About personal science

Just because government agencies or other professionals can’t agree doesn’t mean the rest of us can’t make up our own minds. Personal science is the idea that anybody can use scientific techniques to find actionable insights for everyday problems, including how to respond responsibly in the next pandemic.

We publish every Thursday. If you have other topics to discuss or comments, let us know.