This week we evaluate OpenEvidence, a site that promises high-quality medical search results—for doctors only.

More and more people are using LLMs for medical questions. According to OpenAI’s January 2026 report, more than 5% of all ChatGPT messages globally are about healthcare — billions of messages a week. About 40 million people ask ChatGPT a healthcare question every day. Naturally, this is making a lot of experts nervous. What if the chatbot gets it wrong? What if a patient acts on bad advice?

Personal scientists take the worry seriously but draw a different conclusion. We trust no one — not a chatbot, not an “expert,” not a glossy consensus guideline — because nobody cares more about your health, or your family’s health, than you do. It’s true that a year or two ago, LLMs were often deceptively inaccurate, but they’ve gotten substantially better, even over the past six months, and they continue to improve.

Still: when you really need the right answer, maybe a general-purpose chatbot isn’t enough, so on the recommendation of a doctor I respect, I tried OpenEvidence.

OpenEvidence is a retrieval-augmented LLM optimized for clinical decision-making. Give it a clinical question and it searches peer-reviewed literature — NEJM, JAMA, The Lancet, Cochrane, plus 300+ journals and FDA/CDC guidance — then synthesizes an answer grounded in those papers with inline citations you can click through to the abstracts. And it’s very popular: About 40% of US physicians use it, across more than 10,000 hospitals.

One catch: it’s only for doctors. Registration requires a National Provider Identifier (NPI) — the 10-digit number CMS issues to healthcare providers. No, you can’t just get your own NPI. It’s a federal crime to imitate a doctor, so don’t even try.

The gate isn’t really about protecting the public from dangerous information. It’s a condition of the publishing deals — and a feature of the business model. OpenEvidence is monetized through pharmaceutical and medical-device advertising at reported CPMs of $70 to $1,000+ (vs. $5–15 for social media). The audience being sold to advertisers is specifically prescribers. Non-prescribers aren’t the customer, so keeping us out is the point.

PSWeek has discussed other academic-focused research tools over the years — Elicit (PSWeek240222), Consensus (PSWeek250601) , FutureHouse (PSWeek250612). Most of them are happy to take anyone’s money. OpenEvidence is the first one I’ve run into that actively refuses civilians.

Testing It Anyway

Fortunately, OpenEvidence’s homepage accepts one query without full registration — presumably rate-limited by IP, though I couldn’t confirm how many you get before being cut off. Enough to run one real test.

I gave both OpenEvidence and Claude the same question — a topic I’ve been thinking about as a possible self-experiment (see PSWeek240905):

This is a good stress test: it spans diagnostic and therapeutic literature, the age qualifier forces the model beyond easy geriatric studies, and the evidence base is genuinely complex.

What OpenEvidence delivered

A clean, authoritative, well-cited response. Every claim linked to a specific PubMed ID. The key references were exactly what a cardiologist would cite.

The quantitative specifics were precise:

• 7% increased cardiovascular risk per 1 m/s increase in aortic PWV after full adjustment,

• approximately doubled risk in participants aged 60 or younger,

• aerobic exercise reductions of −0.75 to −1.02 m/s in central PWV,

• typical programs of 40 minutes × 3 days/week × 11 weeks.

It reads like an UpToDate entry — and for a physician with fifteen minutes between patients, that’s exactly what you want.

What Claude delivered

A longer, more discursive, and far more epistemically honest answer.

Where OpenEvidence stated that PWV predicts events in adults under 50 with high confidence, Claude flagged exactly where that evidence gets thin: the key meta-analyses skew heavily toward older cohorts, the subgroup data for younger adults has wide confidence intervals, plus other caveats.

Claude also raised points you won’t see in a clinical summary: a physics-level critique of the statistical methodology. Similarly, with interventions, Claude was more nuanced: Aerobic exercise got the strongest endorsement (consistent with OpenEvidence), but Claude was honest about the weaker evidence like statins and omega-3s.

Best of all, Claude added something OpenEvidence never would: a personal science section on what it would actually take to run a self-experiment on PWV, including the measurement-noise problem and the distinction between consumer brachial-ankle PWV devices and the gold-standard carotid-femoral method.

Why Claude Wins (For Personal Scientists)

The comparison illustrates the difference between “what” knowledge and “how” knowledge — a distinction we’ve been exploring in PSWeek for a while now.

OpenEvidence knows how to answer your question — efficiently, accurately, well-cited. Claude helps more on the what to look for in the first place— something only a human can decide.

To be fair to OpenEvidence, part of the divergence is by design. A tool built to serve a physician at the point of care should deliver a crisp, actionable summary, not a rumination on the limits of the literature. But that’s also the limitation: the same interface that is perfect for a busy clinician is wrong for a personal scientist deciding whether a biomarker is worth tracking, Claude’s critical thinking is far more valuable — precisely because it’s less confident and more epistemically careful. The best medical-AI experience for a personal scientist turns out to be the general-purpose frontier model we already have.

My recommendation for 2026: Use frontier models as your primary medical research tool. If you want extra confidence in the results — especially for anything you might act on — ask a different frontier model to check the work. Run the same question through Claude and ChatGPT (or Gemini, or Grok). Where they agree, you can be fairly confident. Where they diverge, you’ve found exactly the places where the evidence is genuinely uncertain — which is the most valuable information of all.

Personal Science Weekly Readings

Speaking of medical advice, we should also mention three recent studies that argue the case against trusting chatbots:

Tiller et al. in BMJ Open red-teamed five popular models with 50 questions each on cancer, vaccines, stem cells, nutrition, and athletic performance; Half the answers were “problematic” and none got all the answers. But come on! these were running ancient February 2025 free models with with prompts deliberately designed to elicit bad answers. I’m not persuaded.

Rao et al. in JAMA Network Open tested 21 frontier LLMs (including GPT-5, Claude 4.5 Opus, Grok 4) on 29 standardized clinical vignettes and found that even top models fail more than 80% of the time at generating alternate diagnoses— though they nail the final diagnosis once enough information is on the table.

The most uncomfortable result comes from Bean et al. in Nature Medicine, a preregistered randomized trial with 1,298 UK participants: the LLMs alone correctly identified the relevant condition in ~95% of cases, but participants using the same models got it right only ~34% of the time — no better than people Googling the NHS website. But again, they were using models from 2024 (ancient history) and these were not sophisticated users.

My main complaint with all three studies: none of them includes a physician baseline on the same tasks. The interesting question isn’t whether the chatbot is sometimes wrong — it’s whether it’s wronger than the alternative the patient would otherwise have used. In my experience, a skeptical, well-prompted user with a frontier general-purpose LLM and the habit of cross-checking can do better than 95% of the population using the exact same model. The implication for personal scientists is the one we keep relearning: the tool is fine; the bottleneck is the person operating it.

About Personal Science

The NPI wall isn’t going away — but neither is the basic truth behind personal science: you are the most interested, most patient, and most appropriately paranoid researcher of your own body. No credentialing system selects for that. The good news is that the tools a curious amateur can reach in 2026 are genuinely good enough to do the work — and in some dimensions, better than the ones gated behind professional credentials.

As we like to say: nullius in verba — take no one’s word for it. Not even the AI’s.

Personal Science is delivered every Thursday to anyone who thinks science is just as useful for everyday, personal reasons as it is for professionals. If you have topics you’d like us to explore, please let us know.

Leave a comment

Now that I have an accurate HRV sensor (see PSWeek260409), I’ll describe some new ways I’ve been trying to optimize my sleep—and one surprising result from my data.

The Hypothesis I Didn’t Expect

I’ve known for years that I feel more rested all day when I wake up with a low resting heart rate. But how can I reliably get there? My first guess was magnesium—a common biohacker recommendation—because it seems connected to sleep in some people and latest blood magnesium tests showed I had, at best, average levels.

A good personal science rule of thumb is that when something works, it works, and you’ll notice a difference quickly. If you need to do a complicated statistical analysis to show an effect, well, there’s probably not much of an effect.

I noticed an immediate effect from the magnesium: diarrhea! Effect on sleep: not noticeable. Conclusion: not a good tradeoff.

After some additional searching, fortunately, I discovered that magnesium comes in many different forms. My gut issues went away immediately when I switched to magnesium bisglycinate (480mg). It can take a few weeks, though, for the body to absorb it enough to make a difference, so I figured that I might as well use that time to conduct a serious experiment. I started feeding weeks of detailed sleep, food, and activity data into a Claude project I’ve set up for my personal health.

After a week of this, an unexpected variable kept surfacing as the strongest predictor of my deep sleep duration: the amount of protein I ate the day before.

That, plus the type and intensity of exercise, seems to reliably affect my resting heart rate and how I feel the next morning.

How I Track All This

My setup combines several tools, each doing what it does best:

Sleep: Apple Watch (built-in sleep staging), plus a Polar H10 chest strap with the Kubios HRV app each morning. I put on the strap when I wake up, wait a few minutes for my body to settle, then run a 3-minute HRV reading. Kubios doesn’t have an export feature, so I just screenshot the result and show it to Claude.

Food: I use Grok for meal logging—it’s lightning-fast and I’ve prompted it to give me exactly the macronutrient breakdown I want. I’ll use a thread titled “April 9th” and come back to it each time I eat something. A typical entry: “½ cup Greek yogurt with a handful of pepitas.” At the end of the day, I paste Grok’s summary into Claude.

The AI workflow: This is where the true power comes in. I have a Claude project loaded with everything I can think of—blood test results, EHR records from clinic visits, optometrist and dental records, typical food diary, family history. At bedtime, I start a new thread, paste the day’s food log, and say I’m going to sleep. When I wake up, I share the Kubios screenshot and ask Claude to pull my Apple HealthKit data for the night, compare it to previous nights, and give me its analysis. I save a summary to Obsidian, and Claude consults that context the next time. Sleeping happens every night, so at that rate it doesn’t take long to build up a real dataset.

The Protein Signal

After 20 nights of tracking I learned that the variable that modulates N3 (deep sleep) duration most is dietary protein intake—and it’s not a subtle, linear dose-response. It’s closer to a step function with a threshold around 90g.

Last week provided a clean demonstration. Six consecutive nights, same bedtime, same supplements, same person. The only meaningful variation: prior-day activity load, diet, and the resulting sleep architecture.

Thursday night (Apr 10→11) was the best night of the week by nearly every metric—despite being the lowest-activity day (4,691 steps, half of Wednesday’s). It produced 52 minutes of N3 (the week’s high, +17 min above the mean), 96% sleep efficiency, only 14 minutes of wake-after-sleep-onset, and a resting heart rate of 55 bpm. I woke up feeling great. The dietary profile that day: high protein (~120g), simple meals (whey smoothie, pork, spinach salad, milk), dinner before 7 PM, no alcohol.

From the full 20-night dataset, here are the variables that seem to modulate my N3 duration, ranked by estimated effect:

Helpers: Adequate protein (>90g): +17 min. Rest day after a training block: +12 min. Total sleep time above 5.5 hours (a gate condition): +12 min. Resistance training same day: +7 min. Early dinner (before 7 PM): +6 min.

Hurters: Threshold-intensity exercise (Zone 3/4): −18 min. Low protein (<80g): −15 min. Late dinner (after 7 PM): −8 min.

Protein shows up as the largest lever in both directions—biggest helper when adequate, second-biggest hurter when deficient.

Why Might This Work?

The textbook explanation for diet and sleep is tryptophan: eat it, convert it to serotonin, then melatonin, then sleep better. But that story has problems. High-protein meals flood your blood with large neutral amino acids that compete with tryptophan for brain uptake, and only 1–2% of dietary tryptophan converts to brain serotonin anyway. Minor contributor at best.

A more compelling candidate is the GHRH pathway. Ask your own LLM to explain the details (or see this recent study in Cell), but the basic idea is that GHRH neurons modulate sleep architecture directly via circulating amino acids, and there’s a threshold involved. This would explain the step-function shape I see in my data. Below a protein threshold, the signaling system doesn’t fully engage, and N3 is suppressed. Above it, the system activates normally. It also explains why the best N3 nights cluster on days with both high protein AND prior resistance training—the brain monitors peripheral anabolic demand and adjusts accordingly.

My LLM also suggests other plausible mechanisms, each testable. For example, if it’s GHRH-driven, a bolus of high-protein whey 30 minutes before bed should acutely increase N3 independent of daytime intake. If it’s total anabolic demand, timing shouldn’t matter.

I’m testing those now.

The Skeptic’s Caveat

I should be clear about what this is and isn’t. It’s an n-of-1, 20-night observational dataset with a lot of confounders I can’t fully control. The “effect sizes” in my plots are estimates from my body, not from a randomized trial. Protein days might also be days when I ate better overall, or felt less stressed, or any number of other things.

And people’s bodies are dynamic. Even if this pattern is real today, it could shift as my training adapts, my diet changes, or I simply age another year. The history of personal science is full of interventions that “worked” for a while and then didn’t.

But that’s the point. The goal isn’t to discover a universal law—it’s to find what works for me right now, track it honestly, and update when the data says otherwise. Twenty nights of structured tracking with Claude as an analytical partner has already taught me more about my sleep than years of casually wearing a watch.

Personal Science Weekly Readings

• Speaking of experiments with unexpected results, everyone knows that penicillin was discovered by ‘accident’, when an absent-minded Alexander Fleming left some mold out too long. A long Asimov article reviews the record and concludes it was actually a more deliberate search than he made it sound at the time. “Chance favors the prepared mind”.

• An excellent Nature article Investigating the analytical robustness of the social and behavioural sciences, had hundreds of authors read academic papers and try to replicate them from the raw data. About half the replications got roughly the same results, but about 25% reached a different conclusion from the original.

• Speaking of protein, a narrative review in Nutrients confirmed that higher-protein diets are associated with better objective sleep quality (more deep sleep, higher efficiency), though the mechanisms remain debated. Doherty et al., 2022 and higher-protein diets improved sleep quality scores in two randomized controlled trials of overweight adults. Interestingly, slow-wave sleep increased on the high-protein arm. Zhou et al., 2016

• For previous PSWeek sleep experiments: I tested CBD (PSWeek241024—no effect), ProdromeGlia (PSWeek240111—significant deep sleep improvement), SleepScore sonar tracking (PSWeek251218), and compared AI research tools on my 3AM wakeup problem (PSWeek250612).

About Personal Science

Personal scientists approach sleep the way we approach everything: with open-minded skepticism and a willingness to experiment on ourselves. We track not because the numbers are perfect, but because tracking forces the attention that leads to insight.

We publish every Thursday. If you’ve found dietary patterns that affect your sleep, let us know.

Leave a comment

This week I’ve been comparing it to my latest toy, the Polar H10 heart rate monitoring chest strap.

How Much Does Your Wrist Lie?

As we discussed recently in PSWeek260326, I’ve worn an Apple Watch continuously for years. It tracks my resting heart rate, flags illness before I feel it, and logs my sleep. For everyday health monitoring, it’s the best single device you can buy. Quantified Scientist YouTuber Rob ter Horst agrees.

But I’ve been wanting to go deeper on heart rate variability (HRV), the beat-to-beat variation that many researchers consider one of the best single markers of autonomic health. I’ve been following personal scientists experimenting with vagus nerve stimulation to improve HRV and I’ve been curious whether my Z2 cycling sessions actually shift my parasympathetic tone the way the academic scientists suggest.

The problem: my Apple Watch can’t give me the HRV data I need. Not because it’s bad — it’s fine for overnight trends — but because wrist-based photoplethysmography (PPG) has fundamental limitations that become obvious once you strap on a proper ECG sensor.

So this week I got a Polar H10 chest strap (~$100) and took it on a two-hour cycling ride alongside my Apple Watch Ultra 2. The results taught me more than I expected.

Two Devices, Two Very Different Stories

Here’s what both sensors reported for the same 119-minute, 32 km ride:

The Apple Watch never saw my actual peak heart rate. It missed a 164 bpm surge entirely, and it never registered the sustained 159 bpm effort that followed a few minutes later. Instead of the spiky, dynamic trace the Polar captured at 1 Hz across 7,149 samples, the Watch produced a smoothed, flattened version that made a hard threshold ride look like a gentle tempo session.

This is the textbook PPG failure mode during cycling: motion artifact from handlebar vibration and grip changes causes signal dropouts, and the Watch’s algorithm interpolates through them, clipping peaks and compressing the range. It’s impressive engineering — the Watch does a remarkable job reconstructing something from a noisy optical signal — but it’s still interpolation, not measurement.

For resting HR and walking? The Watch is excellent. For cycling, and especially for capturing the rapid HR transitions that matter for interval training and HRV analysis, it’s not enough.

Why This Matters Beyond Heart Rate

Heart rate is actually the easy part. The bigger issue is HRV.

The Polar H10 is functionally a single-lead ambulatory ECG. It measures the electrical depolarization of the heart muscle directly, sampling internally at 1000 Hz and deriving R-R intervals (the time between heartbeats) at roughly 1 millisecond precision. That’s the raw material you need for proper HRV metrics: SDNN, RMSSD, frequency-domain analysis, Poincaré plots.

The Apple Watch measures something different: pulse rate via green LED reflectance at the radial artery. Pulse arrival time is delayed 100–200 ms from the actual R-wave, and that delay varies with blood pressure, vasomotor tone, and arterial compliance. The resulting jitter — maybe 10–20 ms on an 800 ms beat interval — washes out when you’re averaging for heart rate, but it corrupts the fine-grained beat-to-beat variability that is HRV.

For overnight HRV trends (which Apple reports as a single nightly SDNN value), this is probably fine. But for exercise HRV, post-exercise recovery kinetics, and real-time biofeedback during Z2 training, you need true interbeat interval data. That’s what the chest strap gives you.

The Calorie Puzzle (And What It Taught Me About Fitness)

The calorie discrepancy — 663 kcal (Watch) vs. 1,545 kcal (Polar) — is striking, and both numbers are probably wrong. Neither device is measuring real exercise—they’re just using algorithms based on heart rate to estimate how hard my body worked.

The Watch’s 663 kcal is built on an under-measured HR trace, so it’s too low. The Polar estimate uses the Keytel 2005 equation, which at my HR average of 126 spits out a number that’s too high: ~1,545 kcal. And neither measure takes into account how the heart responds to a longer ride, especially in people who are less fit.

A power-based estimate (the most defensible method) suggests 800–1,100 kcal for a 20-mile ride at ~10 mph average.

But here’s the non-obvious part: the mismatch between my low average speed (10 mph) and high heart rate (40% in Z4) is itself diagnostic. It means my heart is working hard to produce modest power output — the classic signature of a detrained cardiovascular system.

I thought I was in decent shape. The Polar H10 data, honestly analyzed, says otherwise. That’s exactly the kind of uncomfortable truth a personal scientist should want to find.

The $90 Upgrade That Beats Oura

Back in PSWeek220616 we mentioned Oura as the winner of a head-to-head competition for heart rate accuracy — largely because the ring’s PPG sensor sits on the finger, where blood flow is better and motion artifact is lower than the wrist.

But the Oura Ring 4 starts at $349 plus a $5.99/month subscription ($70/year), and it's still PPG. It's still measuring pulse waves, not electrical signals. It still can't give you true R-R intervals with the millisecond precision you need for serious HRV work.

The Polar H10 costs about $100, has no subscription, uses a CR2025 battery that lasts over a year, and gives you actual ECG-grade data. It stores a workout internally if you leave your phone behind, connects via Bluetooth and ANT+ simultaneously, and works with essentially every fitness app and device on the market.

The tradeoff is convenience: you won’t wear a chest strap 24/7. But that’s actually fine. Think of it like a Keto-Mojo blood ketone meter versus a CGM — one is a spot-check instrument for when precision matters, the other is continuous but lower-fidelity.

My recommendation: Keep the Apple Watch for 24/7 passive tracking (RHR, overnight HRV trends, sleep staging, step counts, illness detection). Add the Polar H10 for dedicated cycling sessions, structured interval work, and any experiment where you need clean HRV data.

What I’m Testing Next

That difference between the two devices was actually proof that I’m out of shape. My ride data shows I spent only ~15 minutes in true Zone 2 (98–114 bpm) — far below the 90–150 min/week target that the endurance pros recommend for building aerobic base. What I thought was a Z2 ride was actually a Z3/Z4 threshold session, because I was relying on Apple Watch data that understated my HR by enough to shift the zone classification.

With the Polar H10 providing cleaner real-time feedback, I can now actually hold the narrow Z2 band — which, for someone in my current fitness state, means riding at what feels uncomfortably easy: maybe 12–14 mph on flat terrain. The hypothesis: sustained Z2 work → improved parasympathetic tone → lower resting HR → better sleep consolidation. I’ll be tracking all of it.

Personal Science Weekly Readings

A Harvard physics professor successfully got Claude Code to write a significant paper, the kind that would normally take a second-year grad student a year or so. Claude made lots of mistakes, so the final result required substantial hand-holding from a knowledgeable human. Importantly, this was theoretical physics, where innovation comes from new ideas rather than new data — the tools are already powerful enough that a determined personal scientist could probably make a real contribution in domains that previously required an academic lab.

Evolutionary biologist Bret Weinstein The Dark Horse podcast has a fascinating segment disputing that Jackson Lab study that seemed to show mice live longer when exposed to long-term cellphone radiation. The catch: laboratory mice are specifically bred for tumor susceptibility, which Weinstein argues makes them paradoxically more likely to live longer by affecting the cancer/longevity tradeoff. Be very skeptical of experimental results on lab mice.

About Personal Science

The lesson from this week’s experiment is one that applies across all of personal science: your measurement tool shapes what you see. The Apple Watch showed me a comfortable tempo ride. The Polar H10 showed me a threshold session with a detrained cardiovascular system struggling to produce modest power. Same ride, same body, different story — and the uncomfortable version was the more useful one.

Personal scientists don’t just collect data. We question whether the data collection itself might be the source of our blind spots.

If you have topics you’d like us to explore, please let us know.

This year’s cold/flu was milder than last year, partly thanks to some of the personal science techniques I applied. This week’s post will explain how I studied the different bouts to knock it more quickly.

The terms “cold” or “flu” are the everyday way we refer to an illness that leaves us coughing, sneezing, runny nose, often accompanied with an achy feeling. The worst symptoms last for a couple of days and then the whole thing passes after a week or two. COVID confused us all about exactly which term we should apply, so now I prefer the simple “upper respiratory infection” (URI) or “Influenza-like Illness” (ILI).

My URI began innocently on a Saturday afternoon with an unexpected sneeze. I felt completely fine, so I hoped it was maybe some dust, but the next day it was obvious that I was coming down with something.

My symptoms progressively worsened, reaching the nadir by Monday night. I slept terribly—headache, fever, the works—and I was useless the next day. I “worked from home”, but mostly from my couch, answering Slacks and emails between naps. I tested myself with one of my leftover COVID kits; it was negative of course.

A cold (URI) normally lasts a week, but through my carefully-honed personal science techniques I was able to shorten that to a mere seven days. By the weekend I was fine.

Compared to what

Rather than waste time in bed, I decided to use the new power of LLMs to study my illness carefully and then compare it to last year.

I have a Claude Project (“Personal Health for Richard”) where I have uploaded literally every health- and medical-related document I can think of: old lab tests, fitness and exercise routines, my typical diet, medications and supplements, and on and on.

Of course my project is also connected to Apple Health data, which itself is connected to many other devices and services including my Apple Watch, my doctor’s MyChart data, my bluetooth-enabled bathroom scale, ketone meter, blood pressure cuff, and much more. Again, the rule is: if it’s medical- or health-related: just throw it in there. (see more in PSWeek260326)

This repository of my personal health data makes it trivial to ask day-to-day health-related questions like “how well did I sleep” or “what specific supplements or medications might be good in this situation?”. In my case, especially at the beginning of the infection, I especially wanted to know (1) how long will this last, and (2) what can I do to feel better more quickly.

In March 2025, I documented a similar illness that knocked me out for a full week and left a lingering cough for three more. (see PSWeek250424) Thanks to a full year of continuous Apple Watch data, I can do something I couldn't do before: quantitatively compare the two illnesses head-to-head using resting heart rate, heart rate variability, and sleeping wrist temperature.

The Data

The 2025 illness (probably influenza) hit much harder: resting heart rate spiked 38% above baseline, peaking around 80 bpm. This year’s URI (probably rhinovirus) was gentler — a 14% rise to about 64 bpm. HRV tells the same story: it cratered to 32 ms in 2025 versus a nadir of 46 ms this year. And wrist temperature, my favorite new metric, showed a dramatic difference: one sharp single-night spike to 37.6°C in 2026 versus a sustained multi-day fever plateau last year.

The overall shape is different too. Last year’s RHR chart had two peaks — and that second one, a week and a half after symptom onset, was my own fault. I felt well enough to hike the Grand Canyon, which was clearly stupid in retrospect. The extra exertion triggered what looked like a relapse: elevated heart rate, return of cough, another week of feeling lousy. My 2025 illness dragged on for 25+ days total, with a cough that lingered three weeks. This year? Seven days, start to finish. No cough. No relapse. Partly because the virus was milder, but partly because I didn’t do anything dumb.

Lesson learned: don't hike the Grand Canyon before you're fully recovered. The data couldn't be clearer. That second RHR peak in 2025 was entirely self-inflicted. My body was fighting an immune war, and asking it to also haul up and down a canyon is like opening a second front. This year I imposed a strict rule: no exercise until RHR stayed at or below 60 bpm for two consecutive days.

Could I have nipped it in the bud?

A 2022 study in the Journal of Infectious Diseases found that wearable sensors detected influenza infection an average of 58 hours before symptom onset in 94% of inoculated volunteers — using exactly the kind of multi-variable anomaly detection (heart rate + activity) that our Apple Watches already collect. The challenge is that the algorithm needs a robust personal baseline, and the signal-to-noise ratio for a mild rhinovirus is much lower than for influenza. In my case, there was no detectable difference in resting heart rate, heart rate variability, or wrist temperature.

Still, the question nags: if I’d noticed something early could I have done anything? The honest answer is: probably not much. There’s no strong evidence that any intervention during the viral incubation period prevents illness once you’ve been exposed. Zinc lozenges within 24 hours of symptom onset might shorten a cold by a day or so, but the evidence is mixed and the effect is small. Extra sleep and reduced stress during that window are probably the best bet — which, ironically, is what my body was already telling me to do by making me sedentary.

On the worst night I tried magnesium bisglycinate with 2 mg melatonin (Solaray brand). It worked well — fast sleep onset, improved deep sleep from a miserable ~20 minutes to ~27 minutes the next night. The melatonin wasn’t just for sleep: there’s decent evidence it modulates the NLRP3 inflammasome pathway, which is part of the innate immune response. Whether that mattered for a mild URI is debatable, but it didn’t hurt and the sleep was better. (A second night's experiment with a different magnesium blend (oxide/citrate) was a mistake — osmotic diarrhea. Stick with bisglycinate.)

About Personal Science

Everybody gets sick eventually, and there’s nothing a personal scientist can do to change that. But having a year of continuous wearable data turns a miserable week into a case study you can actually learn from — comparing severity across illnesses, testing which supplements help your sleep, and knowing when it’s truly safe to resume normal activity. The biggest lesson from my two bouts wasn’t a supplement or a hack. It was simpler than that: respect the data, respect the recovery, and don’t hike the Grand Canyon.

If you’ve tracked your own illness with wearable data, or have a personal protocol that works for you, we’d love to hear about it. Let us know.

Leave a comment

This week we’ll point to a deep analysis made by a fellow personal scientist, and show how new AI tools make it possible to do a similar report for yourself in minutes.

Alex Chernavsky (see PSWeek251113) is at it again with a report showing how he tracked five years of exercise https://www.self-experiments.org/five-years-of-exercise-tracked/

With the assistance of Claude Code, I analyzed more than 640 hours of running, cycling, and elliptical workouts collected over five years. The clearest gains in fitness came early: my resting heart rate fell by about eight beats per minute in the first year and stayed lower afterward. Running pace improved quickly before leveling off, even as training continued. Despite a long break from running due to plantar fasciitis, cross-training preserved most of my aerobic capacity. Different activities produced distinct intensity patterns, with the elliptical often matching or exceeding the effort of cycling. Although Claude Code exhibited some flaws, it ultimately proved to be a very useful partner in making sense of the data.

Any good personal science analysis should produce results that are

1. Non-Obvious

2. Actionable.

Too many self-experiments use complicated math to “discover” obvious advice your mother already gave you for free (“running will improve your fitness”), or information that you can’t do anything about anyway (e.g. “I ran faster when I was younger”).

So it’s inspiring to see that Alex’ analysis gave him some worthwhile takeaways: elliptical training works better for him than sprint cycling, and a prolonged injury doesn’t wipe out his gains.

Let Claude do everything

But to get his data, Alex used a cumbersome data export via the Auto Health Export app (see PSWeek230413). Once he had the data on his desktop, he used Claude to write software to do the analysis which led to the charts you see. Although new LLMs make programming much easier than in the past, he still estimates it took him 15-20 hours over a couple of weeks—and time and effort commitment that few of us can repeat, inspiring as it is.

Now that Claude can access Apple Health directly, I wondered if I could skip that export step and jump straight to the analysis.

Here’s my entire prompt:

My friend just did an analysis of his health data and wrote about it here. I think you have direct access to my Apple health. Can you show me a plan for how you might make the same kind of report only except using Alex and his data, use my data and what you know about me. His report is https://www.self-experiments.org/five-years-of-exercise-tracked/. For the first version I’d like a report that simply focuses on exercise and the items mentioned by Alex (exercise vs RHR over time and exercise intensity). I don’t do running so those items are less necessary.

After a few minutes of churning, Claude came back with a report as detailed as the one Alex had painstakingly generated but using my own Apple Watch data.

It concluded with this about me, which is more concerning:

For your age group, normative SDNN is approximately 40–80 ms; current values remain well within the population range, but the direction of change combined with rising RHR suggests declining parasympathetic tone rather than a benign aging effect alone.

It’s now worthwhile for me to dig more deeply into my exercise habits, where I’ll look especially at some actionable results. Another task for Claude!

Personal Science Weekly Readings

If you listen to any popular wellness-related podcast, you’ll have heard about AG1 (”Athletic Greens”). Scott Carney did a deep-dive takedown of the “science” behind their claims in a 3 1/2 hour video that also reviews the hype from other top health influencers. tldr; don’t trust any of their health claims. Like we said when we first mentioned AG1 back in PSWeek241017, “it’s an expensive way to get extra nutrition”.

NNT (Number Needed to Treat) is one of the simplest measures of how well a medical intervention actually works. The free website TheNNT has compiled NNT metrics for hundreds of drugs. For example, you’d need to give statins to 104 healthy people for 5 years in order to prevent a single heart attack, but meanwhile about 10 of them would suffer unwanted side effect and 50 would develop diabetes. For people who’ve already had a heart attack, the NNT is a more favorable 83, with similar rates of harm.

Finally, for well-referenced and curated LLM details about a specific condition, I’ve recently been trying OpenEvidence, thanks to a recommendation of Dr. Adam Rinde. More in a future post.

About Personal Science

Your wearable has been quietly collecting data for years. Until recently, making sense of it required either serious programming skills or the patience to wrestle with clunky export tools. Now an AI can do the analysis in minutes — but only if you know the right questions to ask.

That’s always been the core of personal science: not the tools, but the curiosity that drives you to use them. Nullius in verba — take no one’s word for it, including the AI’s. Check its work, question its conclusions, and look for what’s non-obvious and actionable in your own data.

Personal Science Week is published each Thursday for anyone who’d rather test a claim than take it on faith. If you have topics you’d like to see covered, let us know.

Leave a comment

So I tried one.

Result: nothing. Actually, worse than nothing.

This isn’t my first ketone experiment. I’ve been curious about exogenous ketones for years, first with a 10-day keto diet in 2020 and later testing Metabolic Switch ketone esters, which I reviewed in PSWeek220714. The esters did reliably raise my blood ketones — at the cost of spiked LDL cholesterol and a nasty taste. Ketone-IQ uses a different molecule, R-1,3-butanediol, a precursor that your liver converts into the ketone body BHB. In theory, it’s a gentler, more physiological approach. In practice — at least in me — it did essentially nothing.

The Experiment

Sunday morning, fasted 10+ hours, one cup of coffee with a teaspoon of coconut milk. Baseline finger stick with a Keto-Mojo meter: BHB 0.4 mM, glucose 73 mg/dL. At 6:45 AM I drank the shot.

Peak rise: +0.2 mM. Published pharmacokinetic data for butanediol predict a rise of +1.5 to 2.5 mM in fasted subjects. My 0.2 is within the Keto-Mojo’s own measurement error (~10–15% coefficient of variation). In other words: statistically indistinguishable from zero.

Worse, by T+90 my BHB had dropped below baseline — from 0.4 to 0.3. One plausible explanation: the 70 calories in the shot triggered just enough insulin to suppress my body’s own ketone production, without the butanediol providing adequate replacement. Net effect: I ended up less ketotic than before I drank it.

Subjectively? Hunger and mild agitation around the one-hour mark. No cognitive clarity, no appetite suppression — the opposite of what’s promised on the label. I also measured my reaction time using BRT at each blood draw: 341ms, 324ms, 398ms. Normally you’d expect reaction times to improve with repeated testing (the well-known practice effect in psychometrics), yet my worst score came last — at the point when the ketones should have been working hardest.

Why It Failed: Check Your Genes

At this point my ever-present Claude LLM pointed out something important: Butanediol doesn’t become BHB on its own — your liver has to convert it using the enzyme alcohol dehydrogenase (ADH), the same enzyme family that metabolizes ethanol. And ADH activity varies enormously by genotype.

That same afternoon I checked my 23andMe data. If you have 23andMe, you can look this up yourself in about two minutes.

The key SNP is rs1229984 (that link goes directly to your 23andMe raw data browser — you’ll need to be logged in). This encodes the ADH1B Arg48His variant, the single biggest determinant of how fast your liver converts butanediol into BHB.

Here’s the one tricky part: 23andMe reports on the positive (+) strand of the reference genome, while most research literature and SNPedia describe this SNP on the minus strand. So you need to take the complement of what 23andMe shows you:

My result: C;C — the slow metabolizer. The enzyme activity of this genotype is roughly 40–100× lower than the fast ADH1B*2 variant. The butanediol is sitting in my system waiting for an enzyme that works at a fraction of the speed the product designers assumed.

You can optionally also check rs698 for the ADH1C Ile350Val variant (a secondary factor — ADH1C*2 is about 1.5–2× less active than ADH1C*1), and rs671 for ALDH2 — the “Asian flush” gene, which affects the next step of the pathway. All three together paint a fairly complete picture of your alcohol/butanediol metabolism.

This is a textbook case of pharmacogenomic non-response — the kind of individual variation that population-level marketing completely ignores. Those impressive published results? They’re averages across mixed populations where fast metabolizers pull the mean way up. But I’m not an average—I’m me!

The Broader Lesson

Personal scientists know that individual variation is the whole point. A product that “works” in a clinical trial works on average, across a population. You’re a specific person with a specific genotype, and a $4 experiment plus a free look at your existing 23andMe data can tell you whether a whole category of supplements is worth your time — or not.

For me, butanediol-based products (Ketone-IQ, deltaG precursor formulas) are predicted to be ineffective. If I want exogenous ketones, I’d need a direct BHB salt or ester formulation that bypasses the liver entirely — like the Metabolic Switch I tested in 2022 (which did raise my BHB, though with the LDL caveat). Or I could simply fast for 36–48 hours and achieve 1.5–2.0 mM BHB endogenously, at zero cost. The Lindy approach wins again.

The real value of this experiment wasn’t the ketone shot. It was the structured attention: an hour of finger-sticking, a table of numbers, and a genetic lookup that took five minutes. Total cost: $4 and a morning. The insight — that my ADH genotype makes an entire class of products useless for me — is permanent.

Do not repurchase.

Personal Science Weekly Readings

Speaking of stuff that doesn’t work, imagine my surprise when earlier this week I decided to take some extra Vitamin C “just in case” that weird tickling in my throat turns into a full-blown cold. I asked Claude if it’s okay to mix it with hot tea and it said sure, but it’s a waste! It pointed me to well-established research that shows your body can’t handle more than 200mg of Vitamin C at a time — the rest is just peed out.

But some other personal science ideas may work better:

• Vaccine beer? Chris Buck and his brother used their homebrewing skills to make a vaccine-in-a-beer by genetically modifying brewers yeast. See the details in their Substack post.

• AI for healthcare: Both Claude and OpenAI now offer HIPAA-compliant products, though they’re aimed at practitioner workflows more than patient diagnosis. Dr. Adam Rinde also recommends OpenEvidence as another good tool. We’re working on a review for a future PS Week.

• The missing sense: Mark Hay writes a lengthy overview at JSTOR of why your doctor should be giving you a smell test — olfactory decline is one of the earliest signals of neurodegeneration. Asimov Press “Scent, In Silico” is a recent, detailed overview of new technologies that make it easier to study. I bet personal scientists could come up with “good enough” tests for smell that are better than what clinicians use today.

About Personal Science

Collect your own data, think for yourself, and be skeptical. Those are some of the many characteristics of people who practice science for personal reasons. We publish weekly, every Thursday, but we intend most of our issues to be useful for years to come. You may especially want to browse past posts on blood tests, conspiracy theories, storing your data, and much more.

And if you’re in the Seattle area on March 24th, join us at 5pm for our Quantified Self Happy Hour. We’ll be meeting near Pioneer Square. RSVP for more details.

Any other ideas for genomics-led predictions for which products will work for you? Let us know

Leave a comment

This week I’ll summarize what I learned.

I attended the 2026 edition (March 3–6, Anaheim) as a guest of my friends at Tastermonial, whose $100+ experiment kits come with a clinical-grade continuous glucose monitor, food product samples, and an app that walks you through your own glucose-response experiment. I’ve learned a ton from my own Tastermonial experiments — with Biostrips (PSWeek250807), Apple Cider Vinegar (PSWeek240725), and CGMs (PSWeek230406) — so I was especially watching for products that could become the basis of new self-experiments.

Expo West isn’t open to the public. Everyone there is in the industry — distributors, retailers, brand founders — so exhibitors give out a lot of samples to attract business partners. Walking the tightly-packed vendor aisles is like Trick-or-Treating on Halloween: you start with an empty bag and it fills up fast. Here’s a small selection from just one afternoon:

Trends Worth Testing

A few of my personal science takeaways:

Dates as sweetener. Companies like Just Date and Datefix are pushing date syrup as a “whole food” sugar replacement. Their pitch is that because dates contain fiber, their glycemic impact is lower than refined sugar. I’m skeptical. Date syrup is still roughly 70% sugar by weight; the fiber content per serving is modest. But the glucose curve might genuinely differ from honey or table sugar, and that’s what matters for an n-of-1 experiment. It’s a testable claim — exactly the kind of thing a CGM experiment could answer.

Mushrooms in everything. Functional mushroom products were everywhere — in coffee (Four Sigmatic, Malama, MUD\WTR), in chocolate, in gummies. The claimed benefits (focus, immune support, adaptogenic stress relief) are mostly backed by in vitro or animal studies, not robust human trials. Lion’s mane has the most promising human evidence for cognition, but even there the studies are small and short-term. Still, the risk profile is low, and the products taste surprisingly good.

Protein on everything — fiber next? The “protein” label has been slapped on everything from boba tea (Boba Tea Protein) to chips and cookies. An industry insider told me the next wave will be fiber. I saw hints of this — Catalina Crunch had installed an airport billboard for the occasion — but it hasn’t fully arrived yet. As with most of these fads, protein labeling is often misleading (whey isolate added to junk food doesn’t make it health food), and fiber claims may follow the same pattern.

Manuka honey. Supposedly, this honey from New Zealand bees who pollinate the manuka bush has potent antibacterial properties, primarily due to its high concentration of methylglyoxal (MGO). Honey Droplets turns that into a throat lozenge that you can take any time. I can believe the claims for topical wound healing, but for oral consumption as a health food, I’m less certain. But I’m intrigued and would like to test it more.

Energy drinks and mixes. These were so abundant I couldn’t carry them all home — airport liquid restrictions did me a favor. Most are repackaged caffeine-plus-B-vitamins with varying amounts of nootropic pixie dust. A few interesting outliers existed, but the category as a whole felt saturated and undifferentiated.

What I Didn’t See

Where are the peptides? I had expected the supplements aisles to overflow with peptide-themed products, given the buzz in the biohacking community. The absence likely reflects the regulatory gap: most serious peptides require injections and don’t fit neatly into the “natural products” framing of Expo West. Or the industry just hasn’t figured out how to package and market them yet.

More broadly, I was disappointed by the lack of genuine innovation in supplements. Most booths offered variations on fish oil, magnesium, and concentrated vitamin drinks. The supplement aisle felt like it’s been in a holding pattern for years — lots of new packaging, not many new ideas.

The Ultimate Personal Science Tool

One thing that has changed dramatically is my workflow for processing trade shows. Rather than collect piles of brochures, I now take a few photos at each booth — hundreds over the course of the event. Later, I have Claude Cowork look through them, identify vendors from logos or signage, visit their websites, and generate structured summaries. I’ve written a custom “skill” that produces pages like this one:

(Also see Day 3 booth visits and the full exhibit summary.)

This is arguably the most personal science thing I did at Expo West. The photos → AI → structured web page pipeline turns an overwhelming trade show into a searchable, referenceable database. Claude Code has become such a large part of my daily workflow that very little of my personal science work doesn’t pass through an LLM at some point. Thanks to integration with Apple Health, I’m running near-daily experiments — more on that in future posts.

Personal Science Weekly Readings

People Science is on a roll with new studies. Sign up for tests with probiotics, joint health, women’s health and more.

Adam Kroetsch has an excellent analysis of Why clinical trials are inefficient. It’s not technology, or even regulation. One obviously wasteful practice: the policy of 100% source data verification (accounting for 25–40% of trial costs, which the FDA has publicly opposed for over a decade) persists because managers at Big Pharma happily overspend to minimize any chance of blame. Then there’s what the article calls the “sacred science” problem: treating trials as purely scientific endeavors rather than business processes makes it socially awkward to even discuss cost reduction, and reform proposals balloon into unfocused “everything bagel” agendas instead of targeting efficiency directly.

One initiative I’m optimistic about: THRIVE, led by Stanford’s Michael Snyder, secures up to $34.5M to develop the first FDA-grade “intrinsic capacity score” for ARPA-H’s PROSPR program. Includes our friends from OpenCures.

Finally, How Metrics Make Us Miserable: a conversation with philosopher C. Thi Nguyen forces journalist Derek Thompson to rethink his relationship with his Oura ring. If life is a game, who decides the objective and how it’s measured? (Listen w/transcript). I think this is a good reminder of the distinction between measurement attention vs. measurement precision — the Oura ring is useful not because its HRV numbers are gospel, but because it makes you pay attention.

About Personal Science

We’re open-minded and skeptical — that’s the guiding principle of personal science, where we apply the truth-seeking power of science to everyday situations.

If you’ve tried any of the products above, or have ideas you’d like to discuss, let us know.

Leave a comment

This week I experimented with another one: Yuka.

Yuka is a French-made app (60 million users worldwide) that scans barcodes and instantly rates food from 0 to 100. Green means “Excellent” (75+) or “Good” (50-75), orange is “Poor” (20-50), and red is “Bad” (below 20). It also does cosmetics, but I only tested the food side. The free version gives you unlimited scans; a paid tier ($10-20/year) adds search and offline use.

The scoring formula is straightforward: 60% nutritional quality (based on Nutri-Score, a European front-of-pack labeling system), 30% additives (each one classified by risk level), and 10% organic bonus. Nutri-Score itself penalizes calories, sugar, saturated fat, and sodium, while rewarding fiber, protein, and fruit/vegetable content. If a “high-risk” additive is present, the score is automatically capped at 49 regardless of nutritional quality.

Simple. Fast. And sometimes baffling.

My Kitchen Test

I grabbed a handful of items from my kitchen and scanned them all:

Most of these are reasonable enough. Quinoa and coconut milk landing in the green? Fine. Salt getting dinged? Understandable (it’s literally sodium).

But then there’s the pepitas.

Pumpkin Seeds: “Poor”?

Great Value Pepitas—roasted and salted pumpkin seed kernels—scored 35/100. That puts them in the same “Poor” tier as Morton sea salt.

Meanwhile, Barilla Cellentani Pasta scored “Excellent.”

Think about that for a second. Pumpkin seeds are one of nature’s most nutrient-dense foods: roughly 30% protein by weight, rich in magnesium (one of the hardest minerals to get enough of), high in zinc, iron, and healthy fats. They’ve been a dietary staple across Mesoamerican civilizations for thousands of years. If you had to pick one food to survive on, pepitas would be a far better choice than refined wheat pasta.

So what happened? The problem is Nutri-Score’s fat penalty. The algorithm dings foods for total calories and saturated fat content per 100g. Seeds and nuts are calorie-dense by nature—that’s what makes them nutritious. A 100g serving of pepitas has roughly 550 calories and 5g of saturated fat (alongside about 19g of polyunsaturated fat, which Nutri-Score mostly ignores). The “roasted and salted” part adds a sodium penalty too. Add in the fact that seeds don’t benefit from the same “fruit, vegetables, and legumes” positive score that nuts sometimes get (Nutri-Score’s own categorization here is fuzzy), and you end up in “Poor” territory.

This is a known weakness. European researchers have published peer-reviewed papers specifically documenting how Nutri-Score underrates nuts and seeds, and the Nutri-Score Scientific Committee has been trying to fix it since 2022. The updated algorithm moved nuts into a dedicated “fats, oils, nuts and seeds” category—but whether Yuka has adopted these revisions is unclear.

The pasta, by contrast, sails through: low fat, low sodium, modest calories per serving, no additives. Never mind that it’s essentially refined carbohydrate with minimal micronutrient content. Nutri-Score rewards absence of “bad” things more than presence of good ones.

The Deeper Problem

This pepitas-vs-pasta absurdity illustrates a pattern that registered dietitians and nutrition researchers have been pointing out:

Yuka’s scoring conflates “low in things to avoid” with “healthy.” Cheese gets dinged for saturated fat despite being an excellent source of protein and calcium. Natural peanut butter scores poorly because it’s “high in fat”—which is the whole point of peanut butter. Grass-fed beef jerky with three ingredients (beef, salt, vinegar) scores 43/100 while Chef Boyardee ravioli gets 63/100.

The 10% organic bonus is also questionable. Being organic says nothing about a food’s nutritional value—it’s a production method, not a health indicator. Giving automatic points for an organic label is a values judgment dressed up as science.

And the additive scoring, while well-intentioned, applies a precautionary principle that doesn’t account for dosage. A trace amount of a “moderate risk” additive caps your score at 50, even if the actual quantity is far below any level shown to cause harm.

What Yuka Gets Right

I don’t want to be entirely negative. Yuka has some genuine strengths for the personal scientist:

It’s free, fast, and frictionless. Point your phone at a barcode and you get an instant result. The UI is beautifully designed. Compared to apps like Sift or Fig that limit you to 5 free scans per month, Yuka’s unlimited free scans lower the barrier to entry dramatically.

It flags additives you’d never notice. This is where the app genuinely adds value beyond the nutrition label. Even if you disagree with the risk weighting, having someone highlight the dipotassium phosphate in your oat milk or the BHT in your cereal is useful information.

It’s independent. Yuka claims to take no money from food companies, funding itself through premium subscriptions. In a world where most “nutrition” apps are basically advertising platforms, that matters.

It’s changing industry behavior. In France, supermarket chain Intermarché reportedly removed 140 additives and reformulated over 900 products in response to Yuka scores. Chobani has done similar reformulations in the US. Even if the scoring is imperfect, the competitive pressure to improve ingredients is real.

About Personal Science

Here’s how I think about Yuka, and food-scoring apps in general: they’re useful as one signal among many, but dangerous as a sole authority.

As we discussed back in PSWeek241212, the whole premise of one-size-fits-all food scoring has the same fundamental flaw as mandatory nutrition labeling: it assumes everyone has the same dietary needs. A 35/100 for pepitas is meaningless for someone tracking their magnesium intake. A pasta score of “Excellent” is misleading for a diabetic monitoring blood sugar spikes.

The more interesting apps—like Zoe, which we covered in PSWeek250515—try to account for individual variation by scoring foods in the context of your whole meal and your personal biology. That’s harder to build, but it’s the direction that actually serves personal scientists.

In the meantime, Yuka is worth downloading for the additive-flagging alone. Just don’t let it convince you that pumpkin seeds are bad for you and refined pasta is health food. Your grandmother already knew better than that.

Leave a comment

I started writing this on Sunday afternoon from the 11th floor of an Airbnb in Guadalajara, Mexico. From my window I can see pillars of smoke rising to the south — one of them in the direction of the airport. The street below, which two days ago was full of well-dressed families and teenagers celebrating quinceañeras, is nearly empty. The few people out are burly and somewhat scruffy-looking men. No women — which, if you’ve spent time in places where things go wrong, you know is never a good sign.

I didn’t learn about any of this from Google News or the New York Times. After a quiet Sunday morning, we’d prepared a full itinerary for the day: the ceramics museum in Tlaquepaque, lunch at Casa Fuerte, folk art shopping. But after hopping in an Uber, the driver handed me his phone. On the screen was a Google Translate message warning of cartel activity at our proposed destination. Do we really want to go? he asked. I did my own quick online check to confirm. Uh no, we decided as we had him turn around and bring us back.

That’s how information works in a crisis. Not a push notification from Google News. A driver showing you his phone.

Finding What’s Actually Happening

What had happened: earlier that morning, the Mexican Army killed Nemesio “El Mencho” Oseguera Cervantes, leader of the Jalisco New Generation Cartel, in an operation near Tapalpa — about two hours south. The cartel retaliated with coordinated arson and roadblocks (narcobloqueos) across Jalisco and several other states. The governor declared Código Rojo (Code Red). Schools closed. The U.S. Embassy told Americans to shelter in place.

But “shelter in place” doesn’t answer the questions I actually needed answered: am I safe at this location? Where can I get food? How long will this last?

For that, I did what any personal scientist would do: I cast a wide net and filtered hard. Repeated queries to X and Grok pulled up a stream of real-time posts — some from people clearly on the ground in Guadalajara, others from armchair analysts thousands of miles away. I wanted specific, observed detail that tells you something.

Being open-minded means following every lead. On X I found posts speculating about conspiracies — Navy SEALs had supposedly been spotted doing a joint training exercise last week in Campeche. Coincidence? Being skeptical means noticing that the same source got other verifiable details wrong. Open-minded enough to read it; skeptical enough not to share it. That’s the line.

As Arnold Kling puts it: “We decide what to believe by deciding who to believe.” In normal life, we outsource that decision to institutions — the Times, the CDC, Wikipedia. In a crisis, the institutions can’t tell you what you need to know, and the question of who gets very local, very fast. The Uber driver. The street vendor who sold us chorizo because the grocery chains were all shuttered. Our own eyes can see the smoke in the distance, the empty streets, the boarded shops.

Meanwhile, plenty of dramatic images circulated online — some real, some apparently AI-generated, many impossible to verify from my apartment. But I found that the most reliable signals came from tools nobody thinks of as "news." The Uber app showed "no rides available" to the airport. Rappi — Mexico's version of DoorDash — showed "no stores available." These aren't editorialized, they can't be faked, and they told me more about actual conditions on the ground than any headline. When your delivery app becomes your intelligence source, you know the institutions have hit their limits.

Travel Is Not Tourism

I’ve written before (PSWeek250529) about how travel doesn’t automatically “open the mind.” Philosopher Agnes Collard argues in her Case Against Travel (archive) that “Tourism is what we call travelling when other people are doing it.” Even Seneca, writing to Lucilius two thousand years ago, warned that changing your location doesn’t change your mind — you travel with yourself. A Columbia University study found that people who travel broadly (many countries briefly) actually show small decreases in moral behavior, while those with depth experiences don’t.

Someone once told me the test for real travel: (1) use public transit, (2) visit a grocery store, (3) spend time at a local person’s house. The idea is that these force you past the curated surface and into how people actually live.

I’d add a fourth: travel with a purpose. My trip to Mexico back in 2018 wasn’t tourism — I was tracking my microbiome while eating chapulines and drinking pulque, testing whether foreign travel changes your gut in measurable ways (it does: Proteobacteria spike, diversity increases, everything bounces back in a few weeks). That kind of intentional observation is the opposite of what Collard criticizes. You’re not collecting experiences like souvenirs. You’re running an experiment.

A crisis, it turns out, is another kind of purpose — one you didn’t choose. When the metro shuts down and the grocery chains close and you’re buying chorizo from a street vendor because it’s the only food available, you are no longer a tourist. You’re navigating the same reality as five million locals. And the personal science mindset — skeptical, observational, trusting your own data — turns out to be exactly what you need when the information environment itself becomes unreliable. The question isn’t “what does the New York Times say?” It’s “what do I actually see?”

It’s now Thursday morning and I’m headed back to the US today. The situation here has become more stable and I don’t expect any more problems. But I’ll be paying attention.

Personal Science Weekly Readings

The MIST (Misinformation Susceptibility Test) lets you test your own ability to distinguish real from fake headlines. I scored in the 36th percentile — not because I’m gullible, but because I was too skeptical.

Stanford STORM is a free, open-source tool that generates Wikipedia-style articles with citations on any topic — think of it as deep research you can run yourself. To test how well early crisis information holds up, I asked it to analyze how accurate early COVID-19 predictions turned out to be.

Eric Topol questions the VO2 max craze, arguing that the widely-cited study linking it to longevity has serious methodological problems. As someone whose Apple Watch VO2 max readings bounce around enough to be essentially meaningless, I found this a reassuring counter-argument to Peter Attia’s claims that it’s the single most important metric for overall health and longevity.

And speaking of longevity, among the items found at the compound of that dead cartel leader El Mencho was Tationil Plus 3000mg — an intravenous glutathione injection made by Swiss Healthcare Pharmaceutical Ltd. You can't get the injectable kind in the US, but apparently it's quite popular in Latin America, where it's promoted for two purposes: skin-whitening (it inhibits tyrosinase, reducing melanin production) and longevity. Unfortunately if he was hoping it would help him live longer, it doesn't seem to have worked.

About Personal Science

Personal scientists think for themselves, approaching health and life decisions with open-minded skepticism. We collect our own data when possible, remain curious about mechanisms, and remember that averages don’t define individuals — your information environment in a crisis might be completely different from mine, and that’s exactly what makes personal science valuable.

We publish each Thursday for anyone interested in applying scientific thinking to everyday life. If you have questions, experiences to share, or topics you’d like us to explore, please let us know.

Leave a comment

This week I retake my hearing test and explore what’s new.

My Updated AirPods Hearing Test

Back in PSWeek241114 I showed how easy it is to run the AirPods Pro audiogram test (it’s built into every iPhone with Airpods) and noted that my left ear looked worse — probably because I forgot to shut the door. This time I did it properly: quiet room, door closed, late at night with minimal ambient noise.

And I guess there’s been no change: my right ear is slightly better—in fact, even better than last time.

The test itself has improved since my last attempt. Apple’s hearing features received FDA authorization in late 2024, and the clinical validation data is impressive: the median deviation between AirPods and professional audiometry was just 1.8 dBHL for the standard 4-frequency pure tone average. Classification agreement with professional audiologists was 86% for identical WHO grade and 100% within ±1 grade. That’s remarkably close to what you’d get in a sound booth.

More importantly, the AirPods now function as an over-the-counter hearing aid for mild-to-moderate loss. Take the test, and if the results suggest you need help, your personalized hearing profile automatically boosts frequencies in real time — during conversations, music, phone calls. No audiologist visit required.

Know Your Number

Now from the Johns Hopkins Bloomberg School of Public Health there’s a more sophisticated way to test: Hearing Number app. It’s free (iOS and Android) and takes about five minutes: after double-checking that your headset is working properly, it plays a series of tones at different volumes and you tap the screen when you hear something.

The result is a single trackable metric: your 4-frequency pure tone average (PTA4) in each ear.

The PTA4 is the average of your hearing thresholds at 500, 1000, 2000, and 4000 Hz — the frequencies most important for understanding speech. Children and young adults with healthy hearing score as low as -10 dB. The number creeps up with age. The WHO uses these same thresholds to classify hearing loss:

• < 20 dB: Normal ← That’s me

• 20–34 dB: Mild loss

• 35–49 dB: Moderate loss

• 50–64 dB: Moderately severe

• 65+ dB: Severe to profound

The app was created by Frank Lin, MD, PhD, who also led the landmark ACHIEVE trial on hearing and cognitive decline (more on that below). His pitch is simple: you know your blood pressure, your weight, your step count. Why don’t you know your hearing number?

Hey Wait a Minute!

The Apple test and the Hearing Number tests disagree! There’s a gap of 7–19 dB depending on the ear, and the two tests don’t even agree on which ear is better. What’s going on?

The most likely culprit is calibration. Apple has an enormous advantage here: they control both the hardware (AirPods Pro) and the software, so they can calibrate precisely for that exact transducer. The Hearing Number app, built by Mimi Hearing Technologies under contract to Johns Hopkins, has to work across many headphone models. This seems to be a problem noted in all the research studies. Even among devices of the same model, studies find reference sound levels can vary by about 4 dB — and that compounds with other error sources.

Testing method matters too. Apple uses a dynamic adaptive algorithm that samples across the entire frequency and decibel range simultaneously — a departure from the traditional one-frequency-at-a-time approach. The Hearing Number app likely uses a more conventional ascending method. Different methods can produce systematically different thresholds.

Then there’s the asymmetry question. My Apple test shows a 12 dB gap between ears; Hearing Number shows them equal. Either one test caught a real difference the other missed, or environmental noise was affecting one ear more than the other — and Apple’s active noise cancellation may have handled that better. (Both apps warn that ambient noise is the biggest confounder.)

Here’s what matters, though: both tests classify me as normal (< 20 dB). The clinical category agrees even when the specific numbers don’t. And this is actually the more important insight for personal science. A single hearing test number, like a single blood test result, is less meaningful than most people assume. What you want is a trend line: the same test, under the same conditions, repeated over time. If my Apple PTA4 creeps from 11 to 15 to 20 over the next few years, that’s a signal. If it bounces between 8 and 14, that’s noise.

This is a theme we keep returning to in PSWeek: measurement precision matters less than you think, but measurement consistency matters more than you think. Pick one test, standardize your conditions (quiet room, same headphones, same time of day), and track over time. The absolute number is less important than the trajectory.

Hearing and Dementia

And speaking of hearing, a major study on dementia prevention says that the biggest single modifiable risk factor from mid-life is hearing loss, with an estimated 37% increased risk of incident dementia and a dose-response relationship of roughly 16% increased risk per 10 dB of worsening hearing.

The 2024 Lancet Commission on Dementia Prevention, led by psychiatrist Gill Livingston at University College London, is the most comprehensive review of modifiable dementia risk factors ever assembled. They list 14 modifiable risk factors that together account for roughly 45% of all dementia cases worldwide, of which hearing is the most important.

That’s what interests Frank Lin at Johns Hopkins (the same researcher behind the Hearing Number app). His ACHIEVE trial, published in The Lancet in 2023 was the first large randomized controlled trial to show that actually treating hearing loss slowed cognitive decline over three years. Lin and Livingston are independent researchers at different institutions, arriving at the same conclusion from different directions: Livingston through epidemiological meta-analysis of hundreds of studies, Lin through a prospective intervention trial. When independent teams converge, the signal gets harder to dismiss.

Other Hearing

Speaking of hearing tests, here are a couple more worth checking out (see PSWeek240818 for more)

Soundly Online Hearing Test — Web-based, works with any headphones in a quiet room. Pure-tone test with visual audiogram. Free. Good option if you don’t have AirPods or a compatible phone. (See PSWeek221103)

How Old Is Your Hearing? — The fun one. Plays tones of increasing pitch and guesses your age based on your high-frequency cutoff. Not clinically useful, but surprisingly accurate and a great conversation starter.

About Personal Science

Testing your hearing is part of a broader theme in personal science: using acoustic measurements to understand your own body.

Here's the thing about hearing: it declines so gradually that most people don't notice until they've already lost decades of data they could have been tracking. The measurement itself may be the intervention. Once you know your number, you start noticing — the restaurant where you can't follow conversation, the ear you favor on phone calls, the volume creep on your headphones. That structured attention is what personal science is for. Not to replace your audiologist, but to make you a better-informed patient when you finally visit one.

For more on acoustic self-measurement see PSWeek221103 and PSWeek240418, the AirPods audiogram in PSWeek241114, and voice analysis with Praat in PSWeek251023.

If you have other hearing-related tests, let us know.

Leave a comment

This week we’ll look at how AI has upended the way personal scientists should think about notes, why the format of your notes now matters more than the app, and what you should actually be writing down.

Why We Used to Take Notes

For most of history, we took notes to save facts we might need later. We clipped articles. We copied quotes. We bookmarked pages. The whole point was retrieval: you wrote it down because you couldn’t trust yourself to remember it, and finding it again in a library or filing cabinet was hard.

This created what Christian Tietze called the “Collector’s Fallacy” — the illusion that saving information is the same as learning it. We warned about this back in PSWeek230330: don’t fool yourself into thinking you really absorbed something just because you clipped it into Evernote. Most of those clippings were never looked at again. The act of collecting became a substitute for the act of thinking.

But at least the collector’s fallacy made some sense in the pre-AI world, because retrieval was genuinely difficult. If you didn’t save that article about sleep hygiene, how would you find it again? Maybe you’d remember the journal, maybe you’d reconstruct the Google search, but probably you’d just lose it.

That rationale has now completely collapsed. An LLM can find, summarize, and synthesize virtually any published fact in seconds. Saving a clipping “in case I need it later” is like keeping a drawer full of maps when you have GPS. The information isn’t going anywhere — and the AI is better at finding it than you ever were.

So if notes are no longer primarily for storing retrievable facts, what are they for?

Notes as a Record of Your Own Thinking

Here’s what I think has changed: the purpose of note-taking has shifted from collecting other people’s ideas to recording your own. Facts are cheap now. What’s expensive — what no AI can retrieve for you — is your own reactions, hypotheses, connections, and half-formed insights. The note you write after reading a paper, where you say “wait, this contradicts what I found last month with my sleep data” — that’s irreplaceable. The paper itself? Claude can find that in two seconds.

This is particularly important for personal scientists. Raw data matters, of course, but my interpretations, suspicions, and hunches matter more. I can always re-download a paper on melatonin. I can’t re-download the fleeting thought I had at 3am about why my magnesium supplementation seemed to correlate with my wakeup times only in winter.

In other words, AI has made the collector’s fallacy even more dangerous than before, because now it extends to the illusion that the AI “knows” your thoughts too. It doesn’t. It knows published facts. It knows what you’ve told it. But it doesn’t know what you were thinking while reading that article unless you wrote it down.

So: take fewer notes about what other people said. Take more notes about what you think about what they said. The first kind is now redundant. The second kind has become the most valuable thing in your entire knowledge system.

Files Over Apps

If the content of your notes has changed, so should the format. And here the shift is equally dramatic.

Steph Ango, the CEO of Obsidian, articulated a principle he calls “File Over App”: if you want your writing to still be readable on a computer from the 2060s, it should be readable on a computer from the 1960s. Apps are ephemeral; plain text files endure. (Read a much more rigorous analysis of this point in Martin Kleppmann and colleagues 2019 “Local-First Software: You Own Your Data, in Spite of the Cloud”.)

When you depend on apps (or worse, web apps), you are entrusting your data to cloud services that can shut down, change pricing, or lock us out. Vint Cerf, one of the inventors of the internet, has made the same warning. And anyone who’s lost data to a discontinued app — Google Trips, HealthifyMe, the old Zoe app — knows this isn’t theoretical.

The core idea is simple: your data should be more durable than any tool that touches it. If the tool dies, the data survives. If a better tool comes along, you can switch without losing anything. The files are the permanent thing. The apps are interchangeable views into those files

What to Do About It

I switched from Joplin to Obsidian about two years ago. Obsidian stores everything as plain Markdown files in a normal folder — which means any tool can read them: grep, Python scripts, Git, and crucially, AI assistants. I now routinely point Claude Code at my notes folder and ask it to cross-reference my sleep data with my supplement experiments across two years of daily entries. As I described in PSWeek250814, my “repo” isn’t code — it’s self-experiment notes, health records, PDFs of lab results, and years of PSWeek drafts. It just works, because the files are right there.

Gary Wolf’s Zotero-to-Obsidian workflow and tools like Basic Memory (an MCP server giving Claude persistent vault access) show where this is heading. And if Obsidian disappears tomorrow, every note is still a readable text file.

But the tool matters less than the habits. Keep it simple — a plain text file you write in every day beats an elaborate system you abandon after a week. I still keep a daily food/sleep/exercise log in a single yearly text file. Write your reactions, not just the facts — when you read something interesting, write why it’s interesting, what it reminds you of, what you’d test differently. That’s the part no AI can reconstruct. Prefer plain files — Obsidian is my choice, but Logseq is fully open-source and file-based, and even a folder of .md files in VS Code works fine.

Personal Science Weekly Readings

Heying and Weinstein’s “Don’t Look It Up” argues that easy access to information can undermine empirical thinking skills if it replaces rather than supplements your own reasoning. The same caution applies to AI-assisted notes — the goal is to think better, not to outsource thinking.

Don’t forget the “Google Effect” we discussed in PSWeek260108: we tend to remember where to find information rather than the information itself. LLMs amplify this massively. If you’re offloading all your factual memory to AI, what remains uniquely yours is your thinking about those facts.

Martin Kleppmann et al., “Local-First Software: You Own Your Data, in Spite of the Cloud” (2019) — the rigorous academic case for why your data should outlive your apps, with seven ideals for local-first software.

Steph Ango, “File Over App” — the short, accessible version. Key line: “It’s a delusion to think [Obsidian] will last forever. It’s the plain text files I create that are designed to last.”

About Personal Science

There’s a pattern here that extends beyond note-taking. When Function Health wouldn’t let me export my blood test data (PSWeek240912), I was frustrated for the same reason: that data belongs to me, not to whatever app happened to collect it. When a CGM stores readings in a proprietary format, it limits what I can learn from my own glucose data.

File-over-app is really just data sovereignty applied to your own thoughts. Personal scientists already insist on owning their health data. We should insist on owning our knowledge base too — and now, thanks to AI, the stakes for getting the format right are higher than ever.

If you have a note-taking workflow that works well with AI tools, or if you’ve experienced data loss from a discontinued app, let us know.

Leave a comment

This week we’ll look at how much you can trust the label on your supplement bottle.

You probably know that in the United States there’s little oversight on the manufacturing and sale of over-the-counter (mail order) supplements. For all practical purposes, a bottle sold as “Vitamin C” could have anywhere from zero to many times the amount or purity claimed. Nobody checks.

As a personal scientist, I don’t particularly want a legally-mandated testing system for supplements. I’d rather make up my own mind, both on the trustworthiness of the manufacturer as well as the makeup and efficacy of the pills themselves. I trust my ability to evaluate the reputation and believability of the seller. Or to put it more precisely, I want the flexibility to put whatever I want into my body, without some “expert” deciding for me what’s okay what’s not. I’m of course happy to listen to what the experts think, but experts disagree on just about everything, so ultimately I’ll need to decide for myself.

Regulation means less innovation

Pharmaceuticals are regulated under laws passed since the 1962 Kefauver-Harris amendments that required efficacy proof (not just safety) before you’re allowed to try a new drug. Your first reaction might be to assume this is a basic requirement of living in a civilized society. After all, without some legal oversight, profit-driven companies have a built-in incentive to overpromise and even cut corners on quality and safety—perhaps at your expense.

That may be a perfectly legitimate attitude for non-scientists who don’t want to be bothered with the messy process of getting at the truth. But personal scientists know that no expert cares as much as you do about something that directly affects you or a loved one.

And those FDA rules come at a cost, a steep one for those of us who want to see more innovation and are willing to trust our own judgement to decide if something works or not.

Economist Sam Peltzman studied this rigorously in his landmark 1973 paper. Before 1962, the pharmaceutical industry introduced an average of 40 new chemical entities (NCEs) per year. After the amendments took effect, that number dropped to about 16 per year—a 60% reduction that Peltzman’s statistical model attributes directly to the new regulatory burden. (Paid subscribers know we discussed this in Unpopular Science 240317)

The chart shows Peltzman’s predicted vs. actual NCE introductions. His model, built on pre-1962 data, accurately tracked the market until the amendments hit—then actual introductions diverged sharply downward.

Worse, the new laws didn’t even improve drug quality. Peltzman found that the proportion of ineffective drugs reaching the market remained roughly constant (~10%) before and after 1962. The market was already filtering bad drugs through physician experience, patient outcomes, and liability concerns. The FDA requirements added enormous costs without meaningfully improving the drug pool.

Casey Mulligan’s 2021 NBER update applied this framework to COVID vaccines and estimated that the 48-day delay between Pfizer’s clear efficacy results (November 5, 2020) and Emergency Use Authorization (December 22, 2020) cost 6,000-10,000 additional nursing home deaths. The total economic cost of vaccine delay: roughly $1 trillion.

The FDA faces what Alex Tabarrok calls the “invisible graveyard” problem: when the agency approves a harmful drug, there are identifiable victims, lawsuits, and Congressional hearings. When it delays or blocks a beneficial drug, the victims are statistical—they never knew the treatment existed. This asymmetry creates systematic bias toward excessive caution, regardless of net health outcomes.

None of this means the FDA does no good. But the tradeoffs are real, the costs are larger than commonly acknowledged, and the same logic applies when people propose stricter supplement regulation.

How to tell for yourself

It’s important to remember that “no regulation” is not the same as “no oversight”. Partly because the government doesn’t monopolize the testing of supplements, there is a healthy market of third parties who compete to evaluate supplement quality.

Examine.com is an independent educational organization that claims to have the largest database of nutritional and supplement information on the internet. As we noted back in 2022, Elizabeth Van Nostrand studied their site and concluded that, while the summaries are well-done and extensive, they’re incomplete. “If a particular effect is important to you, you will still need to do your own research,” she concludes. (By the way, Elizabeth’s writing at Aceso Under Glass is a great resource for Personal Science in general, definitely worth following).

The three main independent testing labs are ConsumerLab (consumerlab.com), Labdoor (labdoor.com), and SuppCo (supp.co). I’ve tried them all, but I don’t have any reason to recommend one over the other (see PSWeek250731). But I’d like to know if, as with blood, there are some supplements that are particularly difficult to manufacture consistently.

Evaluating supplement quality

I asked Claude to summarize what ConsumerLab has learned from testing over 7,000 supplements since 1999. The overall failure rate? About 22%—roughly one in four or five products fails to meet quality standards.

The failure rates vary dramatically by category, but digging deeper I think the answer is obvious. Some “non-chemical entities” are just easier to manufacture and transport than others. Here’s what I got when I double-checked Claude against other facts:

In general, anything that’s a mineral requires little processing (zinc and calcium are basically glorified rock). Something that requires more chemical reactions for processing, or that relies on harvesting biological matter (e.g. fish oil) will be much harder to get right.

Bottom line: choose generic brands for the items at the top of this chart, but carefully consider the independent reviews for the others.

More on pharmaceuticals

Speaking of regulation, although the FDA requires drug manufacturers to submit to all kinds of paperwork before they can sell their products, it can be difficult for normal people to see the details. For example, do you know exactly where your prescription drug was manufactured?

Propublica rx Inspector lets you type in a generic prescription and it will tell you. For example, here’s what it says about the drug I took for Lyme. Click “View Details” and it tells you the FDA notes on each of their inspections.

Personal Science Weekly Reading

Claude keeps getting better and better. Their iPhone app now integrates with Apple Health and I’m now able to ask more intimate questions easily and fluidly. This is a game-changer that I’ll discuss in future posts.

Speaking of Claude, please check out a new app I “vibe-coded”: TextKeep is a simple app for MacOS that lets you save your iPhone text messages as easily-editable files on your computer. For technical reasons, Apple doesn’t allow iMessage apps in the App Store, so you’ll need to download it directly from my site. But it’s a real Mac app, notarized and virus-free. Please check it out and let me know how you like it!

The cool part is how easy it was to build this app: literally less than an hour. My mind is racing over all the other cool apps that are now possible thanks to how easy LLMs have made it to do serious software development.

People Science is running a sleep study to test a new type of probiotics for its effectiveness in people with sleep troubles. Check out their website and application form to see if you quality. You’ll need to take a couple of before/after microbiome tests and take their probiotic for 15 weeks, but if you complete everything they’ll give you an Oura Ring and $100 for your trouble. (You need to have pretty serious insomnia—they didn’t approve some of the sleep-deprived people I know)

About Personal Science

The hard part of personal science isn’t finding information — LLMs have made that almost trivially easy. The hard part is figuring out which ideas are worth your time in the first place. What should you actually track? Which supplement claims deserve a real experiment versus an eye-roll? When does a new study matter for your situation?

That's what we try to do here each Thursday: not rehash what you could ask Claude yourself, but highlight the ideas and tools that a curious, skeptical person might want to follow up on.

Is there something else you think is interesting about supplements? Let us know

Leave a comment

This week we’ll explore what makes fermentation so well-suited to personal experimentation, share some surprising new research about geography and microbes, and review my own experiments with sourdough, koji, sauerkraut, and kefir.

About Fermentation

Part of my interest in home fermentation comes from a friend whose chronic gut problems seemed to stabilize after drinking kefir, the yogurt-type drink you can buy anywhere these days. When he got tired of paying $3/day for the store-bought kind, he switched to making it at home and discovered a whole new level of healing: his gut responded dramatically better with the homemade kind compared to the commercial ones.

This anecdote made me start thinking of fermentation—and microbes in general—as an “external digestive system” that mediates between food in your immediate environment and the delicate process your stomach uses to make it useful for your body. Just like humans use fire to chemically transform food into easier nutrition, fermentation is how we use microbes to do something similar.

Sauerkraut

Last week I finally set up the traditional Chinese-style fermentation jar I received at Christmas. My first project was a simple fermented cabbage (aka sauerkraut). If you’ve not done this before, you’re missing out! It’s incredibly simple: shake a little salt on some shredded cabbage pounded briefly to expose the juices. Stuff it into a container and wait a week.

Sure enough, I now have maybe a gallon of fresh and crunchy sauerkraut.

The lactobacillus bacteria already on the cabbage do all the work. Salt concentration (around 2%) and temperature (room temp is fine) are the only variables that matter.

Sauerkraut is the perfect “minimum viable fermentation experiment”—if you can’t make sauerkraut, you’re overthinking fermentation.

Sourdough

I’ve written about my sourdough bread experiments in PSWeek250911 but here’s the short version: making sourdough from scratch is trivially simple. You literally just mix flour with water and let it sit at room temperature for about a week, “feeding” it daily by discarding half and replacing with fresh flour and water. I’ve continued to experiment several times a week, making sourdough bread whenever I travel and have access to a kitchen.

My “external digestive system” idea matches what people think of as terroir—the idea that San Francisco sourdough tastes different because of unique local wild yeasts. Unfortunately for my theory, a landmark 2021 study in eLife analyzed 500 sourdough starters from four continents and found something surprising: geography doesn’t matter. The researchers concluded that “in sharp contrast with widespread assumptions, we found little evidence for biogeographic patterns in starter communities.” Sorry, San Francisco.

What does predict your sourdough’s character? How you maintain it—feeding schedule, hydration ratio, flour type, and the microbial interactions within the starter itself. The study also highlighted acetic acid bacteria, previously overlooked, as key drivers of aroma and rise rate. Starters with more AAB produce stronger vinegar aromas and slower rises

Koji

Another fermentation experiment I’ve been trying is with a Japanese traditional marinade flavoring called koji. It technically isn’t fermentation—it’s Aspergillus oryzae mold growing on rice grains, producing enzymes that break down proteins and starches. When you apply the resulting enzymes for a couple hours (or overnight) to something like chicken or beef, the food takes on a notably softer and sweeter flavor.

To make the marinade, you need to buy koji grains. I bought a one-pound container of Cold Mountain grains for about $10, but you can buy similar brands through Amazon. It looks like ordinary rice, but you mix it with water and salt (4.5% salinity), stirring daily for a week until it tastes both salty and sweet—that’s how you know it’s ready.

I’ve been using it on chicken thighs, cooked sous vide at 148°F for about five hours. The result is noticeably more tender and flavorful than unmarinated chicken. Unlike microbiome changes (which require expensive lab tests to verify), koji success is immediately obvious to your taste buds.

Kefir and Kombucha

The ferments above are all about transforming food. But one of my favorite personal science experiments happened a few years ago when I tried to directly measure the effects on my gut microbiome.

I’ve made milk kefir regularly by dropping kefir grains into milk and leaving it at room temperature for 24 hours. The grains multiply, requiring regular disposal of excess—a nice problem to have.

From my microbiome testing: when I drink kefir regularly, Leuconostoc jumps from near-zero to significant levels. Fusicatenibacter appeared only during kefir consumption periods—it wasn’t detectable otherwise.

Kombucha on the other hand, made no dent in my microbiome. I ran a week-long experiment drinking 48oz/day of GT’s Gingerade. Result? Bacillus coagulans (from the added probiotic) showed up transiently in my gut tests, but quickly disappeared.

The lesson: just because microbes from a fermented food show up in your gut doesn’t mean they’re doing anything obvious. The microbiome effects are detectable but their significance remains unclear—at least for a healthy person like me. My friend with chronic gut issues had a different experience, which suggests that individual variation matters enormously.

More About Fermentation

• The 2021 eLife sourdough study that analyzed 500 starters from four continents—fascinating methodology and freely available data

• The Sourdough Framework on GitHub: reproducible protocols for bread-making, written by a programmer for programmers

• A Global Review of Geographical Diversity of Kefir Microbiome (2025): Comprehensive review of how kefir varies (and doesn’t vary) worldwide

• My Personal Science Guide to the Microbiome, including detailed kefir and kombucha experiment protocols and results

• Previous PSWeek posts: Bread/Sourdough, Coffee Metabolism, Apple Cider Vinegar

Personal Science Weekly Readings

We’ve been preaching for a long time that someday a personal scientist will win a nobel prize, but now thanks to LLMs it’s looking especially likely. AI Street summarized the state of AI discovering new science and says it’s looking pretty good.

At the national level, the United States has just launched the Genesis Mission, an attempt to build a scientific infrastructure in which AI helps drive simulation, data analysis, and experimental workflows. At the system level, platforms such as FutureHouse Kosmos and AI Scientist-v2 automate increasingly more of the research workflow. And at the level of active scientific projects, frontier models such as GPT-5 are already contributing concrete, verifiable steps across mathematics, physics, astronomy, biology, and materials science, as documented in OpenAI’s Early Science Acceleration Experiments.

About Personal Science

My friend's dramatic improvement from homemade kefir and my own measurable-but-imperceptible microbiome changes aren't contradictory—they're exactly what personal science predicts. Don’t rely on population-level studies that claim some affect; do your own experiment to see how it affects you.

Fermentation is an ideal way to start because the feedback is fast, the stakes are low, and the techniques are Lindy-tested. If something has kept humans alive for thousands of years, be skeptical of anyone telling you it's dangerous—but also be skeptical of anyone promising miracles. The only way to know what works for you is to try it.

Have you experimented with fermented foods? Questions or suggestions? Let us know.

This week we’ll explore the surprisingly wide allowable error margins in standard blood tests, and what this means for those of us who track trends.

Back in PSWeek250904 we mentioned the best deal in blood testing: as low as $142 for a comprehensive 100+ biomarker lab draw available through SiPhox and LabCorp. That offer is unfortunately no longer available—SiPhox has since decided to focus exclusively on at-home tests—but hopefully some of you, like me, got yours in time.

As long-time readers know, I’ve done regular at-home blood testing with SiPhox (PSWeek221117), phlebotomist-drawn tests from LabCorp (PSWeek240125) and Quest, a whole series with Function Health (PSWeek240912), and various at-home cholesterol tests (PSWeek250213).

But how much can I trust these results?

The CLIA Reality Check

My new lab report says my testosterone is 830 ng/dL and my triglycerides are triglycerides of 63 mg/dL. Both are considered excellent numbers for my age, but there’s a huge caveat: I must assume the values are accurate. That seems reasonable: after all, these results come from regulated medical laboratories using sophisticated equipment.

But the federal regulations governing lab accuracy are far more permissive than you’d expect.

The Clinical Laboratory Improvement Amendments (CLIA) set the standards US labs must meet. This table from Westgard QC shows allowable variance for proficiency testing that took effect January 1, 2025:

If my testosterone had been slightly lower, say 400 ng/dL, the actual value could be anywhere from 280-520 and the lab would still be CLIA-compliant. That’s a range spanning “clinically low” to “perfectly normal.”

How many people come back from a single blood test thinking either (1) everything’s great, or (2) I’m in trouble — when the real issue is simple testing error!

Lab vs Lab

The situation gets messier when you compare results from different labs.

As we noted in PSWeek231109, different labs can legally report wildly different values while still being CLIA-compliant. Vitamin D is notoriously variable: one lab might report 30 ng/mL while another reports 50 ng/mL for the same blood sample. The main criterion for certification is consistency within the same lab, not measurement against an objective external standard.

So if you switch from SiPhox to Function Health, or from Quest to LabCorp, you can’t compare the numbers directly. Your carefully maintained spreadsheet essentially starts over.

Actionable Thresholds

Doctors make treatment decisions based on cutoffs. Statins, testosterone replacement, thyroid medication—these are all prescribed based on specific levels.

But if that LDL measurement could legitimately be 135 or 165 (with a reported value of 150), you might be on either side of a treatment threshold purely due to assay variance.

Worse, this variability applies to research labs too. If a peer-reviewed study makes a health claim based on Vitamin D levels under 30 ng/mL being “low,” you can’t compare your own results unless you know precisely which lab did the analysis.

In fact, this high variability has me questioning the entire premise of Vitamin D supplementation. (see PSWeek251210)

The Trend is Your Friend

The solution is that you need to track biomarkers over time rather than obsess over single values.

1. Use the same lab, consistently. Different labs may give systematically different results. Stick with one provider to maximize comparability.

2. Don’t panic over small changes. A 15-20% change in most biomarkers is within the noise floor. Look for large, persistent trends.

3. Establish a baseline range. Consider 2-3 measurements under similar conditions before attributing changes to interventions.

4. Know which tests are precise. Glucose, HbA1c, and hematology values are reliable. Hormone tests and inflammatory markers are not.

5. Focus on effect sizes that matter. Design experiments with interventions likely to produce big effects—or accept you’re collecting hypothesis-generating data, not definitive proof.

6. And finally, compare yourself to yourself, which is often more informative than comparing to population averages. (see PSWeek240509)

Here’s what happens when I asked Claude to regenerate my data using the error bands that my labs should have put there.

My actual results are somewhere in those shaded bands and in this case, you can tell that I was doing some kind of intervention in the top chart, where the triglycerides go up and down, versus the bottom chart where my overall levels are essentially unchanged.

Personal Science Weekly Readings

Speaking of how to get better results from imperfect data, What If Everyone Knew Which Science to Trust? by Paul Litvak introduces a new project to create AI tools that will systemically go through academic papers and “peer review” them according to the rigorous unbiased standards that only an AI can do.

evidence.guide hosts an API that turns a PDF into structured (JSON) data with fields like p-value, number of experiments, etc. making it easier to automatically compare studies.

refine.ink is a $50 site that “devotes hours of compute to help you find and fix the issues that matter most to readers and reviewers.”

Speaking of bad research, I was intrigued by a new paper that was summarized by Eric Topol as “Multilingualism and Extending Healthspan”. The idea is exciting: learning another language makes you live three years longer! The paper is based on an analysis of almost 90K people in Europe, which sounds impressive until you see their methods. Their definition of “multilingual” is basically self-reported, so people who studied Spanish in high school could count.

Smithsonian compiles its list of Ten Celestial Events for 2026 including a lunar eclipse on March 3rd and a solar eclipse partially visible in Alaska, Canada, and the Northeastern US on August 12

About Personal Science

Personal scientists use the tools and methods of science for personal rather than professional reasons. That includes understanding the limitations of those tools, not just their capabilities.

Lab tests are powerful instruments, but they’re noisier than most people realize. The confidence doctors express about precise numbers (”your B12 is 412”) is often epistemically unjustified. Knowing this doesn’t mean we should stop testing—it means we should interpret results with appropriate humility.

We publish this newsletter each Thursday for anyone who prefers thinking for themselves. If you have other topics you’d like to discuss, please let us know.

This week I’ll briefly summarize a few of the devices I saw that might be interesting to personal scientists.

The annual Consumer Electronics Show is a zoo. For a city that lives on hosting big conventions, it’s the biggest one of the year with almost 150,000 attendees, every one of whom needs a place to stay, something to eat, and a way to get around. It’s the first time in my life I’ve experienced a cellular network outage; the Uber app actually told me its network was over capacity and I’d have to retry later. Unbelievable.

All those people descend on Las Vegas to glimpse the latest-greatest consumer products, as thousands of aspiring companies bring their best for three grueling days of demos and talks. I’ve been to CES a few times, so I already knew it’s more efficient to stay home and watch the news coverage, where you can see the same companies and demos captured by people paid to attend and summarize the best of what they see. Sadly, this year I had to bite the bullet and attend in person. After crawling through hundreds of exhibition booths spread over dozens of football-sized show floors, I know I barely scratched the surface—so my summary here is just a brief look at a few products that caught my eye as being of interest to personal scientists.

Bionic Legs: The E-Bike for Walking

The most impressive product for me was the Dephy Sidekick, a powered ankle wearable that comes with special shoes to augment your walking. At around $4,000, it’s not cheap, but the company has spent years selling to rehabilitation facilities and is now pivoting to consumers. It’s basically the e-bike equivalent for your legs: not replacing your own effort, just giving you a boost when you need it.

Workers on their feet all day, hikers wanting extra range, anyone with aging knees who wants to stay active. When you try it, your first reaction is a little like floating: with each step you feel a little push-back that makes it feel like you’re not really walking. The company claims the batteries last for four hours.

Another company, Dnsys, went bigger with full exoskeleton legs under the tagline “Upgrade Humans.” They had a walking track at their booth where attendees could try them out. These use motors strapped to your knees and thighs, reducing strain when walking or climbing. I can imagine these becoming popular for long-distance hiking.

And speaking of turning into robots, CES was covered with companies that basically want to automate everything in the physical world. I admit feeling a little sad to see the regular exhibit hall janitorial crew working amid the high-tech gadgets that you know are coming for their jobs.

Personal Hygiene Tech

Several booths focused on reimagining everyday hygiene routines, some more plausibly than others.

Y-Brush is an $80 French product that promises to brush your teeth in 10-20 seconds using a Y-shaped silicone mouthpiece that fits over all your teeth simultaneously. Sonic vibration does the work. Sounds appealing—makes brushing faster and easier—though I think they’ll have a hard time getting people to change such a basic habit.

One product that I hope takes off is the $200 Prinker portable temporary tattoo printer. I mean, if you really want to get a tattoo, go for a trial run first. And this one has another advantage: cosmetic applications. Color-matching for skin imperfections, scars, or vitiligo. For personal scientists interested in non-permanent body modification, this seems like a good place to start.

I saw several vendors selling products to turn your bathroom into a private biolab. Pondo offers a toilet-mounted AI device that analyzes stool and hydration levels, delivering insights to your phone—”your gut feeling, now backed by gut data.” Starling Medical’s “Pete” does something similar for urine, optimized for UTI detection, at around $200-300 with insurance often covering the cost.

Speculative Science

Every CES has products that blend ancient practices with modern technology, hoping the combination yields credibility. Two caught my attention—not because I necessarily think they work—but because personal scientists might actually be positioned to test them.

IriHealth from Korea offers AI-powered iris analysis based on iridology, a 3,000-year-old practice claiming the iris reflects the health of internal organs. They claim to have trained their image analysis algorithm on thousands of iris images, yielding a useful diagnostic signal. I’m skeptical — the practicing iridologists I’ve met seem more like astrologers to me—but the CEO seems to know his stuff on both biology and AI, so I’m open-minded.

RaDoTech takes a similar approach with acupuncture meridians. Their handheld device claims to assess internal organ health by scanning “bio-electric current” at acupuncture points. They won a CES Innovation Award, which tells you something about how those awards work. I have the same personal science skepticism here : ancient practice plus modern gadgetry doesn’t equal evidence. But again—if a somebody tracks RaDoTech readings against validated health markers over time, we’d learn something, even if that something is “this doesn’t work.”

Other Notable Finds

A few quick mentions:

Withings Body Scan 2 ($600) continues that company’s evolution from simple smart scales to comprehensive body composition analysis: segmental measurements, vascular age estimation, even nerve health assessment. Oh, and like everything else these days, it’s advertised as a “longevity” product.

Elemind offers a $400 acoustic stimulation headband for sleep optimization. It detects when your brain has fallen into deep sleep and then it fires an acoustic signal that claims to enhance your sleep. They claim 15 years of research and clinical trials showing 76% of participants fall asleep significantly faster.

Vagus nerve stimulation devices were everywhere—at least three vendors (Zenowell, Pulsetto, OhmBody) targeting stress, sleep, pain, and focus. It’s a crowded category with real science behind it, though the consumer claims often outrun the evidence. I’ll cover VNS separately in a future post.

And finally: Pebble is back. The beloved open-source smartwatch platform has returned with new models.

Healthtech Sessions

CES also features tons of keynotes and talks, but the best one I attended was “Always On: How Continuous Health Data is Transforming Care," featuring CEOs from Dexcom, Oura, and Rimidi. As you’d expect, despite these companies all doing data collection, they emphasize the importance of actionable takeaways. Tom Hale of Oura argued that wearables save clinicians time by enhancing patient self-awareness; when people arrive already informed about their own patterns, doctors can ask better, targeted questions. I liked the responses when somebody asked the panelists for sensor data they wish they could collect: blood pressure, kidney function, and hormones continuously. In general, good sensors are about "making the invisible visible." But the best advice came at the end: "DWYMTY"—Do What Your Mother Told You. Eat well, sleep, exercise. All the biosensors in the world won't help if you ignore the basics.

About Personal Science

Personal scientists believe in using the tools and methods of science for personal rather than professional reasons. We’re curious about the world around us and prefer to run our own experiments when possible, always maintaining healthy skepticism about expert claims, no matter the source.

CES is designed to overwhelm you with novelty and promise. For personal scientists, that just means more questions: What can I actually measure? How would I know if it works? And am I willing to be the experiment?

We publish this short newsletter each Thursday for anyone who likes to use science in their daily lives. If you have other topics you’d like to discuss, please let us know.

Leave a comment

This week I tried to measure my own Google effect as I speculate on what LLMs will do as they become more integrated in our lives.

The Original Google Effect

In 2011, psychologist Betsy Sparrow and colleagues published a provocative paper in Science called “Google Effects on Memory.” The findings were elegant and slightly alarming:

1. When we expect to look something up later, we don’t bother remembering it. Participants who believed trivia facts would be saved had significantly lower recall than those told the facts would be deleted.

2. We remember WHERE better than WHAT. People were better at remembering which computer folder contained information than remembering the information itself.

3. Hard questions prime us to think about search engines. In a Stroop-style task, participants were slower to name the color of tech-related words (GOOGLE, YAHOO) after answering difficult trivia questions—suggesting their brains had already turned toward computers as the answer source.

Sparrow called this “transactive memory”—the same phenomenon that happens in couples where one spouse remembers birthdays and the other remembers directions. The internet had become our cognitive partner.

The Replication Problem

Before we panic about digital amnesia, an important caveat: the “Google Stroop” effect failed to replicate in a major 2018 study published in Nature. The researchers couldn’t reproduce Sparrow’s findings.

This doesn’t mean cognitive offloading isn’t real—the save/delete memory effects make clear intuitive sense and have been more robust across studies. But the specific mechanism remains contested.

This is why n=1 experiments matter. Population-level effects may wash out individual variation. Maybe the Google Effect is stronger in some people than others, depending on how much they’ve internalized search engines as cognitive partners.

Testing It On Myself

Rather than just read about the Google Effect, I asked Claude to build me a self-experiment tool based on Sparrow’s original methodology. Because I want to be able to compare my results to those of the original paper, I specifically asked that the test include similar tasks.

• Think about 20 trivia statements (half “saved to folders,” half “deleted”)

• 90 seconds of math problems to clear working memory

• Free recall: write down what you remember

• Recognition test: old vs. new statements

• Location memory: which folder held each saved item?

• Stroop task: name colors of tech words (CLAUDE, CHATGPT, GOOGLE) vs. neutral words after hard vs. easy trivia questions

The whole test takes about 10 minutes.

Results

My results were ambiguous. Although in one attempt, the tech words slowed me down more after hard questions—consistent with the Google Effect hypothesis—in other attempts the results were negative.

I’m guessing that it’s a problem with the test itself. I’m trying too hard to replicate the design of the original paper rather than think from first principles about the effect I really want to measure.

Take the test yourself and see what you find.

From how to where to what

The idea behind the “Google Effect” is that technology has replaced the way we remember specific present-at-hand facts we need about our world. Instead of storing the fact itself, we store where to find that fact.

Google Maps changed how we get from one place to another because it’s just better than we are at knowing all possible routes and traffic. When we want to go someplace, we no longer care how we’ll get there; we just remember where we’ll get the directions: the Maps app. (I doubt most young people these days could find their way home from the grocery store if something happened to disable Google maps).

As I use LLMs more and more, I notice myself taking another step where I no longer even bother remembering where to look for an answer. I know I’ll be using the LLM, and I trust it’ll give me the answer. The only question is what I’ll do with the results.

Back in PSWeek250227, we wrote how LLMs have shifted the balance of personal science power from knowing how to do things to knowing what to do in the first place.

• Google offloads the facts: recall, directions, phone numbers

• LLMs offload the HOW: reasoning, synthesis, expression

If machines handle the HOW, what remains uniquely human? I think it’s the WHAT in a different sense—knowing what questions to ask, what problems matter, what to do with the answers.

Before LLMs, I never would have thought to ask “what’s the ratio of unmarried men to women in Seattle by neighborhood?” The question wasn’t worth the research effort. Now I can ask anything—and the question-space itself has expanded.

Maybe we’re not getting dumber. Maybe we’re reorganizing what we consider worth knowing.

But we still have to decide what we want to do in the first place.

Personal Science Weekly Readings

Speaking of LLMs and their affect on our brains, several groups are actively trying to investigate whether LLMs are making us dumber (eg. the 2025 MIT “Brain on ChatGPT” study) . But these studies tend to be full of researcher bias, and LLMs are changing so quickly that I’m not sure I believe any of the results.

Instead, I think it’s better to focus on the more timeless ideas of philosophers who have been pondering this problem for a long time. See Extended Mind Thesis - Clark & Chalmers 1998.

Speaking of other things to track, did you know you need a federal permit to put a band on a wild bird. There’s no fee, but you have to renew every three years. It applies to almost any bird you can think of. But the good news is that this enables a national database showing where all the birds are. Here’s a map of American Robin

But there are no regulations on banding an insect (like butterflies)

Cellular Technologies will sell you a $175 BlūMorpho – Ultralight 2.4GHz Radio Tag that fits on a butterfly wing and can be detected from 300 feet away with special bluetooth receivers. For $2/month you can register your tag and get notified when it shows up on their network of sensor stations.

About Personal Science

Personal scientists approach their own biology and behavior with the same rigor professionals apply to research questions: careful observation, controlled experiments, and healthy skepticism.

We’re particularly interested in what happens when you test things on yourself rather than waiting for the population-level studies. Sometimes the effect is real but too small to matter. Sometimes it’s real for you but not for the average person. And sometimes the act of measuring changes the behavior more than any intervention.

If you tried the Google Effect test and want to share your results, let us know. We publish these weekly updates for anyone who prefers nullius in verba — take no one’s word for it.

Leave a comment

To kick off 2026, here’s a look at what’s in the hopper—ideas I’m actively researching and will develop into future posts as they mature.

Testing & Measurement

SiPhox results incoming: Regular blood testing is one of the most insightful diagnostics a personal scientist can do and it’s time for a comprehensive update. (See PSWeek221117). I recently completed another round of SiPhox testing and will report the results soon—still waiting on answers to a few questions about interpretation. Between that and my previous Function Health panels (PSWeek240912), I have enough comprehensive blood data that I’m not planning to try the newer competitors (Mito Health, UltraHuman). Unless somebody convinces me otherwise, the marginal information isn’t worth the marginal cost.

Grip strength with calipers: I’ve been playing with digital calipers to measure wrist/grip strength—a surprisingly robust longevity biomarker. Expect a post on methodology and what the numbers actually mean.

Nitric oxide test strips: Someone pointed me toward at-home NO measurement, and I have test strips waiting to be tried. Nitric oxide is involved in everything from vascular health to exercise performance, but I’m skeptical about what a simple strip can actually tell you. We’ll see.

Reaction time as cognitive biomarker: Following up on the Seth Roberts BRT work (PSWeek250703), I’m collecting more data on what actually moves the needle on cognitive performance. Alex Chernavsky’s intriguing results with Metformin have me excited about trying more things for myself. Sleep timing, specific supplements, stress—all candidates for systematic testing now that LLMs democratize the tools and analysis.

and speaking of cognitive performance, I’ve been playing with a bunch of vagus nerve stimulation tools (see PSWeek250625), so this will be the year that I put them to a serious test.

AI & Personal Science Tools

Doubling down on Claude: I continue to use ChatGPT and Grok a fair bit, but this year I’m going deep on Anthropic’s tools, especially Claude Code. Of all the frontier models, Claude feels the most cutting-edge for the kind of work personal scientists do—iterative reasoning, careful analysis, and the ability to work through complex problems step by step. I expect significant improvements throughout 2026 and will continue to track them closely.

Beyond Claude, I’m watching several AI tools relevant to personal science: FutureHouse for literature synthesis and hypothesis generation (see PSWeek250612), Elicit for academic paper analysis (PSWeek240222), PodSnacks to keep up with the torrent of interesting podcasts (PSWeek251120), and the various “Deep Research” modes now appearing in ChatGPT, Perplexity, and Grok (PSWeek250227). The landscape is evolving fast, and I want to keep up on what actually proves useful versus what’s just impressive demos.

The bigger question: AI agents that can design and iterate on experiments. For personal scientists, the dream is an AI that proposes hypotheses, designs n-of-1 protocols, analyzes results, and suggests next steps. We’re not there yet, but the trajectory is clear.

Nutrition & Metabolism

Home Cooking: My experiments with home breadmaking (PSWeek250911) raised bigger questions about why something so fundamental to civilization could possibly be unhealthy, as many health gurus claim. Are there other examples of traditional foods that, with some tweaking, might be healthier than we think? How about different varieties of rice, potatoes, or the shelf of fats like butter or lard?

Fermentation: I’ve been making koji, sourdough, and of course kefir, but maybe this year I’d like to try kombucha again. And maybe some homemade “farmer” cheeses.

The “attention as active ingredient” hypothesis: A theme that keeps emerging—whether it’s tracking apps, dietary frameworks, or biohacking protocols—is that structured attention itself might be doing most of the work. The measurement creates the behavior change. I want to explore this more systematically.

Philosophy of Personal Science

The Google Effect and instant answers: What does it mean to live in a world where any factual question can be answered in seconds? This isn’t just about convenience—it fundamentally changes how we should think about knowledge, expertise, and learning. When answers are free, what becomes valuable? When you can look up anything, what’s worth memorizing? How should we move through life differently when uncertainty about facts is optional?

Books vs. LLMs—a personal reckoning: My book-reading has plummeted since LLMs arrived, and I want to understand why. Is it that I’m getting the same information faster? That dialogue with an AI is more engaging than monologue from an author? That I’m losing something important—depth, serendipity, the slow accumulation of a thinker’s worldview? Or is this simply a more efficient way to learn, and nostalgia for books is just that—nostalgia?

Philosophy of science, properly: I want to go deeper on epistemology this year—how we know what we know, the structure of scientific revolutions, the limits of empiricism. I’ve already explored Strevens’ The Knowledge Machine (PSWeek221229) Hasok Chang (PSWeek251023), Heidegger (PSWeek240822) and others. Lately I’ve become interested in Bas van Fraassen, who Redditors rank as one of three most important living philosophers. The question is whether I’ll learn more from reading the primary texts or from lengthy investigations with Claude. Probably both, but the ratio is an open question.

Expert disagreement and epistemic humility: We’ve discussed replication crises, citation chains built on sand (e.g. PSWeek250723 on the spinach/iron myth), and the limits of peer review. But the bigger question remains: how should personal scientists decide who to believe when credentialed experts disagree?

Ideological Turing Tests for health claims: Personal scientists are open-minded but skeptical. Unless you understand all sides of an argument, you don’t understand anything (See PSWeek230824). There’s a lot more we can do to expand the range of interesting and non-intuitive ways we can approach controversial subjects.

What’s Not in the Hopper (Yet)

Some topics that keep getting suggested but haven’t made my active list—either because they’re outside my expertise, my risk tolerance, or simply not where my curiosity currently leads:

Women’s health and hormones: At-home hormone testing (like Eli Health’s Hormometer for cortisol and progesterone) is increasingly accessible, and there’s real personal science potential for tracking cycles, fertility, and menopause. But this isn’t my area. If a reader wants to collaborate on a guest post about women’s health self-tracking, I’d love to hear from you.

Rapamycin and other longevity pharmaceuticals: Interesting but I need better ways to figure out if they’re actually working. I know the current set of bioage measurements don’t cut it (See PSWeek250327).

Psychedelics for cognitive enhancement: Same concern—too much regulatory and personal risk for me. (See PSWeek240822)

Genetic engineering DIY: Fascinating (the Bionomica Labs stuff), but I lack the wet lab skills.

Guest contributors: Many of our readers are doing experiments that are beyond my expertise and interest. If you’re doing rigorous self-experimentation and want to share your work, let us know. This could be anything from the topics above to something entirely different. The only requirements: genuine curiosity, honest data, and a willingness to be proven wrong.

About Personal Science

The motto remains: Nullius in verba—take no one’s word for it.

We’re skeptical about everything, including our own conclusions. We listen to experts but think for ourselves. We run our own experiments because waiting for large-scale studies means waiting forever for questions that matter only to us.

Personal Science Week is delivered each Thursday to people who use science for personal rather than professional reasons. If you have topics you’d like to see covered—or experiments you’re running that deserve attention—let us know.

Here’s to a year of curious, skeptical, open-minded experimentation.

What’s in your hopper for 2025?

Leave a comment

Personal scientists are allowed to think a little more open-mindedly, so in honor of Christmas, this week we’ll make the case for why it’s okay to be a believing Christian and still remain true to science.

The History of Science and Religion

Virtually all great thinkers we associate with Scientific Revolution and well into the Industrial Age (Descartes, Galileo, Newton, Maxwell, Mendel) were practicing Christians, as sincere in their religious beliefs as they were in their professional work. In fact, many of them considered their scientific inquiries to be a direct consequence of their theological ones. When the Protestant Reformation exposed different interpretations of traditional beliefs about scripture, many thinkers turned to the natural world of God’s Creation as a way to read additional clues that could settle theological arguments. A personal God who made us in His Image surely must have designed His Creation in a way that will afford answers to those who seek diligently.

Contrast this with

• Islamic occasionalism: God directly causes every event; no “laws of nature” to discover

• Greek/Chinese cyclical cosmos: everything repeats; no concept of progress or discovery

• Animist worldviews: nature is populated by capricious spirits, not governed by laws

Of course, after Darwin the picture became more complicated, but it’s important to remember Darwin himself was immersed in a Christian world. His alma mater, University of Cambridge, required all students to take an Oath of Supremacy; ability to read and comment on the Gospels (in Greek!) was a requirement for graduation. He almost certainly knew more about the Bible and Christianity than you do!

You may know that in the US all the top Universities began as divinity schools, but did you know that well into the 19th and 20th centuries they required all students to attend chapel on Sundays? Harvard ended compulsory daily prayers in 1886, but Yale didn’t follow until the 1920s. Brown and Princeton were among the last Ivies to require chapel: Brown ended compulsory chapel in 1959 and Princeton phased out the last mandatory Sunday services by 1964.

In other words, whatever the personal beliefs (or anti-beliefs) of most important scientists until well into living memory, they all had significant Christian exposure.

On the other hand

Modern science would be incomplete without the Enlightenment, many of whose key thinkers famously positioned religion (especially Christianity) as outdated and even harmful: Voltaire, Hume, Gibbons, Thomas Paine. Even those who weren’t explicitly anti-religious were comfortably secular, hiding behind “Deism” (God might exist but he doesn’t interact with the world) or what today we’d call agnosticism. Thomas Jefferson, for example, was able to write about rights “endowed by their Creator” in the Declaration of Independence, but downplayed any specifics.

Many, perhaps most modern thinkers go further. If you’re up-to-date on popular science, you’ll have no trouble rattling off all the reasons why religion is bad. For some people the term itself is a synonym for “superstition”, with additional suspicions thrown in: **bigotry, oppression, closed-mindedness.

But I wonder how much of this is a generations-long reaction to the overwhelming hold that Christianity had on everything in western society.

Aaron Renn describes Christianity in America as proceeding in three phases over the past fifty years. Until the 1990s, it was a positive, where public officials needed to pay homage to their Christian beliefs whether or not they were sincere. By the early 2000s, perhaps in reaction to the George W. Bush presidency as well as the perceived excesses of the Moral Majority years, American elites shifted to Christianity as a neutral force. It’s fine if you’re Christian, fine if you’re not. By the 2010s and especially after the rise of Trump, Christianity is seen as a negative. Self-respecting elites gain prestige by being actively anti-Christian.

I wonder if the same thing happened in Academia a generation or two ago before this wider political shift.

Compared to what?

Early 20th century Progressives, animated by Darwin’s theories of Natural Selection, were proud to use the language of science to reject many long-standing Christian principles, including the fundamental dignity of every human being. By actively rejecting religion as “anti-science”, eugenics must have seemed a natural consequence of their reason-based, scientific approach to crafting a better society.

Go down the list of your favorite Progressive forbearers: Jane Addams, Irving Fisher, David Starr Jordan, Theodore Roosevelt, Woodrow Wilson, John Maynard Keynes, all the founders of American economics (and today’s prestigious AEA), sociology, and revered legal scholars like Oliver Wendell Holmes. Every one of them was a proud eugenicist — and proud that they were fighting the Catholic Church, which condemned eugenics from the start.

The scientists who developed the birth control pill didn’t care that they were violating long-standing Catholic teaching against contraception; their first clinical trial was conducted on unsuspecting poor Puerto Rican women. One of the only groups to publicly oppose the Japanese internment in WW2 was the Baptist Church, whose missionaries were the few white Americans with direct interaction with them.

In a world plagued with scientific fraud, would it be a bad thing to live in a society where everyone attends a one-hour lecture on morality and ethics each Sunday?

Personal Science Weekly Readings

Speaking of religion, see PSWeek241017 where we wrote about our visit to a Creationist Theme Park. Paid Subscribers can also read our Unpopular Science posts for even more controversial takes on science and religion.

This is a good week to read books, so here are a few to consider:

Steven Pinker’s Enlightenment Now is the definitive case for why science and religion should have nothing to do with one another. Although I’ve been one of his biggest fans since his first book The Language Instinct and The Blank Slate, I thought this book was poorly argued, misrepresenting Nietzsche for example and ignoring the much Bad Stuff done in the name of the Enlightenment (e.g. French Revolution, Communism, Fascism)

Charles Freeman The Awakening is a textbook-level look at the role Christianity played in both the decline of the Greco-Roman world and later in what he calls “The Reopening of the Western Mind”. He argues convincingly that many of the early scientific thinkers were less influenced by Christianity than many scholars claim.

Rodney Stark is a sociologist who debunks many of the common stories you’ll hear about historical Christianity. Bearing False Witness: Debunking Centuries of Anti-Catholic History warns that many of our history narratives are passed down from 16th century (Protestant) European countries who were at war with (Catholic) Spain-Italy-France and tended to embellish stories about Galileo vs. the Church (the Church was fine with heliocentrism but after the Reformation they weren’t fine with Galileo’s politics) or the Inquisition (which had fewer actual executions for heresy than Britain at the time).

Merry Christmas

There are other religions, of course. You’ve probably attended a scientific meeting that began with a word of prayer land acknowledgement or reading from Holy Scripture a code of conduct. But today, of all days, it should be okay to post about the one religion that has undeniably had the closest connection to science of all.

Good personal scientists are skeptical about everything, and of course all religions—including Christianity—are a rich target. But it’s important to be humble about what we don’t know. Remember that people way smarter than you have struggled with every possible scientific objection and many of them came away as firm believers.

You don’t have to be a Christian to be a good scientist, but you don’t have to be anti-Christian either.

About Personal Science

If you studied science in college and grad school, you might now work as a professional scientist, planning and executing experiments, publishing papers and attending conferences. That’s your job.

Personal scientists use the same tools and ideas as the professionals, but we turn science back at topics of direct interest to our daily lives.

We publish every Thursday. If you have other topics you’d like to discuss, please let us know.

Leave a comment

This week I’m testing a new sleep app that doesn’t require a wearable. Along the way, I’ll explain why I keep tracking despite the contradictions—and why 80% of sleep improvement has nothing to do with gadgets.

The SleepScore Experiment

I’ve been trying SleepScore, the app from sleep.ai that NYTimes Wirecutter calls the best sleep app. The main selling point: no wearable required. You just point your phone’s speaker toward your bed and its “patented sonar technology”—inaudible sound waves bouncing off you— will track you throughout the night.

I tried it and it actually works! The app even comes with a test screen where you wave your hands silently in front of the phone. Cover the camera, stay quiet, and it still detects movement. Cool!

Setup quirks: My nightstand sits slightly lower than my mattress, so the first night the sonar wasn’t aimed at me and detected nothing. After repositioning, I started getting consistent results. You can try it out with the free version. For $60/year or $10/month you’ll get extras like more in-depth insights.

I’ve not been using it long enough for a rigorous review, but after a few nights of testing, the numbers don’t match my other trackers.

For example, here’s a typical night

Three apps. Three different stories. Which is true?

The Dirty Secret About Sleep Tracking

Here’s what years of tracking have taught me: none of them are reliably accurate, and they often don’t match how I actually feel. AutoSleep will report a “great” night when I woke up groggy; Apple Watch will show I was “awake” during periods I don’t remember at all.

This isn’t a SleepScore problem—it’s a fundamental limitation of inferring sleep stages from movement or heart rate. Even the gold-standard polysomnography (PSG) requires electrodes on your scalp measuring brain waves. Everything else is estimation.

So why keep tracking?

Two reasons:

First, long-term trends matter more than nightly scores. Any single night’s data is noisy. But patterns over months and years might reveal something real—how my sleep changes as I age, whether a new habit actually moved the needle, whether that supplement did anything (see examples below). In the decades I’ve been doing tracking, for example, I’ve learned that my long-term nightly sleep average (about 6.5 hrs) is remarkably stable.

Second, and more importantly: the tracking forces attention. This is a core insight of all my work on personal science. The “active ingredient” in most sleep improvement isn’t the gadget or the supplement—it’s the structured attention itself. When you’re tracking, you notice things. You start wondering whether that afternoon coffee mattered, whether your variable bedtime is a problem, whether alcohol is disrupting your REM.

The 80% That’s Just Obvious

I’m regularly surprised at how many people with sleep problems haven’t tried the basics. They’ll spend $300 on an Oura Ring while ignoring fundamentals that cost nothing:

• Consistent bedtime (within 30 minutes, every night)

• Dark room (blackout curtains, no LEDs)

• Cool temperature (65-68°F)

• No screens in bed

• Caffeine cutoff (earlier than you think—I calculated I had ~110mg still active at bedtime from an afternoon latte)

• Alcohol awareness (helps you fall asleep, wrecks your REM)

When I ran regression analysis on years of my own data using ChatGPT (see PSWeek241024), coffee was the only statistically significant predictor—reducing my sleep by over 30 minutes. Not CBD (no effect). Not most supplements. Just coffee timing. (More on caffeine genetics in PSWeek230622.)

This isn’t dismissive—it’s liberating. You don’t need expensive hardware to fix most sleep issues. You need to pay attention to the obvious stuff first.

What’s Left After the Basics

For me, after optimizing the fundamentals, one problem persists: a 3AM wakeup. I fall asleep fine, but somewhere around 3AM I’m wide awake, mind racing, and struggle to fall back asleep. This has gotten worse since 2020—now happening 5-7 nights per week.

I’ve tested various interventions:

• CBD: No effect (p=0.24, later confirmed with larger dataset). See PSWeek241024.

• ProdromeGlia: Actually showed statistically significant improvement in deep sleep. See PSWeek240111.

• Qualia Night: No noticeable difference after 7 nights. See PSWeek241003.

I’ve also found non-pharmacological approaches helpful: the 4-7-8 breathing technique (inhale 4 sec, hold 7 sec, exhale 8 sec) to lower heart rate below 60, and remembering that quiet rest (NSDR) provides roughly equivalent benefit to actual sleep—so lying there isn’t wasted time.

Will SleepScore help with the 3AM problem? Too early to say. But I’ve signed up for their sleep study, so at least my data might contribute to something useful.

Personal Science Weekly Readings

• Dr. Michael Breus has an excellent 5-minute video on what to do when you wake up at 3AM with encouraging words on why it may be less of a problem than you think.

• Rob ter Horst’s YouTube channel remains the best source for rigorous sleep tracker comparisons. (tldr; get an Oura Ring)

• For the genetics-curious: mutations in NPSR1 and ADRB1 are associated with people who need very little sleep (it’s not me)

• I previously experimented with the SleepSpace app and its founder Dan Gartenberg in PSWeek230209

• For a deep dive into using AI research tools to investigate sleep problems, see PSWeek250612 where I compared FutureHouse, Perplexity, and other LLMs on my 3AM wakeup issue

• A comparison of “Deep Research” tools across multiple LLMs is in PSWeek250227

About Personal Science

Personal scientists approach sleep the same way we approach everything: with open-minded skepticism, and a willingness to experiment on ourselves. We track not because the numbers are perfect, but because tracking forces the attention that leads to insight.

We publish every Thursday. If you’ve found sleep interventions that work for you, let us know.

Leave a comment

This week we’ll step back and think about the what science can do, and what it can’t.

The Naturopath’s Wall

Back in PSWeek250213 I mentioned that I share my mother’s lifelong high blood cholesterol, and how her sudden need for a stent operation early this year has me digging further into family medical history. So recently I joined my father on his annual medical visit, this time to a well-regarded naturopathic doctor as he reviewed Dad’s latest blood test results. I wanted to know how his advice would differ from an LLM, so I gave ChatGPT the same panel that the doctor was reviewing and received a detailed answer concluding all is well except for some borderline kidney function (eGFR), which was out of line for his age.

The doctor saw something different. Barely glancing at the eGFR, he was concerned instead about the “low” vitamin D level of 37 ng/mL (which most reference ranges say is just fine). That was the only number he needed to launch into his theory about inflammation, cholesterol irrelevance (”taking a statin is like shutting off the water to the whole house because of a leak”), and the necessity of taking Vitamin D and K2 together.

How is this doctor’s single-minded focus on Vitamin D any different from my cardiologist’s focus on blood cholesterol? Both of them treat a single blood parameter as if it’s the most important predictor of health.

On the wall hung a framed magazine profile of the doctor, from 2002, back when he was much younger. I found myself wondering: what was he prescribing twenty-five years ago. Probably not the vitamin D + K2 protocol—that wasn’t fashionable yet. Maybe antioxidants? Fish oil? The specific molecule rotates with the decades; but the doctor’s confidence remains constant.

Values in Science

Here’s the uncomfortable truth that philosophers of science have been pointing out for decades: science is not value-free. What questions we ask, what counts as “significant,” what we consider adequate evidence—all shaped by values. The naturopath and the cardiologist aren’t just interpreting the same data differently; they’re asking different questions entirely.

The cardiologist asks: “What interventions reduce mortality in large populations?” The naturopath asks: “What underlying imbalances might be causing symptoms?” These aren’t the same question, and the same blood panel won’t answer both.

This sounds like a problem. If science can’t give us neutral, value-free truth, what good is it?

But for personal scientists, this is actually liberating. We don’t have to pretend we’re doing view-from-nowhere objective research. We can be explicit: I care about my own health, my own energy levels, my own longevity. That’s the value that drives us. Unlike the professional researcher who must obscure their motivations behind grant applications and publication pressure, we can name what we’re after.

The medical professionals mistake isn’t having values. It’s not being explicit about them—and not being honest about the limits of his experience.

Escaping Nihilism

So if “science” and experts disagree, and even peer-reviewed studies fail to replicate—where does that leave us? Some people land on nihilism: nothing is knowable, believe whatever feels right. Instead, here are my favorite ways to deal with uncertainty:

The Lindy Principle: Things that have survived a long time will probably survive longer. Olive oil, fermented vegetables, walking, sleep—these have passed through the filter of centuries. By contrast, Vitamin D and statins have only decades of proponents. And don’t get me started on Rapamycin or NAD.

Chesterton’s Fence: Before you tear down a fence, understand why it was built. Before you reject religious fasting as superstition, notice it’s been Lindy-tested for millennia.

Skin in the Game: Grandmother’s heuristics beat nutritionist’s models because grandmothers had skin in the game across generations. The practices that killed people got selected out.

Or maybe the active ingredient is attention itself. Kosher laws, Ayurvedic doshas, macro counting, carnivore, vegan—wildly different content, often similar outcomes for adherents. What do they share? A framework that makes you think about what you put in your body. You’re not on autopilot.

The naturopath’s vitamin D fixation, the cardiologist’s cholesterol obsession, the biohacker’s supplement stack—perhaps they all “work” partly because they force structured attention to health. The specific theory matters less than having a theory that keeps you engaged.

The real question isn’t which evidence to trust. It’s whether your framework—whatever it is—keeps you paying attention.

Personal Science Weekly Readings

Speaking of Vitamin D, here’s your regular reminder that different testing companies can report very different values while still being compliant with regulations. See PSWeek231109 for examples of how different labs can legally report wildly different levels of Vitamin D (30 ng/mL vs 50) because the main criteria for certification is consistency within the same lab, rather than measurement against an objective standard.

Scott Alexander’s December Astral Star Codex links describes why the popular genetic testing company Nucleus might not be as quality as they claim.

So which genetic test should you get? These days, I’d just ask my favorite LLM but you can get a (slightly old) summary of what’s involved with geneticist Razib Khan’s 2022 So you want to test your DNA.

I wish every academic paper included a section like this one

I want to hear the case for why Hepatitis B vaccines should be given to infants. Unless the mother is in a high-risk group (e.g. drug user), I just don’t understand why this would be mandatory for the millions of babies born each year. The best, most recent “pro case” seems to be Hepatitis B Vaccination: A Remarkable Success Story That Must Continue but this fact-check makes me wonder how that made it through peer review or how a responsible doctor would recommend it for everyone.

Speaking of misrepresenting the truth, a recent NPR headline should have you outraged at government cuts in science, until you hear that the “researcher” is a 76-year-old PI who hasn’t done anything original in years and basically operates a factory for cheap labor from post-docs who slave away at boring unpromising tasks.

About Personal Science

Personal scientists prefer to figure things out for ourselves. We listen to experts not because they’re necessarily right, but because they’ve had more experience than the rest of us. Still, experts are often wrong and frequently disagree with one another, so whether you listen to experts or not, you’ll need to make up your own mind.

The philosophy of science reminds us that “science” isn’t a magic oracle delivering truth. It’s a set of methods—useful ones—with real limitations. Understanding those limitations doesn’t undermine personal science. It clarifies what we’re actually doing: making bets under uncertainty, tracking what works for us, and staying revisable when the evidence shifts.

Nullius in verba—take no one’s word for it. Including mine.

If you have other topics you’d like to discuss, please let us know.

Leave a comment