Personal Science Week - 251106 Grokipedia
Which is better: Wikipedia or the new Grokipedia?
However much personal scientists want to prioritize our own first-hand evidence, we often need to rely on general sources of information. Until recently, Wikipedia was one key asset, however imperfect. Now there’s a competing encyclopedia, Grokipedia, built with powerful AI algorithms that are supposedly more neutral and unbiased than humans.
This week we tested Grokipedia to Wikipedia on a few key terms to see which is better.
Facts vs. Opinions
The term “alternative facts” is often derided as a politically-biased way to sweep away the truth under the guise of seeming to be neutral. But there really are alternative, equally factual ways to describe things (e.g. “2+2=4” and “1+3=4” are both true). Often the difference between two perspectives boils down to which facts are emphasized (e.g. the undocumented immigrant was “from El Salvador” vs. “a resident of New Jersey”).
But we personal scientists, striving to be both open-minded and skeptical, really want to get to the truth of the matter, which means following the motto of The Royal Society, established in 1660: Nullius in verba: “take nobody’s word for it.” That’s why we almost always prefer first-hand, self-collected evidence over what we read or heard from somebody else, however many credentials they have.
Still, there’s no avoiding the need to evaluate and at least tentatively trust some of what we read. Until recently, one of our most reached-for sources was Wikipedia, not because we think it’s necessarily correct (we’re skeptics!) but because it’s free and comprehensive. It might not always be reliable, but it’s a good place to start. Even when compared to LLM chatbots, sometimes it’s nice to start fresh with a well-curated summary of a topic.
Wikipedia is broken
Unfortunately, Wikipedia has been broken for a long time. What used to be the “free encyclopedia than anyone can edit” is now largely controlled by a mostly faceless team of volunteer administrators who are clearly biased. You’re not even allowed to cite sources—however factual—that Wikipedia editors don’t approve. This is why their list of perennial reliable sources rates Times of India — the most influential newspaper in India—a “yellow”, less reliable than (the openly biased) Mother Jones (green). Similarly, MSNBC is green but Fox News is yellow.
When political science researcher David Rozado studied thousands of Wikipedia entries, he showed that on almost every issue, Wikipedia uses positive, non-neutral language to describe positions associated with left-leaning people and negative language to describe the right-leaning ones.
Grokipedia vs. Wikipedia
It’s fine to have political opinions, and maybe the truth really is on your side. But personal scientists want you to be up-front about your biases, and we won’t believe you until after we’ve heard all sides.
That’s why we were happy to see the launch of Grokipedia: an AI-generated alternative to Wikipedia that supposedly applies a neutral evaluation of every topic in a way that no human editor can. But is it really neutral?
I was hoping to do a full-scale comparison, similar to David Rozado’s work, using the “vibe coding” techniques from PSWeek250814: choose, say, 500 topics, run a sentiment analysis on the two sources and plot the results. Unfortunately I ran out of time this week, so I’ll leave that as a exercise to some other ambitious personal scientist.
Instead I picked three topics I’ve been following personally: John Ioannidis (scientific biography), Lyme disease (medical), and face mask efficacy (contested science) . I placed the text entries from each source into Claude and asked it to evaluate based on (1) neutrality, (2) how up-to-date it is, (3) comprehensiveness, and (4) readability.
John Ioannidis Biography: Grokipedia vs. Wikipedia
Despite his status as one of the most-cited scientists of all time, and a long-time distinguished editor on all the top medical journals (JAMA, JNCI, Lancet), Wikipedia’s entry on Stanford Professor John Ioannidis opens with “accused of promoting conspiracy theories.” His foundational work— quantifying publication bias, calculating positive predictive values under realistic conditions—is measured against one Wired magazine writer who confidently declares that future generations will regard him as a “fringe scientist” for his opposition to COVID lockdowns.
By contrast, Grokipedia maintains professional neutrality, explaining Ioannidis’s arguments: PPV calculations, quantified publication bias, meta-analysis proliferation and much more.
Which source helps you evaluate his claims on their merits? Wikipedia seems to care more about the social controversy than it does about the scientist’s actual accomplishments. For personal scientists who just want to know the facts, Grokipedia’s technical depth is a better start. Wikipedia exemplifies the ad hominem fallacy Ioannidis himself critiqued.
Lyme Disease Evidence: Wikipedia vs. Grokipedia
On this relatively apolitical medical topic, Wikipedia narrowly outperforms Grokipedia. Both maintain strong neutrality and current data (2025 vaccine trials), but Wikipedia provides superior decision-relevant detail with specific treatment protocols.
Grokipedia has less dosing specificity. Its conspiracy theory debunking (Plum Island, RFK Jr.) adds helpful context but introduces editorial stance Wikipedia avoids.
For personal scientists conducting n-of-1 experiments or making treatment decisions, Wikipedia’s actionable thresholds (when to seek antibiotics, what tests mean at which disease stage) are more valuable than Grokipedia’s descriptive richness. Use Wikipedia for clinical decision-making; cross-reference Grokipedia for the rest.
Face Mask Efficacy Evidence: Grokipedia vs. Wikipedia
Grokipedia is the clear winner for objective, unbiased presentation of the facts. While Wikipedia shows pro-mask framing and dismisses RCT evidence as “not gold standard,” Grokipedia maintains strict neutrality, presenting null and positive findings with equal rigor.
Grokipedia provides quantitative detail essential for decision-making: effect sizes, confidence intervals, p-values, and compliance rates.
Wikipedia fragments evidence without explaining contradictions. For personal scientists weighing individual protection versus population mandates, Grokipedia delivers the analytical framework you won’t get from Wikipedia.
Bottom line: Trust No One
Interestingly, on a reasonably non-political subject like Lyme Disease, Wikipedia’s human-sourced methods are a pretty good place to start. Grokipedia (which is trained on Wikipedia) almost sounds like it was trying too hard to correct a bias that may not exist. Yes there are conspiracy theories about Lyme origins, but that’s not the main thing you care about if you want to know how to treat the condition.
But why choose? Both sites are free for everyone, and ad-free. Wikipedia has more entries (for now) and includes more helpful graphics and cross references (for now). Grokipedia is still early and getting better every day, but for now I’ll just use both.
Personal Science Weekly Readings
I’ve been addicted to Claude Code for months now and I can’t recommend it highly enough. If you’re not sure why it’s the best $20/month (or $100 like I do for Max), Lenny’s Newsletter has a nice summary of very useful tips like finding more space on your computer, figuring out why my Mac is slow, improve the quality of screenshots, and much more. Just now I got it to recommend a way to lower my memory consumption by almost 1 GB just by switching to a different app that had never occurred to me.
A serial healthtech company founder, Justin Mares, has a detailed recommendations for a healthier life with many non-obvious, actionable ideas including reasons to be more careful with your clothing choice (dyes and fibers touch your skin all day and who knows what the chemicals are doing) and magnesium for better sleep, and lots more. (The Washington Post, which is currently free to read using the (free) Comet browser, says magnesium might help )
Speaking of sleep, TheSleepDoctor.com, Dr. Michael Breus, read our post from PSWeek250612 and offered this excellent 5-minute YouTube on what to do when you wake up in the middle of the night. Top tips:
Remember that just lying in bed silently (aka “Non-sleep deep rest”) is pretty good too: an hour of NSDR is about equivalent to an hour of “real” sleep.
Practice 4-7-8 breathing (inhale 4 sec, hold 7 sec, exhale 8 sec) to lower your heart rate below 60, which is what your body really needs in order to fall asleep.
About Personal Science
Personal scientists don’t wait for professional researchers to tell us what’s true. We run our own experiments, collect our own data, and evaluate sources skeptically—whether it’s Wikipedia, Grokipedia, or Personal Science Week.
We publish each Thursday. If you have other topics that interest you, let us know.
Richard, there’s so much eye-opening info in this post. Thank you, after a quick read I’ll definitely be back to it to study in more detail.
This was useful information for me! Though I must criticize for using an AI to judge an AI. Also was Grok mainly funded by Elon Musk? Grok lets me click a link and not get sent to references which is great.
Comparing the page on composer Tchaikovsky. Wikipedia references books and PHD dissertations. Grok may be very good at finding in depth sources but at the same time it doesn't vet them as well. It cited tchaikovsky-research.net, a wiki, which may be very in depth and has lots of sources but no editorial over-site or history of being reliable. It does claim to be written by foremost historians on the subject. Wikipedia can't reference other wikis but uses offline written material (books) that Grok cant. Also Wikipedia uses magazines and newspapers which is something Grok doesn't seem to either. Grok's links lead to sites where the music can be listened to.
When looking for a page on reasoning Grok offers a bunch of specific types of reasoning not a single page on the subject.
I have an even better idea than Grok wiki. Kialo.com manages notability issues without requiring editors and so their bias. It also manages to let no one be left out or wonder if anything was left out. Detail gets put up a tree or voted very low so as not to bother those who are not looking for it. User can decide who's votes count so there is no mass or bot bias either. Same methods can be used on very unpopular ideas. An essay internet page can be built out of Kialo's outlines (usually of debates) based on preset preferences. As the user reads they can change those preferences for specific sections to go into more or less depth and controversy. Right and as a debate site Kialo courts controversy.