I Tried to Screen Drugs for Multiple Sclerosis on a Mini PC. The Tool Kept Fooling Me — and That Turned Out to Be the Point.

A few months ago I read about a team at MIT and McMaster using an AI tool called DiffDock to predict how a new antibiotic latches onto its target in bacteria. The tool was free. The code was public. And a thought lodged itself in my head that I couldn't shake:
Could I do that at home? On a cheap little computer? For multiple sclerosis?
I want to be upfront about my qualifications for this, because they're relevant: I had none. I work in technology audit. When I started, I could not have told you what "remyelination" meant. I'd never run a scientific pipeline, never touched a protein structure, never heard of the databases this field runs on. What I had was a mini PC the size of a paperback, a monthly AI subscription, and a stubborn curiosity.
This is the story of what happened. It is not a story about curing anything. It's a story about what it's actually like to do real technical work in a field you don't understand — and about the single most important thing I learned, which is that the hardest part wasn't getting answers. It was figuring out which answers were real.
A quick note on two very different kinds of "AI"
Because this trips people up, let me draw one line and hold it for the whole post. Two completely different kinds of AI show up in this story:
- ​DiffDock — the docking tool. This is the thing I was testing. It takes a protein and a molecule and predicts how they fit. It is not a chatbot; it's a specialized prediction model. Whenever I say "the docking tool" or "DiffDock," I mean this.
- ​AI assistants (like Claude) — the chatbots I worked with to actually get anything done. One assistant wrote and ran all the code. Others I used to check the work. Whenever I say "an AI assistant," I mean these.

The lottery ticket

Here was the fantasy. There are about 3,000 drugs already approved for human use — proven safe, sitting on shelves. What if one of them, designed for something else entirely, happened to fit a protein involved in MS? You could find it by computer, cheaply, and hand it to real researchers to test. This is a genuine strategy in drug discovery. It's called repurposing.
Now, I knew the odds were bad. Pharmaceutical companies have screened those 3,000 drugs against the major targets already. The chance that a guy with a mini PC finds something the entire industry missed is, frankly, a lottery ticket.
But here's the thing about lottery tickets: people win. Not often, not predictably, but if enough people play, someone eventually hits. I figured the worst case was that I'd learn a lot and find nothing — which is exactly what happened, except the "learn a lot" part turned out to be worth far more than I expected.
I nicknamed the project msdock.
The proteins: what we were aiming at, and why it matters for MS
To even start, I needed to understand what we were aiming at. So here's the crash course an AI assistant gave me, which I'll pass along.
Multiple sclerosis is, roughly, an immune system malfunction. Your nerve fibers are wrapped in insulation called myelin — think of the plastic coating on a wire. In MS, the immune system attacks and strips that insulation, and the signals traveling those nerves start to short out. That's where the symptoms come from.
There are really two problems in MS, and this distinction turned out to be the key to the whole project:
1. ​The immune system is attacking the myelin. Nearly every approved MS drug targets this — they calm, suppress, or redirect the immune system.
2. ​The bo​dy fails to repair the stripped myelin. Here's the striking part: essentially no approved drug does anything about this. Nothing on the market actually rebuilds myelin. That repair gap is one of the great unmet needs in MS — the holy grail is a drug that makes the body's own myelin-making cells get back to work.
The proteins I picked (ten in all) map onto three strategies:
Calming the immune attack: ​BTK is a signaling switch inside the immune cells that drive MS — it's the target of a hot new drug class (tolebrutinib, evobrutinib) precisely because it reaches cells inside the brain. S1PR1 is clever: it works by keeping immune cells penned up in your lymph nodes so they can't travel to the brain and cause damage — this is how the drug fingolimod works. CXCR4 helps immune cells move around the body.
Rebuilding myelin — the exciting, neglected frontier: PDGFRA marks the precursor cells (called OPCs) that are supposed to mature into myelin-makers but stall out in MS. CHRM1 (the "M1 receptor") acts as a brake on that maturation — block it, and the precursors can grow up; this is the story behind clemastine, an old over-the-counter antihistamine that actually showed myelin-repair signals in a real trial. GPR17 and LINGO1 are other brakes on repair. LPAR1 is an inflammatory signal that suppresses it.
Protecting the nerves: RGMA blocks nerve fibers from regrowing; PIEZO1 senses the physical stiffness of tissue, which influences whether those precursor cells mature.
Here's why the dream was seductive: if some existing, already-safe drug happened to hit one of those repair targets — say, release one of the brakes on myelin-making — you might get something no approved drug delivers: actual repair. That's the lottery ticket that would matter.
Setting up: the AI assistant did almost everything
I need to be honest about how little of this I did with my own two hands, because it matters for how much you should trust the result.
I didn't write the pipeline. I described what I wanted, and an AI assistant installed the software, wrote the scripts, downloaded the protein structures, and ran everything. When something broke — and things broke constantly — the assistant diagnosed it and fixed it. When results came back, an assistant interpreted them and told me what they meant. My job, mostly, was to sit in the middle: read what the assistant told me, decide whether I believed it, and tell it what to try next.
That arrangement is the whole story, so hold onto it. At almost every layer, I was relying on AI I could not independently verify. I can't read a protein structure. I can't check the chemistry. I couldn't tell you if a docking score of −1.5 is good or bad by any intuition of my own. I was, in the most literal sense, flying on instruments I didn't build and couldn't calibrate myself.
If that sounds like a recipe for fooling yourself — it is. And it did, over and over.
How do you test a tool you can't understand? (the actual method)
This is the part I most want people to get, because it's the heart of doing honest work with a tool over your head.
The naive plan is: run all 3,000 approved drugs against an MS protein through the docking tool, take the top scores, and shout "leads!" That plan is worthless, and here's why: a score is only meaningful if you first know the tool can be trusted for that protein. DiffDock will happily hand you a number for any drug-protein pair. The number might be signal. It might be noise. You cannot tell by looking at it.
So before you go treasure-hunting, you have to check the metal detector. The way you do that is beautiful in its simplicity: you bury some known coins and some bottle caps, and see if the detector can tell them apart. If it can't distinguish treasure you already know about from junk, you'd be a fool to trust it on treasure you don't. Here's what that looked like, step by step, for each protein:
1. ​Get th​e protein's 3D structure. These come from public databases — either experimentally mapped structures, or ones predicted by another AI (AlphaFold). This is the "lock."
2. ​Gather known coins ("actives"). From a giant public database of lab results (ChEMBL), pull molecules already proven in real experiments to bind this protein. These are our known-good keys.
3. ​Gather b​ottle caps ("decoys"). Pull a set of molecules matched to the actives in size and general chemistry, but presumed not to bind. Crucially, they have to be convincing look-alikes — if the fakes are obviously different, the test is too easy and proves nothing. (Getting this wrong bit us more than once.)
4. ​Run th​e docking tool on everything — every known binder and every decoy — against the protein.
5. ​Score the ​tool's performance. Did DiffDock rank the real binders above the decoys? This gets boiled down to a single number between 0.5 and 1.0. 0.5 means a coin flip — the tool is useless for this protein. 1.0 means perfect separation. Somewhere around 0.75+ and you might start to trust it.
6. ​The ki​ller test — surprise it with real drugs. Take actual approved drugs known to hit the protein that were held back from the calibration, and see if the tool flags them. This is where things got humbling (more below).
7. ​Decide the p​assing grade in advance. One hard rule I learned: commit to what counts as "good enough" before you look at the results — otherwise you'll unconsciously move the goalposts to make your data look like a win. (Everyone does this. Pre-committing is the only defense.)
8. ​Add er​ror bars. Finally, check whether the score is even real or just luck, by re-running the math thousands of times on shuffled samples. This last step, added late, quietly demolished several conclusions I'd been ready to believe.
Only if a protein passed all of this would it make any sense to run the 3,000 unknown drugs against it and take the top hits seriously. The calibration is the whole game. The treasure hunt is the easy part you only earn by proving the detector works first.

The tool kept handing me beautiful, wrong answers

Early on, I got a fantastic result. One target — S1PR1 — came back with a score suggesting the docking tool could clearly tell real drugs from decoys. It looked like the pipeline worked.
It didn't. When we ran the killer test — surprising it with actual approved MS drugs that hit that protein — three out of four scored terribly. The impressive result was an illusion: the tool had accidentally learned to recognize one family of look-alike molecules, not to understand the biology. The great score was measuring the wrong thing. We'd never have known if we hadn't held those drugs back to surprise it with.
That became the pattern of the entire project. Again and again, the pipeline produced a confident, plausible, wrong conclusion:
- ​A target "failed" for a specific chemical reason — except when we dug in, that wasn't the real reason at all.
- ​A batch of molecules appeared un-processable by the tool — until we found the actual cause was a memory glitch on my mini PC killing the program mid-run, disguised as a scientific failure.
- ​A confident explanation for a strange result turned out, on inspection, to be flatly false.
Every one of these arrived dressed as a reasonable, expert-sounding story. If I'd taken any at face value — which, remember, I had no domain knowledge to challenge — they'd have gone straight into my conclusions. Several nearly did.
What actually caught the mistakes
Here's the part I'm proudest of, and it's not a "look how smart I was" part — because I wasn't the smart part. The safeguard was structural.
Without quite planning it, the project ended up running on a loop of different AI assistants that checked each other:
- ​One assistant ran the experiments and interpreted the results.
- ​A second assistant — a separate conversation, with no stake in the first one's answers — had the job of distrusting those interpretations and demanding proof. It caught a lot, precisely because it wasn't the one that produced the work.
- ​And when that second assistant itself got something wrong and overclaimed — which it did, on the single most important finding — a third, fresh assistant, brought in cold with no attachment to the earlier reasoning, caught the overreach.
- ​I sat on top, deciding which challenges to run and when a claim had survived enough beating to keep.
That's the real discovery of this project, and it has nothing to do with MS. The honesty didn't come from me being rigorous. It came from the setup — separate AI assistants auditing each other, with a human conducting. I couldn't verify the chemistry, but I could insist that no single assistant got to be both the author and the judge of the same claim. The rigor lived in the structure, not in my expertise — which is genuinely hopeful, because a structure is something anyone can copy.
What we actually found
After months of this, here's the honest result — with the important caveat that once we added those error bars, several things I'd been ready to believe dissolved.
We calibrated the docking tool carefully against four MS proteins. Once you account for statistical noise, none of the four could be reliably told apart from the others, and none was good enough to trust for finding new drugs. The differences between them that I'd been narrating? Mostly noise. Humbling.
There was exactly one finding solid enough to survive every check, and it's strange and interesting. Inside a single protein (BTK), the tool's accuracy didn't depend on the protein — it depended on the shape of the drug being tested. For one family of rigid molecules, it was nearly flawless. For a different family of floppier ones, it was worse than random — actively wrong. Same protein, same pocket, wildly different reliability depending purely on the chemistry of the key. A single overall score hides this completely. That's a real lesson about these tools: their reliability is far more uneven, and far more hidden, than any headline number suggests.
One protein (CXCR4) was worse still — the tool didn't just fail, it inverted, reliably ranking real drugs below random ones. We confirmed that across three different structures of the protein. Even that has an honest caveat: we can't fully rule out that it's an artifact of the specific decoys we compared against, rather than the tool truly failing on that protein.
And the headline fantasy — a repurposing hit, the lottery ticket? No. Nothing. Because none of the proteins passed calibration, we never even earned the right to run the 3,000-drug treasure hunt and believe the results. Which, given everything above, is exactly what an honest pipeline should return. If I'd announced a hit, you should have distrusted it immediately.
The one thing worth sending back to the tool's creators
There was a single observation clean enough that I wrote it up as a question for DiffDock's authors — the BTK finding, where the tool places one family of molecules in a perfectly sensible spot but then rates its own good work with strangely low confidence.
I want to be scrupulously careful here, because I wasn't careful enough at first, and it's the best illustration of the whole project. My initial write-up confidently claimed this was a flaw in the tool's confidence scoring. Then a fresh AI assistant — brought in specifically to attack my conclusion — pointed out that I couldn't actually prove that. My evidence was weaker than I'd assumed, and the low confidence might be the tool being correctly cautious about a pose that only looked right to me. So the version I'm actually sending is a question, not a claim: "here's something odd I observed — is this expected, or worth a look?" That correction, forced by an AI I convened to disagree with me, is the project in miniature.
The big caveat, stated plainly
You should read everything above as an honest account of an AI-driven experiment run by a non-expert — not as science you can lean on.
I could not independently verify a single piece of the underlying biology or chemistry. Every number came from AI-run code I trusted but couldn't audit line by line. Every interpretation came from an AI assistant that was, demonstrably, capable of being confidently wrong. Even the error-catching came from other AI assistants. The entire result is only as trustworthy as that loop of machines checking each other — which caught a lot, but which I'd be foolish to claim caught everything. Two of the most important corrections only surfaced because I brought in outside review. Which means, almost certainly, there are things it didn't catch that a real expert would spot in five minutes.
That's not false modesty. It's the actual situation, and pretending otherwise would betray the whole point. Blind reliance on AI is dangerous specifically because the AI is so good at being convincing when it's wrong. The only defense I found was to never let one AI assistant have the last word — and to treat every clean, satisfying result as guilty until proven innocent.
(In the spirit of that honesty: I drafted this post with AI help too. Of course I did.)
Why I think this was worth doing anyway
For a long time, the barrier to contributing anything real in a technical field was expertise — years of training just to ask a sensible question. AI hasn't erased that barrier. But it has moved it. Someone who didn't know what remyelination meant reached the working edge of a specialized science in a matter of weeks and asked its tool-makers a question sharp enough to be worth their time.
The domain knowledge came from AI. But the thing that made the result trustworthy — the honesty — did not. That came from the setup: AI assistants made to check each other, a human refusing to accept the first flattering answer, and a hard commitment to reporting what actually happened, including the disappointing parts.
And that's the genuinely hopeful part, the reason I'm writing this at all. The new barrier isn't expertise. It's intellectual honesty. And unlike a PhD, honesty isn't gated by anything except your willingness to be honest. The proof this project "worked" isn't a cure — it's that I can hand you a clear map of exactly where this popular docking tool can and can't be trusted, and defend every number on it, including the ones I wish were better.
That's smaller than a discovery. It's also far more solid. And it's reachable by anyone willing to do the unglamorous thing: point the machines at each other, pre-commit to the truth, and check.
I didn't win the lottery. But I learned exactly how to buy an honest ticket — and I think a lot more of us could be playing.
This project was run entirely on consumer hardware with free, open-source software and commercial AI assistants. No wet-lab work was done; no claims are made about any drug, compound, or treatment. Every result here concerns the behavior of a prediction tool, not biological truth, and should be treated as a hobbyist's documented experiment rather than peer-reviewed science. If you're an expert and you spot where I went wrong — please tell me. That's the whole idea.