Spec-Driven AI Development

I haven’t read a code diff on my edtech app in weeks.

Not because I’m reckless. Because reading diffs stopped being the thing that keeps the system correct. Here’s the bet I made:

Write the requirement. Keep it alive as you build — AI makes that cheap. Regenerate the code when the requirement changes.

The second half of the bet is the part people miss: I write requirements not just for the product but for the architecture. The technical constraints live in the spec too, right next to the feature descriptions.

Combined with heavy integration tests and a mutation-testing trick I’ll get to, I’ve stopped reviewing code entirely. That should make you nervous. Let me show you why it works — and where it doesn’t.

The bet

It’s an edtech app, so the crown jewel is the learning layer. That’s the brain: everything about how a learner is assessed and advanced. The apps on top of it — about 20 of them — are supposed to be dumb. Presentational. No education logic smuggled into a button handler.

Keep the brain decoupled and you get one source of truth: change how learning works once, and all 20 apps inherit it. Let logic leak into the apps and you get 20 subtly different definitions of “correct,” discovered one bug report at a time.

So the whole game is: guarantee that a change lands in the right layer. In the beforetimes I’d guarantee that by reading every diff. At 20 apps, reading every diff is how you stop shipping.

The problem

How do I know an app stayed dumb — that it’s delegating to the brain instead of hard-coding logic or quietly reimplementing it — without looking at the code?

The trick: kill the brain on purpose

Mutation testing, but blunt. In the app’s test setup, swap the real learning layer for a saboteur — every method becomes a no-op:

// Replace the brain with a corpse: every method returns nothing.
function neuter<T extends object>(real: T): T {
  return new Proxy(real, { get: () => () => undefined });
}

const learningLayer = neuter(realLearningLayer);
// Now run the app's suite. Anything still green is a suspect.

Think about what a passing app test means once the brain is dead. The app shouldn’t be able to do anything correct — so why is the test green? Because the test mocks the learning layer, or the app hard-codes the answer, or it duplicates logic that was supposed to live in the brain.

That catches architectural leaks. For everything finer-grained — did this method actually do the thing — I run normal mutation testing on top. Two different blades: one for “is the logic in the right place,” one for “is the logic right."

"Wait, you don’t review code?”

Right. But not reading diffs is not the same as not verifying. The review didn’t disappear — it moved. Here’s where every job a code review used to do actually goes:

• Are the tests any good? Two adversarial reviewers per test, each an LLM arguing the other’s test is wrong. Any single reviewer has a failure rate, so quality oscillates — but a modern frontier model (GPT‑5.4‑class and up) can review a unit test without hallucinating it into a false pass. Then I manually test on top. Yes, I’m QA now. That’s fine.
• Security? The apps use simple token-based auth — small surface, not much for a diff review to catch that a spec can’t.
• Performance? Performance tests, same as on my other projects. They fail the build, not my eyeballs.
• Accessibility? Lighthouse, gated in CI.
• Regeneration churn? Regeneration is per requirement, and a requirement is usually a small slice — not some monster that re-emits the whole app. So I leave the existing tests in place and let the regeneration round run as TDD: the old tests are the spec the new code has to satisfy. Non-determinism has to get past a wall of green before it reaches me.

Where this bites

The honest version of the pitch: for the first few months this project ran the traditional way — AI generates, human reviews the diff, repeat. It worked, and then it didn’t, because the number of apps grew faster than my capacity to read. Spec-driven development plus tests plus mutation testing is what let me stop reading diffs and keep trusting the system.

The code review didn’t go away. It turned into writing better requirements, writing adversarial tests, and building fitness functions that scream when the architecture drifts. Turns out that’s a more leveraged place to spend your time.

But this is still an experiment in progress. The sharp edges are real: This is not a 100k-line, 100-engineer monolith. I’m not claiming this scales to a huge org. It’s one person moving fast across many small, disciplined apps.