If you’re building something small, single-use, or disposable with an LLM, you can cut corners. If you’re building something ambitious, you need guardrails.

Human code review, unit tests, linting, security scans, hooks, CI - the usual moves still help. A lot of it can be automated. But none of it covers everything. Especially not on the front-end.

Why human review breaks down

LLMs produce a lot of code. Fast. And the code they produce is usually very similar to working code - statistically close to the real thing. If you rely on human code reviewers to do a final pass on the code, you might be falling into a trap: LLM-generated code often looks plausible. You can read a hundred lines of working code and miss the one subtle thing that isn’t quite right.

And the better LLMs become at writing code, the harder it becomes to spot the subtly wrong.

Human reviewers can easily be set up to be accountability sinks, set up to fail.

Cory Doctorow has an excellent essay that goes further on this. Seriously worth a read if you’re planning to use human eyes as a quality gate in any AI-powered workflow.

That said, human attention is very valuable. The trick is to build systems that amplify and support that attention. Not systems that expect humans to have machine-like attention spans.

A picture is worth a thousand words

If you are building something with a modern web frontend, the way it looks and feels is pretty hard to test. You can easily put a lot of effort into making automated tests for your frontend, and then miss the fact that it sucks to use on mobile devices because the text is really, really tiny.

Often, problems in code are not very visible at all. They only appear when you interact with the front end. You look at the thing with your eyes, click around, and find the things that don’t quite do what you expected.

Of course, automated tests help, but only so far. Playwright can check that buttons do what they’re supposed to when clicked. It can verify that menus appear and elements exist. But there is a limit.

If you build something with a frontend, you'd best look at the frontend.

Some teams have QA engineers who handle this kind of work, but it can become a pretty big, repetitive task. Manually clicking through the front end every time you want to ship is expensive in time and patience. You can’t rely on this kind of thing fully, you really shouldn’t, but it’s an important part of the toolbox.

Quick primer on spec-driven development

This post is about how I build QA functionality into my spec-driven development workflow. I realise that not everyone has heard of spec-dd, so here is a lil primer:

Spec-driven development means you don’t jump straight to code. First, you work with an LLM to write a spec — a markdown description of what needs to be built. Then you turn the spec into a plan — a concrete sequence of steps for implementing it, with references to files, tests, and code. Then you execute the plan.

A well-known example is spec-kit. I encourage you to go look at it very closely. But don’t assume it’ll work for you on your project by default. I’ve played around with a few Spec DD implementations and couldn’t get one that fitted my needs. So I decided to make my own.

I recommend you do too.

Where QA fits in

In the workflow I’m currently using, an LLM is used to create an implementation plan AND a QA plan from a spec file.

A QA plan is a markdown file. It’s a list of steps a competent human QA could follow to verify that everything works. It’s a write-up of a bunch of manual click-tests that explain things like:

• Which pages to visit

• Which buttons to click

• Which menus to explore

• Which tables should exist, and what they should contain

• What to check on different screen sizes (full end-to-end on a large screen, visual-only spot-checks on smaller ones? )

It’s written alongside the spec and the implementation plan, not after the code is done. It’s part of the design.

I can look at this QA plan and make sure it captures what the feature is actually meant to do — complicated user flows, edge cases, all the weirdness. I can use it to see if any weird assumptions have slipped in; it lets me see the LLM’s “understanding” of a feature before it gets built.

Running the QA plan

Once the code is in place and looks about right, there is a command to run the QA test plan.

The command ensures the dev server is up, then passes the plan to a QA agent. The QA agent uses Playwright MCP to drive the browser — clicking, typing, navigating through the pages — and takes screenshots as it goes. The screenshots land in the same directory as the plan.

If a step needs data in the database — say, a feature that touches a large cohort of learners — the QA agent asks a QA helper subagent to make it. The helper has access to the dev database, and it can CRUD whatever it needs to.

The QA helper subagent is encouraged to use existing factories to create the structures it needs, rather than writing long setup scripts in the codebase. I’m a big fan of factory_boy for this: it lets you create a bunch of related objects at once, and the factories end up doubling as documentation for how your models relate.

I already have a bunch of factories set up that are used in the unit tests. The QA helper subagent uses those same factories. The object hierarchies it creates are officially sanctioned rather than hacky.

You are a human…

There’s one constraint on the QA agent that’s really important: it needs to behave like a human QA tester. It should not write code, run automated tests, fiddle with the database directly or any of that stuff. It can only interact with the browser as a human would.

If you let it act like a developer, it will cheat. It will write QA tests instead of clicking around. It will run the existing test suite and call that QA. It will add tests that duplicate what’s already in place, creating rigidity and coupling, and slowing down the whole suite. All of this is faster than actually walking through a front-end and looking at screenshots, so it’s what it’ll reach for.

So tell it, plainly, that it’s a human who can’t write code. If it needs help setting up test data, it can ask the QA helper. Otherwise, its job is to walk the plan, take screenshots, and report what it sees.

This costs tokens, and it costs time. The benefits are worth it: actual pictures of how the feature behaves, across the screen sizes your users will hit, without you having to click through every page yourself. Often, the QA report surfaces problems that did not show up in the unit tests. It’s really good at spotting issues, and it’s patient enough to check every screen and interaction thoroughly.

The report

When the QA agent is done, it writes a report.

The report covers every step in the plan — passed, failed, or partially passed — with the relevant screenshots alongside. And it flags extra things the agent noticed along the way. Regressions in unrelated parts of the UI. Bad decisions elsewhere that only became visible because the agent passed through, or bugs and edge cases you didn’t consider before.

The functionality under test worked great, the report might say — but it noticed a problem with something on the next page over. Free bonus bug reports.

You can find a bunch of qa_report.md files that this flow has generated here.

Dealing with QA test failures

If the report flags a real problem with the thing we’re actually building, then we need to fix it. Usually, TDD is the way to go:

• Red: write a test that fails because of the bug.

• Green: fix the bug so the test passes.

• Refactor: Never forget this

Some minor issues can be dealt with directly, without the full TDD dance. But TDD is the default.

The plan and report as project history

The QA plan and the QA report both live in the repo as markdown files alongside the spec file. That matters more than it might sound. Old QA reports become useful reference documentation - they tell you how things are meant to behave from an end-user’s perspective.

Zoom out, and it’s true for any artefact of spec-driven development. Specs, plans, QA plans, QA reports — each one is a small record of what was built, why, and how it behaved.

Want to learn more?

I’m running a zero-to-hero spec-driven development course:
Spec Driven Django Development with Claude Code

You need a solid set of foundational skills to build effective workflows for your projects. There isn’t a one-size-fits-all solution. So this course has 2 parts:

1. An intense 2-day workshop that starts with the foundations and builds up to more advanced skills and techniques. You’ll get familiar with all sorts of tools and techniques, and you’ll see why they matter and where to use them

2. One month membership in an Agentic Django Mastermind group. The real learning will begin when you start taking your skills and applying them to real projects. This mastermind group is a place to get continued support and to interact with peers on a similar journey to you.

This space keeps changing. There is still a lot to discover. We’re building a place of discovery and mutual support.

Here is a piece of advice that I got early in my career. It really stuck with me. It changed a lot about how I think about code:

Always code as though the next developer who’ll touch or use your code is your client. There is usually somebody paying for the development. There is usually an end client, but the next developer in the chain should be treated as though they are as important.

That next developer might be your future self. Or somebody else entirely.

These days, the next developer is likely either a coding harness or somebody making use of a coding harness.

The principle still holds. But “readable” now means a harness can parse it. “Usable” means a harness can set up the project, find the scripts, and follow the conventions without you babysitting it.

What I’m building

I’m happy to report that I’ve been writing a lot of code lately. I’ve been working on an LMS — a learning management system — called FreedomLS. It’s a Django project designed to be extended by other Django projects. Think Wagtail, not WordPress: highly flexible, but to really make it dance, you need software development skills.

Other Django projects install FreedomLS and build on top of it. Everything is configurable, everything is overridable. You can cut out entire sections of the application and wire up your own stuff.

The problem: other projects need to understand yours

FreedomLS is not a simple piece of tech. It has certain technical requirements, standards, and a certain shape. So when somebody installs FreedomLS in their own project, their coding harness needs to be FreedomLS-aware — it needs to know the conventions, the scripts, the way things hang together.

Working code is great and all, but if the coding harness doesn’t know how to work with it, then things can get hairy.

My strategy: a plugin that ships with the code

Inside the FreedomLS repo, I have a directory called fls-claude-plugin. This is where the magic is. It’s a Claude Code plugin that can be installed in any project that uses FreedomLS — including FreedomLS itself.

You can see the actual plugin here. It’s still a work in progress, but most things are.

The plugin has an init command: /fls:init that sets up everything the harness needs to know. Here’s what that actually does:

• Permissions: The plugin recommends certain permissions and merges them into the project’s .claude/settings.json. If you’re using a plugin that does this kind of thing, it’s on you to be careful and make sure you’re happy with what’s been added.

• Configuration: The plugin requires certain variables — the site name, default admin credentials for development, that kind of thing. The init script creates a .claude/fls.md config file for the concrete implementation. FreedomLS has two concrete implementations in development right now — Bloom and First Class — and they each have different configurations.

• CLAUDE.md updates: The init script updates the project’s CLAUDE.md with the basics of how the plugin works.

• Wrapper scripts: The plugin contains shell scripts for what FreedomLS always needs to do. But concrete implementations are usually complicated enough to need extra steps — things before, after, or around what the base script handles. So instead of asking people to edit the plugin scripts directly, FreedomLS creates wrapper scripts in the project root that call the plugin scripts. The wrapper is where those project-specific steps go.

Here’s a concrete example. The plugin includes a script called install_dev.sh for getting a development machine set up — I use it whenever I launch a new git worktree so we can do work in parallel (I’ll talk about worktrees in another post). It looks like this:

#!/bin/sh

uv sync

npm i

npm run tailwind_build

# Set up per-branch database

./dev_db_init.sh

# Apply migrations

uv run manage.py migrate

When you run /fls:init, a wrapper install_dev.sh gets created in the project root:

#!/bin/sh

# install_dev.sh — Generated by fls plugin init

# PLUGIN_DIR is set during fls:init to where the plugin is installed

# (e.g., “submodules/Freedom-LS/fls-claude-plugin”)

PLUGIN_DIR=”?”

# === FLS base setup ===

“$PLUGIN_DIR/scripts/install_dev.sh” “$@”

# === Project-specific setup ===

# Whatever you want to do, you do here. Eg setting up some demo data in the development database.

The base script handles what FreedomLS always needs. The wrapper gives concrete implementations a place to add their own steps — initialising databases, creating demo data, whatever makes sense for that specific project.

When any Claude skill or command needs one of those scripts, it references the wrapper in the project root — @install_dev.sh rather than $PLUGIN_DIR/install_dev.sh.

How concrete implementations pick it up

For now, concrete implementations pull in FreedomLS as a git submodule — so the plugin lands as an actual folder on disk.

Each concrete project gets its own claude.sh launcher that points at the plugin directory. Run /fls:init, and the harness knows about the conventions, the scripts, the config. After that, you just use the plugin — same commands, same workflow, tuned to that project’s specific setup.

Transferrable Tricks

None of this is FreedomLS-specific.

If you’re writing a package or library that other developers depend on, a plugin like this makes your code more usable.

If you’re building a standalone tool — something with an API or a user interface that a coding harness, or other AI-based automation tool might need to interact with directly — publish the markdown files that explain how to use it right alongside the tool. That makes it more usable for the end user. If the end user is an LLM, so be it.

A summary of the tricks:

• Build a plugin alongside your project

• Make an init command to set everything up

• Make a way to configure the plugin for each project that uses it

• Create scripts in the plugin - and if they’ll need to be overridden, create wrapper scripts that can be edited in project-specific ways

This is good for humans too!

The fact that project docs are organised within a Claude plugin does not mean that those docs are not useful for humans.

Something that’s quite interesting to me in general is that, in this age of LLM-assisted coding, many software engineering best practices are actually becoming more important, not less. Documentation saves lives. People often neglect documentation, but now it’s become critically important for harnessing modern tools.

I’d argue that this approach is not only good for coding harnesses, but for people too.