Building with AI · Part 4

The Dance: running a seven-app fleet with agents

Posted May 1, 2026

The question I had a few months ago was "can an agent build an app?" - the last two months answered that. It certainly "can" (especially if you're not too picky about what "build" means) but that turns out to be one of the easier parts of software engineering. Producing something that works versus something that is maintainable, secure, and performant is where the real challenge lies. AI can spit out something that appears to be a one-shot, fully-functional app, but what I most often see is a really flashy facade that falls apart under any real-world usage or scrutiny.

Now the question is evolving. AI can produce code, but it can produce too much. The question now becomes "can one person keep a fleet of apps and their shared template healthy, continuously, using agents?" The answer turned out to be yes, but only after the two most humbling weeks of this whole project. This post is about the pipeline that emerged (I've taken to calling each run of it a dance) and the three hard lessons that shaped it: kill automation you can't review, verify before you file, and never trust "it compiles."

The pipeline crystallizes

Spernakit v3.0.0 (March 16) shipped the infrastructure a fleet needs: a shared/ workspace so frontend and backend can't disagree about data shapes, config schema validation that fails at boot instead of at 3 a.m., OpenAPI-versus-frontend-type validation, and an automated template-upgrade script. Then the audit machinery got turned loose on every derived app at once. Deeper alone generated 114 findings, and the week after v3.0 produced 815 commits across twelve repositories. Real fixes, too: MFA signing-key separation, timing-safe comparisons, SQL injection hardening, 130+ accessibility corrections.

By late March the workflow had settled into a repeatable cycle: bump the template → upgrade every app → refactor drift → run the automated tester → convert bugs to features → review the features → remediate → run the full end-to-end test → back-propagate improvements → commit → sync. March 27's tester sweep across six apps in parallel surfaced thirty bugs in one session. When the same bug appeared in three or more apps, that was the tell it was a template bug. so the flow worked in reversed, too: improvements discovered in individual apps got upstreamed into the template. With a little encouragement, the ecosystem was learning from itself.

One number from March still delights me: 69 audit findings resolved in one session using five parallel agents. About 20 minutes of wall time instead of five sequential hours. Parallel agents are the unlock. But before I could trust parallelism, two things had to break.

Lesson one: kill automation you can't review

The auto-apply template-upgrade script had shipped with v3.0 and worked about 80% of the time. The other 20%, it silently clobbered files that apps had deliberately customized. On April 10, a single --apply run stripped domain code out of aidd and cascaded into ten-plus typecheck errors. On April 11, I spent roughly thirteen hours on what should have been one workflow and at the end of it I ended up deleting the script entirely from the template and all six apps.

Two details make this more than a war story. First, the replacement was slower in theory and faster in practice: manual cherry-picking through six parallel subagents takes about nine minutes per fleet pass, is deterministic, and produces diffs I can actually read. Automation that silently corrupts state is worse than no automation at all.

Second, and this is the embarrassing part, my own notes had already warned me about the --apply issue but I ran it anyway, to save time. Writing lessons down isn't the hard part but reading them back at the right moment is.

Lesson two: verify before you file

April 14 was the false-positive gauntlet. A full tester sweep filed a batch of bugs; triage rejected 5 of 12 as false positives. 42%. Testers had "observed" regressions that a curl against the running backend disproved in seconds. Meanwhile the filings that had been verified against the live API had a false-positive rate of exactly zero.

The fix was a discipline, not a tool: no bug gets filed without reproducing it against the running system. The three test agents re-dispatched with an explicit "verify with curl first" instruction produced zero bad filings. One memorable "critical truncation bug" turned out to be a 64-character token visually clipped by a 54-pixel-wide input box. The token was fine. The testers had transcribed the visible prefix and filed the resulting 404. Believe the symptom but distrust the obvious cause...and check!

The same week added a third layer: a feature-review pass that reads specs before implementation caught five concrete contradictions - an endpoint that didn't exist, a schema file a spec falsely claimed was already on disk - each one an implementation dead-end averted for the cost of one more read. Defense in depth, with each layer catching a different failure mode.

Lesson three: "it compiles" is not a gate

The quietest, most important realization of April: smoke:qc - typecheck, lint, build, format - is not the commit gate. The supertest is: reset, build the production Docker image, boot it, crawl every route, screenshot everything. The two are genuinely orthogonal. On April 25, all six apps passed smoke:qc and three failed supertest.

Supertest caught a service calling a function it never imported (smoke:qc had waved it through on a stale cache). It caught a Docker base image that had quietly stopped being published upstream. It caught a three-month-old SQLite bootstrap race that no unit test would ever reach. On one particular aidd run it took four consecutive retries to get a pass, each retry surfacing a distinct real bug, including an audit-log write per crawl request that drove SQLite into disk I/O error under load. Three of those four would have shipped silently if I'd given up after the first fix.

Two smaller April bugs earned permanent places in the lore because both were invisible in development. Bun releases the client socket before the response-logging hook runs, so audit logs recorded null IPs - in production only. The fix captures the IP at request arrival in a WeakMap keyed by the request object. And the browser-automation helper's mouse.click() fires mousedown and mouseup as a pair, which Radix dropdowns interpret as open-then-close. Weeks of "flaky" tester runs, explained by one dual-fire click. Every mysterious flake is a real bug wearing a disguise.

The signal to stop

Between April 17 and April 19, the fleet went through five full dance cycles in under 36 hours, shipping seven template versions. The fifth cycle surfaced zero bugs. Four passes earlier there had been fifty-plus. The zero-bug cycle is a signal, not a coincidence: the fleet had been driven to a local quality maximum, and the right next move wasn't a sixth lap. It was a new scope.

I need to end the churn with the template and finally stop polishing it. It's time to point the machine at the machine - aidd v2 is coming, it's outgrowing the current architecture - I started with a pretty simple bash script, afterall.