Most of the tedious testing on this project is now done by a sub-agent we call Sleepy Hollow, the Headless Rider — it rides the game server at night so a human never has to grind the manual test loop. You hand it a bug or a question (“does this recipe still quietly do nothing when the player is carrying a spare bucket?”, “what’s the exact ingredient threshold below which this recipe stops appearing?”, “audit every hand-craft recipe in the mod for the duplication class”) and it reproduces the behaviour, A/Bs a fix against throwaway variants, verifies with evidence, and reports — no human in the loop. This is what it can do, what it’s built on, and — the more useful part — what it structurally cannot see.
What it’s built on
The agent isn’t clever in isolation — it’s clever because it’s handed a curated pile of hard-won knowledge and told to build on it rather than rediscover it:
- ~50 numbered engine rules — the accumulated “this is how Build 42’s crafting actually behaves, as opposed to how the wiki says” list. Consumption arithmetic, flag interactions, the shapes that silently deadlock, the ones that crash a headless server.
- An operating agreement — one change per test, a fresh game world every single time, read the server log before theorising, don’t over-claim “working,” escalate a scope trade-off early instead of quietly guessing a workaround. This exists because ignoring it once cost about two weeks on a single recipe.
- A running log of every bug already found — so a new investigation starts from “here are the seventy recipes already flagged and why” instead of from zero.
When it finishes, it appends its own quantified finding back into that knowledge base. The pile grows.
What it can do
- Reproduce a bug headlessly first. It provokes the reported behaviour against a synthetic server-side character, varying inputs — charge level, fluid amount, distractor items, first run versus second run — until the exact trigger and threshold are pinned to a number.
- A/B a fix without touching anything real. Candidate fixes go into throwaway recipe variants. It confirms the fix works and doesn’t reintroduce a known failure class.
- Sweep the whole mod in seconds. A full pass over a couple hundred hand-craft recipes takes about three seconds — enough to answer “did this change break anything” before the server would have finished rebooting.
- Recommend exactly one option, with the numbers inline — or say honestly that “accept this as a limitation and document it for players” is the right answer. It leads its report with the reproduction, the mechanic in one paragraph, and the single recommendation; evidence follows.
Hard limits
It exercises the server crafting path — the one that’s authoritative in multiplayer — and nothing else. Not the crafting UI. Not the client/server item round-trip. Not timed-action interruption. Nothing visual: models, placement, tinting, animations. And a dedicated server never loads the translation table at all, so any rendered tooltip or menu string still needs one human to eyeball it. These get stated in every report, not buried.
The dark spots
The hard limits are the ones you can plan around. The dark spots are the ones that produce a confident, wrong report:
- A green test can encode the same misunderstanding as the bug. One check — “does pouring out a still return the empty pot” — passed cleanly for a long time because it was asserting the wrong thing. The assertion and the bug shared an assumption, so the test agreed with the bug. A passing suite is evidence, not proof.
- “Works headless, fails in-client.” Recipe and translation files load once, at game start. A player who has been through three server redeploys in a session is running stale scripts — and the agent has no way to see that. Several “bugs” it was asked to fix turned out to be exactly this; the fix was “restart your client.”
- One player, never two. Everything runs against a single synthetic character. Race conditions in multiplayer item sync are completely invisible to it.
- The engine’s own unfinished edges. Some server-side actions add an item to an inventory without sending the client a sync packet — there is a literal “todo: handle syncing” comment in the game’s source at that spot. “The item is there server-side” and “the player can see it” are different claims, and the agent can only verify the first.
- World-state mechanics are simulated, not observed. Encumbrance, oven heat, spoilage, cook timers — the synthetic character has none of the surrounding world state, so the agent drives these directly (set the heat value, age the item by N days). That tests the mechanic’s logic; it does not test whether a real wood-fired oven in a real save ever reaches that heat.
- It can be wrong in the write-up. The reasoning is only as good as the evidence it chose to gather. The defence is to read the numbers in the report, not just the verdict line.
Worth it anyway
With all of that on the table: a bug that used to take a day of spawn-click-watch-restart now takes a couple of minutes, an exploit that hid for weeks because “works once then doesn’t” looked like user error is now a one-line diff, and a whole-mod regression check runs faster than a server reboot. The dark spots mean a human still walks the final in-client mile — but they walk it once, at the end, instead of on every iteration.