Agent-Driven QA Loop
An AI agent on the night QA shift
Put an AI agent on overnight QA: a test charter with hard prohibitions, happy-path plus wrong-thing session scripts, screenshot-based failure inspection, severity-scored triage, and a regression set that compounds weekly.
Install it in 3 steps
No sign-up. No CLI tools. Pick your AI, copy one line, paste it over.
Prefer doing it yourself? Buy the skill ($4.99), download the ZIP from your checkout confirmation, unzip it into the directory shown above, restart your AI session — same result. Already bought it? Your AI assistant can download it for you.
What's inside
One SKILL.md (the playbook) + one starter template. A preview:
Agent-Driven QA Loop
When to use
- Your build needs playtesting and your QA team is you (or is asleep).
- You want regression coverage on every nightly build without writing a full test suite.
- Bugs keep shipping that "looked fine on my machine" — visual, timing, and state bugs.
- You're already using an AI agent in development and want it pulling QA shifts too.
Workflow
- Define the QA contract (input: build → output: test charter)
Write what the agent is allowed to do: launch the build via the exact command, play for N minutes per session, capture screenshots at fixed intervals and on errors, read logs, and file bug reports — but never modify source, never touch networked services, never spend money or create accounts. A QA agent with write access to your repo is not QA, it's a liability. - Script the play session (input: charter → output: session script)
Give the agent a concrete session: start new game, play the core loop for 5 minutes pursuing the win condition, then deliberately do the wrong things (mash buttons, pause mid-animation, alt-tab, resize the window, idle for 2 minutes). Wrong-thing testing finds the crashes; happy-path testing finds nothing. Include the oracle: what "correct" looks like (target frame rate, no error dialogs, state transitions listed). - Set up screenshot inspection (input: session script → output: inspection protocol)
The agent compares screenshots against expectations: HUD elements present, no rendering artifacts (black boxes, stretched textures, z-fighting), text readable and unclipped, menus navigable. Teach it your game's specific failure signatures with 3–5 example screenshots of past bugs ("this is what the inventory soft-lock looks like"). Vision-based QA beats log-based QA for the bugs players actually see. - Run the overnight loop (input: protocol → output: bug inbox)
Schedule: build at midnight, agent plays three 10-minute sessions with different seeds/characters, files one report per distinct bug with screenshot + repro steps + log excerpt. Cap the run (timebox, max sessions) so a crash loop doesn't burn your API budget by morning. In the morning you triage, not debug — the agent already attached the evidence. - Triage with severity math (input: bug inbox → output: fix queue)
Score each bug: crashes and soft-locks first, then progression blockers, then visual glitches, then polish. For each: can a player hit it in normal play (multiply severity), and did it appear in the last build (regression = fix today)? Anything the agent found that you can't reproduce in 5 minutes goes back to the agent with "reproduce this deterministically" before it earns a place in the queue. - Close the loop into the build (input: fix queue → output: regression set)
Every fixed bug becomes a permanent check: add its screenshot signature and repro to the agent's protocol. Over weeks this compounds into a regression suite you never had to hand-write. Review the protocol monthly — delete checks for systems you removed, or the agent will cry wolf about ghosts.
Template preview
Agent-Driven QA Loop Worksheet
Set this up once; it compounds into a regression suite.
1. Test charter
- Launch command:
- Session length × count:
- Agent may: play, screenshot, read logs, file reports
- Agent may NEVER:
- Timebox / budget cap:
2. Session script
Happy path (core loop, N minutes, pursuing the win):
Wrong-thing list (button mash, pause mid-animation, alt-tab, resize, idle 2 min):
Correctness oracle (frame rate, no error dialogs, state transitions):
3. Inspection protocol
Your game's failure signatures (with 3–5 example screenshots of past bugs):
1. 2. 3.
4. Overnight run
- Build time:
- Sessions (count × length, seeds/characters):

