AI Services Market › Use cases › Games & Simulations
Games & SimulationsDone with Claude Oct 29, 2026
Play Game Boy games through an emulator
A developer built an agent where Claude watches game frames from PyBoy, decides actions and presses buttons, using memory tools and periodic summaries. · Oct 29, 2026
- What They Did
- You supply a ROM, an Anthropic API key and optionally a saved state. Claude 3.7 gets recent screenshots from PyBoy and answers with tool calls. It sends button sequences, sets a goal, and adds, updates or removes memory items. In the default turn-based mode, the emulator advances only when Claude sends inputs. Every 30 turns by default, it writes a progress summary.
- How It Went
- The README gives no results or progress numbers. It describes a working agent with frame logging for debugging. Movement is purely button-based, with no coordinate helpers.
- Worth Knowing
- Context is capped by settings: 30 messages and 5 screenshots by default. Thinking budget is 16000 tokens per turn, so API costs may add up.
Try it yourself with Claude
Build me a Python agent that plays [Game Boy game] through the PyBoy emulator using Claude's API. Each step it should send the current screen frame to Claude, get back a button press, and apply it, with a simple memory file and a summary written every [number] steps. It's done when I can run one command, watch it play for a few minutes, and see the summary log; tell me which ROM and API key setup I need to provide.