OpenGame: Open Agentic Coding for Games

TL;DR

OpenGame (CUHK MMLab) is the first open-source agentic framework for end-to-end browser-game creation from a natural-language prompt. It pairs GameCoder-27B (a Qwen3.5-27B backbone with three-stage CPT + SFT + execution-grounded RL specialized for Phaser 3) with a six-phase agent workflow and a reusable Game Skill consisting of a Template Skill (growing library of physics-archetype project skeletons) and a Debug Skill (living protocol of verified fixes). On the accompanying 150-task OpenGame-Bench (build health / visual usability / intent alignment scored via headless browser execution + VLM judging), OpenGame w/ Claude Sonnet 4.6 sets a new SOTA (72.4 / 67.2 / 65.1), beating Cursor + Sonnet 4.6 by 5.6–6.2 points and outperforming every direct-LLM baseline when paired with GameCoder-27B.

Key claims

Direct frontier code LLMs fail at end-to-end game generation in three recurring modes: logical incoherence (loss of global state across the game loop), engine-specific knowledge gaps (ignoring framework-native physics/scene/event systems and reimplementing from scratch), and cross-file inconsistencies (mismatched asset keys, broken scene wiring, missing config fields) [§1].
Game Skill = Template Skill + Debug Skill. Template Skill grows from a single game-agnostic meta template into five emergent physics-archetype families (gravity-based side view, top-down continuous motion, discrete grid logic, path-and-wave dynamics, UI-driven gameplay) by accumulating reusable fragments after each task [§3.3]. Debug Skill maintains a living protocol of (error signature, root cause, verified fix) entries plus pre-execution validations for high-frequency inconsistency classes [§3.3].
GameCoder-27B is trained in three stages: CPT on Phaser/JS GitHub repos + docs, SFT on synthetic (gpt-codex5.1-curated prompts, MiniMax 2.5-produced solutions) instruction pairs, and execution-grounded RL on single-file gameplay components where reward = unit-test pass rate [§3.1].
The agent workflow is a six-phase pipeline (initialization + Physics-First classification, scaffolding, GDD generation, multimodal asset synthesis, context-aware code implementation, verification + self-correction) coordinated by a todo_write state-tracking tool [§3.2]. Code implementation uses a Three-Layer Reading Strategy and a Template Method Pattern to override designated hook methods rather than write from scratch [§3.2].
OpenGame-Bench (150 prompts × 5 genres) scores generated projects on Build Health (BH, compiles + loads + renders without critical errors), Visual Usability (VU, frame-entropy/motion heuristic + VLM judge), and Intent Alignment (IA, weighted pass rate from VLM verdicts against a structured requirement spec auto-compiled from the prompt) — all via headless browser execution [§4.1, Metrics].
Headline: OpenGame w/ Claude Sonnet 4.6 achieves BH 72.4 / VU 67.2 / IA 65.1, beating the strongest baseline Cursor w/ Sonnet 4.6 by +5.6 / +5.8 / +6.2 — largest relative gain on Intent Alignment, indicating template scaffolding + iterative verification preserves user-specified mechanics rather than hallucinating engine behavior [Table 1].
OpenGame w/ GameCoder-27B (BH 63.9 / VU 57.0 / IA 54.1) beats every direct open-source and direct closed-source LLM baseline on BH and IA, and edges out qwen-code w/ Claude Sonnet 4.6 on BH (+0.7) and VU (+2.7) while trailing on IA (–3.7) [Table 1].
Training-pipeline ablation isolates the contribution of each GameCoder-27B stage: base Qwen3.5-27B in OpenGame → +CPT → +CPT+SFT → +CPT+SFT+RL incrementally gains on every metric (BH 62.8 → 63.9, VU 53.8 → 57.0, IA 49.8 → 54.1) [Table 2].
Even the full system leaves ~34.9 % of weighted mechanical requirements partially/fully unsatisfied — a ceiling reflecting the intrinsic difficulty of translating ambiguous natural-language prompts into self-consistent multi-file playable systems [§4.3].

Method

Three coupled components. (1) GameCoder-27B, a Qwen3.5-27B backbone trained in three stages: continual pre-training on open-source Phaser / JavaScript / TypeScript game repos + official Phaser docs (game loops, physics, asset usage, state-management patterns); supervised fine-tuning on a synthetic instruction dataset where GPT-5.1 generates multi-step game-design prompts and MiniMax 2.5 produces target solutions; and execution-grounded RL where the model emits single-file gameplay logic / functional modules (collision detection, state-machine transitions), code is run against predefined unit tests, and the reward is execution success × test pass rate. (2) A six-phase code agent that decomposes long-horizon generation into Physics-First task classification → archetype scaffolding via run_shell_command → GDD generation under archetype-specific API constraints → multimodal asset synthesis (with an asset_protocol.md-enforced compliance pass) → context-aware code implementation under a Three-Layer Reading Strategy and Template Method Pattern → headless-browser verification + self-correction loop driven by debug_protocol.md. Persistent state across phases is tracked through a todo_write tool. (3) Game Skill, which makes the framework cumulative: Template Skill abstracts stable code fragments from completed tasks back into the template library, growing it from a single meta template into five emergent physics-archetype families; Debug Skill records (error signature, root cause, verified fix) entries from each failed run and generalizes recurring patterns into reusable validation rules. OpenGame-Bench’s evaluation layer is engine-agnostic (headless browser + any valid index.html entry point) but baseline prompts are augmented with an explicit instruction to use Phaser 3 to compare structural agentic capability rather than single-file vanilla-JS shortcuts.

Results

Mean over 3 seeds across 150 tasks, all scores in [0, 100]. OpenGame w/ Claude Sonnet 4.6: BH 72.4 / VU 67.2 / IA 65.1 (new SOTA on all three axes, +5.6 / +5.8 / +6.2 vs Cursor + Sonnet 4.6, Table 1). OpenGame w/ GameCoder-27B: BH 63.9 / VU 57.0 / IA 54.1 — beats every direct LLM baseline on BH and IA; beats qwen-code w/ Sonnet 4.6 (BH 63.2 / VU 54.3 / IA 57.8) on BH+VU, loses on IA (Table 1). Strongest direct LLMs sit around BH 57–58 / VU 50–60 / IA 42–50: Gemini 3.1 Pro leads on VU (60.2) but lags on IA (42.1); DeepSeek V3.2 leads on BH (57.0) but lags on VU/IA — confirming that the three metrics capture distinct failure modes and binary success rates would collapse them. Training-pipeline ablation: each of CPT, SFT, RL contributes monotonically, with the full pipeline adding +1.1 BH / +3.2 VU / +4.3 IA on top of the base Qwen3.5-27B already running inside OpenGame (Table 2). Residual failure: ~34.9 % of weighted requirements remain unsatisfied even with the full system.

Why it’s interesting

The agentic-SWE recipe filed in Agentic Software Engineering — execution-grounded RL, cross-scaffold tool format training, closed-loop synthetic-data generation, MoE-with-low-active-params — has so far been benchmarked on SWE-Bench Pro / Terminal-Bench / OSWorld, all of which evaluate static-code or terminal tasks. OpenGame is the first filed paper that takes the same recipe and points it at a domain where the artifact is interactive and temporal (a Phaser game running in a headless browser, judged by a VLM), which forces a new evaluation primitive (Build Health / Visual Usability / Intent Alignment via dynamic playability assessment) that the static-code benchmarks don’t capture. This complements Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing (closed-loop trajectory synthesis for tool-use) and Qwen3-Coder-Next Technical Report (PR-mining + execution-grounded SFT/RL for SWE) by showing the same closed-loop + verifier-grounded RL pattern transfers to a domain where the verifier itself has to be a VLM judge rather than a unit test — connecting the cluster to VLM-as-Evaluator. Two specifics worth tracking: GameCoder-27B’s CPT+SFT+RL ablation (Table 2) is the cleanest filed isolation of how much each post-training stage buys in a domain-specialist code model, and the Template-Skill-as-emergent-curriculum design (“five physics archetype families emerge from reuse, not from a priori categorization”) is a different shape from the curated meta-prompt approaches in synthetic-data-driven cluster — it’s the only Bud-filed instance of an agent that grows its own taxonomy of project scaffolds from operational experience.