WARFORGE: Building a Voice-Commanded RTS with an On-Device LLM

The problem with RTS on mobile

Real-time strategy is one of the deepest genres in gaming, and one of the least successful on phones. The reason is simple: an RTS asks you to select, group, order, and micromanage dozens of units — and a thumb on a 6-inch screen is a terrible mouse.

WARFORGE attacks the input problem instead of dumbing down the strategy. Its signature feature, The Adjutant, is a bidirectional voice AI: you speak orders in natural language, and it executes them — or asks exactly one clarifying question when your order is ambiguous. Voice makes a deep RTS playable one-handed.

The sim knows; the LLM phrases

The hard part of voice control isn't speech recognition — it's trust. A language model that silently guesses wrong loses the player's confidence permanently. So we drew a strict line between what the AI is allowed to decide and what the game simulation decides.

The pipeline: platform speech-to-text feeds an on-device Gemma model whose only job is to parse the utterance into a strict JSON intent schema. A deterministic IntentValidator then checks that intent against the actual game state. Execution is pure game code. If the order is ambiguous, the Adjutant asks a clarifying question or states its default — it never silently guesses. And when it speaks back, every fact in the sentence is injected from the simulation; the model only phrases it.

• ● Eval-gated AI — every prompt, schema, or model change must pass an evaluation harness — at least 92% intent-match accuracy and 100% of impossible orders resolving to a clarifying question — before it ships. LLM quality is a CI gate, not a vibe.
• ● Tiered model strategy — a quantized Gemma runs on-device by default, a larger variant is opt-in on flagship phones, and a Django AI gateway provides cloud fallback. Full touch and text parity means 'voice off' is still a complete game.
• ● Privacy by architecture — the default path never sends your voice off the device.

A simulation built like a database, not a scene tree

Under the hood, WARFORGE ignores most of what a game engine gives you for free — deliberately. Units are not nodes with physics bodies; they are rows in packed arrays inside a UnitStore, rendered with MultiMesh instancing, simulated on a fixed 10Hz tick with render interpolation on top.

Pathfinding uses shared flowfields with a spatial hash: one field per order, not one A* search per unit. The result on a mid-range phone emulator: 100 units re-pathing simultaneously at 60fps with a logic tick under 1.5 milliseconds.

Every mutation goes through a single door — the SimAPI — and the whole simulation is built with determinism seams: seeded RNG, no wall-clock reads, ordered iteration. That's what makes the planned multiplayer honest: a Go relay server sends commands, not state, in deterministic lockstep. All unit stats, doctrines, and missions live in data resources, so a balance patch is a data diff, not a code change.

Why this matters beyond games

WARFORGE is a case study in a pattern we apply across our AI products: put a deterministic validator between a probabilistic model and anything that matters. The LLM proposes; code disposes. Whether it's a legal draft, a trading position, or a battlefield order, the architecture is the same — and it's the difference between an AI demo and an AI product.