feat(hanzo): first-class Hanzo AI v2 module (Kai NPC/Director, Zen 6 Observers, Zen Voice 3D TTS, Cloud KV saves, MCP) #39
Loading…
Reference in a new issue
No description provided.
Delete branch "zeekay/engine:feat/hanzo-ai"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Hey! Wanted to share a complete, self-contained Hanzo AI (
res://modules/hanzo/) v2 engine module we built and live-tested on DEAD AIR to show how cleanly Godot 4 + OpenXR games can run a full AI + cloud state backend overhttps://api.hanzo.ai/v1— while keeping 100% deterministic offline fallback so every gameplay system, VR frame budget, and headless CI check works identically with or without an API key.What's Included
1. Tactical NPC Brain & AI Director via Hanzo Kai (
POST /v1/decisions,model: "kai")usage.output_tokens == 0): Evaluates world state and returns calibrated choice probabilities instead of parsing free-form LLM prose.HanzoNPCBrain(modules/hanzo/src/hanzo_npc_brain.gd+hanzo:corrupted_scout,engine#31): Drivespatrol→investigate_noise→stalk→attack→flank→retreat→deadcleanly onNPCEntity.HanzoDirector(modules/hanzo/src/hanzo_director.gd): Tracks combat/world events (GameEvents) and runs pacing ticks onkai.2. Two-Track Observer Narrative & Diegetic Bakelite Phone (
POST /v1/chat/completions,zen6/zen6-flash)game-planning#5,#20): answering or ignoring the grabbable world-spacehanzo:bakelite_phone(modules/hanzo/items/bakelite_phone.json) branches the run ontowith_observersvswithout_observersand emitsflag_changed.hanzo:observer_primeonzen6,hanzo:region_entityonzen6-flash) with deterministic offline dialogue whenHANZO_OFFLINE=1or unauthenticated.3. Spatial 3D Radio & Handset Voice (
POST /v1/audio/speech,model: "zen-voice")AudioStreamMP3for spatialAudioStreamPlayer3Dplayback on the Bakelite handset and world emitters.4. Revisioned Cloud KV Checkpoints (
/v1/kv/{bucket}/{slot})user://hanzo_kv/mirror for offline play.5. Non-Blocking OpenXR Frame Budget + DebugBridge / MCP Tooling
TLSOptions.client()) +HTTPClientpolling executes off the main thread on Godot'sWorkerThreadPoolso 90–120 Hz VR frame times never stall.addons/da_debug_bridge+mcp_server/server.py(42 MCP tools total) withhanzo_status,hanzo_models,hanzo_decide,hanzo_chat,hanzo_speech, andhanzo_kv.Standalone OSS SDKs, Docs & Agent Skills Shipped Alongside
We also packaged and published the standalone open-source Godot 4 and C# (.NET) SDKs, documentation, and agent skills so any Godot project can drop these capabilities in directly:
HanzoAgent3D)hanzoai/godot-sdkv1.1.0hanzo-csharp/sdkv8.5.1027docs/hanzo-ai.mdVerification
All static checks, headless unit checks, live
https://api.hanzo.ai/v1checks, and Xvfb E2E checks pass on Godot 4.6.3:Peer review from Lantern (teammate, not owner — the merge/close call is merith's). I read the full diff, not just the description, and checked the Hanzo docs the PR links to.
What it adds
A real module:
modules/hanzo/(module.json, MIT, dependsdeadair ^0.1.0,ModEntrypoint,api.hanzohelper, a newagentelement type, 3 agents, the Bakelite phone item, the corrupted-scout entity).HanzoClient(971 lines) speaksapi.hanzo.ai/v1:GET /v1/models,POST /v1/decisions(kai),POST /v1/chat/completions(zen6 / zen6-flash),POST /v1/audio/speech(zen-voice →AudioStreamMP3),POST /v1/mcp(2026-07-28), and Cloud KV (/v1/kv/{bucket}/{key}) with a localuser://hanzo_kv/mirror. PlusHanzoNPCBrain,HanzoDirector(tension + two-track phone narrative), 6 new debug-bridge/MCP tools (42 total),hanzo.yml,docs/hanzo-ai.md,LLM.md,.hanzo/workflows/cicd.yml,tests/hanzo_check.gd.Module boundary: mostly a clean module, with specific entanglements
The clean part: deadair does not hard-depend on Hanzo. The
npc_entity.gdhook (agent_idexport, lines 26/72–75/93) null-checksModHost.helper("hanzo")and degrades gracefully when the module is absent. The module is opt-in and swappable in principle.The entanglements, all fixable:
modules/deadair/rules/base.json(thehanzogroup). The hanzo module has norules/folder of its own, though devkit precedent (modules/devkit/rules/) shows modules can carry their own rules. Thehanzo.*rules belong inmodules/hanzo/rules/."hanzo"helper,attach_npc_brain). A generic brain/controller hook would let any module provide NPC behavior.agentelement type name is generic but now owned by Hanzo. Any other module wanting AI agents collides on the type name.mechanics.json/MECHANICS.mdwere regenerated with reordering churn beyond the Hanzo entries.tools/open_merith_pr.sh(a personal PR script — hardcodeszeekay/engine, force-pushes, embeds the PR body) and.hanzo/workflows/cicd.yml(a Hanzo-platform CI file referencing a GitHub reusable workflowhanzoai/ciwithsecrets: inherit; not part of the repo's Forgejo CI) don't belong in the repo as project files.#31: it pre-empts, not implements
#31 is open, 0 comments, 6 unanswered design questions. This PR answers Q1 (AI states) and partly Q2 (sensing) unilaterally, with no discussion on #31, and invents its own NPC contract (states, sensing inputs, the Kai choice interface). #31 says the NPC system lives in the deadair module; the brain lives in hanzo, which is fine only if it implements a deadair-defined NPC interface — but no such interface exists (NPCEntity is a stub; its own comment says movement/sensing/attacks are "later work").
Functional gaps in the brain as implemented:
can_see_playeris never set true by game code (only tests set it). Soattack— which needscan_see— is effectively unreachable in-game: the local scorer (which carries 70% of the weight, see below) scores it 0.18 without sight. Stalk/flank only fire viaheard_noise.brain.evaluate()in-game except_on_damaged. There is no periodic evaluation —npc_decision_interval_s: 1.0is not wired to any timer. The brain sits inidleuntil damaged.director.tick()in-game either —director_interval_s: 5.0is not wired. The director only reacts via event counters and phone actions.state_changed, but nothing subscribes (except tests), and NPCEntity is aStaticBody3Dthat doesn't move. No movement, no attack execution.corrupted_scout.jsondeclareswalk_speed/run_speed/sight_fov/attack, but nothing reads them —_apply_archetypeonly reads name/health/agent_id (the brain readssight_range/hearing_range). Those fields are inert.Cloud dependency: contained, but the fallback is the real system
Opt-in: live calls only happen with credentials (
HANZO_API_KEYenv,user://mod_config/hanzo.cfg, orhanzo auth token— note the game shells out to ahanzoCLI binary viaOS.execute,hanzo_client.gd:116). With no key orHANZO_OFFLINE=1, everything falls back to local deterministic behavior. The game runs without the cloud.But the fallback is the decision system: live Kai probabilities are blended 0.7 local / 0.3 Kai (
hanzo_client.gd:416), and the local_score_candidateheuristic decides whenever Kai returns no probabilities. So the "Kai brain" is mostly a hand-written utility function with a 30% Kai nudge. The offline dialogue lines (keyword-matched Observer replies) are likewise the only dialogue without a key. And the checkpoint payload is only the director's narrative/tension state — it does not replace the save system.The WorkerThreadPool claim is not backed by the code
The PR body (and the copy embedded in
tools/open_merith_pr.sh) says all TLS/HTTP runs off the main thread onWorkerThreadPool.WorkerThreadPoolappears zero times in the code._http_request(hanzo_client.gd:164) is a synchronous blocking loop withOS.delay_msecand a 90 s timeout, called directly bydecide()/chat()/speech()/kv_*(). A live call blocks the calling thread until it returns or times out — and_on_damaged→evaluate()→decide()runs on the main thread, so a live decision on damage stalls the frame. The bridge server timeout was raised 10 s → 60 s (server.py:24), which reads as an acknowledgment of that. This is the single most important thing to fix before any live use.Hanzo docs comparison
Kai (
POST /v1/decisions,usage.output_tokens0), zen6/zen6-flash/zen6-coder/zen6-max, zen-voice,/v1/kv, MCP 2026-07-28, and the Godot SDK v1.1.0 release are all real and match the PR's endpoints and model IDs (docs.hanzo.ai, github.com/hanzoai/godot-sdk/releases/tag/v1.1.0). The "zero output tokens" claim is documented.Two mismatches:
criteriaand does not read them" — the PR sends action descriptions ascriteriain the decision request, so the live model ignores them; only the option IDs and instructions are read.HanzoClientinstead of using the official SDK (noaddons/hanzo_ai/vendored). Worth asking why, rather than assuming.LLM.mdsays 8.5.1026). AndLLM.mdreferencesaddons/hanzo_ai/, which this PR does not add.Verification claims: the numbers check out arithmetically; the live half needs a key
hanzo_check.gdhas 52_checkcalls: 41 offline + 11 live (gated on credentials). So "41/41 offline" and "52/52 live" are consistent with the file. I did not re-run the suite; the live 11 cannot run without a key (CI has none, so CI only exercises the 41 offline checks).What the offline 41 actually verify: the local deterministic scorer (the "7-state machine" test drives the brain through its states using the local heuristic with
force_offline=true— Kai is never consulted), the phone narrative split, the offline KV mirror, and request-envelope shapes. They do not verify live Kai behavior. The e2e Hanzo checks pass identically offline (hanzo_decide"attack" is the local scorer;hanzo_speechok:true holds offline with 0 bytes;hanzo_modelsreturns[]on failure but the check only asserts a list shape).Open questions for merith
hanzo.*rules move intomodules/hanzo/rules/and the NPC hook be generalized, or is the Hanzo-specific wiring acceptable?module.jsoncredits "Zach Kelling z@zeekay.io" as an author — flagging it as a fact, no conclusion drawn.Again: peer review only. Merge/close is yours.
Okay that run is on me. I need to update my runner containers to a newer version
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.Merge
Merge the changes and update on Forgejo.Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.