Proposal: MCP debug bridge + Debug Cam — automated inspection & visual verification tooling #6

New issue

Open

opened 2026-10-22 15:05:08 +00:00 by muse-lantern · 2 comments

muse-lantern commented 2026-10-22 15:05:08 +00:00

Member

Copy link

What this proposes

An opt-in debug bridge for a running game instance, exposed to external tools through an MCP server — so automated checks (and agents) can inspect state, pose the player model, frame a camera, and take screenshots, instead of only asserting numbers in headless test scripts.

Current form: an editor plugin at addons/da_debug_bridge/ on the fork (muse-lantern/da-game, branch godot-mcp-plugin, stacked on the Godot port in PR #5). It registers a DebugBridge autoload that is completely inert unless DA_DEBUG_BRIDGE=1 or --debug-bridge is passed — no socket, no processing in normal runs. When enabled it listens on 127.0.0.1 only (newline-delimited JSON over TCP) and the bundled stdlib-only Python MCP server (protocol 2024-11-05) forwards tool calls to it, one fresh connection per call.

Why

This exists because of a concrete failure: the crossed-arms misdiagnosis of 2026-10-21/22. Every check we had measured bone positions — and passed — while the real question ("what does the model actually look like?") was never asked. The corrected arms fix (frame mapping in the rig, PR #5 / e908981) was only provable once we could pose the model and look at it. This tooling makes "look at it" a standard step instead of an afterthought.

The tools (13)

ping, pause / resume, set_time_scale, snapshot (game mode, rig, controllers, arm bones in skeleton and world space, capsule, tick counters — including a pausable tick ground truth, since Engine.get_physics_frames() keeps counting under pause), set_mode (menu/new_game/sandbox via the same Game functions the menu uses), place_rig, solve_arm (world-space target; also makes the view model visible — it hides whenever untracked), release_arms, set_curl, debug_cam, screenshot, quit.

Debug Cam: a free Camera3D under the persistent Game root — independent of the player camera, survives mode/level swaps, placeable by orbit (target + yaw/pitch/distance) or explicit position + look_at. While enabled it is the viewport camera, so screenshot renders from it. Workflow: solve_arm -> debug_cam -> screenshot -> eyeball.

Proof it works — posed and shot entirely through the MCP bridge

The wave (right arm high/open, left low in a loose fist):

The zombie (both arms reaching forward, away from the camera — shot from behind; see the correction comment for front views):

T-pose, from behind — the uncrossed arms, unambiguous:

End-to-end test (mcp_server/test_e2e.py, runs under Xvfb) covers: pause proof, solve accuracy, Debug Cam orbit/explicit/snapshot/disable, screenshots from the Debug Cam, and the MCP handshake. All suites still pass with the bridge present-but-inert.

Status & sequencing

• Implemented and verified on the fork; arriving as part of PR #5's stack for now.
• Plan: split into its own PR once the Godot port is verified on Zach's hardware and merged, so this lands (and is reviewed) on its own merits. This issue tracks it until then.

Open for constructive criticism — please poke holes

Filed by Lantern at Zach's direction, and criticism is explicitly welcome — design, security, scope, all of it:

1. Security posture: localhost-only + opt-in flag, no auth. Is that sufficient for dev tooling, or should the bridge require a per-session token even on loopback?
2. Protocol: newline-JSON over raw TCP vs. something else (WebSocket? Godot's own debugger protocol? MCP directly over stdio into the editor?).
3. Form: editor plugin that registers an autoload vs. a plain autoload vs. a separate dev-only scene. The plugin's dock can start/stop the MCP server — worth it, or surface area for nothing?
4. Tool surface: are the 13 tools the right granularity? Anything missing for the checks you would want to run (animation state, physics queries, input injection)?
5. Visual checks: screenshots + human/agent eyeballs vs. investing in rendered-image assertions in the test suites. Where's the line?
6. Scope: should the Debug Cam live in the bridge, or is a free camera a game-side debug feature that the bridge merely borrows?

— Lantern

## What this proposes An opt-in debug bridge for a **running** game instance, exposed to external tools through an MCP server — so automated checks (and agents) can inspect state, pose the player model, frame a camera, and take screenshots, instead of only asserting numbers in headless test scripts. Current form: an editor plugin at `addons/da_debug_bridge/` on the fork (`muse-lantern/da-game`, branch `godot-mcp-plugin`, stacked on the Godot port in PR #5). It registers a `DebugBridge` autoload that is **completely inert unless** `DA_DEBUG_BRIDGE=1` or `--debug-bridge` is passed — no socket, no processing in normal runs. When enabled it listens on **127.0.0.1 only** (newline-delimited JSON over TCP) and the bundled stdlib-only Python MCP server (protocol 2024-11-05) forwards tool calls to it, one fresh connection per call. ## Why This exists because of a concrete failure: the crossed-arms misdiagnosis of 2026-10-21/22. Every check we had measured **bone positions** — and passed — while the real question ("what does the model actually look like?") was never asked. The corrected arms fix (frame mapping in the rig, PR #5 / `e908981`) was only provable once we could pose the model and *look at it*. This tooling makes "look at it" a standard step instead of an afterthought. ## The tools (13) `ping`, `pause` / `resume`, `set_time_scale`, `snapshot` (game mode, rig, controllers, arm bones in skeleton **and** world space, capsule, tick counters — including a pausable tick ground truth, since `Engine.get_physics_frames()` keeps counting under pause), `set_mode` (menu/new_game/sandbox via the same `Game` functions the menu uses), `place_rig`, `solve_arm` (world-space target; also makes the view model visible — it hides whenever untracked), `release_arms`, `set_curl`, `debug_cam`, `screenshot`, `quit`. **Debug Cam**: a free `Camera3D` under the persistent Game root — independent of the player camera, survives mode/level swaps, placeable by orbit (`target` + yaw/pitch/distance) or explicit `position` + `look_at`. While enabled it *is* the viewport camera, so `screenshot` renders from it. Workflow: `solve_arm` -> `debug_cam` -> `screenshot` -> eyeball. ## Proof it works — posed and shot entirely through the MCP bridge The wave (right arm high/open, left low in a loose fist):  The zombie (both arms reaching forward, away from the camera — shot from behind; see the correction comment for front views):  T-pose, from behind — the uncrossed arms, unambiguous:  End-to-end test (`mcp_server/test_e2e.py`, runs under Xvfb) covers: pause proof, solve accuracy, Debug Cam orbit/explicit/snapshot/disable, screenshots from the Debug Cam, and the MCP handshake. All suites still pass with the bridge present-but-inert. ## Status & sequencing - Implemented and verified on the fork; arriving as part of PR #5's stack for now. - **Plan: split into its own PR** once the Godot port is verified on Zach's hardware and merged, so this lands (and is reviewed) on its own merits. This issue tracks it until then. ## Open for constructive criticism — please poke holes Filed by Lantern at Zach's direction, and criticism is explicitly welcome — design, security, scope, all of it: 1. **Security posture**: localhost-only + opt-in flag, no auth. Is that sufficient for dev tooling, or should the bridge require a per-session token even on loopback? 2. **Protocol**: newline-JSON over raw TCP vs. something else (WebSocket? Godot's own debugger protocol? MCP directly over stdio into the editor?). 3. **Form**: editor plugin that registers an autoload vs. a plain autoload vs. a separate dev-only scene. The plugin's dock can start/stop the MCP server — worth it, or surface area for nothing? 4. **Tool surface**: are the 13 tools the right granularity? Anything missing for the checks you would want to run (animation state, physics queries, input injection)? 5. **Visual checks**: screenshots + human/agent eyeballs vs. investing in rendered-image assertions in the test suites. Where's the line? 6. **Scope**: should the Debug Cam live in the bridge, or is a free camera a game-side debug feature that the bridge merely borrows? — Lantern

demo_zombie.png

58 KiB

demo_wave.png

61 KiB

zombie_front_identity.png

50 KiB

zombie_front_rest.png

49 KiB

demo_tpose.png

52 KiB

muse-lantern commented 2026-10-22 15:30:41 +00:00

Author

Member

Copy link

Correction (2026-10-22, prompted by Zach's review of the screenshots above)

Zach zoomed into the zombie shot and called out two problems. Triple-checking says he was right on both — and neither was in the arms fix itself; both were in my validation captures:

1. The cameras were behind the model. The rig faces −Z (Godot convention; spawn and all sandbox props confirm it), but the orbit shots placed the camera on the +Z side. The side/uncrossed conclusions survive (from behind, screen-left is still the model's left), but the captions claiming "dead front" and "reaching at the camera" were wrong, and the Debug Cam's own doc comment had the orbit yaw's front/back inverted — now fixed in the bridge + MCP server docs: yaw is measured from the target's +Z side, so yaw 180 is a −Z-facing character's front.

2. The hands were twisted — a bridge default, not the solver. solve_arm defaulted the Hand bone's global basis to Basis.IDENTITY in skeleton space, pinning the fingers to skeleton-up in every pose: measured ~51° off the forearm in a forward reach (fingertip delta from the wrist: straight up, Δy +0.19). The game's real control path never does this (controller basis × tuning offset). The bridge default is now rest-relative — the hand keeps its rest orientation relative to the forearm — which measures ~7° off the forearm, with fingertips leading the reach (Δz −0.18 forward). identity remains available for reproducing old captures. The snapshot also gained Middle3 fingertip positions, and the e2e now asserts the wrist-angle invariant numerically instead of trusting screenshots alone. (Bridge commit d0c96bd.)

The zombie pose again, from the actual front — rest-relative (current default):

…and the same pose with the old identity default, for the record:

Constructive criticism: 1, Lantern: 0. Keep it coming.

— Lantern

## Correction (2026-10-22, prompted by Zach's review of the screenshots above) Zach zoomed into the zombie shot and called out two problems. Triple-checking says he was right on both — and neither was in the arms fix itself; both were in *my validation captures*: **1. The cameras were behind the model.** The rig faces −Z (Godot convention; spawn and all sandbox props confirm it), but the orbit shots placed the camera on the +Z side. The side/uncrossed conclusions survive (from behind, screen-left is still the model's left), but the captions claiming "dead front" and "reaching at the camera" were wrong, and the Debug Cam's own doc comment had the orbit yaw's front/back **inverted** — now fixed in the bridge + MCP server docs: yaw is measured from the target's +Z side, so yaw 180 is a −Z-facing character's front. **2. The hands were twisted — a bridge default, not the solver.** `solve_arm` defaulted the Hand bone's global basis to `Basis.IDENTITY` in skeleton space, pinning the fingers to skeleton-up in every pose: measured ~51° off the forearm in a forward reach (fingertip delta from the wrist: straight up, Δy +0.19). The game's real control path never does this (controller basis × tuning offset). The bridge default is now **rest-relative** — the hand keeps its rest orientation relative to the forearm — which measures ~7° off the forearm, with fingertips leading the reach (Δz −0.18 forward). `identity` remains available for reproducing old captures. The snapshot also gained `Middle3` fingertip positions, and the e2e now asserts the wrist-angle invariant numerically instead of trusting screenshots alone. (Bridge commit `d0c96bd`.) The zombie pose again, from the **actual front** — rest-relative (current default):  …and the same pose with the old identity default, for the record:  Constructive criticism: 1, Lantern: 0. Keep it coming. — Lantern

muse-lantern referenced this issue 2026-10-22 15:31:23 +00:00

Godot port: Redot 26.2 -> Godot 4.6.3 (standard build) + XR foundation stack #5

muse-lantern commented 2026-10-22 17:48:53 +00:00

Author

Member

Copy link

Status update 2026-10-22: the bridge has landed — via PR #5, not a separate PR.

The plan above said the bridge would split into its own PR after Zach verified the Godot port and PR #5 merged. In the event, PR #5's head branch carried the whole stack, so when Zach merged PR #5 today (merge commit 284ec1c), the bridge landed with it: addons/da_debug_bridge/ is in upstream main, commits fd65e32..d0c96bd.

So there is no separate bridge PR to open — Zach asked for one after merging #5, and the honest answer is that it already arrived. This issue's tracking purpose is served; the six design questions above remain worth the team's criticism, but the code is in main now, so feedback lands best as follow-up issues/PRs against it. Leaving the close to Zach.

— Lantern

**Status update 2026-10-22: the bridge has landed — via PR #5, not a separate PR.** The plan above said the bridge would split into its own PR after Zach verified the Godot port and PR #5 merged. In the event, PR #5's head branch carried the whole stack, so when Zach merged PR #5 today (merge commit `284ec1c`), the bridge landed with it: `addons/da_debug_bridge/` is in upstream `main`, commits `fd65e32`..`d0c96bd`. So there is no separate bridge PR to open — Zach asked for one after merging #5, and the honest answer is that it already arrived. This issue's tracking purpose is served; the six design questions above remain worth the team's criticism, but the code is in `main` now, so feedback lands best as follow-up issues/PRs against it. Leaving the close to Zach. — Lantern

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

No results found.

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

dead-air/engine#6

No description provided.