Model integration (FUTURE — code first) #10

New issue

Open

opened 2026-11-15 01:10:09 +00:00 by muse-lantern · 0 comments

muse-lantern commented

2026-11-15 01:10:09 +00:00

Member

The plan

FUTURE. Do not build this now. This issue tracks the decision-model work so the shape it needs is kept in mind while the heuristic system is built — and so nobody starts it early.

The build order is code first, then model integration. The heuristic system has to be proven on the simulation scenarios before a model touches either decision slot. When the time comes, the plan is below.

The shape (from the design doc)

Commander and Director are both decision roles with a heuristic default and a decision-model alternate behind the same interface. A model sits as Director and Commander — both layers, not one:

Commander slot: team tactics, small state (team knowledge plus local environment), fast cadence — roughly one decision per team per one to two seconds.
Director slot: base coordination, big state (union of Commander knowledge plus the map picture), slow cadence — far less often than a Commander.

A model in either slot sees only what that layer knows. A Commander model sees team knowledge and local environment — never the player's true position, never anything an NPC has not perceived. A Director model sees the union of its Commanders' knowledge — never anything outside it. The model is optional in both slots, off by default, and the game runs fully on heuristics without it. If the model is absent, slow, or wrong, the layer falls back to heuristics — instantly, with no visible break. The model never sits in the per-frame loop and never blocks a decision: every call has a timeout, and a timeout means the heuristic decides.

The work, when it starts (in order)

Define the decision interface both layers implement — what goes in (the layer's knowledge, in the ledger's shape) and what comes out (orders for a Commander, objectives for a Director), so the heuristic and the model are interchangeable behind it. The interface depends on the order-vocabulary and Director-objective issues being settled first.
Commander slot first. Smaller state, faster feedback, easier to measure. One model per team.
Evaluation harness. The simulation scenarios are the tactic data: a model Commander is measured against the heuristic on the same scenarios. The metric itself is still an open question (see below) — it must be settled before any model result means anything.
Director slot. Same interface shape, bigger state, slower cadence.
Ship as an option, off by default. The model is never a core dependency.

Research backing

Two research notes back the slots (delivered as Markdown, not filed here): the Hanzo service offering and the self-hosted decision-model survey. The findings that constrain this work: there is no local Kai and no local Jev — both are API-only with closed weights, so neither can run in these slots as shipped; a cloud model could only sit in a slot behind a developer-run relay, which is not part of this design. The self-hostable candidates are small open-weight decision models copying the state-plus-typed-questions contract, with the 4B tier as the practical ceiling on a player PC (a GPU already rendering PCVR fits only small models). Runtimes: llama.cpp / llama-server is the most shippable; ONNX Runtime is the most game-friendly in-process option. No decision model is trained on combat or horror game state — combat value is unproven until measured on this game's own tactic data, which is what the scenarios are for.

What done looks like (future)

A model Commander beats or matches the heuristic on the scenario suite by the settled metric, with the timeout fallback proven (kill the model mid-decision and the heuristic decides with no visible break), shipped as an option that is off by default.

Open questions

The interface shape: exactly what state goes in and what orders or objectives come out?
Evaluation: how do we measure whether a model beats the heuristic — what tactic data, what metric, what counts as good enough to ship as an option?
Which model first — a 4B decision model per the survey, or something smaller?
Who runs the evaluation — the scenario suite as a benchmark, or a human reviewing replays?

## The plan FUTURE. Do not build this now. This issue tracks the decision-model work so the shape it needs is kept in mind while the heuristic system is built — and so nobody starts it early. The build order is code first, then model integration. The heuristic system has to be proven on the simulation scenarios before a model touches either decision slot. When the time comes, the plan is below. ## The shape (from the design doc) Commander and Director are both decision roles with a heuristic default and a decision-model alternate behind the same interface. A model sits as Director and Commander — both layers, not one: - **Commander slot:** team tactics, small state (team knowledge plus local environment), fast cadence — roughly one decision per team per one to two seconds. - **Director slot:** base coordination, big state (union of Commander knowledge plus the map picture), slow cadence — far less often than a Commander. A model in either slot sees only what that layer knows. A Commander model sees team knowledge and local environment — never the player's true position, never anything an NPC has not perceived. A Director model sees the union of its Commanders' knowledge — never anything outside it. The model is optional in both slots, off by default, and the game runs fully on heuristics without it. If the model is absent, slow, or wrong, the layer falls back to heuristics — instantly, with no visible break. The model never sits in the per-frame loop and never blocks a decision: every call has a timeout, and a timeout means the heuristic decides. ## The work, when it starts (in order) 1. **Define the decision interface** both layers implement — what goes in (the layer's knowledge, in the ledger's shape) and what comes out (orders for a Commander, objectives for a Director), so the heuristic and the model are interchangeable behind it. The interface depends on the order-vocabulary and Director-objective issues being settled first. 2. **Commander slot first.** Smaller state, faster feedback, easier to measure. One model per team. 3. **Evaluation harness.** The simulation scenarios are the tactic data: a model Commander is measured against the heuristic on the same scenarios. The metric itself is still an open question (see below) — it must be settled before any model result means anything. 4. **Director slot.** Same interface shape, bigger state, slower cadence. 5. **Ship as an option, off by default.** The model is never a core dependency. ## Research backing Two research notes back the slots (delivered as Markdown, not filed here): the Hanzo service offering and the self-hosted decision-model survey. The findings that constrain this work: there is no local Kai and no local Jev — both are API-only with closed weights, so neither can run in these slots as shipped; a cloud model could only sit in a slot behind a developer-run relay, which is not part of this design. The self-hostable candidates are small open-weight decision models copying the state-plus-typed-questions contract, with the 4B tier as the practical ceiling on a player PC (a GPU already rendering PCVR fits only small models). Runtimes: llama.cpp / llama-server is the most shippable; ONNX Runtime is the most game-friendly in-process option. No decision model is trained on combat or horror game state — combat value is unproven until measured on this game's own tactic data, which is what the scenarios are for. ## What done looks like (future) A model Commander beats or matches the heuristic on the scenario suite by the settled metric, with the timeout fallback proven (kill the model mid-decision and the heuristic decides with no visible break), shipped as an option that is off by default. ## Open questions - The interface shape: exactly what state goes in and what orders or objectives come out? - Evaluation: how do we measure whether a model beats the heuristic — what tactic data, what metric, what counts as good enough to ship as an option? - Which model first — a 4B decision model per the survey, or something smaller? - Who runs the evaluation — the scenario suite as a benchmark, or a human reviewing replays?

muse-lantern referenced this issue

2026-11-15 01:10:25 +00:00

Build sequence: code first, then model integration #11

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

dead-air/daisy#10

No description provided.

Rows
Columns