Defer AI tuning until game completion and set adjacent difficulty targets

Update the goal objective and synchronized repository guidance at the user's request. Remove the weekly usage cutoff, routine account-usage checks and mandatory token-counter tracking while retaining efficient work, verified Git pushes and selective subagents. The objective file remains outside the repository; private before/after copies are saved.

Keep existing AI profiles and minimal legal participation for feature testing and critical correctness fixes. Defer full AI implementation, strategic refinement and calibration until all non-AI game features are complete. Then aim for approximately 75% higher-tier wins separately for Medium/Easy, Hard/Medium and Brutal/Hard in balanced pairwise evaluations, with swapped factions/seats, independent evaluation seeds and reported uncertainty. Preserve broader multiplayer acceptance, hidden-information boundaries and human-seat pacing. Historical calibration results are not evidence of these targets.

Documentation-only validation: reviewed the changed instructions, checked 73 local links, verified all three target pairs and preserved Git/subagent requirements, and passed git diff --check. No runtime behavior changed; tests/build were not rerun.
This commit is contained in:
Quintine committed 2026-09-20 14:59:14 +10:00
1 parent 20cc6057e9
commit 8f7a62cd4c
6 files changed
+91 -15

No files matched your search

+9 -3
View File
@@ -29,11 +29,17 @@ for the in-memory SQLite tests. `tsx` is a pinned development dependency.
existing checklist. Prototypes need usable controls, a legal AI path and saved existing checklist. Prototypes need usable controls, a legal AI path and saved
continuation; placeholders and disconnected helpers are not completed features. continuation; placeholders and disconnected helpers are not completed features.
- Keep focused checks for crashes, deadlocks, legality, custody, privacy and save - Keep focused checks for crashes, deadlocks, legality, custody, privacy and save
integrity. Defer exhaustive combinations, AI calibration and visual polish integrity. Defer exhaustive combinations and visual polish until broader
until broader functional coverage, unless current failures require them. functional coverage, unless current failures require them.
- Defer full AI implementation, strategic refinement and difficulty calibration
until all non-AI game features are complete. Until then, maintain only minimal
legal participation and fix critical playability/correctness issues. Afterward,
target approximately 75% higher-tier wins for each adjacent pair: Medium/Easy,
Hard/Medium and Brutal/Hard. See `docs/AI_DEVELOPMENT_PLAN.md` for evaluation.
- Preserve required checkpoint checks, selective subagents and independent - Preserve required checkpoint checks, selective subagents and independent
review for complex rules/privacy/persistence. Commit and push verified review for complex rules/privacy/persistence. Commit and push verified
checkpoints under standing authorization; stop at 80% weekly usage consumed. checkpoints under standing authorization. No user-imposed usage cutoff or
routine account-usage check remains.
- Keep material rulings explicit and prototype independent work while pending. - Keep material rulings explicit and prototype independent work while pending.
Prototype evidence does not open mode or publication gates. Prototype evidence does not open mode or publication gates.
+6
View File
@@ -14,6 +14,12 @@ npm run dev -- --host 0.0.0.0
Open `http://localhost:3000`. Create a room, share its eight-character invitation, and add human or AI seats. The host can configure Easy, Medium, Hard and Brutal AI opponents. Current calibration demonstrates a substantially weaker Easy policy; it does not yet demonstrate a consistent ordering among the upper three levels. Open `http://localhost:3000`. Create a room, share its eight-character invitation, and add human or AI seats. The host can configure Easy, Medium, Hard and Brutal AI opponents. Current calibration demonstrates a substantially weaker Easy policy; it does not yet demonstrate a consistent ordering among the upper three levels.
Full AI implementation and tuning wait until all non-AI game features are
complete. The [AI development plan](docs/AI_DEVELOPMENT_PLAN.md) then targets
approximately 75% higher-tier wins for Medium/Easy, Hard/Medium and Brutal/Hard.
Until that stage, AI work focuses on minimal legal participation and critical
playability fixes.
Choose **Learn to play** for the [interactive introduction](docs/INTERACTIVE_INTRODUCTION.md) Choose **Learn to play** for the [interactive introduction](docs/INTERACTIVE_INTRODUCTION.md)
at `/learn`. Practice shipping, battle plans and collection before entering a at `/learn`. Practice shipping, battle plans and collection before entering a
room; lesson progress stays in your browser and does not change saved games. room; lesson progress stays in your browser and does not change saved games.
+6
View File
@@ -1,5 +1,11 @@
# AI calibration: supported basic games # AI calibration: supported basic games
**Current plan:** full AI implementation, strategic refinement and calibration
wait until all non-AI game features are complete. Then target approximately 75%
higher-tier wins for Medium/Easy, Hard/Medium and Brutal/Hard. See the
[AI development plan](AI_DEVELOPMENT_PLAN.md). The studies below remain historical
evidence and do not establish those pairwise targets.
Measured 6 September 2026 (Australia/ACT), using the local engine and bot policy in this checkout. **All 120 seeded games completed; no deadlock, action-limit, exception, or checked invariant failure occurred.** Easy won once in 120 seat appearances. Medium and Brutal each won 41 times, and Hard won 37 times. These observations do **not** establish an increasing Medium → Hard → Brutal strength ladder. Measured 6 September 2026 (Australia/ACT), using the local engine and bot policy in this checkout. **All 120 seeded games completed; no deadlock, action-limit, exception, or checked invariant failure occurred.** Easy won once in 120 seat appearances. Medium and Brutal each won 41 times, and Hard won 37 times. These observations do **not** establish an increasing Medium → Hard → Brutal strength ladder.
This is a bounded simulation study of the current implementation, not a verification of the published game's complete rules, a browser/recovery test, or a human skill benchmark. No bot, engine, UI, or package configuration was changed for this study. This is a bounded simulation study of the current implementation, not a verification of the published game's complete rules, a browser/recovery test, or a human skill benchmark. No bot, engine, UI, or package configuration was changed for this study.
+51
View File
@@ -0,0 +1,51 @@
# AI development order and difficulty targets
Updated 20 September 2026 at the user's request. This is the current development
plan; the [earlier calibration studies](AI_CALIBRATION.md) are historical results,
not evidence that the new targets have been met.
## Complete the game features first
Full AI implementation, strategic refinement and difficulty tuning start only
after all non-AI game features are complete: Basic and Advanced rules, all twelve
factions, the three expansions, applicable optional modules and interactions,
multiplayer functions and the required player-facing features. Do not spend time
tuning strategy against incomplete rules.
Until then, preserve existing profiles and provide only the minimal legal AI
participation needed to exercise new features. Fix crashes, deadlocks, illegal
actions, privacy violations and saved-continuation failures. Focused legal-action
and complete-game smoke checks remain appropriate; strength studies and tuning
are deferred. Working AI paths remain part of a connected feature prototype.
## Tune the completed game
Finish faction/module strategy, planning, risk assessment and negotiation, then
aim for these separate adjacent-pair results:
| Matchup | Target |
| --- | --- |
| Medium against Easy | Medium wins approximately 75% |
| Hard against Medium | Hard wins approximately 75% |
| Brutal against Hard | Brutal wins approximately 75% |
These are statistical targets over repeated games, not a promise for every four
games or a 75% overall win rate in a table containing all four difficulties.
Use legal head-to-head configurations with balanced faction/seat assignments,
swapped assignments and multiple seeds. Keep evaluation seeds separate from
tuning, report sample sizes and uncertainty, and break results down by rules
configuration so aggregate wins do not hide a broken matchup. Define outcome
accounting before running the evaluation and report draws or shared wins
explicitly. Do not treat unfinished or failed games as ordinary wins or losses.
Separately verify the completed strategies in supported two-through-six-player
tables, including alliances and expansion/module combinations. Pairwise strength
does not establish multiplayer competence. Preserve private information, legal
resources and the authoritative rules at every level; never give higher levels
unauthorized information or rule exceptions to reach a target. Retain the
existing 1–2 second action pacing when AI plays a human-owned seat.
Final AI acceptance needs measured evidence for all three approximate 75% targets
after feature completion, plus legal continuation across the requested scope.
The current policies and old studies do not satisfy that acceptance gate.
+7 -4
View File
@@ -89,7 +89,7 @@ These links define bounded working behavior, not complete module certification.
| Basic core | Ordinary Guild repricing/transport settlement, broader truthful commitments, unresolved special timing and integrated rules/human/AI acceptance. | | Basic core | Ordinary Guild repricing/transport settlement, broader truthful commitments, unresolved special timing and integrated rules/human/AI acceptance. |
| Six-faction Advanced | Base gaps, provisional special-Karama outcomes, remaining timing audits and authentic multiplayer/browser acceptance. Setup matrices and complete samples do not open public starts. | | Six-faction Advanced | Base gaps, provisional special-Karama outcomes, remaining timing audits and authentic multiplayer/browser acceptance. Setup matrices and complete samples do not open public starts. |
| Twelve factions and all optional modules | Every missing effect, remaining skill bands, Ecaz card variant, Kull Wahad, unfinished Richese cards, Terror effects, occupation entitlement, Nexus interactions, contested Discoveries and complete games across valid combinations. Detailed boundaries remain in the checklist/decision index. | | Twelve factions and all optional modules | Every missing effect, remaining skill bands, Ecaz card variant, Kull Wahad, unfinished Richese cards, Terror effects, occupation entitlement, Nexus interactions, contested Discoveries and complete games across valid combinations. Detailed boundaries remain in the checklist/decision index. |
| AI | Legal continuation for every supported function and distinct Easy/Medium/Hard/Brutal strategy; consistent upper-level strength ordering remains unproven. Defer broad calibration until functional coverage warrants it. | | AI | Maintain minimal legal participation and critical correctness fixes while game features are unfinished. Full implementation, strategy refinement and calibration wait until all non-AI features are complete; then target approximately 75% higher-tier wins for Medium/Easy, Hard/Medium and Brutal/Hard. See the [AI development plan](AI_DEVELOPMENT_PLAN.md). These targets remain unverified. |
| Multiplayer | Broader disconnected/abandoned-seat recovery, unattended continuation and network-failure acceptance, while preserving authoritative versioning, custody, privacy and saves. | | Multiplayer | Broader disconnected/abandoned-seat recovery, unattended continuation and network-failure acceptance, while preserving authoritative versioning, custody, privacy and saves. |
| Components and experience | Full component inventory/text verification, all readable inspectors, original artwork, animation/sound/accessibility/mobile polish, complete internal guidance and end-to-end human play. | | Components and experience | Full component inventory/text verification, all readable inspectors, original artwork, animation/sound/accessibility/mobile polish, complete internal guidance and end-to-end human play. |
@@ -102,11 +102,14 @@ record their measured scope. [README](../README.md) retains publication gates.
Prototype every remaining function in dependency order, using existing controls, Prototype every remaining function in dependency order, using existing controls,
legal AI and the saved-state model. Continue independent functions while material legal AI and the saved-state model. Continue independent functions while material
rulings are pending. Then run integrated games, repair failures, deepen combination rulings are pending. Then run integrated games, repair failures, deepen combination
and recovery coverage, calibrate AI and polish. Keep focused checks for crashes, and recovery coverage, and complete all non-AI features. Only then fully implement
deadlocks, legality, custody and privacy; complete required broad checkpoint checks. and refine AI strategy and calibrate the adjacent difficulty pairs. Keep focused
checks for crashes, deadlocks, legality, custody and privacy; complete required
broad checkpoint checks.
Use selective subagents and independent review for complex rules/privacy/persistence. Use selective subagents and independent review for complex rules/privacy/persistence.
Commit and push verified checkpoints under **“Push all from now on”**, verify push Commit and push verified checkpoints under **“Push all from now on”**, verify push
success, and stop pursuit at **80% weekly usage consumed** with the goal unfinished. success. The user removed the usage cutoff and routine account-usage checks;
efficient work, Git delivery and selective subagents remain required.
The user-authorized 13 September reset happened once; preserve all games created The user-authorized 13 September reset happened once; preserve all games created
since it. Keep `.wrangler/state`, credentials and private backup artifacts out of since it. Keep `.wrangler/state`, credentials and private backup artifacts out of
+12 -8
View File
@@ -64,20 +64,24 @@ prototyped, integrated, verified and polished functions, and choose connected
batches by dependency across core and expansion systems. A prototype includes batches by dependency across core and expansion systems. A prototype includes
a working action, usable controls, legal AI participation and saved continuation. a working action, usable controls, legal AI participation and saved continuation.
Track functions newly playable and issues discovered during integration, alongside Track functions newly playable and issues discovered during integration, alongside
tokens and rework; do not count placeholders as delivered functions. rework; do not count placeholders as delivered functions. Earlier token counters
remain historical measurements, not a required per-checkpoint task.
Keep focused invariant and smoke checks during prototyping and required checks Keep focused invariant and smoke checks during prototyping and required checks
at stable checkpoints. Use integrated games to find defects; defer exhaustive at stable checkpoints. Use integrated games to find defects; defer exhaustive
combination testing, AI calibration and final visual polish to refinement unless combination testing and final visual polish to refinement unless an active defect
an active defect requires them sooner. Preserve material rule questions, mode requires them sooner. Full AI implementation, strategic refinement and calibration
gates, selective subagents and automatic verified Git pushes. The earlier batch wait until all non-AI features are complete; earlier AI work is limited to minimal
legal participation and critical correctness fixes. Then pursue the three
approximately 75% adjacent-pair targets in the [AI plan](AI_DEVELOPMENT_PLAN.md).
Preserve material rule questions, mode gates, selective subagents and automatic
verified Git pushes. The earlier batch
estimates remain historical planning evidence and should be reassessed for this estimates remain historical planning evidence and should be reassessed for this
broader prototyping workflow, without an assumed savings percentage. broader prototyping workflow, without an assumed savings percentage.
Check account usage after each verified checkpoint. The user explicitly confirmed On 20 September the user removed the prior usage cutoff and routine account-usage
the stop point as **80% weekly usage consumed (20% remaining)**. Keep the goal checks. Preserve Git and subagent requirements and leave the removed recurring
unfinished when stopping for usage, preserve Git and subagent requirements, and automation absent. Earlier usage counters in this log are historical only.
leave the removed recurring automation absent. After `32791ed`, usage was 3% used.
Use [current status](CURRENT_STATUS.md), [rule decisions](RULE_DECISIONS.md) and Use [current status](CURRENT_STATUS.md), [rule decisions](RULE_DECISIONS.md) and
[verification workflow](VERIFICATION_WORKFLOW.md) as short entry points. Historical [verification workflow](VERIFICATION_WORKFLOW.md) as short entry points. Historical