# 090 — Verification and evidence

## Scope and authorization

This document defines the verification contract. Checks actually performed during the authorized implementation are recorded in `110`; no tests were executed during the earlier planning-only phase. Matrix is the requested primary demo target, with voice off. Unrelated broad suites, paid AI/TTS, browser automation, other device testing and deployment still require applicable authorization.

Real Kodi visual comparison is a release gate for changed layouts; if permission is not yet available, record that gate as pending and ask Standa in final chat. Do not label the UI verified from XML or headless screenshots. Do not perform unrelated full-skin baseline runs.

## Behavioral checks

Implemented tests are consolidated in `tests/test_story_game.py` and `tests/test_story_transport.py`, using existing Kodi stubs and mocked transports. Run them as isolated scripts; combined discovery can mix pre-existing fake-module installations. Test behavior, not a line-for-line duplicate of implementation.

| Area | Required assertions |
|---|---|
| Content | Exact counts/groups, required/types/limits, distinct A–D, correct letter, identity/scope, unavailable, no silent defaults |
| Lives | Start at three; exactly one lost per wrong; no loss on API error; win with mistakes; final zero-life loss precedence |
| Input/state | Illegal/double inputs ignored; correct counters; wrong answer still advances plot; no autoplay or countdown |
| Pack lifetime | No cross-attempt content reuse or history; replay resets; author demo fixed; generated pack released |
| Generation | One call per explicit preparation; no answer/feedback calls; request failures, truncation, cancellation, late result ignored |
| Entry | Remembered eligible mode; fallback without preference overwrite; person/mix exclusion; demo source never becomes AI |
| Scope | Movie/year, whole series, episode with type `tv`/`series`, existing season scope if actually supplied; query round-trip |
| View | Complete keys/clearing; chapter progress uses answered count; answers/ending hidden until allowed; stale properties absent |
| Settings | Story difficulty independent; shared voice unchanged; invalid stored mode/difficulty defaults; no Story spectator rows |
| Modal | One active adapter, valid focus, retirement on switch/Back/close, late callbacks cannot update a replacement |
| Accounting | Shared provider records call once; Story code does not double-track, hardcode models, or persist packs |

Use provider stubs with known ready/unavailable/invalid responses. Do not use paid calls to discover local state-machine bugs.

## Existing regression to select by changed seam

| Changed seam | Existing checks to select |
|---|---|
| Canonical entry/subject | `tests/test_millionaire_entry.py` |
| Settings helpers/title settings | `tests/test_millionaire_settings.py` |
| Option-placement helper | `tests/test_millionaire_ai_content.py` |
| Modern generator/property/focus behavior | `tests/test_millionaire_modern.py`, focused golden scenarios |
| Demo launch/source | `tests/test_millionaire_demo.py` and the Matrix Millionaire harness run |
| Modal/adapter lifecycle | `tests/test_millionaire_golden.py` and relevant existing error/exit scenarios |

Current simple entry tests use patched constructors and source-pattern assertions. Adjust narrowly for real shared-launch behavior if needed; do not keep a meaningless assertion solely because it matches an old file location. Preserve public compatibility and actual outcomes.

The existing command `python3 scripts/game_harness/game.py golden verify` replays Millionaire headless goldens. Run it when shared gameplay/lifecycle/extraction changes justify it; do not repeatedly broaden tests after relevant checks pass.

## Static/artifact checks

- Compile touched Python without triggering network/provider/UI work; use the repository's normal targeted approach.
- Generate modern XML from its source, parse it, and run `xmllint --noout` when available.
- Detect duplicate control IDs, nonexistent default/focus/navigation targets, and mode visibility overlap.
- Verify cs/sk/en new strings and settings references; no duplicate translation IDs.
- Validate all three authored packs through the production validator.
- Check release inclusion of `wcs/games/story` and `resources/demo/story` when a targeted dry build is justified/authorized. Do not publish or bump versions manually.
- Inspect the final diff for unrelated changes, copied logic, missing cleanup, model changes, new caches, and credentials.

## Headless integration

Drive the real common modal + Story adapter + engine + demo source with existing Kodi stubs. Run every deterministic scenario in `080-matrix-demo.md`. Capture structured state/property/focus evidence and compare authored text against the fixture. Verify zero provider/TTS calls.

Implemented Story demo CLI commands are documented in `080`. Keep existing Millionaire coverage intact. The future paid model comparison will need a bounded Story review runner and explicit content-capture policy; current Millionaire prompt override/resend commands are not Story tooling.

## Authorized visual checks

Use the existing Kodi/dialog/game harness skills and capture tools; do not invent browser tests for a Kodi modal. Compare with baseline frames from the same current installation and resolution. Required states: title both modes and selector focus, settings/rules, loading, scenes at chapter changes, A–D focus, correct/wrong results, life counts 3/2/1/0, win/loss/results, error, exit confirmation, long text/title and missing art.

Confirm shared edges and gaps, text readability, no clipping, reachable scrollbars, no focus on hidden lifelines, no jump to stale controls, no lost artwork, and video continuation beneath a launch from playback. Pixel-identical comparison applies to unchanged Millionaire gameplay screens; the intentionally added launch selector is reviewed as a deliberate change.

Store paths and concise geometry/diff findings in `110-progress-and-handoff.md`. Preserve baseline references; do not overwrite a golden merely to make a failed comparison pass.

## Optional paid content review

Only on request: report selected provider/model, run one bounded Matrix pack test, inspect all scenes/reveals/ending for facts, order, coverage and leaks, and report actual latency/usage/errors. This verifies that observed pack, not a universal accuracy guarantee. Testing every provider/model is outside initial acceptance unless requested.

## User acceptance

Give Standa a short Czech numbered checklist covering mode memory, all three difficulties, one/two/three mistakes, immediate result/Continue, five chapter markers, replay, exact selected episode, and unchanged Millionaire. Record the actual reply; silence/tool `accepted` is not approval.