diff --git a/docs/development.md b/docs/development.md index 3bc0b9b5..04db63fb 100644 --- a/docs/development.md +++ b/docs/development.md @@ -49,7 +49,7 @@ implementing changes. It owns implementation discipline and local commit rules. | Output, accepted-result reuse, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. | | External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. | | Repository-specific release commands or diagnostics | [Source Releases](release.md) | The runbook supplies publication commands and recovery procedures. | -| Implementation status or future scope | [Implementation plan](roadmap/implementation.md), [native migration target](roadmap/promptkit.md), and [separate backlog](roadmap/future.md) | The plan records verified work and remaining acceptance; the target owns cutover requirements and the backlog keeps other work separate. Implemented behavior belongs in canonical documentation. | +| Future scope and planning | [Future work](roadmap/future.md) | The backlog records proposed work; active feature and implementation plans live alongside it when present. Implemented behavior belongs in canonical documentation. | For an existing subsystem, also inspect its focused tests and the package-local types and contracts before changing behavior. diff --git a/docs/roadmap/archive/implementation.md b/docs/roadmap/archive/implementation.md deleted file mode 100644 index 5238d5dd..00000000 --- a/docs/roadmap/archive/implementation.md +++ /dev/null @@ -1,322 +0,0 @@ -# Policy Alignment Implementation Plan - -## Purpose And Execution Contract - -Implement [Standard Repository Policy Alignment](policy.md). That feature -roadmap owns the target policy choices and justified Notarius exceptions; this -document owns implementation order. It replaces the completed kit-upgrade plan, -not the implementation or contracts delivered by that earlier work. - -Each numbered stage is one bounded prompt for a gpt-5.6-terra coding agent. -Complete stages in order. Inspect the worktree before each stage, preserve -unrelated work, validate the resulting documentation, and commit the coherent -stage under agent policy. Report changed files, checks, and any concrete blocker. -Do not push, tag, publish, or retire unrelated roadmaps. - -This is a documentation/policy migration. Do not change application code, -dependency pins, tests, release scripts, runtime contracts, or release gates. -No production credentials or model calls are needed. - -## Fixed Sources And Adoption Rules - -Use upstream full commit -`d84a38ebab26f048cafbe59a2246700c35d29843` throughout. -Begin with its -[adoption guide](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/adoption.md). -Read applicable local policies through `docs/development.md`; when revising a -policy, also read the upstream counterpart and affected cross-policy provisions. -Do not reconstruct historical adoption baselines. - -Retrieve pinned files through public URLs or `git show` against that exact -commit in an available checkout. Do not use an unverified sibling working tree -or silently select newer upstream content. An unavailable source is a concrete -blocker to copying it accurately, not a reason to invent its text. - -For every adopted policy: - -- Copy upstream wording verbatim except for required template removal and the - agreed, substantive project-specific customizations in the feature roadmap. -- Retain its upstream Base Policy Version; set Last Updated to the actual date - the local document is adopted, not the review date by habit. -- Add Upstream Source pointing to its own file at the full selected commit: - `https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/FILE.md`. -- Remove the entire Base Policy Template section and perform its instructions. - Keep upstream generic rules outside it, including commit-pinned example links. -- Explain substantive local exceptions briefly where they apply, or retain an - existing rationale/ADR link. Do not create an exception registry or preserve - stylistic paraphrases merely because they already exist. - -Current public project and adoption-guide links are discovery links; they may -follow main. Source, example, and attribution links stay commit-pinned. Preserve -accurate preexisting example pins supplied by the selected OSS template. -Metadata must reflect only documents actually adopted by the current stage. -The transition may temporarily contain mixed versions without claiming otherwise. - -## Stage 1 — Establish Provenance And Copied-Policy Licensing - -**Status:** Complete (2026-09-28). - -**Goal:** Establish the self-contained adoption and licensing foundation. -One prompt limited to two short policies and their actual notice materials. - -**Read:** Selected upstream `docs/policy/meta.md` (1.0), -`docs/policy/oss.md` (1.8), `COPYRIGHT`, -`examples/licensing/policy-attribution.md`, and the adoption guide; -local `COPYRIGHT`, `LICENSE`, and the feature roadmap. - -**Files:** Create `docs/policy/meta.md`, `docs/policy/oss.md`, -`docs/oss/attribution.md`, and -`docs/oss/licenses/CC-BY-SA-4.0.txt`; update `COPYRIGHT`. - -1. Adopt meta and OSS using the fixed-source rules. Keep public discovery URLs - in meta and link to the existing local developer guide and new attribution. - Point project-specific OSS context to Notarius's grant and release runbook; - do not import generic example projects. -2. Create attribution with `# Open Source Attribution Notices` followed by - the canonical policy entry. Replace all three UPSTREAM_COMMIT placeholders. - Initially identify the two newly adopted policy files as adapted material. - Later stages expand this statement as their material is actually adopted; - do not imply that the older five files have already been refreshed. -3. Copy the complete CC BY-SA 4.0 standard text from the pinned reference asset. - Preserve the canonical copyright/SPDX/notices/source arrangement and its - modification indication. Do not add research or compliance prose. -4. Add a distinct scope statement to COPYRIGHT: policy material adapted from - Standard Repository Policies remains CC-BY-SA-4.0, with attribution defining - the affected material. Keep the existing GPL-3.0-or-later application grant, - warranty disclaimer, and third-party statement; leave root LICENSE unchanged. -5. Do not create placeholder dependency entries. Dependency inventory is assigned - to Stages 4–5, and remains tracked here rather than asserted complete. - -**Acceptance:** Both policies have correct versions, date, file-specific source -links, and no template-only section. All new relative links resolve. Attribution -describes this stage's actual material, and code licensing is unchanged. - -**Validation:** Compare copied policy bodies and license text to the pinned -sources; inspect deliberate substitutions. Run `git diff --check` and review -the COPYRIGHT diff. No Go tests are needed. - -## Stage 2 — Standardize General Policies And Selective Orientation - -**Status:** Complete (2026-09-28). - -**Goal:** Adopt the general rules and their reading workflow. One prompt: -three largely verbatim policy updates plus small orientation edits. - -**Read:** Local agent/testing/documentation policies; pinned upstream versions -1.3, 1.3, and 1.11 respectively; upstream `agents/AGENTS.md` and the adoption -guide's Policy Reading Guide; local `AGENTS.md` and `docs/development.md`. - -**Files:** Update `docs/policy/agents.md`, `testing.md`, -`documentation.md`, root `AGENTS.md`, `docs/development.md`, and -`docs/oss/attribution.md`. - -1. Adopt agent and testing substantive wording without local paraphrases. - Remove the redundant agent preamble, additional generic contract-preservation - prose, and ADR-0017 illustration. Its decision remains in architecture/ADR; - this is not permission to change it. -2. Adopt documentation wording and ownership rows. Customize the consumer row - for existing subprocess guidance, not a hypothetical public Go API. - Omit the inapplicable HTTP contract row; preserve applicable CLI/config/ - operations/integration/internal ownership. Remove the local duplicate - implementation-details-versus-policy section. -3. Base root AGENTS.md on upstream `agents/AGENTS.md`, not upstream root - AGENTS.md. Preserve applicable existing instructions. Require orientation - through the developer guide and agent policy, but not full-suite reading. -4. Add the adoption guide's seven-row selective Policy Reading Guide and its - selection/reuse/scope-expansion rules to docs/development.md. Retain existing - subsystem navigation and validation guidance; remove redundant competing - policy-trigger rows from the old table rather than creating two authorities. -5. Expand the attribution material statement to identify the newly refreshed - policy files, adapted AGENTS.md, and copied policy-reading-guide portion of - docs/development.md. Do not license the unrelated remainder as copied policy. - Ensure the COPYRIGHT scope statement covers these identified portions. - -**Acceptance:** General rules match upstream, except required documentation -ownership customization. All seven policy triggers are available; existing -subsystem links still work. Reading is selective and public discovery is reachable. -No test changes or stricter testing/commit requirements were introduced. - -**Validation:** Diff each policy against its pinned source, inspect AGENTS and -developer-guide navigation, check attribution scope and relative links, and run -`git diff --check`. Search affected files for leftover template instructions. - -## Stage 3 — Align Architecture And Release Policies Without Contract Changes - -**Status:** Complete (2026-09-28). - -**Goal:** Complete the customized policies and release navigation. One prompt -focused on preserving existing decisions rather than redesigning them. - -**Read:** Local architecture and release policies, docs/release.md, relevant -linked ADRs; pinned upstream architecture 1.3 and release 1.4. Use -docs/internal/overview.md and existing canonical contracts only where needed to -resolve an ownership question. - -**Files:** Update `docs/policy/architecture.md`, `docs/policy/release.md`, -`docs/release.md`, and `docs/oss/attribution.md`. - -1. Adopt upstream general architecture ownership/approach guidance. Replace its - application-specific template guidance with Notarius's existing decisions. - Preserve the feature roadmap's enumerated pipeline, module, assets, domain, - identity, validation/retry, scheduling, configuration/provenance, state, - secret-handling, permissions, platform, and distribution contracts. - Keep existing application-specific sections unless an edit is necessary for - the adopted structure; do not turn this into a broad architecture rewrite. -2. Adopt release wording with narrow project customizations: source only, - lightweight tags, main/origin, branch-before-tag, no prereleases, and no - hosted/asynchronous deliverables. Keep the native CLI/version check and - GOWORK=off rationale. Preserve the historical no-retrospective-notes decision. - Retain unchanged generic requirements verbatim. -3. Add the OSS footer rule for new notes. Update the runbook with a copyable - three-link COPYRIGHT | LICENSE | ATTRIBUTION footer using absolute - `https://gitea.maximumdirect.net/eric/notarius/src/tag//...` paths. - Explain substitution of the selected release tag and local path checking. - Do not require a future tag to resolve or change publication scripts. -4. Expand policy attribution to the complete adopted docs/policy set, retaining - the AGENTS/developer-guide scope from Stage 2 and any relevant adaptation - indications. All seven per-document source links now identify this adoption. - -**Acceptance:** Existing architecture and release choices are unchanged and their -local reasons remain clear. Upstream generic wording is used where applicable. -There are no new gates, platforms, artifacts, retrospective notes, or ADR edits. - -**Validation:** Review policy diffs specifically for added/removed requirements, -check runbook agreement and affected links, and run `git diff --check`. -Do not run a release/publication command. No runtime tests are warranted. - -## Stage 4 — Attribute The Kit Dependencies And Catalogs - -**Status:** Complete (2026-09-28). - -**Goal:** Add the first bounded group of component notices: ConfigKit, -PromptKit, and their two backend catalogs. One prompt for four related projects. - -**Read:** Adopted OSS policy; go.mod/go.sum, resolved build dependency metadata; -exact selected module COPYRIGHT/LICENSE/NOTICE/attribution files and VCS metadata. -The reviewed pins are ConfigKit v0.4.1, PromptKit v0.11.1, and both catalog -modules v2.20260926.0; verify the actual selected versions before writing. - -**Files:** Update `docs/oss/attribution.md`; add applicable texts under -`docs/oss/licenses/`. - -1. Use read-only discovery such as `GOWORK=off go list -m -json all`, - `GOWORK=off go list -deps ./cmd/notarius`, and selected module contents. - Dependency acquisition may populate the module cache but must not change - dependency pins or introduce a local replace directive. -2. Establish each selected version's exact source commit from module/VCS - metadata; peel annotated tags to commits. Do not substitute a tag object ID, - main-branch head, assumed sibling HEAD, or version tag in attribution URLs. -3. Add factual House Format entries with supplied copyright statements, - applicable SPDX expressions, local standard-text links, required upstream - notice links, and exact-commit Source links. Preserve alternatives and - exceptions; determine scope from contents rather than filenames. -4. Include LGPLv3 and its GPLv3 companion for the kit grants, and MIT for the - catalogs where confirmed by the pinned sources. Reuse shared local standard - texts; leave root project LICENSE unchanged. -5. Inspect upstream dependency inventories for leads, but do not copy them - wholesale or treat them as substitutes for Notarius's own inclusion scope. - Hand remaining selected third-party components to Stage 5 through the stage - report, not editorial prose in attribution. - -**Acceptance:** These four components have verified source/notice links and -complete applicable local texts. No dependency changes, guessed notices, or -new distribution obligations were invented. - -**Validation:** Check exact revision correspondence, upstream filenames, every -new local license link, and byte fidelity of standard texts. Run -`git diff --check`; confirm go.mod/go.sum are unchanged. Report discovery -commands and evidence; do not add a persistent inventory tool or audit artifact. - -## Stage 5 — Complete Attribution And Connect Reader Navigation - -**Status:** Complete (2026-09-28). - -**Goal:** Finish remaining incorporated-component coverage and canonical -navigation. One prompt for the remaining small dependency set and doc links. - -**Read:** Adopted OSS policy, remaining selected module/source notices, -README.md, integration guides, and repository evidence of copied code/assets. -Start with jsonschema/v6 v6.0.2, x/text v0.40.0, yaml.v3 v3.0.1, and yaml/v4 -v4.0.0-rc.6, then resolve any additional actual incorporation revealed by -Stage 4. A go.sum entry alone is not proof of incorporation. - -**Files:** Update `docs/oss/attribution.md`, applicable -`docs/oss/licenses/*`, `README.md`, -`docs/integrations/pkg-configkit.md`, and -`docs/integrations/pkg-promptkit.md`. - -1. Inspect direct/transitive incorporation and copied material separately. - Apply the same exact-revision, factual-entry, and faithful-text rules as - Stage 4. Include relevant components, not every tool used in development. - Do not add generic entries with unresolved versions or copyrights. -2. Use the OSS format for omitted copyright/SPDX fields where the source - supplies no applicable value. Preserve required project-specific notices. - Raise only a concrete unresolved obligation or rights issue; do not seek - blanket approval for the already selected linked-notice arrangement. -3. End README with the prescribed GPL-v3-or-later COPYRIGHT link and the - Open Source attribution notices link. Do not imply that every incorporated - component or document has the application's license. -4. Replace competing license/notice paragraphs in both integration guides with - concise links to the relevant canonical attribution anchors. Keep dependency - pins, behavior, compatibility, and public API links in the guides. -5. Keep the attribution document to its title and factual component entries. - Do not include the inventory research, legal conclusions, or completion logs. - Do not copy unused upstream examples or standard licenses. - -**Acceptance:** Attribution covers the verified relevant components and copied -policy material, with only applicable texts. README and integration links point -to the canonical inventory; no user-visible API or dependency settings change. - -**Validation:** Review coverage against resolved dependencies and inspected -copied-material evidence, validate affected relative/source links and text -fidelity, and run `git diff --check`. No Go suite, scanner, or SBOM is required -solely for documentation changes. - -## Stage 6 — Verify The Complete Adoption - -**Status:** Complete (2026-09-28). - -**Goal:** Perform the cross-document check and correct scoped documentation -defects. One prompt; no fresh feature work or automated policy-sync system. - -**Read:** The feature roadmap; the seven resulting policies and their pinned -sources; AGENTS.md, developer guide, COPYRIGHT, README, attribution/license -materials, release runbook, and affected integration links. - -**Work and acceptance:** - -1. Compare all seven policies directly against the selected revision. Classify - remaining differences as metadata, template removal, required customization, - or a feature-roadmap exception. Replace accidental paraphrases with upstream - wording. Preserve established local requirements. -2. Confirm versions 1.3/1.3/1.11/1.3/1.4/1.0/1.8 for agents, architecture, - documentation, testing, release, meta, and OSS respectively. Verify each - Upstream Source targets its own file and Last Updated reflects actual work. -3. Confirm discovery links follow public project/main-guide URLs while adopted - source, examples, and component attribution remain accurately commit-pinned. - Check no placeholder UPSTREAM_COMMIT, template-only section, or local - checkout-dependent reference remains in adopted materials. -4. Check selective policy routing, local authority, canonical documentation - ownership, copied-material scope/modification status, unchanged application - grant/root LICENSE, and directly reachable license/companion texts. -5. Verify source-only publication, selected gates, optional diagnostics, - release completion, and prospective-only footer guidance still agree. - Do not modify historical notes, accepted ADRs, or scripts to satisfy this check. -6. Review changed-file scope: only intended policy, notice, and supporting - documentation files should have changed. Correct in-scope documentation - defects, report unresolved concrete issues, and leave unrelated roadmap - retirement and release preparation to an explicit later request. - -**Validation:** Run `git diff --check`; inspect file/link destinations and -compare local standard texts with verified sources. A search for -`Base Policy Template` or `UPSTREAM_COMMIT` in adopted files should be empty; -do not confuse explanatory roadmap text with an unfilled adopted template. -No production tests, new test framework, recurring synchronization gate, or -release execution is needed. - -## Open Questions - -None at the agreed baseline. Policy choices and exceptions are settled. -A newly discovered concrete notice/source or rights issue should be reported -with evidence and viable treatments rather than resolved by inventing a broader -requirement. That contingency does not authorize reopening settled policies. diff --git a/docs/roadmap/archive/policy.md b/docs/roadmap/archive/policy.md deleted file mode 100644 index 649e125f..00000000 --- a/docs/roadmap/archive/policy.md +++ /dev/null @@ -1,252 +0,0 @@ -# Standard Repository Policy Alignment - -## Purpose - -Bring Notarius's adopted policies into alignment with Standard Repository -Policies, using identical upstream language wherever a substantive, -project-specific decision does not require customization. Preserve Notarius's -application contracts, operator choices, selected code license, and source-only -release process. - -This roadmap defines the agreed target and update scope. It does not itself -adopt policies, alter runtime behavior, or authorize a release. - -## Selected Upstream Revision And Current Baseline - -The agreed review, performed on 2026-09-28, uses full upstream commit -`d84a38ebab26f048cafbe59a2246700c35d29843` of -[Standard Repository Policies](https://gitea.maximumdirect.net/eric/policy). -Follow the [adoption and update guide at that revision](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/adoption.md). -Use that fixed revision for implementation, not whichever revision is then on -`main`. - -The comparison is between the current Notarius documents and that revision. -Historical adoption baselines need not be reconstructed or assessed. Introducing -source metadata records the revision actually incorporated by this update, not -a claim about previous adoptions. - -Notarius currently has five policies, no `meta.md` or `oss.md`, and no -`docs/oss/` attribution collection. Its current agent entry point and developer -guide provide orientation but lack the complete selective policy-reading guide. -The project grant is GPL-3.0-or-later; root `LICENSE` contains the GPLv3 text. -README licensing navigation and prospective release-footer instructions are -missing. - -Public upstream discovery and attribution are now expressly approved. This -supersedes the earlier instruction to omit references to the policy source; -local checkout paths are neither necessary nor appropriate public references. - -## Alignment Rules - -- Start from the selected upstream wording, not a paraphrase of it. Adopt - non-substantive wording changes verbatim. -- Retain local variations only for substantive, locally significant - requirements or decisions. Explain their purpose in the relevant policy - section or link to existing rationale; do not add a separate exception registry. -- Follow each template's adoption instructions and remove its entire - `Base Policy Template` section. Replace required project-specific guidance - rather than leaving template instructions in an adopted policy. -- Preserve deliberately selected contracts. Simplifying duplicate wording does - not authorize changing architecture, runtime behavior, or release mechanics. -- Keep local policies authoritative. Upstream changes take effect only through - deliberate adoption; no automatic synchronization is introduced. - -## Target Policy Set And Justified Customizations - -| Local file | Current version | Target version | Required treatment | -| --- | --- | --- | --- | -| `docs/policy/agents.md` | 1.1 | 1.3 | Adopt substantive upstream wording verbatim. Remove redundant local preamble and the ADR-0017 illustration; architecture already owns the deployment-permissions decision. No substantive exception is needed. | -| `docs/policy/architecture.md` | 1.1 | 1.3 | Adopt general purpose, ownership, architectural approach, and constraint-versus-implementation guidance while retaining the application-specific decisions described below. | -| `docs/policy/testing.md` | 1.1 | 1.3 | Adopt substantive upstream wording verbatim. Existing risk-based testing remains intact; no project-specific exception or test rewrite is needed. | -| `docs/policy/documentation.md` | 1.3 | 1.11 | Adopt upstream wording and licensing/provenance ownership rows. Customize consumer ownership for subprocess use and omit the inapplicable HTTP-contract row. Remove the duplicate local implementation-details-versus-policy section; agent policy owns that rule. | -| `docs/policy/release.md` | 1.1 | 1.4 | Adopt upstream wording with the existing source-only choices and selected gates below. Add prospective OSS-footer guidance. | -| `docs/policy/meta.md` | Absent | 1.0 | Add upstream substantive wording, including public discovery links, local authority, and local attribution navigation. | -| `docs/policy/oss.md` | Absent | 1.8 | Add upstream substantive wording, commit-pinned example links, and required Notarius context through canonical documents. | - -No existing policy remains entirely unchanged: all five need wording and/or -provenance updates. This does not imply new substantive testing or agent rules. - -### Architecture Exceptions - -The architecture template expressly requires application-specific content. -Retain Notarius's fixed stage sequence rather than an arbitrary DAG, -artifact-family ownership across stages, domain-neutral framework, inward -dependency direction, external adapter boundaries, and content-only assets -package. Retain adopted validation/retry, scheduling, identity, evidence, -configuration, and provenance contracts. - -Preserve operator-owned filesystem access policy and the separately adopted -correctness boundaries, cache/checkpoint eligibility, output/debug lifecycle, -secret handling, and platform/distribution decisions. These express existing -product requirements and ADR-backed choices, not stylistic variations. -Use existing architecture and ADRs as their authority; this migration does not -authorize strengthening or weakening them. - -Keep concrete package inventories and implementation mechanics in their current -internal-documentation homes. No architecture redesign, new threat model, or -new abstraction is part of this work. - -### Documentation Exceptions - -Notarius's consumer interface is primarily a subprocess CLI, not a public Go -API. Retain `docs/consumers/` ownership for that workflow and its consumer -responsibilities. Retain integration contracts as the owners of durable receipt -and artifact shapes. No HTTP service documentation or placeholder API document -is needed. - -Other editorial variations should use upstream language, including the current -versus future behavior and accepted-ADR rules. Preserve their meaning without -retaining duplicate local formulations. - -### Release Exceptions - -Preserve source-only delivery, lightweight tags, `main` on `origin`, -branch-before-tag publication, and no prereleases. There are no binaries, -hosted release objects, or asynchronous artifacts to publish or recover. -Use upstream generic language where applicable, but do not introduce those -channels or atomic-push requirements. - -Retain the native CLI build and injected-version check with `GOWORK=off`: -it protects the executable entry point and release version, which package tests -alone do not verify. Preserve its documented rationale and existing optional -diagnostics. No additional validation gate is warranted. - -Retain the decision not to create retrospective notes for v0.1.0, v0.2.0, or -v0.3.0. Apply new footer formatting prospectively; do not rewrite published -history to retrofit it. - -## Provenance And Selective Reading - -Each of the seven policies must retain its actual target `Base Policy Version`, -set `Last Updated` to its actual local adoption date, and add an -`Upstream Source` link to its own source file. For example: - -```text -https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/testing.md -``` - -Substitute the correct filename for each policy; do not link every header to -the repository root or adoption guide. Retain provenance through subsequent -local edits. - -Use current public discovery URLs in `meta.md`: - -- https://gitea.maximumdirect.net/eric/policy -- https://gitea.maximumdirect.net/eric/policy/src/branch/main/docs/adoption.md - -Discovery links intentionally follow the current project and guide. -Adopted-source, example, and attribution links identify immutable commits. -The OSS template already includes commit-pinned example references; retain -accurate pinned references rather than converting them to branch links. - -Adapt upstream `agents/AGENTS.md` into root `AGENTS.md`, preserving applicable -existing instructions. Direct readers to `docs/development.md`, require agent -policy before recommending/planning/implementing changes, and explicitly avoid -reading the whole policy suite by default. - -Merge the upstream adoption guide's Policy Reading Guide into -`docs/development.md`. Include all seven task triggers, especially policy -updates, dependency/licensing changes, validation selection, and compatibility -decisions. Preserve focused subsystem navigation without duplicating policy -triggers across competing tables. Reuse policies already in context, expand -reading when scope expands, and read affected cross-policy provisions when -revising a policy. - -## Licensing And Attribution Target - -### Copied Policy Material - -Follow the selected upstream grant and canonical policy-attribution example: - -- Create `docs/oss/attribution.md` with the House Format title and insert the - Standard Repository Policies entry using the selected full commit for its - source, COPYRIGHT, and LICENSE URLs. -- Identify actual copied/adapted material and modification status. Include - `docs/policy/`, the adapted root `AGENTS.md`, and the copied policy-reading - guide within `docs/development.md` as applicable. Do not claim unrelated - developer documentation was copied or relicense unrelated material. -- Preserve supplied copyright, license, and disclaimer notices through the - specified arrangement; keep the entry factual rather than adding compliance - or research commentary. -- Copy the complete, verbatim `CC-BY-SA-4.0.txt` to `docs/oss/licenses/`. -- Add the separately licensed policy-material scope to `COPYRIGHT`, pointing - to attribution for exact scope and notices. Adapted policy material remains - CC-BY-SA-4.0; application code remains GPL-3.0-or-later. Leave root `LICENSE` - unchanged. - -The upstream licensing-format examples are CC0; using their original formatting -alone does not require a separate attribution entry. This does not exempt -copied policy prose from its stated terms. - -### Incorporated Dependencies And Assets - -Complete the previously planned inventory of relevant direct/transitive -dependencies and copied code/assets using `go.mod`, resolved build dependencies, -and exact pinned source contents. Neither direct imports alone nor every -`go.sum` entry defines the inventory. - -Current starting points are ConfigKit v0.4.1, PromptKit v0.11.1, both PromptKit -backend catalogs v2.20260926.0, jsonschema/v6 v6.0.2, x/text v0.40.0, yaml.v3 -v3.0.1, and yaml/v4 v4.0.0-rc.6. Verify actual incorporation, terms, notices, -and exact commits; this list is not a pre-approved attribution inventory. - -Use the upstream OSS House Format without editorial elaboration. Copy only -applicable standard and companion license texts, retaining their wording. -Link component-specific notices and sources at the exact imported commit. -Apply the upstream house interpretation of linked notices without inventing -extra local-copy requirements, recurring approvals, or licensing conclusions. -Handle a concrete uncovered obligation separately if one is discovered. - -### Navigation And Canonical Ownership - -- End `README.md` with the project-grant and attribution links prescribed by - OSS policy, using Notarius's GPL-3.0-or-later grant. -- Update `docs/release.md` with a copyable prospective - `COPYRIGHT | LICENSE | ATTRIBUTION` footer using absolute Notarius URLs at - the release tag or corresponding immutable source revision. Check intended - local paths before publication; do not require an unpublished tag to resolve. -- Update `docs/integrations/pkg-configkit.md` and - `docs/integrations/pkg-promptkit.md` to link to canonical attribution rather - than maintaining competing notice inventories. Keep behavioral/API references - in those integration guides. - -## Local Materials And Upstream-Only References - -Keep the seven adopted policies, adapted agent entry point, project-specific -developer guide, actual notices, and applicable license texts locally. - -Keep the upstream README, root AGENTS.md, contributor guide, adoption guide, -and example collection upstream. Link to them when appropriate instead of -copying an additional documentation tree. Do not copy unused license texts, -example projects, or placeholder documents. Root Notarius `AGENTS.md` is based -on upstream `agents/AGENTS.md`, not upstream's own contributor instructions. - -## Implementation Ownership - -[Implementation Plan](implementation.md) owns the ordered work stages and their -validation. This document owns the target end state, scope, and policy choices. - -## Scope Boundaries And Acceptance - -- No application code, dependency pins, tests, release scripts, or published - history change merely to adopt policy wording. -- No new ADR, recurring policy-sync job, SBOM, scanner, blanket header sweep, - security audit, platform matrix, approval round, or release gate is required. -- All seven policies use the selected versions and correct file-specific source - links, with actual adoption dates and no remaining template-only sections. -- Non-substantive differences use upstream wording; substantive variations are - confined to justified Notarius responsibilities and decisions. -- Orientation supports selective reading and public upstream discovery without - requiring a sibling checkout. -- Attribution accurately identifies copied material, modifications, selected - dependency revisions, notices, and faithful local license texts. Application - and policy-material license scopes remain distinct. -- Canonical navigation works without duplicate notice inventories, nonexistent - interface documents, or copied upstream guide/example collections. -- Validate affected relative and public links, metadata targets, license-text - fidelity, and `git diff --check`. Ordinary document review is sufficient; - production model runs and new runtime tests are not warranted. -- Resolve only concrete newly discovered rights or scope issues with the user; - do not reopen settled choices through blanket approval requirements. - -There are no blocking open design decisions at the agreed review baseline. diff --git a/docs/roadmap/archive/remediation.md b/docs/roadmap/archive/remediation.md deleted file mode 100644 index 0dc3ed5d..00000000 --- a/docs/roadmap/archive/remediation.md +++ /dev/null @@ -1,124 +0,0 @@ -# Kit Upgrade Review Remediation - -## Resolution - -Completed 2026-09-26. No production changes were needed. - -- `internal/framework/llm/promptkit_observation_test.go` now covers retries in - both initial and repair generations, aggregate versus legacy accounting, - unknown/known-zero/partial usage, retry-wait cancellation, exhausted execution, - preparation failure, and overlapping request isolation. -- `internal/framework/pipeline/debug_execution_test.go` exercises the real - scheduled client and debug wrapper together. It verifies returned and persisted - summaries for provider errors and cancellation without a body, split debug and - fallback content, terminal references, and explicit usage-presence JSON fields. -- Temporary regression probes confirmed that discarding failed responses in the - scheduler and collapsing usage presence into known zero cause the new tests - to fail. Those probes were reverted. -- Focused LLM/pipeline tests, their race-enabled suites, and - `GOWORK=off go test ./...` passed. Existing no-debug and redaction coverage was - retained. Runtime budgets, scheduler policy, and output contracts are unchanged. - -The original finding below is retained as review context. - -## Review Scope - -Reviewed implementation commits `31b48d2` through `7ac9d94` against -[the feature roadmap](upgrade-kit.md) and the completed upgrade work recorded -in those commits. -The review covered configuration selection/defaults, production client wiring, -error translation, execution observation, scheduler/debug propagation, current -documentation, and tests. No production model calls were made. - -The implemented architecture is consistent with the intended design: ConfigKit -v0.4.0 owns composition and overlay provenance behind the configuration boundary; -PromptKit v0.11.0 owns transient retries and structural repairs behind the LLM -adapter. Notarius retains stage retry/disposition policy, completion-scoped -scheduling, application-owned error/observation types, and opt-in debug output. -No new runtime defect or unrequested restriction was established in this review. - -## Finding 1 — Complete The Missing Retry And Observation Verification - -**Priority:** Medium. This is a verification gap, not a demonstrated runtime -failure. Stages 5–7 are marked complete, but their consequential cross-boundary -acceptance tests were not implemented. - -### Evidence And Risk - -- `internal/framework/llm/promptkit_client_test.go:58` adds only a successful, - single-attempt observation assertion with positive prompt-token usage. It - does not exercise absent versus partial/known-zero usage, failed/canceled - execution summaries, preparation failure, or concurrent request isolation. -- `TestPromptKitClientAppliesConfiguredBuiltInTransientRetries` covers basic - status recovery, explicit zero, exhaustion, and a noneligible status. It - discards the response, so does not protect terminal accounting. There is no - combined transient-retry/structural-repair scenario or cancellation while - waiting on Retry-After. -- `internal/framework/llm/scheduled_client.go:40` now preserves a response when - the inner call fails. No corresponding test protects that behavior. Returning - an empty response again would silently remove precisely the failed-call - observations this feature is intended to retain. -- `internal/framework/pipeline/debug.go:267` and `:277` add execution summaries - to response artifacts and call references, plus fallback/content-splitting - propagation. Neither the debug nor terminal-debug tests were extended to - exercise these fields. Passing existing tests does not prove that failed-call - metadata reaches persisted records. - -These are Notarius-owned integration risks, not a request to re-test PromptKit's -backoff algorithm. Losing failed-call observations or misrepresenting incomplete -usage would be difficult to detect in ordinary successful runs. A focused set -of behavioral tests is justified under the testing policy. - -### Required Remediation - -Use existing test assets, local HTTP fixtures, scheduler wrappers, and debug -recorders. Combine related assertions; no fixed test count or new test framework -is required. - -1. Add an adapter scenario with a transient failure, a structurally invalid - completion, and a successful repair. Exercise retries in both generations - with small explicit budgets, and assert terminal attempts/retries/repair - counts without double-counting. Use controlled usage fields to distinguish - known zero, unknown, and partial aggregate usage. Preserve legacy usage - semantics independently. -2. Cover failed and canceled completions, including cancellation during a - Retry-After wait. Verify caller cancellation identity, bounded completion, - and retained execution metadata. Use signaling and a generous bounded test - timeout, not exact sleep durations or jitter assertions. Cover a preparation - failure with no execution summary. Add a small concurrent-call check for - request-local summary isolation, which race tests alone do not prove. -3. Exercise an error response through the real scheduled client and existing - debug wrapper/recorder seams. Assert that the summary reaches response - artifacts and terminal call references even without a provider response - body. Cover the content-splitting/fallback path with a representative response. - Reuse existing no-debug and redaction tests rather than duplicating their - whole inventories. Keep new summary assertions metadata-only. - -Retain existing scheduler policy, stage/validator budgets, cancellation -disposition, warning semantics, and debug opt-in behavior. Do not introduce a -total deadline, new retry caps, telemetry artifacts, or live API requirements -to make these tests easier. Fix production code only if these checks reveal a -violation of the already selected behavior. - -**Acceptance:** The added tests fail if failed responses lose their execution -summary at an adapter/wrapper boundary, usage presence is collapsed into zero, -or retry/repair totals are double-counted. They pass with the intended behavior -and provide evidence for the previously unverified stage acceptance criteria. - -**Validation:** Run focused LLM/pipeline tests, race-enabled tests for those -packages, and `GOWORK=off go test ./...`. No production credentials are needed. - -## Checks Completed During Review - -- `GOWORK=off go test ./...` -- `GOWORK=off go vet ./...` -- `GOWORK=off go build ./...` -- `GOWORK=off go mod tidy -diff` -- `GOWORK=off go test -race ./internal/framework/llm ./internal/framework/pipeline` -- CLI configuration validation for the minimal and complete examples, including - the complete example with `--config-overlay development`. -- `git diff --check` - -All passed. Existing green tests do not substitute for the missing behavioral -coverage above. Temporary diagnostic test material used during review was -removed; no application code was changed. diff --git a/docs/roadmap/archive/upgrade-kit.md b/docs/roadmap/archive/upgrade-kit.md deleted file mode 100644 index acef22ff..00000000 --- a/docs/roadmap/archive/upgrade-kit.md +++ /dev/null @@ -1,337 +0,0 @@ -# ConfigKit And PromptKit Upgrades - -## Purpose - -Adopt ConfigKit v0.4.0 and PromptKit v0.11.0 in one coordinated work set: - -- Consistently call ConfigKit's selected configuration layer an **overlay**, - distinct from PromptKit LLM profiles. -- Adopt configurable transient HTTP retries, provider-neutral failure - classifications, and execution observations from PromptKit. - -Preserve application/framework boundaries and keep the two upgrades as distinct -implementation workstreams. Neither requires a redesign of pipeline execution. - -This feature roadmap records the target state and scope. Its original -implementation plan was completed and later replaced by a new work plan. -Neither document authorized release publication. - -## Current Baseline - -The reviewed baseline is Notarius v0.7.1 (`53d9284`): - -- `go.mod` pins ConfigKit v0.3.0 and PromptKit v0.10.0. -- `internal/core/config/load.go` passes `LoadOptions.Profile` into - `configkit.Options[Document].Profile`. -- `--config-profile` selects `profiles/.yml` beneath the configuration - root for `run`, `config validate`, `config effective`, `config explain`, and - `pipelines list`. -- The maintained selected-layer example is - `examples/profiles/development.yml`; configuration and CLI tests create - equivalent directories and exercise selection and inspection. -- ConfigKit owns composition and inspection metadata. Notarius owns its - version 5 document, defaults, validation, source-relative path resolution, - runtime conversion, and CLI discovery. - -A dependency-only trial against v0.4.0 fails to compile at `load.go:33` because -`configkit.Options.Profile` no longer exists. The review found no direct -Notarius references to `LayerProfile`, `CodeProfile*`, or `LoadSecretsDir`. - -An isolated dependency-only trial against PromptKit v0.11.0 passes the complete -Notarius test suite and `go vet ./...`. That proves baseline compatibility, not -adoption of the new controls or observations. The current adapter does not -configure transient retries or attach an execution observer. It classifies -generation errors broadly and records successful-result usage and structural -repair counts. The outer scheduler holds a permit for the whole completion. - -## Upstream Changes - -### ConfigKit v0.4.0 - -The [ConfigKit v0.4.0 release guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.4.0/docs/releases/v0.4.0.md) -defines four breaking naming changes: - -| Surface | Previous | Target | -| --- | --- | --- | -| Selection API | `Options.Profile` | `Options.Overlay` | -| Selected file | `profiles/.yml` | `overlays/.yml` | -| Provenance layer | `LayerProfile` / `profile` | `LayerOverlay` / `overlay` | -| Diagnostic identifiers | `CodeProfile*` / `profile.*` | `CodeOverlay*` / `overlay.*` | - -There are no legacy aliases or directory fallback. Explicit imports from a -directory named `profiles` remain valid ordinary imports. Merge rules, -precedence, lifecycle, source-origin APIs, minimum Go version, and YAML parser -dependency are unchanged. - -The intervening v0.3.1 release makes the opt-in `LoadSecretsDir` API ignore -dot-prefixed entries. Notarius does not use this API; adding secrets-directory -bootstrap is outside this work. - -### PromptKit v0.11.0 - -The [PromptKit v0.11.0 release guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/releases/v0.11.0.md) -introduces additive runtime APIs and one changed default: - -- Built-in generation defaults to two additional transient HTTP attempts for - statuses 429, 500, 502, 503, and 504. Engine and request controls can override - the count, including zero. Injected `LLMClient` implementations are not - automatically replayed; a custom HTTP client on the built-in path still is. -- Backoff is cancellable and honors valid provider `Retry-After` timing. - Generation and HTTP timeouts apply per attempt; the caller context bounds - the whole operation. Transport failures, malformed responses, other HTTP - statuses, and invalid generated content are not transient-retry candidates. -- `RunRequest.Observer` reports lifecycle metadata, attempts, waits, outcomes, - and presence-aware usage, including work before a failed execution. It works - with prepared execution; preparation itself emits no execution events. -- `ErrTransport`, `ErrMalformedResponse`, `ErrRetriesExhausted`, and - `RetryExhaustedError` support typed classification without parsing error text. -- The license changes to LGPL-3.0-or-later. Catalog selections and the minimum - Go version are unchanged from v0.10.0. - -## Target End State - -### Dependency And Ownership - -Pin ConfigKit v0.4.0 and PromptKit v0.11.0 and tidy the dependency graph. -Continue using only ConfigKit's -public root API behind `internal/core/config`. No compatibility parser, -application-owned composition engine, or new configuration abstraction is -needed. PromptKit remains behind `internal/framework/llm`; its types must not -leak into domain modules or pipeline contracts. Retain the existing prepared -execution path, profile choices, and D&D model settings. - -### Consistent Overlay Selection - -- Replace the configuration-selection CLI flag with `--config-overlay` on all - commands that currently accept `--config-profile`. -- Rename application-owned fields, variables, helpers, help text, and test - descriptions that specifically represent this selection to overlay terms. -- Pass the selection through `configkit.Options.Overlay`. -- Select files only from `overlays/.yml`, following ConfigKit's contract. -- Preserve empty and `default` selection as the composed base configuration - without a selected overlay. -- Use a clean cutover: no old CLI alias, directory fallback, automatic file - moves, or custom migration machinery. Normal unknown-flag handling is - sufficient for the retired flag; explain migration in documentation. - -This rename is deliberately narrow. Preserve `llm_profile`, `--llm-profile`, -`promptkit.profile_dir`, `promptkit.profile_file`, `base_profile`, embedded -`dnd-extraction` profiles, and `examples/llm-profiles/`. Other application -concepts named `Profile`, including pipeline/runtime types, must not be renamed -merely because they share the word. - -### Preserved Behavior - -Retain configuration version 5, with the additive retry setting described below. -Preserve existing defaults other than the explicitly adopted transient retry policy, -imports, overlay merge behavior (including sequence replacement), explicit CLI -override precedence, strict decoding, domain validation, and secret redaction. -Do not add new restrictions or relax existing explicit contracts as part of the -rename. - -Relative paths continue to resolve from the file supplying the surviving -declaration, including overlay files. Invocation-relative CLI paths and -main-file-relative built-in defaults retain their current behavior. - -Effective configuration, explanations, and configuration-load diagnostics use -the new upstream overlay provenance and diagnostic vocabulary. Preserve source -file labels and actionable error context; do not translate the new metadata -back into legacy profile terminology. - -Pipeline stage order, semantic acceptance, LLM selection, domain artifact -schemas, and subprocess result receipts remain unchanged. No blanket checkpoint -invalidation is added merely for the ConfigKit rename. Update the existing -PromptKit-version identity used by checkpoint fingerprinting with its dependency -pin, following the current adapter convention; no cache redesign is needed. - -### Transient Retry Policy - -Expose `promptkit.transient_retries` as a nonnegative integer setting with a -Notarius default of **2 additional attempts**. Explicit zero disables transient -retries. Apply the resolved setting to the shared PromptKit engine so modules, -normalizers, validators, and semantic correction calls receive the same policy. -Include the effective value in configuration inspection. Do not impose an -arbitrary application maximum or add per-binding controls in this work set. - -Maintain three separate budgets and owners: - -| Budget | Owner | Repeated work | -| --- | --- | --- | -| Transient retries | PromptKit | The same generation request after eligible HTTP failures | -| Structural repairs | PromptKit | A corrective generation for invalid structured output | -| Producer/validator retries | Notarius | The configured stage or validator operation, with semantic feedback where applicable | - -Do not silently reduce, repurpose, or remove existing stage/validator budgets. -Each structural repair generation receives its own transient budget. For a -single-completion producer, two additional stage attempts, one structural -repair, and two transient retries permit up to 18 HTTP attempts, excluding any -separate validator calls. Document this multiplication and the possibility of -additional cost; retries do not imply exactly-once provider execution. - -Preserve caller cancellation throughout generation and waiting. Do not describe -a profile's generation timeout as a total completion deadline: provider -`Retry-After` can extend waits beyond local backoff, and each HTTP attempt has -its own timeout. A new total-deadline configuration surface is outside this -upgrade; accurately document the current controls and limitations. - -Retain Notarius's completion-scoped scheduler permit during upstream backoff. -PromptKit releases its active backend permit during that wait, but this does -not release the outer Notarius permit. This remains bounded and correct; do -not add an attempt-level scheduler redesign. - -### Provider-Neutral Failure Diagnostics - -Translate the new typed errors into application-owned classifications for -transport failures and malformed provider responses. Represent transient retry -exhaustion separately from its underlying failure/status, retaining the actual -attempt count where available. Preserve existing HTTP-status reporting, -capacity classification, cancellation, deadline handling, and redaction. - -Do not turn a malformed provider envelope into a semantic rejection or confuse -it with a structurally invalid model response. Do not expose PromptKit error -types across the adapter boundary or parse prose to recover classifications. -Existing stage disposition policies continue to decide what happens after a -completion fails; this work does not redesign those policies. - -### Execution Observations - -Attach a request-local observer through the existing prepared-execution path. -Translate selected metadata into Notarius-owned completion/debug summaries: - -- dispatched attempt, transient retry, and structural repair-call counts; -- capacity-wait, generation, retry-delay, and total execution durations; -- terminal outcome, failure classification, HTTP status, and exhaustion state; -- observed token counts with explicit known/complete indicators. - -Retain available summaries on failed completions, including cancellation, rather -than discarding them with the absent `RunResult`. Preparation failures have no -execution observation; do not fabricate one. Keep existing successful-result -usage fields and their meanings unchanged. Observation totals are separate -presence-aware metadata, not a second set of tokens to add to existing usage. -Unknown counts are not known zero; partial totals are not a billing ledger. - -Use lightweight per-request accumulation with no unbounded event history or -asynchronous queue. Callbacks are synchronous: avoid blocking I/O or acquisition -of scheduler resources inside them, and isolate concurrent requests. Persist -the projection through the existing opt-in debug path, including safe terminal -summaries where applicable. Do not serialize upstream event structs directly; -their JSON layout is not a public contract. Allowlist metadata and preserve -existing identifier/content handling and redaction boundaries. - -Successful transient recovery is ordinary success, not a warning. This work -does not add routine console messages, change the warning policy, introduce a -new durable telemetry artifact, or add complete cost accounting. - -## Required Changes - -### Code And Examples - -- Update `go.mod` and `go.sum` and the configuration loader/options boundary. -- Update CLI selection plumbing in `internal/cli/run.go` and - `configkit_commands.go`, including usage, flag parsing, argument classification, - explicit empty-value handling, and error paths. -- Move the maintained example to `examples/overlays/development.yml` and update - example discovery and related fixtures. Inspect relative paths when moving - files; do not assume all occurrences of `profiles` are configuration overlays. -- Update affected configuration and CLI tests, particularly document paths, - operational defaults, safe inspection, command selection, diagnostic source - labels, run contracts, and maintained-example coverage. -- Add the retry setting through the typed configuration document, defaults, - validation, runtime conversion, and CLI client-construction boundary. Update - the complete example to demonstrate it without adding unnecessary settings - to the minimal example. -- Update the PromptKit adapter's engine construction, request observation, - error conversion, and checkpoint version identity. Extend transport-neutral - response/debug contracts only with the application-owned metadata needed. -- Carry observations through the pipeline debug wrapper and safe terminal - reporting without double-counting usage or losing failed-call summaries. - -### Documentation - -Update current documentation with the implementation, keeping canonical -ownership under [Documentation Policy](../../policy/documentation.md): - -- `docs/cli.md`: new flag on every affected command and matching help behavior. -- `docs/config.md`: overlay terminology, directory layout, precedence, and the - explicit distinction from LLM profiles; transient retry setting and default. -- `docs/integrations/pkg-configkit.md`: pinned release and supported upstream - naming/inspection contract. -- `docs/internal/configuration.md` and relevant CLI internals: new selection - terminology and unchanged boundaries. -- `docs/consumers/subprocess.md` and any other current callers/examples: updated - invocation syntax. Review operations and README references where applicable. -- `docs/integrations/pkg-promptkit.md`: updated pin, typed errors, observer - support, and retry contract. Refresh pinned upstream links, including its - relocated release-policy document. -- `docs/internal/llm.md` and relevant pipeline/debug internals: observation - projection, failure handling, scheduler interaction, and separate retry - budgets. Remove the obsolete claim that PromptKit has no provider retry loop. -- `docs/operations.md`: latency/cost implications, timeout limits, opt-in debug - evidence, and how to disable transient retries. - -Preserve historical release notes and the accepted text of ADR-0018. Its -ConfigKit ownership decision remains valid; a terminology migration alone does -not require a new ADR or supersession. Current documentation must not portray -the old dependency pin as the current supported version. - -Record PromptKit's LGPL-3.0-or-later change in the integration documentation and -eventual release note. Review its notices and distribution implications within -Notarius's source-only release model. Do not silently relicense Notarius, copy -upstream release policy into this repository, or introduce binary packaging. -Any future binary distribution needs its own license-compliance review. - -## Operator Migration And Release Impact - -This work belongs in the next **minor** Notarius release, not a patch, because -both the documented selected-file layout and CLI syntax change. - -Operators move selected configuration files from `profiles/` to `overlays/` -and replace `--config-profile` with `--config-overlay` in scripts, service -definitions, and subprocess callers. Preserve unrelated PromptKit profile -directories and explicit imports. Review relative paths and explicit imports -that refer to moved files. No YAML body version migration is required. - -The eventual release note must explain this cutover, the newly enabled transient -retries and zero-value opt-out, and the dependency license change. PromptKit's -catalogs are unchanged from v0.10.0; this upgrade adds no further profile-name -migration or change to the D&D model defaults. Release preparation, -tagging, and publication remain a separately requested task under -[Source Releases](../../release.md). - -## Acceptance And Verification - -- Notarius builds against ConfigKit v0.4.0 and PromptKit v0.11.0 without - compatibility shims. -- All five affected commands use the new flag; non-default overlay selection, - omitted selection, empty selection, and `default` retain their intended - behavior. The old flag is not accepted. -- Selected overlays load from the new directory. A legacy-only selected file - does not silently load; ordinary explicit imports remain supported. -- Representative tests demonstrate overlay precedence, CLI overrides, - declaration-relative paths, redacted inspection, and source-aware failures. -- Maintained examples validate, including the development overlay. LLM profile - loading and defaults remain unaffected. -- Tests prove the default transient retry policy, explicit zero, configured - counts, and negative-value rejection through the relevant configuration and - adapter boundaries. Use local HTTP test servers for eligible/noneligible - statuses, exhaustion, cancellation, and interaction with structural repair; - an injected LLM fake alone does not exercise built-in HTTP retries. -- Provider-neutral errors preserve underlying status and cancellation identity - without leaking credentials or upstream types. -- Observations are available for success and failure, preserve unknown versus - known-zero usage, avoid aggregate double-counting, and stay isolated under - concurrent completions. Successful retry recovery emits no warning. -- Representative scheduler/repair/retry checks preserve bounded execution and - cancellation; use existing coverage where it already proves the contract. -- Current documentation, help text, examples, and tests agree. Searches for old - vocabulary are reviewed semantically, excluding intentional historical and - LLM-profile references rather than enforcing a blanket text ban. -- Run focused `go test ./internal/core/config ./internal/cli ./internal/framework/llm ./internal/framework/pipeline`, followed by - `go test ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and - `git diff --check`, with `GOWORK=off`. Run race-enabled tests for the modified - adapter, scheduler, and observation integration. Validate both maintained configurations - and the complete configuration with `--config-overlay development`. - -Use existing behavioral tests where possible, following -[Testing Policy](../../policy/testing.md). No live model calls or production API -credentials are needed to validate this migration. diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index e981d8c0..0ad2f1d2 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -2,16 +2,8 @@ Current Notarius behavior is documented in the canonical README, CLI, configuration, operations, internal, and integration docs. This backlog records -future work outside the [native migration roadmap](promptkit.md), which owns -PromptKit/ConfigKit/CommandKit adoption, continuation, admission, cache, output, -and debug decisions. Items below are ordered roughly by value and specificity, -not as committed release dates or prerequisites for that migration. - -Native shared loading/commands, complete assessment/continuation, concurrent -ready lane completion, accepted-result reuse, and private capture are implemented; -see the [implementation evidence](implementation.md) for final acceptance status. -They are not deferred platform work. Human model-quality evaluation, broader -reference/context policies, and storage/distribution extensions below remain separate. +proposed extensions to that implementation. Items below are ordered roughly by +value and specificity, not as committed release dates. ## Near-Term D&D Pipeline @@ -57,12 +49,12 @@ The sections below keep broader extensions deferred. - Consider a dedicated semantic-policy selection option for a typed reconciliation module when a concrete operator workflow needs more than the - migration's [native prompt replacement](promptkit.md#configuration-sources-and-prompt-assets). + [native prompt replacement](../config.md#promptkit-profiles). - Use native prompt IDs and sources while retaining typed interpretation and deterministic safety rules. Define any additional configuration and domain compatibility checks from that workflow; do not recreate asset registration, prompt fingerprints, or checkpoint dependency tracking. -- Reuse the selected [accepted-artifact cache policy](promptkit.md#accepted-artifact-cache-u8-resolved) +- Reuse the [accepted-artifact cache policy](../operations.md#accepted-result-cache-and-recovery) when evaluating policy changes and the need for explicit refresh. ### Broader Reconciliation Inputs And Module Selection @@ -83,7 +75,7 @@ The sections below keep broader extensions deferred. - Add production LLM-backed validators only when a concrete review policy benefits from model judgment and deterministic checks are insufficient. - Consider additional validator diagnostic views only if a concrete need remains - after the selected [native observations and debug capture](promptkit.md#inspection-and-execution-bookkeeping). + with the existing [native observations and debug capture](../internal/llm.md#native-evidence-and-sensitive-data). - Existing typed resolution already checks validator target, artifact kind, and exact Go-type compatibility. Add further semantic suitability metadata only if a deployment needs a contract beyond those implemented checks. @@ -128,7 +120,7 @@ Define compatibility and migration policy before generated-reference chains must span multiple schema versions or long-lived historical artifacts. The policy should address stable identifier semantics, when an older artifact may be decoded or adapted, and when consumers need regenerated artifacts. For cached -artifacts, use the [selected reuse policy](promptkit.md#accepted-artifact-cache-u8-resolved) +artifacts, use the [current reuse policy](../operations.md#accepted-result-cache-and-recovery) and explicit refresh where needed. Do not add a general migration framework until an actual contract change requires one. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index c408b258..00000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,2372 +0,0 @@ -# Native Migration Implementation Plan - -## Purpose And Execution Rules - -Implement the complete target in [Native PromptKit And CommandKit Migration](promptkit.md). -That feature roadmap owns product choices, defaults, ownership, and acceptance -semantics. This document owns execution order and stage completion. Completion -markers below record finished stages. The inspected pre-migration application -baseline was Notarius commit `b3664d587914b1d7b0377f25fa377b57fa97c815`. -All stages below are complete. Their markers record the landed work and checks; -Stage 46 records the completed whole-roadmap acceptance audit. - -Each numbered stage is one implementation prompt for a Luna coding agent. -Execute stages in integer order as one end-to-end sprint. Intermediate stages -may leave callers, tests, or packages unbuildable. Do not spend prompts preserving -intermediate compilation with compatibility adapters, duplicate loaders, aliases -for retired inputs, or replacement completion abstractions. Final compilation, -behavior, and documentation must satisfy the entire target. The native `run` -alias is a selected new interface, not a legacy forwarding implementation. - -Before each stage, read `docs/development.md`, follow its applicable policy links, -and inspect the stage's named source and focused tests. Prefer the codebase -knowledge graph for code discovery. Use the selected upstream public APIs and -maintained examples, not old dependency APIs inferred from current Notarius code. -Keep code and tests scoped to that stage. Commit each completed nonempty unit -under the agent policy, accurately recording any checks blocked by not-yet-migrated -callers. Do not claim a blocked test passed. Later stages own the corresponding -integration; do not omit their work because an earlier package check was blocked. - -The validation commands below are focused checks, not additional permanent -release gates. Run them when their dependencies can compile; otherwise inspect -the affected code, record the concrete dependency blocker, and continue the -sprint. Stage 46 requires the complete assembled repository to pass. Test with -real kits and a fake `promptkit.LLMClient` or local fake HTTP provider, as appropriate. -No ordinary test requires real credentials, a production transcript, paid inference, -or an external service. Dependency acquisition is separate from offline testing. - -No stage authorizes a release, push, tag, deployment, rewrite of published history, -or changes to sibling repositories. Do not delete legacy cache data on disk. -Do not add security restrictions, filesystem permission policy, schema changes, -backend limits, retry caps, or public reporting guarantees beyond the feature -roadmap. Retain deterministic domain correctness and the existing intrinsic -artifact schemas; retire only machinery and contracts the roadmap replaces. - -## Settled Design And Implementation Boundaries - -- Use PromptKit `v0.24.0`, ConfigKit `v1.4.0`, CommandKit `v0.7.0`, ckconfig - `v0.10.0`, and ckprompt `v0.11.0` from their `go.maximumdirect.net` paths. - Keep the Go consumer minimum `1.27.0`; use stable go-yaml v4 and selected - transitive versions from the feature roadmap. -- Version 6 is the sole application configuration version. One application-owned - ConfigKit result supplies native PromptKit settings, engine construction, and - both inspection scopes. Composition-root loaders live in `internal/cli`. -- Use ordinary `assets` embedding, native source contexts, and - `promptkit.collections.notarius`. Prompt declarations own default output - contracts. PromptKit owns schema preparation and structural repair. -- Keep typed stage interfaces. Model-executing code receives the shared native - engine. A small producer helper in `internal/framework/pipeline` accepts ordinary - module request/interpretation/correction/fallback functions and the pipeline's - typed assessment callback. No new LLM service interface or execution-plan language. -- The pipeline assesses each initial/revised candidate once through the whole - applicable validator chain. Native coordination owns growing conversation - history and turn iteration. `max_additional_turns` resolves binding, pipeline, - then `2`; explicit zero disables semantic correction. Native transient and - structural-repair defaults remain unspecified by Notarius. -- `mode` resolves producer binding, pipeline, then `strict`. A permissive fallback - is qualified output, not ordinary acceptance. Cancellation stops without fallback. - Fallback and affected dependent work skip cache lookup and storage. -- `concurrency.workers`, default `64`, bounds one shared ready-job pool for - extraction and lane completion. Native backend admission is independent; no - injected backend limits or replacement global LLM ceiling. -- Cache only final accepted artifacts/plans under the initial prepared - `MessageContentHash`. The key has no extra dimensions. Hits get ordinary decode/ - usability checks and no new semantic assessment. `auto`, `bypass`, and `refresh` - use the roadmap's exact precedence; cache I/O is best effort. -- Publish domain artifacts at their existing run-specific paths. Report one small - native completion value, including a positive `fallback_count` when applicable. - Retire required manifests, receipts, index files, and reporting sidecars. -- Use one invocation `ExecutionGroup`, native observations, and optional convenient - debug evidence. Detailed debug is not a public serialization contract. Preserve - native credential omissions and ConfigKit secret-aware inspection; do not add - heuristic content rewriting. - -Implementation details such as private helper names and debug filenames may be -refined without changing these choices. Put application outcome/callback types -where typed stage contracts can use them without a `contracts -> pipeline` import -cycle. Do not pass pipeline collaborators through hidden process globals. Modules -still receive no physical roots or CommandKit reporters. - -### Domain Fallback Construction - -Use this construction guidance in the corresponding module stages, under U6: - -| Producer | Supported fallback construction | -| --- | --- | -| D&D scene chunker | One plan entry covering the complete source in document order; use normal materialization and source-addressing rules. | -| Spell, registry, occurrence, combat-turn, and enemy-event extraction | The family's codec-valid empty collection, retaining required top-level shape and any deterministically required context. No invented entity or event. | -| Scene-description extraction | Use the final usable candidate only when it satisfies intrinsic shape, source identity, and downstream consumption invariants. Its artifact requires at least one scene, so an empty list is not a valid fallback. If no such final candidate exists, fail fallback construction; do not fabricate kind/title/summary or change the schema. | -| Semantic registry normalization | Apply compatible safe groups from the final usable proposal afresh to fixed originals, preserving other originals separately. With no usable final proposal, use the deterministic merged input. | -| Deterministic merge/normalization | Reuse suitable incoming/merged data only when the module can construct an invariant-preserving result. An operation error or inability to construct that result fails normally; there is no generic retry. | - -Necessary fallback checks belong to the module; they do not rerun the configured -ordinary validator chain. This table does not promise that every operation can -recover from every failure. Insufficient reconciliation candidates can remain -an ordinary deterministic success; exceeding a required semantic preparation -limit is not silently presented as normally validated model work. - -## Stage 1: Select Dependencies And Update Their Notices - -**Status:** Complete. Selected releases resolve together with `GOWORK=off go list --m all`; Go remains `1.27.0`, with no replacements. Updated commit-pinned notices -for the selected kits and changed runtime dependencies. `git diff --check` passed. -Application API migration remains assigned to the subsequent stages; no application -compilation claim is made here. - -**Goal:** make the reviewed upstream APIs available without a compatibility patch. - -**Inspect/change:** `go.mod`, `go.sum`, `docs/oss/attribution.md`; selected kit -`go.mod`, `COPYRIGHT`, `LICENSE`, and release notes. Read the roadmap's dependency -and evidence sections. Consult the composed CLI example under -`../commandkit/integrations/ckprompt/examples/composed/` as a reference, not a new -Notarius dependency. - -**Work:** add the five selected vanity-path dependencies, retain Go `1.27.0`, -and let Go resolve the reviewed stable YAML/jsonschema versions. Convert import -paths as their callers migrate; do not patch upstream or keep permanent `replace` -directives. Old-path dependencies may remain temporarily while old callers exist; -Stage 35 removes them. Add/update factual attribution with exact selected upstream -commit links and required license material under the OSS policy. - -**Acceptance:** all five selected modules resolve together using declared -versions; no sibling workspace substitutes for a release. Notices describe actual -incorporated dependencies. API incompatibilities are identified as migration work, -not hidden by shims. - -**Validation:** `GOWORK=off go list -m all`, inspect `git diff -- go.mod go.sum -docs/oss/attribution.md`, and `git diff --check`. Do not require the old application -to compile against the new APIs yet. - -## Stage 2: Introduce The Native Version 6 Configuration Definition - -**Status:** Complete. Native settings and composed rules retain the original -application result for direct engine handoff. Version 6 numeric/string loads, -version 5 rejection, application origins, native inheritance, imported message -origins, and secret-aware inspection are covered by the passing -`GOWORK=off go test ./internal/core/config`. `git diff --check` passed. -The old CLI compile check is blocked by old-path ConfigKit importing the removed -`go.yaml.in/yaml/v4/plugin/limit`; later caller migration and dependency removal -own that integration. Source assembly remains assigned to Stage 5. - -**Goal:** preserve one typed ConfigKit load and its original result evidence. - -**Inspect/change:** `internal/core/config/document.go`, `load.go`, -`loaded_config.go`, `document_normalizer.go`, `operational_document.go`, and their -definition/loading tests; PromptKit `Configure`, `Settings`, and `NewFromConfig`. - -**Work:** replace `PromptKitDocument` with native `promptkit.Settings` at -`yaml:"promptkit"`, compose the application definition with `promptkit.Configure`, -and select application version 6 only. Retain the native `configkit.Result` for -handoff and inspection rather than reserializing settings. Preserve application -pipeline normalization, ConfigKit discovery/import/version semantics, and -source-relative application paths. Remove `profile_file`, `profile_dir`, and -`local_backend` consumption; add no separate PromptKit configuration file setting. -Runtime-only client options belong in native `RuntimeConfig`, and invocation -transient-retry overrides use ckprompt's native request override. Do not recreate -the removed configuration wrapper to hold library behavior. - -**Acceptance:** `version: 6` and its native string equivalent follow ConfigKit's -version handling; version 5 has no compatibility translation. PromptKit rules, -including resource identity and secret marking, participate in the same definition. -The whole result can be supplied to `NewFromConfig` without another load. - -**Validation:** focused `go test ./internal/core/config`; test typed `/promptkit` -handoff and unchanged source-origin semantics. Source assembly is completed in -Stage 5; no fake result or YAML round-trip is an acceptable substitute. - -## Stage 3: Resolve The New Application Controls - -**Status:** Complete. Added omission-aware producer controls and final resolution, -invocation-wide cache precedence, independent positive workers, source-relative -state roots, and explicit-empty debug behavior. Removed the superseded application -retry/repair/failure-policy and nested cache configuration surfaces. Updated -focused loading, inheritance, directory, and resolution coverage. -The isolated cache-root test and `git diff --check` pass. Full focused package -tests are blocked by not-yet-migrated pipeline preparation, attempts, and debug -callers of removed fields/types (for example `ValidationPolicy` and `Retries`), -plus old-path ConfigKit's removed YAML `plugin/limit` import. Their scheduled -caller migration and final whole-tree validation remain required; these tests -are not claimed as passing. - -**Goal:** define the selected budgets, modes, workers, and state paths once. - -**Inspect/change:** configuration document/runtime conversion and validation, -`internal/framework/pipeline/profile.go`, `options.go`, `validation_policy.go`, -and policy-resolution tests. - -**Work:** add omission-aware pipeline/binding `mode`, `max_additional_turns`, and -`cache`; root `cache_directory`; and positive `concurrency.workers`. Resolve the -roadmap's precedence/defaults, including explicit zero turns, strict normalization, -and the CLI cache override taking precedence over every binding. Remove producer/ -validator `retries`, application structural-repair fields, failure-category -policies, nested legacy cache settings, `total_llm`, and extraction-worker coupling. -Use native source origins for nonempty config paths. Preserve explicit empty debug -directory as capture-disabled; omitted debug root is `/tmp/notarius-debug`. -Omitted/empty cache root uses `os.UserCacheDir()/notarius`; a configured root is exact. - -**Acceptance:** resolved operations contain final controls rather than repeating -inheritance at execution. Tests distinguish omission, zero, and explicit values. -There are no module-declared permissive defaults, new maximum worker/turn caps, -backend settings derived from workers, or replacement non-LLM retry options. - -**Validation:** focused configuration and pipeline resolution tests, including -binding-over-pipeline mode/turn/cache selection, invocation cache selection, -worker positivity, and explicit-empty path behavior. - -## Stage 4: Convert Embedded Initial Resources And Schema Declarations - -**Status:** Complete. Converted all 16 initial/validator/reconciliation prompt -declarations and the extraction profile to native identified lists. The embedded -`assets/notarius.yml` imports them into the `notarius` collection with its own -filesystem-rooted schemas. Real shared-file paths replace virtual mounts; schema -bytes, prose, message order, and cache hints are unchanged. Native offline loading -and preparation of every declaration pass in `GOWORK=off go test ./assets`, -including exact loaded-schema comparisons and selected profile defaults. -`git diff --check` passed. Module caller/manifest migration remains in its assigned -later stages; no intermediate whole-tree compilation claim is made. - -**Goal:** native declarations become the only default prompt/schema assembly. - -**Inspect/change:** existing `assets/**/prompts/prompt.yaml`, -`assets/dnd/profiles/dnd-extraction.yaml`, `assets/package.go`, and old prompt/schema -manifest tests. Inspect all current content-file paths before changing them. - -**Work:** convert initial producer, validator, and generic reconciliation prompts -to native identified lists in the `notarius` collection, retaining local IDs and -using qualified prompt/profile references. Add one ordinary embedded native source -that imports the resources and declares the collection schema root from the assets -filesystem context. Use real relative paths to shared files, not `sharedassets` -mounts. Set each prompt's response format, validation mode, and schema selection; -remove explicit `repair_attempts`. Keep the default extraction profile's current -model, timeout, service tier, unset reasoning, and unqualified `openrouter` backend. -Remove old resource-version selectors; no new PromptKit version header is needed. - -**Acceptance:** every existing LLM-facing schema is still self-contained and -selected from its actual asset location. Intrinsic artifact schemas remain -unchanged. Shared prose/order/cache controls are preserved except for deliberate -conversation wording changes made with each correction pair later. The assets -Go package remains a content-only leaf, without a behavioral registry. - -**Validation:** native ConfigKit load plus PromptKit construction/preparation with -fake request inputs; cover real shared-file paths and the embedded schema root. -Do not add prose snapshots or fixed shared-prefix-length tests. - -## Stage 5: Assemble One Application Load And Native Engine - -**Status:** Complete. Added invocation-owned selection/result/engine reuse and -one native layered application load with bundled, embedded, and strict peer -operator underlays. Native main-relative imports anchor optional companions to -the selected main source; explicit-empty configuration paths fail. Added real -temporary-file coverage for selection differences, companions/conflicts, -inheritance/imports, captured messages, selective resource replacement, and native -schema contexts. `git diff --check` and the existing native asset tests pass. -CLI/config test execution remains blocked by the recorded legacy YAML import -and pipeline callers of retired policy/retry types. Old command-handler tests -also await the command replacement in Stage 6; no blocked test is claimed passing. -Native standard-command and run wiring remain in their assigned later stages. - -**Goal:** runtime and standard commands consume the same selected resources. - -**Inspect/change:** `internal/cli/configkit_load.go`, -`internal/core/config/loaded_config.go`, new focused CLI loader/engine files, and -old `internal/framework/llm` source assembly. Read upstream shared-loading guidance. - -**Work:** select the actual application main source using the existing explicit/ -environment/native discovery policy. Reject explicitly empty `--config` rather -than discovering another file. Assemble bundled OpenRouter, embedded Notarius, -and operator companions as ordered underlays of the one application load. -Discover optional `prompts.yml` and recursive `.yml`/`.yaml` files in `prompts/` -beside that selected main source, in one strict operator peer group. Preserve -native imports/destinations/source contexts; do not fall back to the working -directory or hide malformed operator sources. Main imports, overlay, and explicit -overrides retain native precedence. Construct `NewFromConfig` from the successful -whole result, with runtime-only options, without callbacks/loading happening again. - -**Acceptance:** invocation-owned reuse distinguishes complete selections, -including overlay/overrides and the two sides of `config diff`. Changes to source -files after capture do not mutate an engine. Shared loaders create no run storage. -No duplicate asset registry, settings reparse, or persistent loader cache remains -on the new path. - -**Validation:** focused CLI/config tests using real temporary files: absent -companions, file+directory conflicts, selective replacement, native inheritance, -imports, source-relative messages, embedded/external schema roots, inline/ -source-relative schemas, and missing-file behavior after root replacement. - -## Stage 6: Install CommandKit And The Standard Configuration Suite - -**Status:** Complete. Added the lazy native application constructor, generated -help/version and Bash/Zsh/Fish completion, and shared `app`/focused `promptkit` -ckconfig scopes with native overrides and comparison. The executable supplies -the release-linked build identity directly. Removed the manual configuration -handlers, dispatcher, and assembled help. Added focused built-in, scope/report, -explicit-empty selection, and diff-status tests. -Build-info/asset tests and `git diff --check` pass. CLI tests and executable -subprocess checks remain blocked by the previously recorded legacy YAML import -and not-yet-migrated pipeline consumers; final assembled verification must run -them. PromptKit-suite convenience aliases are installed with that suite in Stage 7. - -**Goal:** one native application constructor replaces manual command plumbing. - -**Inspect/change:** `cmd/notarius`, `internal/cli/run.go`, `configkit_commands.go`, -`configkit_load.go`, `internal/buildinfo`, and new focused command-composition files. - -**Work:** declare application identity, native help/version, Bash/Zsh/Fish completion, -and presentation through CommandKit. Install ckconfig scopes named `app` and -`promptkit`, using the shared result and native focused-subtree adapter where -appropriate. Install native validation/show/sources/explain/diff and convenience -aliases. Preserve native causes/outcome classification and supplied-value inputs. -Construction itself performs no configuration load, engine build, or storage I/O. -Retire the old configuration command handlers rather than keeping two surfaces. - -**Acceptance:** help/version/completion work without any config or API key; -configuration commands use native report values and the same load evidence. -`config diff` does not accidentally compare a memoized selection to itself. -Release-linked version identity remains available at the real executable boundary. - -**Validation:** focused CLI tests and representative subprocess invocations for -help/version/completion, one app-scope explanation, one focused PromptKit report, -and diff status `3`. Test semantics, not entire upstream help text. - -## Stage 7: Install ckprompt And Reusable Run Overrides - -**Status:** Complete. Installed native resource listings/inspection, requirements, -request validation/preparation, namespace selection, and explicit smoke operations -with ckprompt's builders and scoped convenience aliases. Added reusable native -profile/session/execution flag input and request application, including omitted -versus empty reasoning and native transient-retry overrides. Removed the copied -CLI profile-inspection path and replaced its tests with native command coverage. -The isolated real-kit override test, build-info/asset tests, and -`git diff --check` pass. Assembled CLI inspection/preview/fake-provider smoke tests -remain blocked by the recorded legacy dependency/caller compilation blockers; -their results are not claimed passing. Full pipeline binding remains Stage 33. - -**Goal:** adopt the full selected native diagnostics and execution input suite. - -**Inspect/change:** CLI composition from Stage 6, old `promptkit_profiles.go`, and -ckprompt's maintained composed example and public integration declarations. - -**Work:** use `FromEngine`, request validation/preparation, smoke integration, and -the shared loaders to install all selected resource listings/inspection, -requirements, preview, and explicit smoke checks. Use upstream request builders -unless an application callback is genuinely needed; preserve supplied inputs. -Expose ckprompt's reusable profile/session/execution flags for pipeline run-wide -overrides, carrying omission and explicit empty reasoning distinctly. Use native -`--llm-profile` and `--llm-session-id`; remove copied profile inspectors and -effective-parameter reports. Do not build new native-command wrappers or probes. - -**Acceptance:** standard inspection needs no transcript/provider call. Concrete -request validation/preview performs native request preparation; smoke work is -explicit only and preserves supported/applicable/result distinctions. The reusable -override value can reach every producer, validator, and correction without -mutating captured configuration. Native transient retries replace the former -application-specific override machinery. - -**Validation:** focused CLI tests with a fake provider: listings, partial profile, -explicit preview inputs, no implicit smoke, supplied false/zero/empty values, and -one explicit smoke outcome. Full pipeline binding is completed in Stage 33. - -## Stage 8: Define Typed Producer Collaboration And Resource Dependencies - -**Status:** Complete. Construction now passes the shared native engine. Typed -producer requests carry explicit assessment callbacks, resolved controls, native -execution options, and incoming reuse eligibility; results carry accepted/fallback -disposition, reason, and eligibility without a completion facade. Validators receive -native execution options only. Static prompt-ID dependencies replace correction -transport capabilities and are copied through registration, resolution, and -preparation. Exact artifact type, codec/evidence, option, and reference checks remain. -Added registry callback/native-option, incompatible-type, dependency-isolation, and -preparation coverage. Contracts tests/vet and `git diff --check` pass. Pipeline -tests remain blocked by old ConfigKit's removed YAML `plugin/limit` import; its -production build also encounters unmigrated policy/normalize-retry consumers. -Their scheduled migration remains required; blocked tests are not claimed passing. - -**Goal:** expose only the application contracts required by native execution. - -**Inspect/change:** `internal/framework/contracts/{contracts,typed_pipeline}.go`, -pipeline construction/registries/preparation, module specifications, and typed -registration tests. - -**Work:** replace `ModuleDependencies.LLM` with the shared native engine for -model-executing modules. Add small typed operation collaboration/outcome types -for the pipeline assessment callback, accepted/fallback disposition, fallback -reason, and reuse eligibility. Pass native execution overrides/observers without -creating another completion request/result facade. Adapt typed method arguments -or call options to carry the assessment callback explicitly without an import -cycle. Add static initial/correction prompt dependencies using the same IDs as -request builders; keep dynamic selection runtime-checked. Validators remain -assessment modules, not producer-loop participants. - -**Acceptance:** artifact exact-type checks, codec/evidence compatibility, and -module option/reference validation remain intact. New dependency declarations -are small ID selections, not asset manifests, schema registries, or fingerprints. -Pure domain types and deterministic helpers acquire no PromptKit dependency merely -because surrounding producers do. - -**Validation:** focused contracts/registry/preparation tests, including incompatible -typed registrations and statically declared correction dependencies. This stage -need not migrate every module caller. - -## Stage 9: Centralize Candidate Assessment And Terminal Disposition - -**Status:** Complete. Replaced retrying validator execution and the producer -attempt state machine with one ordered assessment path and strict/permissive -terminal handling. Typed and materialized-chunk assessors give each validator -detached candidate/context views and preserve rejection guidance separately from -failed/unavailable assessments and their native causes. Guidance aggregation has -no hard-coded correction assignment. Fallback construction runs once without -ordinary revalidation, never on caller cancellation, and cannot claim acceptance -or reuse eligibility. Added complete-chain, revised/unchanged candidate, isolation, -guidance-contract, unavailable-assessment, timeout, and cancellation coverage. -Contracts tests/vet, isolated terminal-mode tests, and `git diff --check` pass. -Full pipeline tests remain blocked by the recorded old ConfigKit YAML import; -production compilation also awaits migration of runner/debug/diagnostic consumers -of removed attempt/report types. No blocked test is claimed passing; runner and -native continuation integration remain assigned to Stages 10–11. - -**Goal:** use one complete assessment path for initial and revised candidates. - -**Inspect/change:** `validation_executor.go`, `validation_policy.go`, -`producer_attempts.go`, typed validator execution, and validation-isolation tests. - -**Work:** remove the validator invocation retry loop. Invoke each applicable -validator once per candidate through its public boundary, in configured order, -with immutable candidate views. Collect all rejections/guidance despite earlier -rejections; distinguish reject, fail, and unavailable/skip. Preserve actionable -`CorrectionGuidance` separately from reason codes and operator prose. Expose one -typed assessor to the producer helper. Apply resolved strict/permissive handling -to terminal nonacceptance; unavailable assessment is not a semantic defect. -Fallback construction checks remain module-owned, outside ordinary revalidation. - -**Acceptance:** repeated/unchanged corrected candidates still run the complete -chain. A producer does not acquire acceptance from native execution success alone. -No reason-code text becomes correction instructions, no validator rerun extends -native structural repair, and no final duplicate assessment occurs. - -**Validation:** focused pipeline tests for two independent rejections, earlier -approval followed by a changed candidate, isolated typed validator views, -unavailable assessment, guidance contract errors, and cancellation. - -## Stage 10: Implement The Small Native Producer Helper - -**Status:** Complete. Added one typed helper using native Run/Coordinate and -module-owned interpretation/correction/fallback functions. It checks native -structural status before interpretation, reuses the initial assessment on the -first coordination callback, and assesses every delivered revision once, -including the final allowed turn. Native coordination owns history and budget. -Execution/profile/session selections and existing observers/preparation callbacks -are preserved without duplicate enrollment. Failed continuations cannot recover -earlier candidates; caller cancellation and undelivered responses never become -acceptance or fallback. Real-engine offline fake-provider tests cover acceptance, -zero turns, final-turn acceptance, exhaustion, nil-error structural failure, -operational/unavailable failures, interpretation guidance, cancellation, and -typed native operation timeouts. File-scoped helper tests/vet/race checks and -contracts tests pass. Full pipeline tests remain blocked by the recorded legacy -ConfigKit YAML import and unmigrated consumers; no whole-package success is -claimed. Runner integration, caching, and capture remain in their assigned stages. - -**Goal:** delegate turn iteration while retaining typed Notarius acceptance. - -**Inspect/change:** a focused helper alongside pipeline assessment; upstream -`Coordinate`, `RunRequest`, prepared execution, and native validation status. - -**Work:** accept the module's initial native request, interpretation, correction -builder, supported fallback functions, and the pipeline assessor. Use `Engine.Run` -for uncached work with exchange capture when correction/debug needs it. Check -native errors and `ValidationFailed` before interpretation, including nil-Go-error -failure results. Assess the initial usable candidate, then coordinate actionable -correction from its captured exchange with native `AdditionalTurns`. On the -initial coordination callback, use existing findings; do not reassess the initial -response. Assess each newly delivered result exactly once, including the final -allowed turn. Return typed acceptance/fallback state, not a second retry protocol. -If a continuation fails without a usable response, do not retain an earlier -proposal as the terminal fallback source; use the module's no-usable-response -fallback direction. - -**Acceptance:** native coordination alone counts turns and carries history. -Acceptance stops early; exhausted/stopped are not acceptance. Structural terminal -failure and unavailable assessment do not generate invented semantic guidance. -Cancellation and `Undelivered` do not produce acceptance, fallback, or cache writes. -Distinguish caller cancellation from native execution failures using caller context -state and native typed causes rather than flattening every timeout into cancellation. -No earlier-candidate recovery, cross-turn plan union, or manual assistant replay. - -**Validation:** real PromptKit with a fake provider: initial acceptance, zero turns, -correction acceptance, final-turn acceptance, exhaustion, validation failure with -nil error, operational failure, and cancellation before callback delivery. -Stage 27 adds cache selection to this same helper, not a parallel executor. - -## Stage 11: Connect Typed Runner Operations And Eligibility Propagation - -**Status:** Complete. Runner operations now carry resolved controls, native -execution options, and operation-scoped assessors. The runner consumes native -helper outcomes without outer retries or revalidation; deterministic modules -produce once and are assessed directly, with an optional module-owned fallback -function. Final codec checks remain mandatory; ordinary artifact citation checks -depend on configured validators. Necessary source parsing/materialization and -module/fallback/cache construction checks remain. Fallback eligibility follows -chunk ancestry, lane results, and actual generated-reference consumers without -tainting unrelated work; emitting operations contribute once to fallback_count. -Removed superseded attempt/correction hooks and their debug/policy projections. -The pipeline production package builds. All production files plus the focused -typed registry, codec, assessment, native-helper, and runner tests pass, including -focused vet/race checks; contracts tests and `git diff --check` also pass. -The complete pipeline test package still imports retired dependencies/fixtures -and is not claimed passing. Production module migration, worker-pool replacement, -legacy-store removal, publication, and full capture remain in their assigned stages. - -**Goal:** the runner supplies assessment once and consumes the returned outcome. - -**Inspect/change:** `runner_typed.go`, `typed_execution.go`, `runner.go`, -`runner_chunk_plan.go`, `handoff.go`, and the old state/correction hooks. - -**Work:** bind each resolved producer's assessor and controls to its operation, -route LLM producers through the shared helper, and retain a direct production/ -assessment path for deterministic stages. Remove outer assessment/retry calls -around helper-owned assessment. Carry fallback and cache-eligibility facts through -extract -> merge -> normalize and generated reference handoffs. A chunk fallback -affects all derived lane work; a lane fallback affects that lane and actual -consumers, not unrelated siblings. Keep only the small facts needed for outcomes, -fallback counting, and reuse suppression, without dependency fingerprints. - -**Acceptance:** generated references may carry valid fallback artifacts with -explicit reuse ineligibility. Stable ordering, exact codec handoff, required -references, and fixed pipeline steps remain intact. Each actual fallback operation -contributes once; affected downstream operations are not themselves counted unless -they also emit a fallback. - -**Validation:** focused runner/handoff tests using small typed modules: full-chain -assessment once, deterministic single execution, chunk/lane/generated-reference -eligibility propagation, unrelated predecessors unaffected, and cancellation. -Native production modules are migrated in Stages 12–23. - -## Stage 12: Migrate Spell Extraction As The Representative Producer - -**Status:** Complete. Spell extraction now prepares fixed transcript/reference, -catalog, and NPC projections once and uses the shared native producer helper. -Added the same-schema/default-profile correction prompt, enduring system -principles, explicit run/profile/session propagation, static dependencies, and -the codec-valid empty fallback. Removed spell completion/asset/schema registries -and manifest/fingerprint metadata; retained private-response interpretation and -source-order canonicalization. Migrated focused tests to a native provider fake -and added a leaf-assembled generic-chunker spell contract covering complete ordered -assessment, final correction, strict exhaustion, permissive output, and exact -continuation history without repeated references. -Native asset, catalog/shape, isolated schema/input-helper, and focused pipeline -tests pass; runnable asset/catalog/shape vet/race checks and production pipeline/ -generic/Seriatim builds pass. Full spell/shared/source-validator tests, including -the new assembled contract, are blocked at setup by shared legacy asset helpers -importing old-path ConfigKit's removed YAML `plugin/limit`. Those tests are not -claimed passing. Central registration and remaining legacy fixtures/dependencies -remain assigned to Stages 24 and 35, with final assembled verification required. - -**Goal:** prove native extraction with the real typed module and validators. - -**Inspect/change:** `internal/modules/dnd/extract/spells`, shared extraction input -helpers, spell catalog projection, spell validators, and `assets/dnd/spells`. - -**Work:** prepare transcript, catalog, references, and NPC resolution once; build -native initial requests and interpret native results through the helper. Preserve -source-reference ordering/swapping, contextual entity resolution, spell identity, -catalog semantics, and exact artifact encoding. Add `notarius/dnd.spells.repair` -using the same schema and default profile as the initial prompt. Put enduring -principles in the initial system context and only findings/new correction -assignment in the follow-up. Keep explicit profile/run overrides on both requests. -Implement the valid empty spell fallback and static dependencies. - -**Acceptance:** one generic-chunker spell pipeline exercises native production, -ordered validation, correction, strict failure, and permissive empty output. -Source/ref input bytes are not regenerated during continuation. The helper owns -the operation and the runner does not validate it again. - -**Validation:** `go test ./internal/modules/dnd/extract/spells -./internal/modules/dnd/validate/spells/...`; adapt the assembled spell contract -test to a native provider fake, including catalog rejection and final correction. - -## Stage 13: Refactor The Shared Reconciliation Core - -**Status:** Complete. Removed the reconciliation execution service and its -private completion/correction transport, retry classifications, asset/schema -registration, metadata, and fingerprints. The core now exposes fixed owned -preparation bytes, native initial/follow-up request builders, proposal decoding, -and the existing independent safety assessment/application functions. Candidate -handles, declared ranges, bounded source windows, group-safety rules, typed -application, and actionable issue prose remain; the follow-up asset owns the -replacement assignment rather than an application retry message builder. -Added the generic native correction pair with the unchanged response schema and -matching unset profile defaults, moving enduring protocol/policy into initial -system context. Retained domain-neutral regressions and added native provider/ -preparation, structural-failure, fixed-material, and successive-proposal tests. -The complete reconciliation and asset tests, vet, race checks, and -`git diff --check` pass. Domain normalizer and central registration consumers of -the removed APIs remain assigned to Stages 14–15 and 24; no whole-tree compilation -claim is made at this intermediate boundary. - -**Goal:** separate one-time evidence preparation from proposal interpretation. - -**Inspect/change:** `internal/framework/semanticreconcile/{engine,preparation, -proposal,application,schema}.go` and relevant tests; generic normalization prompt assets. - -**Work:** expose reusable preparation, native request construction, and proposal -decoding/assessment as ordinary functions. Keep fixed candidate-number mappings, -range declarations, namespace guidance, bounded source windows, proposal safety, -and model-facing issue prose. Remove the core's old completion client/correction -transport and response-schema registration/identity machinery. The module's -producer helper invocation owns native execution; do not create a replacement -reconciliation service/exchange wrapper. Convert the generic default prompt and -add its correction pair, reusing the same response schema and native defaults. - -**Acceptance:** each operation prepares candidates/windows once and applies every -complete proposal afresh to those originals. Unknown/overlapping handles and -unsafe groups remain rejected. No source evidence or opaque identity is copied by -the model. Pure proposal assessment/application remains independently testable. - -**Validation:** `go test ./internal/framework/semanticreconcile`; retain the -candidate-number, group-safety, deterministic ordering, limits, and source-window -behavioral tests. Native structural failures terminate at the kit boundary, not -as a request to restart the core. - -## Stage 14: Migrate Item Registry Normalization - -**Status:** Complete. Item normalization now uses one fixed preprocessing and -reconciliation preparation, native initial/correction requests, and the shared -producer helper. Each response applies its own safe proposal to original records -before operation-scoped assessment. Currency compatibility, identity derivation, -evidence ordering, and domain diagnostics remain. Added the same-schema/default- -profile correction declaration and static dependencies; removed module retries, -completion/asset registration, and manifest/fingerprint metadata. -Permissive output uses only the final safe candidate or the supplied merged -artifact after a result-less failure. Fallback construction checks codec, registry -identity, and source usability without ordinary revalidation. Preparation limits -honor resolved mode; insufficient candidates use the existing assessment path -without native requests. Exposed/factored the existing terminal/no-model assessment -functions for these real callers rather than duplicating acceptance policy. -Retained domain regressions and added native-fake correction, final safe partial, -unchanged-candidate, failed-continuation/no-earlier-recovery, structural-failure, -mode/limit, no-model, fallback-usability, override, and cancellation tests. -Focused pipeline tests/vet/race, terminal tests, complete reconciliation/asset and -item-shape tests/vet/race, production framework builds, and `git diff --check` -pass. Item-normalizer/codec/identity/source-validator tests remain blocked at -setup by shared legacy asset helpers importing old ConfigKit's removed YAML -`plugin/limit`; the new module tests are not claimed passing. Their full execution -remains required after scheduled dependency cleanup and in final verification. - -**Goal:** prove a complete native normalization operation and safe fallback. - -**Inspect/change:** `internal/modules/dnd/normalize/itemregistry`, its validators/ -codec/identity helpers, and `assets/dnd/item-registry/normalize`. - -**Work:** connect the core functions to the typed producer helper. Preprocess -fixed originals once, use initial/correction requests with fixed handles, and -apply each proposal afresh before the complete normalization assessment. Add the -domain correction prompt and dependencies. Treat preparation-limit inability and -unusable native output through resolved strict/permissive handling, without -automatic permissive defaults. On permissive exhaustion apply only safe groups -from the final usable proposal; otherwise retain the deterministic merged input. -Reuse already available merged data, never rerun deterministic stages to get it. - -**Acceptance:** a failed correction cannot mutate originals, union earlier groups, -or silently claim complete semantic success. Insufficient candidates avoid a model -call as before. Reuse eligibility distinguishes ordinary accepted normalization -from final safe partial/deterministic fallback. - -**Validation:** focused item-normalizer tests with a native fake provider: initial -success, repaired proposal, unsafe final groups, strict failure, permissive -preservation, unchanged-candidate reassessment, and cancellation. - -## Stage 15: Migrate NPC And Location Registry Normalization - -**Status:** Complete. Both normalizers now use the shared native producer helper -and fixed one-time reconciliation preparation. Each response applies its own safe -groups afresh to the original records, before typed operation assessment. Added -same-schema/default-profile correction prompts and static dependencies; removed -retry transport, schema/policy fingerprints, and manifest-only asset registration. -Name-based NPC identity, evidence-anchored location identity, canonical naming, -source ordering, and domain application rules remain unchanged. Strict handling -does not silently fall back; permissive output uses only the final safe candidate -or the supplied merged artifact, with codec/identity/source construction checks. -Retained domain regressions and added native provider coverage for correction, -unchanged reassessment, final-only safe groups, failed continuation without prior -proposal recovery, structural failure, overrides/history, cancellation, and -unusable fallbacks. Native prompt-pair checks run in the content package without -depending on legacy module helpers. -Asset/reconciliation tests, vet/race checks, focused pipeline producer/runner -tests, production framework builds, and `git diff --check` pass. Both normalizer -package tests remain blocked at setup by shared legacy asset helpers importing -old ConfigKit's removed YAML `plugin/limit`; their new tests are not claimed -passing. Scheduled dependency cleanup and final whole-tree validation must run -them and close that blocker. - -**Goal:** apply the proven normalizer path without domain-policy drift. - -**Inspect/change:** `internal/modules/dnd/normalize/{npcregistry,locationregistry}` -and their normalization assets, identity/grounding helpers, and focused tests. - -**Work:** migrate these two parallel consumers using Stages 13–14's shared core -and producer helper. Preserve each family's canonical naming, proper-name scope, -source grounding, identity computation, compatible-group checks, and output order. -Add each domain correction prompt with the same initial schema/default profile; -retain generic protocol/candidate presentation as ordinary shared files. Remove -their retry directives, schema/policy fingerprints, and manifest-only metadata. - -**Acceptance:** all three registry normalizers share execution and preparation -mechanics but retain their typed application rules. Every turn operates on the -same candidate namespace. Final safe-group/deterministic fallbacks are explicit, -strict remains default, and model errors do not trigger an outer restart. - -**Validation:** `go test ./internal/modules/dnd/normalize/npcregistry -./internal/modules/dnd/normalize/locationregistry`; adapt boundary tests to native -provider fakes and retain domain identity/reconciliation regression coverage. - -## Stage 16: Migrate D&D Scene Planning And Chunk Assessment - -**Status:** Complete. Scene planning now prepares the full transcript/reference -inputs once and uses the shared native producer helper with initial/correction -prompts, matching schema/profile defaults, and static dependencies. Enduring -scene principles remain in initial system context. Removed the private completion, -response-schema/asset registration, and manifest metadata paths. Existing -source-order, coverage, gap/overlap, and boundary interpretation rules remain. -Unresolvable boundaries supply actionable correction guidance; unusable native -structural results never become semantic candidates. Permissive terminal handling -constructs one unannotated whole-source plan using normal materialization; invalid -input, cancellation, and fallback-construction failures do not publish a fallback. -The runner already binds materialization and complete chunk assessment to the -native operation without an outer attempt/replay loop. Added a runnable runner -contract proving ordered full-chain assessment through final-turn acceptance, -strict/permissive exhaustion, no fallback revalidation, and derived reuse exclusion. -Migrated scene regressions to a native provider fake and added correction/history, -override, structural-failure, whole-source fallback, and cancellation cases. -Scene prompt-pair/schema checks run in the asset package. -Asset/source tests and vet/race checks, focused pipeline tests and vet/race checks, -and `git diff --check` pass. Scene-chunker package tests remain blocked at setup -by shared legacy helpers importing old ConfigKit's removed YAML `plugin/limit`; -their runtime results are not claimed passing. Scheduled dependency cleanup and -final verification must execute them. Chunk-cache replacement remains Stage 28. - -**Goal:** one native chunk producer with deterministic materialization. - -**Inspect/change:** `internal/modules/dnd/chunk/scenes`, chunk-plan interpretation/ -materialization in the runner, chunk validation, and `assets/dnd/scenes`. - -**Work:** prepare the full transcript once, produce/interpret a complete native -plan, materialize it through existing source-addressing checks, and assess it -through the pipeline's chunk validator chain. Add the scene-plan correction -prompt using its initial schema/profile. Implement the permissive one-whole-source -plan and materialize it normally; do not fabricate scene descriptions/tags or add -deterministic slicing. Retain source order for nonconsecutive unit IDs. Remove -outer attempt/replay logic around plan generation. Stage 28 replaces storage. - -**Acceptance:** malformed or unusable plans never become ordinary acceptance. -Every revised usable plan gets complete chunk assessment, including the final -turn. A fallback preserves the full source and marks all derived work ineligible -for cache reuse. Empty/invalid input and fallback-construction errors fail normally. - -**Validation:** focused scene chunker, chunk materialization, and chunk-validator -tests with a fake native provider, including reversed/resolvable ranges, invalid -units, full-source fallback, final-turn acceptance, and cancellation. - -## Stage 17: Migrate The Combat-Semantics LLM Validator - -**Status:** Complete. The validator now makes one native request on the shared -engine, preserving the validator profile/default, session, execution overrides, -transient-retry override, observer, and preparation callback. Static prompt -dependencies replace asset/schema registration and manifest/fingerprint metadata. -Native errors and failed validation status are checked before decoding; execution -and interpretation failures produce an unavailable assessment with inspectable -causes, not semantic rejection or invented correction guidance. Cancellation -does not deliver a judgment. Successful classification retains the original -combat policy, scene-local transcript boundary, explanation checks, and actionable -producer guidance. Interpretation and its existing cases are independently -testable; evaluation fixtures and schema/policy assets are unchanged. -Added native provider tests for judgments, override/input isolation, nil-error -failed validation, unavailable provider/capacity, interpretation failure, -cancellation, and nested same-backend admission. A runnable framework contract -proves complete ordered native assessment on a backend limited to one active call, -without holding producer admission or repeating the final chain. -Asset tests/vet/race, independent interpretation/evaluation tests/vet/race, -focused pipeline tests/vet/race, and `git diff --check` pass. The concrete -validator package remains blocked at setup by shared legacy asset helpers -importing old ConfigKit's removed YAML `plugin/limit`; its native runtime tests -are not claimed passing. Scheduled dependency cleanup and final verification -must execute them. No validator-owned semantic continuation loop was added. - -**Goal:** validators use native execution without another semantic retry loop. - -**Inspect/change:** `internal/modules/dnd/validate/scenedescriptions/combat_semantics`, -its prompt/schema assets, registration, and evaluator tests. - -**Work:** build a native request using the shared engine and run-wide overrides. -Use the configured validator profile/default and native output contract. Check -errors and failed structural validation before decoding. Return domain rejection -with actionable correction guidance, or an unavailable assessment when execution/ -interpretation cannot complete. Preserve scene-local contextual inputs and the -combat policy. Remove validator retry/repair fields and private schema/client -adapters; do not give the validator its own producer coordination loop. - -**Acceptance:** native structural repair may occur internally, but Notarius -invokes the validator only once for each producer candidate. Capacity/provider -failures are unavailable assessment, not semantic rejection. A successful rejection -supplies useful prose to the scene producer and no opaque diagnostic code. - -**Validation:** focused validator and ordered-chain tests for approve, reject, -malformed/failed validation, nil-error failure status, unavailable provider, and -nested same-backend admission with a limit of one. - -## Stage 18: Migrate The Three Registry Extractors - -**Status:** Complete. NPC, item, and location extraction now use native initial/ -correction requests through the shared typed producer helper. Transcript and -campaign reference materials are captured once per chunk; every response uses -the existing family-specific canonicalizer and deterministic identity rules. -Added same-schema/default-profile correction pairs and static dependencies, -moving enduring identity/evidence/domain principles into initial system context. -Private completion, schema/asset registration, and manifest/fingerprint metadata -were removed. Canonical naming, currency scope, same-name location anchors, -source-order/evidence behavior, and production validator-chain declarations are -unchanged. Each family has a codec-valid nonnil empty fallback, qualified by the -resolved mode/outcome and excluded from reuse, rather than an invented entity. -Retained domain regressions with native provider fakes and added per-family -citation correction through real typed validators, fixed input/history and run -override checks, strict/permissive exhaustion, ordinary empty acceptance, failed -continuation, structural failure, and cancellation coverage. Prompt-pair and -structural-versus-semantic schema checks run independently in the asset package. -Asset/shape tests and vet/race checks, focused pipeline tests, and -`git diff --check` pass. Updated the location-shape spec comparison for the -slice-bearing native dependency declaration. All three extractor package tests -remain blocked at setup by shared legacy asset helpers importing old ConfigKit's -removed YAML `plugin/limit`; their new runtime tests are not claimed passing. -Scheduled dependency cleanup and final assembled verification must execute them. - -**Goal:** canonical noun extraction follows the representative native path. - -**Inspect/change:** `internal/modules/dnd/extract/{npcregistry,itemregistry, -locationregistry}`, their extraction assets and shared citation/identity helpers. - -**Work:** migrate the three parallel list producers to native builders, -interpretation, and the shared helper. Add domain correction prompts, static -dependencies, and codec-valid empty fallbacks. Prepare transcript/reference -material once per chunk. Keep proper-name scope, item/currency identity rules, -source evidence resolution, deterministic IDs, and required validator chains. -Remove per-module prompt/schema manifests and completion metadata, without -consolidating unlike domain interpretation rules. - -**Acceptance:** each produces its existing typed durable artifact and preserves -source refs/identity across correction. Empty fallback is distinguished from a -normally accepted empty extraction. No module implements its own turn loop or -pins structural repair. - -**Validation:** focused tests for all three extractor packages and one -representative correction/fallback case each, using the shared native fake pattern -from Stage 12. Retain meaningful per-family grounding and identity regressions. - -## Stage 19: Migrate NPC And Location Occurrence Extraction - -**Status:** Complete. Both occurrence producers now use fixed transcript/registry -materials, native initial/correction requests, and the shared typed producer -helper. Added same-schema/default-profile correction pairs and static prompt -dependencies; removed private completion, schema/asset registration, and -manifest/fingerprint metadata. Names-only NPC resolution and complete ordered -contextual location selectors retain deterministic durable identity, supported -kinds, source-order canonicalization, and current-transcript evidence. Unknown -or ambiguous selections supply actionable interpretation guidance before typed -assessment rather than guessing an identity or returning partial output. -Permissive fallbacks are codec-valid nonnil empty collections, explicitly -qualified and reuse-ineligible; accepted work inherits incoming ineligibility. -Retained domain regressions with native provider fakes and added fixed resolution/ -history, source correction, run override, strict/permissive, failed continuation, -structural-failure, and cancellation cases. Prompt-pair/schema tests run in the -content package. Asset and shape tests/vet/race checks, independent NPC response -model tests/race checks, focused pipeline tests, and `git diff --check` pass. -Updated the location-occurrence shape spec comparison for slice-bearing native -dependencies. Both full extractor packages remain blocked at setup by shared -legacy helpers importing old ConfigKit's removed YAML `plugin/limit`; their -native runtime tests are not claimed passing. Scheduled dependency cleanup and -final whole-tree verification must run those tests. - -**Goal:** registry-consuming occurrence lanes retain contextual resolution. - -**Inspect/change:** `internal/modules/dnd/extract/{npcoccurrences,locationoccurrences}`, -their assets, registry resolvers, and source/registry validators. - -**Work:** build initial native requests from fixed chunk/registry material; -interpret contextual selections and attach durable IDs deterministically. Add -the two correction pairs and empty occurrence fallbacks. Keep occurrence kinds, -unknown/ambiguous reference handling, accepted source citations, and compatible -generated-reference expectations. Pass profiles/session/overrides unchanged. - -**Acceptance:** model responses never reproduce opaque application IDs. Source -context and registry resolution remain fixed throughout each correction exchange. -Fallback effects propagate to actual later consumers without invalidating -unrelated registry/predecessor artifacts. - -**Validation:** focused NPC/location occurrence tests covering contextual matches, -ambiguous selection, invalid source refs, corrected final candidate, and fallback. -Do not relax existing domain invariants merely to make the migration tests pass. - -## Stage 20: Migrate Item Occurrence Extraction - -**Status:** Complete. Item occurrences now use native initial/correction requests -through the shared typed producer helper with fixed transcript and names-only -registry materials. Added the same-schema/default-profile correction pair and -static dependencies; removed private completion, prompt/schema registration, -and manifest/fingerprint machinery. Existing private nullable-field mapping, -quantity copying, holder rules, canonical identity, source references, and event -ordering remain unchanged. Unknown selections provide module-authored guidance -before typed assessment, without partial output or invented items. Permissive -terminal output is an explicit codec-valid empty list, not a repaired holder or -ordinary acceptance. Profile/session/execution selections and incoming reuse -ineligibility are preserved. Migrated the holder regression to a native provider -fake with all four typed validators, exact defective-response history, contextual -guidance, final-turn acceptance, and no duplicate final assessment. Added terminal -mode, unknown-item, failed-continuation, structural-failure, cancellation, all-kind, -ordering, and durable nullable-holder cases. Native prompt-pair/holder-principle -and unchanged structural-schema tests run independently in the assets package. -Asset tests/vet/race checks, focused pipeline tests, and `git diff --check` pass. -The extractor and all five occurrence-validator package checks remain blocked at -setup by shared legacy helpers importing old ConfigKit's removed YAML -`plugin/limit`; the new native runtime tests are not claimed passing. Scheduled -dependency cleanup and final assembled verification must execute those tests. - -**Goal:** retain the smaller-model holder reliability fixes in the native path. - -**Inspect/change:** `internal/modules/dnd/extract/itemoccurrences`, associated -holder/registry/source validators and assets, and holder regression tests. - -**Work:** use native initial/correction requests and fixed item registry context. -Preserve `discovered`, `acquired`, `lost`, `consumed`, and `transferred` semantics, -the private response-to-durable holder mapping, party-member handling, and -source-grounded deterministic entity resolution. Add the same-schema correction -pair and valid empty fallback; keep module interpretation guidance separate from -operator messages. Remove only execution/asset registration machinery. - -**Acceptance:** existing acquired/from/to reliability behavior and event ordering -survive. No additional optional model-facing schema fields, synthesized holders, -opaque model IDs, or durable artifact changes are introduced. - -**Validation:** focused item occurrence extractor/validator tests with native -provider fakes, including the previously reported incompatible-holder regression, -correction acceptance, empty fallback, and final semantic rejection. - -## Stage 21: Migrate Combat-Turn Extraction - -**Status:** Complete. Combat extraction now uses one fixed source/NPC projection, -native initial/correction requests, and the shared typed producer helper. Added -the same-schema/default-profile correction pair and static prompt dependencies; -removed private completion, schema/asset registration, and manifest/fingerprint -metadata. Actor/turn-kind semantics, unmatched actors, document-order endpoint -normalization, evidence canonicalization, and exact scene eligibility remain. -Noncombat and unavailable/mismatched scene paths assess their deterministic empty -result directly, retaining the existing warning where applicable, without native -preparation, a model call, or a synthetic cache entry. Explicit outcomes inherit -incoming reuse eligibility. Permissive model failures emit only a qualified, -codec-valid empty combat list; cancellation emits no fallback. -Retained domain/gating regressions with native provider fakes and added complete -ordered-chain correction, fixed eligibility/context/history, override, zero-turn -reversed-endpoint, strict/permissive, failed-continuation, structural-failure, and -cancellation cases. Added a leaf-assembled generic-chunker combat pipeline test -for combat, narrative, recap, meta, and mismatched scenes using provider invocation -counts. Native prompt-pair and unchanged structural-schema checks run in assets. -Asset/shape tests and vet/race checks, independent response-model tests/vet/race, -and `git diff --check` pass. Full combat extractor, source/invariant validator, -and integration package checks remain blocked at setup by shared legacy helpers -importing old ConfigKit's removed YAML `plugin/limit`; native module and assembled -runtime tests are not claimed passing. Scheduled dependency cleanup and final -verification must execute them; remaining old integration fixtures retain their -assigned migration scope. - -**Goal:** native combat production preserves scene eligibility and evidence rules. - -**Inspect/change:** `internal/modules/dnd/extract/combatturns`, combat validators, -scene-description reference selection, and combat prompt assets. - -**Work:** migrate initial native requests, result interpretation, correction pair, -and empty fallback. Preserve combat-only scene eligibility, actor/turn-kind rules, -source-order endpoint normalization, and source/registry context. Skipped -ineligible chunks remain deterministic no-model outcomes, not invented native -requests or synthetic cache entries. - -**Acceptance:** no combat model call is made for an ineligible scene; correction -uses the same fixed operation inputs and schema. The prior resolvable reversed- -endpoint reliability behavior remains without new semantic retry cost. - -**Validation:** focused combat extractor/validator tests and assembled combat -eligibility coverage with a native fake provider. Check no-call cases by provider -invocations, not by obsolete completion-interface mocks. - -## Stage 22: Migrate Enemy-Event Extraction - -**Status:** Complete. Enemy extraction now uses fixed source/reference projections, -native initial/correction requests, and the shared typed producer helper. Added -same-schema/default-profile correction assets and static dependencies; removed -private completion, schema/asset registration, and manifest/fingerprint metadata. -NPC, scene, combat-turn, and combat-opponent reference compatibility, source-free -grounding, contextual subject names, engagement checks, and source-order mapping -remain unchanged. Ineligible scenes produce directly assessed empty results -without native requests, preserving existing unavailable-classification warnings. -Permissive terminal output is an explicit codec-valid empty event list; accepted -work consuming fallback references retains incoming reuse ineligibility. -Retained grounding/domain regressions with native provider fakes and added -full-chain engagement/source correction, fixed multi-artifact context/history, -run selections, empty predecessor handoffs, strict/permissive exhaustion, -failed continuation, structural failure, and cancellation cases. Added a -leaf-assembled native pipeline test retaining the intrinsic codec boundary even -with explicit empty validator chains, and migrated the existing CLI codec -contract's typed request. Native prompt-pair, required-input, and unchanged-schema -checks run in assets. Asset tests/vet/race checks, focused pipeline tests, and -`git diff --check` pass. Extractor, validator, codec, and assembled CLI package -checks remain blocked at setup by shared legacy helpers importing old ConfigKit's -removed YAML `plugin/limit`; their new runtime tests are not claimed passing. -Scheduled dependency cleanup and final verification must execute those tests. - -**Goal:** preserve multi-artifact grounding while changing execution ownership. - -**Inspect/change:** `internal/modules/dnd/extract/enemyevents`, grounding/mapping -helpers, its validators, and enemy-event assets. - -**Work:** move the module to the shared helper and native initial/correction pair. -Keep NPC/scene/combat-turn/NPC-occurrence references, evidence intersection, -engagement checks, contextual names, and durable event order. Prepare reference -projections once per operation; use the codec-valid empty event fallback. Remove -its private completion and manifest/fingerprint metadata. - -**Acceptance:** an operator override applies to initial, correction, and nested -validation without changing domain reference compatibility. Failed/fallback -predecessors affect reuse eligibility but valid fallback references remain usable -in the current run. No fabricated enemy event or expanded extraction scope. - -**Validation:** focused enemy-event extraction/validation tests plus the assembled -enemy-event codec contract, using a native provider fake and current typed codecs. - -## Stage 23: Migrate Scene-Description Extraction - -**Status:** Complete. Scene descriptions now use native initial/correction -requests through the shared typed producer helper with fixed transcript/reference -materials and captured chunk identity/range. Added matching schema/default-profile -correction assets and static dependencies; removed private completion, schema/ -asset registration, and manifest/fingerprint metadata. The existing one-scene -mapping, kind/title/summary contract, and trimming remain unchanged. -Permissive fallback construction uses only the final usable candidate and checks -the unchanged durable codec, exact current chunk identity/range, trimmed fields, -and current-source addressing. It does not rerun ordinary semantic validators, -invent a classification/prose, publish an empty collection, or recover an earlier -candidate after a result-less/structurally failed correction. Strict handling and -cancellation publish no fallback. Added native-provider coverage with the real -combat-semantics validator for ordered complete-chain correction, validator versus -producer profiles, shared run overrides/session, exact history, qualified final-only -semantic fallback, unusable final responses, failed continuation, and cancellation. -Retained mapping/reference/registration regressions and moved native prompt-pair/ -unchanged-schema checks to assets. Independently runnable fallback tests cover -missing/empty/multiple scenes, identity/range/shape failures, and defensive ownership. -Asset, shape, source-reference, invariant, codec, and isolated fallback tests and -vet/race checks pass, as do focused pipeline tests and `git diff --check`. -Full extractor and combat-semantics package checks remain blocked at setup by -shared legacy helpers importing old ConfigKit's removed YAML `plugin/limit`; -their new native runtime tests are not claimed passing. Scheduled cleanup and -final assembled verification must execute them. - -**Goal:** retain the required scene artifact while making fallback limits explicit. - -**Inspect/change:** `internal/modules/dnd/extract/scenedescriptions`, its codec, -shape/source/invariant validators, semantic validator integration, and assets. - -**Work:** use native initial/correction requests with one description for the -operation's source range as currently required. Preserve deterministic scene IDs -and source ranges; keep the kind/title/summary contract unchanged. Add the -domain correction prompt and same-schema/profile dependency pair. Implement the -fallback rule in this plan's table: only a final usable candidate meeting the -intrinsic artifact/downstream invariants can be qualified fallback output. If -the final execution yields no usable scene, fail fallback construction rather -than emitting an empty list, inventing a classification, or recovering an earlier -candidate. Ordinary semantic revalidation is not a fallback prerequisite. - -**Acceptance:** strict rejection fails; permissive handling never publishes an -invalid scene artifact or claims semantic approval. A rejected combat judgment's -guidance reaches correction, and every revised scene gets the full chain. - -**Validation:** focused scene-description/semantic-validator tests with native -fakes: correction, valid-but-semantically-rejected final fallback, structurally -unusable final response, no prior-candidate recovery, and source identity. - -## Stage 24: Finish Deterministic Stage Paths And Production Registration - -**Status:** Complete. D&D, generic, and Seriatim family registration now accepts -only typed module registries; removed prompt/schema/fallback-profile registration -and adapted composition callers. Raw codecs, evidence projectors, all domain -validator defaults, and typed append/noop variants remain intact. Added native -embedded-resource dependency coverage for registered producers and validators. -Moved prompt-composition/schema-compatibility checks to the content package, -using real native preparation without private asset assembly or prefix snapshots. -Deterministic producers already run directly through the migrated runner; updated -the Seriatim typed fixtures to run without an engine and added direct merge-error -and noop-value/cancellation coverage. No synthetic requests or outer retries were -introduced; ordinary errors remain errors even in permissive mode. -All generic and Seriatim package tests, scene-normalizer tests, native asset tests, -focused typed pipeline tests, selected vet/race checks, and `git diff --check` -pass. The D&D registrar, six other deterministic normalizers, and full module -integration tests remain blocked at setup by shared legacy asset helpers importing -old ConfigKit's removed YAML `plugin/limit`; their results, including the new -registration dependency test, are not claimed passing. Scheduled legacy cleanup -and final assembled verification must execute them. Old CLI runtime asset/client -wiring remains assigned to its later migration stages. - -**Goal:** all non-LLM work runs directly, and production registration is asset-free. - -**Inspect/change:** generic unit chunker/append-order merger/noop normalizer, -deterministic D&D normalizers, Seriatim adapter, D&D/generic registrars, typed runner, -and module integration fixtures. - -**Work:** remove outer retries and obsolete correction/repair arguments from -deterministic operations. Preserve typed merge/normalize rules, ordering, domain -validators, generated references, evidence projectors, and raw artifact codecs. -Use resolved mode only for rejected candidates with a supported module fallback; -ordinary parse/materialization/application/encoding errors propagate normally. -Register modules, validator defaults, and static dependencies without registering -prompt files, schemas, or fallback profiles. Complete migration of every LLM role -identified in Stages 12–23, not just the examples. - -**Acceptance:** there is no synthetic PromptKit request for deterministic work -or generic retry setting. Durable artifact-schema validation remains; removing -duplicate model-response schema validation does not remove this application -boundary. Pipeline default-chain replacement, including explicit empty chains, -and exact typed registration still work. - -**Validation:** focused generic/D&D deterministic normalizer, registrar, Seriatim, -and typed integration tests; retain data-integrity tests and remove tests whose -only purpose was the retired generic retry/asset API. - -## Stage 25: Replace Worker Pools And Remove The Global LLM Scheduler - -**Status:** Complete. One lazy coordinator now dispatches ready extraction and -merge/normalize continuations through the same bounded pool. Ready continuations -can run while unrelated extraction remains; prerequisite waits occupy no worker. -Cancellation disables dispatch, drains every dispatched result, and joins workers. -Stable output/error ordering and sequential step barriers remain. Replaced -extraction-only worker inputs with shared workers, using the selected default of -64 consistently with configuration. Removed the global scheduler/scheduled client -and their obsolete tests and CLI wrapping; native engine admission is independent. -Migrated retained concurrency fixtures and replaced legacy scheduler integration -coverage with real native-helper/backend tests. New tests cover mixed-job limits, -serial work, independent backend limits, nested same-backend validation, strict/ -permissive capacity failure without another queue, and cancellation/admission reuse. -The mixed-job regression demonstrably fails with the old pools (peak 3 under -limit 2) and passes with the replacement. Focused pipeline tests/vet and repeated -race checks pass, as do generic/Seriatim/contracts/reconciliation tests and worker -configuration tests. Production pipeline/config builds and `git diff --check` -pass. Full CLI/module integration still stops at the legacy YAML `plugin/limit` -import. The full pipeline suite now reaches remaining retired fixtures such as -`NormalizeRetry`/`runProducerAttempts`; cleanup remains required. After migrating -the configuration extractor fixture, full configuration tests compile but expose -three stale assertions: two use literal backslash-x00 rather than NUL paths, and -one reads the unresolved binding turn pointer rather than resolved controls. -Those full-suite results are not claimed passing; scheduled fixture cleanup and -final verification must close them. - -**Goal:** one ready-job limit and native backend admission govern concurrent work. - -**Inspect/change:** `runner_concurrent.go`, synchronized collaborators, -CLI runtime construction, old `llm/scheduler.go`, and concurrency/cancellation tests. - -**Work:** use the existing coordinator to dispatch ready extraction and ready -merge/normalize jobs to one worker pool controlled by `concurrency.workers`. -Lane completion becomes ready only after all of that lane's extract jobs finish. -Never occupy a worker while waiting for prerequisites, or acquire another worker -for nested validation/correction. Keep sequential input/chunking/publication and -ordered steps. Remove scheduled-client/global permit plumbing and worker coupling; -share the same native engine across every role. Handle native capacity failures -through the producing operation's mode without another queue or backoff. - -**Acceptance:** one worker can complete a multi-lane pipeline without deadlock. -The limit bounds both job types together, independent of backend settings. -Completion timing cannot choose public order or the terminal scoped error. -Cancellation stops dispatch, drains started work, and releases native admission. - -**Validation:** focused runner concurrency tests with test-controlled worker and -backend limits, including overlap, serial work, ready-lane interleaving, nested -same-backend validation, capacity failure, and cancellation. Use focused race -testing when supported; it is a migration check, not a new release requirement. - -## Stage 26: Implement The Small Accepted-Artifact Filesystem Cache - -**Status:** Complete. Added a narrow accepted-artifact cache collaborator and -filesystem implementation storing unchanged JSON bytes at the selected root -under the opaque native hash plus `.json`. Construction/lookup create no cache -directories. Auto reads/writes, bypass performs no cache I/O, and refresh skips -lookup and replaces only a newly supplied valid result; failed writes preserve -previous acceptance. Missing entries are ordinary misses, while malformed, -unreadable, non-file, and unsafe entries return diagnostic misses. Caller-owned -typed decoding/usability and best-effort warning handling remain in the next -integration stage. Reused confined atomic process-permission writes without new -permission policy, envelopes, fingerprints, expiry, or legacy data migration. -Real-filesystem tests cover raw-byte identity, exact root/hash placement, modes, -failed refresh, cancellation, concurrent complete replacement, temporary-file -cleanup, narrow paths/symlinks, unchanged legacy records, umask/setgid inheritance, -and unchanged provisioned root permissions. Cache/fileio/contracts tests, vet, -race checks, focused pipeline tests, and `git diff --check` pass. - -**Goal:** one best-effort cache facility replaces complex persistent records. - -**Inspect/change:** existing `internal/core/fileio` and cache permission/path tests; -add a focused accepted-artifact store under `internal/framework` or the existing -state area, with a narrow lookup/write collaborator for pipeline use. - -**Work:** read/write accepted artifact bytes at `/.json`, -using the native opaque hash unchanged. Use ordinary temporary-file/rename -publication and existing proportionate path/permission correctness helpers. -Keep roots at the CLI/state boundary. The caller owns typed codec/usability -checks. Missing entries are normal misses; unreadable/malformed/unusable entries -are misses with diagnostics. Failed writes warn without failing successful work. -Implement `auto`, `bypass`, and `refresh` I/O semantics; bypass creates no directory. -Refresh cannot remove a previous entry before a newly accepted replacement exists. -Do not add envelopes, manifests, key prefixes, expiry, migration, or collection. - -**Acceptance:** records contain only the accepted artifact/plan; no execution, -configuration, transcript copy, acceptance ledger, or historical usage is required. -Creation respects umask/setgid/default ACLs under ADR-0017, without permission -convergence or added access restrictions. Legacy stores are left untouched. - -**Validation:** focused real-temp-filesystem tests for hit/miss, unusable data, -read/write failure diagnostics, atomic replacement, modes, narrow paths, and -ordinary creation permissions. Reuse existing safety coverage where applicable. - -## Stage 27: Cache Final Typed Producer Artifacts Under The Initial Request - -**Status:** Complete. Eligible producer operations now prepare the initial native -request once, retain its detached details and opaque MessageContentHash, and -execute that same handle on a miss. Auto hits decode the expected durable artifact -with ordinary source-addressing checks and return without interpretation, -assessment, execution events, historical usage, or exchange replay. Prepared -handles are discarded on every unused exit. Only final ordinary acceptance writes -the applied typed value under the initial key, including after correction; -refresh skips lookup and failed refresh preserves previous acceptance. Bypass, -fallback, cancellation, and incoming ineligibility perform no cache writes, with -bypass/ineligibility using ordinary native execution without cache preparation. -Bound typed codecs and optional exact-type evidence projections through runner -extraction/merge/normalization and registry specialization. Cache failures produce -scoped process warnings without failing successful work; artifact encoding and -capture failures remain ordinary errors. Transient scoped cache observations retain -prepared evidence on hits without establishing a persisted cache/report contract. -Real native fake-provider/filesystem tests cover corrected applied output, fixed -initial keys, settings/session reuse versus content misses, frozen prepared -execution, no reassessment/execution on hits, unusable data, source/type checks, -all terminal exclusions, failed refresh, cancellation during hit delivery, -best-effort warning publication, merge/normalization reuse, and predecessor reuse -after downstream failure. Focused pipeline tests/vet/repeated race checks, cache/ -contracts/generic/Seriatim/reconciliation tests, selected vet/race checks, production -framework builds, and `git diff --check` pass. Full pipeline tests still encounter -retired fixtures such as NormalizeRetry/runProducerAttempts; CLI/module integration -still stops at old ConfigKit's removed YAML `plugin/limit` import. Those full -suites are not claimed passing. Chunk-plan cache integration and legacy checkpoint -execution removal remain the next stage; physical CLI runtime wiring retains its -assigned later scope. - -**Goal:** integrate caching into the same native producer helper. - -**Inspect/change:** the helper, typed artifact codecs, operation eligibility, -resolved cache controls, and cache store from Stage 26. - -**Work:** for eligible cacheable LLM work in auto/refresh, prepare the initial -request with `PrepareExecution`; obtain `Details().MessageContentHash`, retain -the prepared details, and keep that key for the complete operation. In auto, -decode/usability-check a found artifact and discard the prepared handle on a hit. -On a miss execute that exact prepared handle using `RunPrepared`, then use the -same interpretation/assessment/coordination path. After final ordinary acceptance, -encode the final typed artifact and write it under the initial key. Normalizers -store the applied artifact, not the reconciliation proposal. Dispose of unused -handles on every exit. Bypass/ineligible work uses ordinary native execution and -performs no cache I/O. Capture must not cause another preparation. - -**Acceptance:** a hit makes no model calls, semantic validation, exchange replay, -or historical usage claims. Models/settings/session/cache hints/schema metadata -add no key dimensions; only the upstream ordered message-content value selects -reuse. Changed policy can deliberately reuse old artifacts until refresh/bypass. -Ordinary usability includes compatibility with the expected codec and current -source addressing; it does not rerun the configured domain/semantic validator chain. -Failed/rejected/incomplete/fallback and affected dependent operations neither -load nor store entries. Corrected accepted output uses the initial key. - -**Validation:** focused helper/store tests: content changes miss, settings-only -changes can hit, ordinary usability failure misses, late accepted write, no write -after terminal failure/cancellation/fallback, eligible predecessor reuse, and -native prepared details retained on a hit without an execution event. - -## Stage 28: Unify Chunk-Plan Reuse And Remove Checkpoint Execution Paths - -**Status:** Complete. Native chunk producers now use the same accepted-result -cache as typed artifacts, storing only the final domain plan under the initial -native message-content hash after materialization and complete assessment. -Hits decode and rematerialize against the current source without semantic -reassessment; unusable entries miss without entering conversation history. -Deterministic chunkers remain uncached. Removed source-digest plan lookup, -checkpoint collaborators/transitions, dependency fingerprint collection, -required-lane hydration, resume barriers, and recomputation policy from runner, -preparation, handoffs, and CLI cache composition. The CLI cache collaborator now -uses the resolved unified root; full native run wiring remains Stage 33. -Replaced canonical-partition/checkpoint tests with native-provider/filesystem -coverage for final correction, content/reference changes, settings-only reuse, -current-source rematerialization, unusable ranges/data, cache modes, terminal -exclusions, failed refresh, and unchanged legacy files. Retained current chunk-map -handoff/schema and deterministic provenance checks. Focused pipeline tests/vet/ -repeated race checks, cache/contracts/source/chunk-map/generic/Seriatim/ -reconciliation tests, production pipeline/cache/config builds, and -`git diff --check` pass. Full pipeline tests still stop at retired fixtures -(including checkpoint helpers and NormalizeRetry); CLI setup still stops at old -ConfigKit's removed YAML plugin/limit import. Those suites are not claimed -passing. Stage 35 owns remaining obsolete stores, module fingerprint declarations, -and fixture cleanup; final assembled verification remains required. - -**Goal:** LLM chunk plans use the same accepted-result cache and no second store. - -**Inspect/change:** `runner_chunk_plan.go`, `chunk_cache.go`, `chunk_plan_store.go`, -checkpoint hooks in runner/preparation/handoff, CLI cache composition, and old -`internal/framework/{chunkplan,checkpoint}` consumers. - -**Work:** cache a generated plan only after materialization and complete chunk -assessment/continuation acceptance, under its initial request's native hash. -On a hit decode/rematerialize it against the current source with ordinary -structural/source-reference usability checks, without semantic revalidation. -An unusable cached plan is a miss, not a defective assistant response. Keep -deterministic chunkers uncached without synthetic requests. Remove source-digest -plan lookup/publication, checkpoint event/state transitions, stage/dependency -fingerprints, resume barriers, and recomputation bookkeeping from execution. -Old records are unread and unmodified, even below the selected new cache root. - -**Acceptance:** changed chunk request messages can select a different plan; -source identity alone no longer pins a partition. Default reruns reuse accepted -predecessors without `--resume`. Chunk fallback and dependent work skip reuse. -No checkpoint identity is retained solely to fill diagnostics. - -**Validation:** focused chunk/runner/cache tests with fake providers: matching -plan reuse, changed references/prompt text, settings-only reuse, invalid cached -source ranges, strict/permissive outcomes, bypass/refresh, and legacy files left -unchanged. Remove old canonical-partition/resume-policy tests as superseded. - -## Stage 29: Publish Only Domain Artifacts At Existing Paths - -**Status:** Complete. The JSON encoder now returns only lane artifacts and -explicitly enabled, available chunk-map/evidence-context files, preserving lane -filename mapping, JSON payloads, intrinsic metadata, codec checks, and auxiliary -schemas. Removed index/manifest/rejected/warnings/diagnostics sidecar generation, -the reporting-model package, manifest/profile/validation ledgers, and reporting-only -output inputs. Runner results expose deliverable files, process diagnostics, and -fallback count; intermediate normalized values remain private for ordered handoffs -and evidence construction. ArtifactCount counts deliverable files only. -Retained filename/collision, JSON/number, auxiliary codec, source ordering/window, -projection/type, ownership, and disabled-output coverage. Added explicit auxiliary -selection/count and no-sidecar checks, migrated evidence preparation fixtures, -and verified evidence reconstruction from a native cached normalization without -semantic reassessment. The Seriatim runner contract now checks the real JSON -domain artifact instead of a manifest or fake output. -Focused pipeline tests/vet/repeated race checks, generic/Seriatim/contracts/cache/ -source/asset/chunk-map/evidence tests, selected vet/race checks, production -pipeline/generic/Seriatim/config builds, and git diff checks pass. Independently -runnable candidate-JSON, combat-turn, NPC-occurrence, scene-description, and -source-range codec tests pass. Full pipeline/CLI and remaining D&D codec suites -still encounter unmigrated callers of the removed reporting package, the old -ConfigKit YAML plugin/limit import, or retired fingerprint declarations. Those -suites are not claimed passing; scheduled command/debug integration and Stage 35 -cleanup must close the blockers before final verification. Native completion -reporting and physical publication/failure outcomes remain Stage 34. - -**Goal:** simplify the logical output boundary without changing payload contracts. - -**Inspect/change:** `internal/modules/generic/output/json/encoder.go`, -`internal/core/artifacts`, output contracts/runner result assembly, evidence output, -and chunk-map publication. Inspect the existing lane filename mapping. - -**Work:** keep run-specific `lanes/.json`, optional `chunk-map.json`, -and optional `evidence-context.json`. Preserve domain schemas, exact lane mapping, -and union-of-cited-source-segments evidence behavior. Remove mandatory index, -manifest, rejected/warnings/diagnostics sidecar generation and reporting-only -models/metadata. Return the small application outcome and logical deliverable -files needed for publication, fallback disclosure, and optional capture. -Intrinsic domain metadata remains intrinsic, not discarded as reporting data. - -**Acceptance:** codecs and normal downstream use need no metadata bundle or -persisted receipt. Artifact count includes actual published domain deliverables, -including reused artifacts and enabled auxiliaries, not intermediates/cache/debug. -Fallback artifacts retain required downstream shape without normal-acceptance claims. - -**Validation:** focused encoder/evidence/chunk-map/codec tests verifying preserved -payloads and paths, no mandatory sidecars, optional artifact inclusion, accurate -counting, and evidence source order/window behavior. - -## Stage 30: Build The Debug Recorder And Session Lifecycle - -**Status:** Complete. Replaced summary/trace allocation and separate terminal -writers with one run-scoped recorder for incremental JSON/text evidence and one -terminal outcome. Allocation creates the absolute run directory and atomically -updates a relative latest link at start; completion never retargets it. Failed -starts retain an available directory rather than deleting captured data. -The recorder is concurrency-safe, retains callback errors for boundary checks, -preserves primary causes alongside capture failures, and finalizes once without -rewriting evidence on repeated calls. Finalization failures are errors, including -after artifact publication; a later presentation failure remains reportable. -Added reusable native debug/debug-dir input binding and selection, preserving -quiet/debug ordering, explicit false/empty values, configured/default roots, -invocation-relative overrides, and nonempty overrides over empty configuration. -Disabled capture performs no allocation or latest update. Removed the legacy -summary/terminal factories and routed run-path allocation/finalization through -the new lifecycle before engine/preflight and before successful presentation. -Real-filesystem lifecycle/concurrency/failure tests, isolated real-CommandKit -flag/selection tests, focused pipeline tests, selected vet/race checks, production -debug/bundle/pipeline/config builds, and git diff checks pass. Full CLI/pipeline -package checks remain blocked at setup by unmigrated imports of the removed -reporting package and old ConfigKit's YAML plugin/limit dependency; their results -are not claimed passing. Full native evidence/observation wiring remains Stage 31, -native run-handler composition remains Stage 33, and final publication/report -outcome integration remains Stage 34. Remaining obsolete fixtures are Stage 35 -cleanup, not a reason to restore the old writers. - -**Goal:** requested capture has one simple recorder and explicit failure semantics. - -**Inspect/change:** `internal/core/debugbundle`, `internal/framework/debug`, -pipeline debug collaborators, run IDs, and CLI debug/state composition. - -**Work:** replace the old summary/trace hierarchy with an application recorder -for run context, operation evidence, and one final outcome. Implement -`//` and start-time relative `latest`, without completion-time -retargeting. Resolve the selected native debug boolean and CLI/config/empty/default -directory semantics. Allocate only on the run path after config/run identity and -before engine construction/preflight. Persist incrementally and finalize once. -Allocation/link failure prevents model work; recording failure surfaces at normal -boundaries; finalization failure fails the invocation. Preserve an existing primary -failure and available data/publication paths; never retry a model for capture failure. - -**Acceptance:** a configured directory alone never enables capture; explicit empty -disables it; a nonempty CLI override can re-enable it. Disabled runs leave `latest` -untouched. Quiet mode and capture selection are independent. No automatic bundle -deletion, fallback directory, capture-depth setting, or new filesystem framework. - -**Validation:** focused temp-filesystem lifecycle tests: absent/empty/nonempty -selection, start failure, failed run retention, concurrent start/link semantics, -incremental recording failure, finalization failure, and no completion retarget. - -## Stage 31: Attach Native Execution Facts And Complete Debug Evidence - -**Status:** Complete. Runner invocations now enroll initial, coordinated, and -nested validator executions in one native ExecutionGroup, preserving native -identity, terminal coverage, and unknown usage without copied totals. Scoped -capture composes existing observers and lightweight preparation callbacks, reuses -split-preparation details on hits, and records native events, prepared contracts, -results, exact response text, and full readable exchanges. Undelivered responses -are explicitly unassessed; result-less and preparation failures retain availability -and error evidence. Capture failures preserve primary causes and cannot trigger -semantic retries or permissive fallback. -Added frozen secret-aware application/PromptKit configuration and source snapshots, -resolved composition, self-contained source/reference/incoming-artifact inputs, -codec candidates, complete findings/guidance, interpretation and correction inputs, -normalization originals/mappings/proposals, deterministic/fallback outcomes, and -cache decisions/artifact bytes. Native credential omissions and exact adopted -configuration sensitivity paths are preserved without scanning application text; -runtime options are not reserialized outside native inspection. Removed the old -completion decorator, synthetic IDs, duplicate accounting, and heuristic content -scrubbing. Evidence is written incrementally without whole-run content retention. -Focused native helper/runner/assessment tests, isolated real-kit configuration -capture tests, vet, repeated race checks, recorder/cache/contracts/generic/Seriatim/ -reconciliation/asset tests, production framework/config builds, and diff checks -pass. Concrete D&D normalizer/validator and full CLI/pipeline suites remain blocked -at setup by legacy removed reporting-package imports and old ConfigKit's YAML -plugin/limit dependency; their runtime results, including the new normalization -capture regression, are not claimed passing. Scheduled cleanup and final verification -must execute them. Native run-handler composition and final reporting remain in -their assigned later stages. - -**Goal:** use native observations/snapshots instead of parallel execution accounting. - -**Inspect/change:** helper/validator request observers, runner operation boundaries, -new recorder, old `pipeline/debug.go`, and PromptKit observation/exchange APIs. - -**Work:** create one invocation `ExecutionGroup`. Enroll the initial request once, -continuations once through `CoordinateRequest.Observer`, and nested validators -once on their own requests; do not also attach the same group to continuation -requests. Compose scope-aware diagnostic/capture observers without replacing -existing ones. Use native RunID/ParentRunID and totals, preserving unknown usage -and terminal coverage. Keep `OnPrepared` lightweight; retain its detached details -for normal boundary writes. For split prepared/cache paths use existing details. - -Record the roadmap's full available inventory: effective secret-aware config and -composition; parsed source; self-contained source/reference/normalization inputs; -prepared requests/output contracts; native events/results/raw response text; -all visible exchanges/readable messages; typed candidates, findings/guidance, -correction inputs, fallbacks, deterministic results, and cache decisions. Label -unassessed `Undelivered` responses and failures lacking prepared/result evidence. -Use convenient indented JSON/text without truncation, fabricated provider bodies, -heuristic scrubbing, or whole-run content accumulation. Exact filenames stay private. - -**Acceptance:** native totals count executions rather than producer attempts; -cache hits add none. Debug needs no second prepare or model call. Shared callbacks -are concurrency-safe; observer write failures reach the next boundary. ConfigKit -secrets/native credentials remain omitted without rewriting ordinary application -content. No duplicate usage accumulator, synthetic call-ID model, or profile ledger. - -**Validation:** focused helper/recorder tests for initial+continuation+nested calls, -result-less terminal failures, cache hit details, canceled/undelivered turns, -full untruncated selected content, native credential omissions, and concurrent -operations. Check meaningful evidence, not the complete native JSON shape. - -## Stage 32: Add Offline Preflight And Pipeline Inspection Commands - -**Status:** Complete. Added native pipeline list/show/validate declarations using -the shared captured application load. List enumerates sorted configuration IDs -without constructing a catalog or engine. Show uses ordinary domain resolution -and reports steps, lanes, modules, controls, validator chains, prompt declarations, -and reference bindings without expanding profiles or serializing runtime options. -Validation resolves the same domain contracts and supplies the captured engine -only for selected model work. The new preflight package inspects every known -initial/correction/validator/input/output prompt and explicit/applicable default -profile, preserves native causes, and aggregates pipeline/step/lane/validator -context. Partial profiles pass; undeclared dynamic selections remain runtime -checks. No request preparation, transcript/reference reads, credential lookup, -provider call, module construction, or run-storage creation is involved. -Focused real-kit preflight and CLI command tests, vet, repeated race checks, -focused pipeline tests, production preflight/pipeline/config builds, and diff -checks pass. Native configuration validation retains no pipeline-specific mode; -added assembled help/config-command regressions. The full CLI suite, including -those assembled regressions, remains blocked at setup by removed reporting-package -imports and old ConfigKit's YAML plugin/limit dependency; it is not claimed passing. -Scheduled cleanup and final verification must execute it. Native run-handler -composition will reuse this preflight boundary in Stage 33. - -**Goal:** execution and inspection share domain resolution and known dependencies. - -**Inspect/change:** new `internal/preflight`, CLI pipeline declarations, catalog, -profile/reference/module resolution, and existing explicit-profile preflight tests. - -**Work:** implement `pipeline list`, `show `, and -`validate `. List only enumerates configuration. Show reports resolved -steps/lanes/modules/validators/references without building an engine merely to -expand every profile. Validation reuses resolver checks and, when LLM dependencies -are selected, the same captured engine and static module declarations as runtime. -Check initial/correction/validator prompts, explicit/applicable/default profiles, -and known dependencies with pipeline/stage/lane context. Remove the old pipeline -mode of `config validate`; that standard command validates resolved configuration. - -**Acceptance:** offline preflight needs no input transcript, reference file bytes, -credential values, provider call, output/cache/debug root, or synthetic request. -Valid partial profiles pass. Dynamic request selections are checked at runtime. -Preflight does not load configuration, register modules, construct engines, or -render CLI reports; it accepts those collaborators/data from composition. - -**Validation:** focused preflight/CLI tests for known missing prompt/profile, -validator and correction dependencies, defaults with no explicit profile, valid -partial profile, no-credential/no-transcript operation, and useful scope context. - -## Stage 33: Wire Native Pipeline Run Inputs And Runtime Composition - -**Status:** Complete. Installed native `pipeline run` and its CommandKit `run` -alias with typed required input, literal repeated lanes, output/cache selection, -reference binds/unbinds, debug controls, and native resource/execution overrides. -One captured application result and engine serve preflight and every model role. -Reference expansion preserves specificity, required slots, generated conflicts, -and CLI/config file origins. Generated sessions remain input-based; explicit -empty profile/session/reasoning and false/zero inputs preserve native behavior. -Debug allocation precedes engine/preflight, which precedes source execution. -Removed the old run dispatcher/parsers/help, retired flags, and legacy runtime -client/asset construction. Retained confined publication helpers for the next -reporting integration. Real-kit/fake-provider run/reference/session/override tests, -focused pipeline and configuration tests, supporting preflight/assets/contracts/ -cache/debug/generic/Seriatim tests, focused vet/race checks, and diff checks pass. -Full CLI/executable checks, including the new subprocess route tests, remain -blocked at setup by `run_result.go` importing the removed reporting package and -old ConfigKit importing YAML's removed `plugin/limit`. The full profile test file -also retains retired `CorrectionProtocol` fixtures. Those checks are not claimed -passing; Stages 34–35 own their remaining integration and fixture cleanup, and -final assembled verification must execute them. Native completion reporting and -publication/presentation failure outcomes remain Stage 34. - -**Goal:** the executable runs the migrated pipeline through one native handler. - -**Inspect/change:** CLI application/pipeline commands, `run.go`, catalog, -reference selectors, `session.go`, run IDs, configuration/engine/state loaders, -and `cmd/notarius`. - -**Work:** install `pipeline run --input-file PATH` and -`commandkit.Alias("run", "pipeline run")`. Use typed supplied values for `--lane`, -`--output-dir`, `--cache`, references/unbinds, debug directory, and ckprompt -overrides. Keep repeated lanes literal (commas are not separators); validate every -singleton occurrence and retain the last successfully parsed value. Expand -reference selectors before framework resolution using the existing specificity, -required-slot, external/generated-conflict, and file-origin policies. Preserve -deterministic input-based session generation, independent of references, and -native explicit session override behavior. Apply run-wide profile/execution -overrides to producers, validators, and continuation requests. - -Reuse the one loaded selection and one engine, resolve/preflight before source -execution, then materialize references and prepare typed modules. Allocate requested -debug before engine/preflight as specified; keep all physical roots at composition. -Remove old dispatch, assembled help, parsers, comma `--only`, bespoke `--json`, -reasoning-clearing flag, resume/recompute/chunk-cache flags, and legacy aliases. - -**Acceptance:** native alias and canonical route invoke the same declarations/ -handler; `--`, operands, repeated flags, modes, and supplied empty/false/zero retain -upstream behavior. No forwarding command, recursive command invocation, fallback -to another config for explicit empty input, or request override mutation of config. - -**Validation:** focused command/reference/session tests and subprocess cases for -the two run routes, multi-reference input, literal comma lane, required input, -native overrides reaching all roles, explicit empty reasoning/debug path, and -input-hash session stability across changed reference bundles. - -## Stage 34: Finish Publication, Native Completion Reports, And Failure Outcomes - -**Status:** Complete. Replaced the receipt/encoder/index and validation-ledger -reporting model with one native completion value and text template. Publication -and requested capture finish before reporting; JSON/YAML share the same value, -omit zero fallback counts and disabled debug paths, and never create reporting -sidecars. Qualified fallback summaries remain visible independently of warning -verbosity. Native progress, configuration diagnostics, and process findings use -stderr without promoting domain advisories to warnings. Operation causes remain -separate from native rendering/output/diagnostic delivery evidence. Failed capture -or presentation retains published files and available paths without model retries -or repeat terminalization. Identifiable preflight selections now retain a typed -classification alongside native causes; cancellation and unknown/mixed failures -remain operational. Replaced obsolete receipt tests with real-kit/fake-provider -coverage for formats, counts, native modes/progress, quiet rendering failure, -partial/failed stdout, failed diagnostics, cancellation, publication/capture -ordering, and primary/capture/delivery cause preservation. -Focused CLI/pipeline tests, supporting preflight/debug/cache/contracts/assets/ -generic/Seriatim tests, focused vet/race checks, and diff checks pass. Added real -executable format/fallback tests using a local fake HTTP provider; their setup and -the full CLI/executable checks remain blocked by legacy `framework/llm` importing -the removed reporting package and old ConfigKit's removed YAML `plugin/limit` -dependency. Those subprocess results are not claimed passing. Stage 35 owns -remaining legacy code/dependency/fixture removal and whole-tree compilation; -assembled and final verification must execute the pending checks. - -**Goal:** subprocesses receive artifacts plus the selected small native report. - -**Inspect/change:** `run_result.go`, `run_terminal.go`, physical output placement, -CommandKit reporter/outcomes, debug finalization, and subprocess receipt tests. - -**Work:** publish domain files in the existing run-specific destination, finish -requested capture, then call `Reporter.Report` with pipeline, artifact_count, -absolute output_directory, optional debug_directory, and positive-only -fallback_count. Supply the selected human text template from that same value. -Remove separate encoders, receipt/error envelopes, automatic report persistence, -warning-count summaries, and durable execution ledgers. Send native diagnostics -to stderr and results to stdout, preserving quiet/mode/progress semantics and -fallback qualification even when warnings are hidden. - -Map native outcomes: completion `0`, operational/cancellation/publication/ -presentation failure `1`, identifiable invalid invocation/configuration `2`, native -diff differences `3`. Preserve typed underlying causes and distinguish operation -failure from report/diagnostic delivery failure. A post-publication capture/report -failure identifies available artifact/debug paths, returns nonzero, and does not -rerun or remove valid artifacts. Failed stdout may be partial; consumers check status. - -**Acceptance:** `--format json --verbose` has report-only valid JSON on stdout -for success. Positive fallback_count counts emitting operations once, not affected -artifacts, attempts, or validators. Quiet suppresses stdout but not useful failure -explanations. Native standard commands have no Notarius-wide envelope. - -**Validation:** focused subprocess tests for text/JSON/YAML success, fallback and -zero omission, verbose stderr, quiet/debug order, render/delivery failure, -publication then capture failure, useful exit categories, and no reporting sidecars. - -## Stage 35: Remove Superseded Machinery And Restore Whole-Tree Compilation - -**Status:** Complete. Removed the retired LLM wrapper, asset/schema registries, -prompt filesystem/mount helpers, copied completion/correction/debug contracts, -checkpoint/chunk-plan source packages, manual diagnostic renderer, reporting -models, and reporting-only module fingerprint methods. No runtime output/cache/ -debug data was deleted. Production has one native execution path; no compatibility -adapter or empty legacy package remains. - -Migrated retained framework, module integration, and CLI fixtures to real native -loaders, engines, provider requests, typed codecs, and published domain artifacts. -Preserved catalog/registration and validator-chain checks, source/annotation -isolation, contextual identity, chronology, generated-reference compatibility, -catalog semantics, evidence union, scene eligibility, complete assessment and -correction, diagnostics, sessions, and intrinsic codec rejection despite validator -overrides. Cross-family contracts retain their black-box integration boundary. -Removed only superseded retry, checkpoint, manifest/index, synthetic completion, -and reporting-ledger assertions; native producer/cache/assessment/capture tests -own replacement behavior. - -Native command/configuration fixtures retain discovery/import/overlay origins, -inheritance, supplied overrides, secret-aware inspection, partial profiles, -offline preflight, and explicit smoke work. Run/state fixtures retain independent -roots, cache modes, group-accessible reuse, narrow identities, collision protection, -selected-data capture, and failure retention without checkpoint/resume machinery. -Maintained-example tests now compile against native commands/reporting; actual -example files are deliberately unchanged until Stage 36. - -`GOWORK=off go mod tidy`, `go list -m all`, whole-tree test compilation -(`go test ./... -run '^$'`), whole-tree vet, production executable build, and -`git diff --check` pass. Removed old-path kits/catalog dependencies and their -notices; retained notices for actual native/runtime dependencies and the Go -minimum of `1.27.0`, without replacements. Scoped removal searches find no current -legacy consumers; remaining fingerprint terminology concerns intrinsic domain -identity, and old-cache names in tests verify legacy data is left untouched. - -All framework, module/integration, configuration, and non-example CLI tests pass, -including real executable route/format/fallback checks with a local fake HTTP -provider. Focused complete pipeline/integration and CLI run/state/command/domain -race checks pass. The full runtime suite is not claimed passing: maintained -example tests still reject the version 5 files, whose relocation/native migration -belongs to Stage 36. Final assembled and whole-roadmap verification remain required. - -**Goal:** delete the old architecture rather than leave dormant alternatives. - -**Inspect/change:** all production/test callers of the feature roadmap's removal -inventory; `internal/framework/llm`, `promptfs`, `readonlyfs` consumers, -checkpoint/chunkplan stores, private completion/correction/schema types, -manifest/receipt models, and old CLI tests/dependencies. - -**Work:** remove now-unused adapters, schedulers, asset/schema registries, -flattening/mount helpers, effective-YAML walking, fingerprints, manual correction -transport/capability checks, generic retries, profile/usage conversions, debug -wrappers, state ledgers, and old command surfaces. Retain useful pure helpers only -where a current consumer needs them, in the actual owning package. Do not keep an -empty `internal/framework/llm` for construction. Adapt remaining tests to native -provider boundaries and final contracts; delete tests solely asserting retired -behavior, not tests protecting unchanged domain correctness. Run tidy and remove -old-path kits/catalogs after their last imports disappear; reconcile notices. - -**Acceptance:** every package and test compiles with the declared dependencies. -Production code has one native path and no legacy compatibility scaffold. Removal -does not delete output/cache/debug directories or rewrite accepted ADR decisions. -Pure domain/codecs remain free of unnecessary library dependencies. - -**Validation:** `GOWORK=off go mod tidy`, `GOWORK=off go test ./... -run '^$'`, -`go list -m all`, and scoped code searches for the removal inventory. Classify -historical release/ADR/archive mentions rather than blindly deleting text matches. - -## Stage 36: Migrate The Maintained Minimal And Complete Examples - -**Status:** Complete. The two version 6 examples now live in separate -`examples/dnd-minimal/` and `examples/dnd-complete/` directories with their own -transcripts and declaration-relative imports/references. Moved data fixtures -without changing their bytes and updated test consumers. Retired the old copyable -configurations/profile file. Minimal uses embedded resources only; complete uses -all D&D families, ordered generated references, optional domain auxiliaries, -native companion profile/backend inheritance, independent workers, and explicit -cache/mode/semantic-turn controls. The limited backend leaves native queue and -diagnostic defaults inherited; development profile/pipeline/overlay variants use -native customization and explicit zero semantic turns rather than repair wrappers. - -Both actual configurations and their pipelines pass native offline validation -without credentials, including the development variant and overlay. Maintained -fake-provider runs verify domain publication and source/contextual identity; -inspection tests verify companion isolation, inherited resources, and overlay -origins without provider work. Whole-tree tests, vet, build, repeated focused race -checks, direct executable validation, and `git diff --check` pass. No live model -or real credentials were used. Later assembled verification and documentation -stages remain pending. - -**Goal:** two useful D&D configurations demonstrate the final native workflow. - -**Inspect/change:** `examples/dnd-{minimal,complete}.config.yml`, pipeline imports, -profiles/overlay files, transcript/reference fixtures, and -`internal/cli/example_contract_test.go`. - -**Work:** organize the two examples as `examples/dnd-minimal/config.yml` and -`examples/dnd-complete/config.yml`, moving their required imports/fixtures and -fixing paths. Separate directories prevent the complete example's companion -resources from changing the minimal example. Minimal uses embedded defaults and -a simple pipeline. Complete uses every configured D&D family and ordered generated -references, with optional chunk-map/evidence output; demonstrate native companion -profiles, inheritance/operator override, and a limited backend without overriding -the native queue default unnecessarily. Use version 6, qualified IDs, new worker/ -cache/mode/turn controls, and no old retry/structural-repair fields. Replace the old -development variant with a native profile/prompt customization or semantic-budget -example, not a compatibility translation. Keep all files secret-free. - -**Acceptance:** both actual config paths load, resolve, and pass offline pipeline -preflight without credential values. Imported paths are declaration-relative. -The minimal example has no accidental companion dependency. Retire superseded -copyable configurations, retaining useful shared test/reference data as needed. - -**Validation:** maintained example-loading tests plus native configuration and -pipeline validate commands for both paths. Fake-provider integration can exercise -them; live OpenRouter/local services are not default validation requirements. - -## Stage 37: Verify Assembled Recovery And Native Command Workflows - -**Status:** Complete. Added two focused native CLI recovery contracts using the -maintained ordered D&D composition, existing provider fake, real domain modules, -typed generated references, filesystem cache, and capture. They include semantic -normalization and the real combat-semantics LLM validator on one limited backend -with one shared worker. A final-turn corrected scene is stored under the initial -prepared message hash; a downstream failure preserves accepted predecessors and -the chunk plan, and fixed reruns execute only the previously failed work. Full -hits retain capture without native executions or historical accounting. Native -failure causes, session/model/backend selections, unknown usage, terminal group -coverage, domain payloads, and source identity remain intact. - -A failed permissive NPC refresh emits one qualified fallback, forces actual -generated-reference consumers to execute without reuse, leaves unrelated lanes -cached, and preserves earlier accepted files. Restoring ordinary mode reuses those -files without provider work. Operator-schema tests now execute the same prepared -handle and verify the custom provider constraint and native validation for inline, -source-relative, and external-root schemas; missing replacement roots do not fall -back to embedded schemas. No compatibility/configuration translation was added. - -Reviewed and retained the existing focused producer/assessment/cache/capture, -worker/admission, command/preflight, and report/subprocess coverage for structural -failure with nil error, result-less/undelivered responses, full ordered guidance, -final-turn assessment, unchanged candidates, native overrides, cancellation, -stream/delivery outcomes, and no duplicate debug preparation. Explicit test repair -budgets use complete native output contracts, without pinning upstream defaults. -All four prescribed package suites and vet pass, as do repeated focused CLI and -shared observer/coordinator/backend race checks and `git diff --check`. No real -credentials, external inference, or model-quality claim is involved. Stage 46 -still owns final whole-roadmap verification after documentation updates. - -**Goal:** prove the cross-boundary behavior most likely to regress in this cutover. - -**Inspect/change:** CLI assembled/command/cache contracts, module integration -tests, and new provider-boundary fixtures. Consolidate existing coverage rather -than creating a second comprehensive fixture framework. - -**Work:** exercise a representative ordered multi-step pipeline with typed -generated references and one native engine. Include the representative extractor, -normalizer, LLM validator, and deterministic paths. Add focused assembled cases -for downstream failure/fix/rerun reuse, correction stored under the initial key, -fallback taint across handoffs versus unaffected predecessors, chunk-plan reuse, -refresh failure preserving an old entry, native limited-backend admission and -serial workers, debug on hits/failure, and report/stream outcomes. Ensure no -unexpected external calls in help/inspection/preflight, or duplicate preparation -for debug. Cover custom operator schemas/contracts through real preparation; -where an actual request-level override is needed, use a complete native -`OutputContractInput` with `RepairAttempts` unset unless the operator supplied it. - -**Acceptance:** behavior follows the feature roadmap's full acceptance matrix, -including result-less failures, unknown usage, native status distinctions, final -turn assessment, all-validators guidance, source/session/reference preservation, -and unchanged intrinsic payloads. Avoid duplicating upstream parser/renderer/ -retry tests, pinning upstream default attempt counts, or snapshotting full help/ -native debug objects. No production model-quality claim is inferred from fakes. - -**Validation:** `GOWORK=off go test ./internal/cli ./internal/modules/integration -./internal/framework/pipeline ./internal/framework/semanticreconcile`; run focused -race checks for shared observers/coordinator when practical. Stage 46 owns the -complete final suite after documentation and remaining corrections. - -## Stage 38: Record Successor Decisions And Align Architecture Policy - -**Status:** Complete. Added sequential ADRs 0021–0024 for native kit/resource and -admission ownership, continuation and application assessment, unified accepted- -result caching, and artifact-first reporting/debug. Updated earlier status links -to distinguish full and partial supersession, including related version/profile, -registrar, reconciliation-fingerprint, and durable-diagnostic clauses. All changed -historical ADR titles, dates, and accepted bodies are byte-for-byte preserved; -ADRs 0012, 0017, and 0020 remain unchanged. Retained content-only assets, contextual -identity, typed/domain/source boundaries, process-warning classification, and -deployment-owned cache permissions. - -Aligned architecture policy with implemented native execution types, one worker -pool, independent native admission, complete candidate assessment, explicit -fallbacks, content-keyed reuse, native reports, and optional private debug evidence. -The adopted upstream baseline is unchanged and the local adaptation is identified. -Reviewed these invariants against the feature roadmap and current loading, -producer/assessment, worker, cache, publication, and recorder code. Local decision/ -policy links and anchors and `git diff --check` pass; existing pipeline, cache, -recorder, and configuration package tests pass (cached). No release/history rewrite, -new runtime contract, access policy, or permanent validation gate was introduced. -Canonical references and internal guides remain assigned to the following stages. - -**Goal:** document deliberate boundary changes without rewriting their history. - -**Inspect/change:** `docs/policy/architecture.md`, ADRs 0005–0007, 0011, 0014, -0016, 0019, and the retained 0012/0013/0015/0017/0020 decisions. Inspect the current -next ADR number before adding sequential records. - -**Work:** write concise successor ADRs for (1) native kit composition/execution/ -admission and prompt-declared assets, (2) native continuation with application -assessment/modes, (3) unified accepted-artifact caching, and (4) artifact-first -publication/native reporting and debug ownership. Use the next available numbers, -starting at 0021 if none have since been added. Change earlier statuses/links to -identify full or partial supersession, preserving accepted decision bodies and -retained content-only asset, contextual-ID, process-warning, deployment-permission, -and ConfigKit ownership decisions. Architecture now describes native library -types as an intentional execution boundary, one worker pool/native admission, -new retry/fallback/cache semantics, optional diagnostic records, and current -output/debug responsibilities. - -**Acceptance:** no current policy still mandates the retired private LLM API, -global scheduler, latest-only replay, source-only plan partition, checkpoint -resume, manifests, or heuristic sanitization. Retained source/domain/type and -filesystem correctness boundaries remain, with no new access policy, storage -guarantee, release gate, or public debug format. Keep policy provenance headers -accurate; this is a local architecture update, not a new upstream policy adoption. - -**Validation:** review each changed invariant against the feature roadmap and -implemented behavior. Check ADR status links and accepted-body preservation; -`git diff --check`. Do not rewrite historical release notes or archived roadmaps. - -## Stage 39: Rewrite Canonical Configuration And CLI References - -**Status:** Complete. Rewrote the canonical configuration and CLI references for -version 6, shared native source selection/companions, whole-record resource -replacement and inheritance, null/presence and schema-root semantics, qualified -profiles, workers, semantic turns, strict/permissive modes, and unified reuse -controls. Documented native pipeline/standard commands, supplied/repeated values, -references, complete run-wide execution overrides, presentation/streams, the small -completion report, and outcome classifications. Removed retired syntax and old -example paths rather than preserving compatibility examples. Native default -repair/retry and backend admission remain upstream-owned. Retained domain module, -reference-slot, output-option, and default-validator definitions, checking all 20 -D&D extract/normalize chains against the real resolved complete pipeline. - -Compared declarations, configuration/resolver tests, actual help, maintained -examples, and the selected upstream module documentation. Twenty direct offline -help/completion/configuration/inspection/preflight/diff checks pass with the -OpenRouter credential variable unset; assignment/explicit-zero examples also -pass. All 46 local links/anchors and 20 pinned upstream destinations/anchors -resolve against repository/module sources. Whole-tree tests, vet, a temporary -executable build with `GOWORK=off`, and `git diff --check` pass. No live inference -or real credential was used. Operations, migration, integration, and internal -documentation retain their following assigned stages. - -**Goal:** operators can configure and invoke only the implemented native surface. - -**Inspect/change:** `docs/config.md`, `docs/cli.md`, final command declarations, -ConfigKit definition, resolver tests, and the two maintained examples. - -**Work:** document native discovery and companions under one result; version 6; -qualified resources, replacement/inheritance/null semantics and schema roots; -native profile/backend controls; worker and semantic-turn controls; strict/ -permissive precedence; and cache settings. Explain no backend concurrency default -is injected and no independent Notarius structural-repair count is pinned. -Document new pipeline commands/alias, required operands, supplied values, repeated -lanes/references, run-wide ckprompt flags, native formats/modes/progress, stdout/ -stderr, and outcome codes. Link upstream-owned standard command semantics where -appropriate, with short Notarius-specific selection examples rather than a -copied upstream manual. Link full copyable files in `examples/`. - -**Acceptance:** current references describe implemented behavior only. Retired -flags/fields occur solely in clearly identified migration material, not usable -syntax. Config fields/defaults belong in the configuration owner, invocation -syntax in the CLI owner; neither duplicates physical state or artifact schemas. - -**Validation:** compare documented commands with native help and focused parser/ -loading tests. Check relative links/anchors and example paths. Test omitted and -explicit values through declarations, not prose-based change detectors. - -## Stage 40: Update Operations And Downstream Consumer Contracts - -**Status:** Complete. Updated operations for independent output/cache/debug roots, -per-user and explicitly provisioned system cache placement, best-effort reuse, -failed-refresh preservation, untouched legacy records, fallback qualification/ -eligibility, whole-source chunk fallback, and required capture with start-time -`latest`. Replaced receipt/index/manifest workflows with native completion reports -and configured domain paths, including partial stdout and post-publication failure -handling. Retired the superseded receipt contract, repaired current navigation, -and linked new native migration guidance for command/configuration/resource IDs, -retry intent, workers/admission, caches, reporting, and private debug evidence. -JSON/chunk-map/evidence contracts retain payload schemas and logical paths without -requiring descriptors or historical execution provenance on reuse. - -The maintained complete native CLI contract now supplies explicit input/output and -pipeline-level campaign/catalog references and verifies all ten domain files plus -both auxiliaries against the twelve-deliverable report using the existing provider -fake. Existing executable fake-HTTP checks retain native format/fallback coverage. -Found and corrected the release validator's retired version flag; its existing -offline test double now rejects the wrong command, demonstrably failing before the -fix and passing afterward. Updated only affected release documentation/command -references, preserving the selected validation gates and adopted policy baseline. -A temporary linked executable reports its injected version via the native command. - -Whole-tree tests and vet, executable build, shell syntax, 147 local links/anchors, -and `git diff --check` pass. Removal searches leave receipt-contract references -only in the active plan and preserved historical release/archive records. No -runtime data, domain schema, tagged note, or archive was changed; no real provider, -credential, release, push, or new provisioning/retention requirement was involved. -Full internal/integration/orientation rewrites remain assigned to the next stages. - -**Goal:** explain the intentional persistence and subprocess breaking changes. - -**Inspect/change:** `docs/operations.md`, `docs/consumers/{subprocess,dnd-pipeline}.md`, -`docs/integrations/run-result.md`, JSON/chunk-map/evidence integration contracts, -and any release-runbook command references affected by the CLI cutover. - -**Work:** describe the independent output/cache/debug roots, per-user cache default -and `/var/cache/notarius` system convention, reuse modes, best-effort I/O, unchanged -legacy data, and explicit refresh for policy/model experiments. Explain fallback -qualification and reuse exclusion, whole-source chunk fallback, debug selection/ -empty paths/start-time `latest`, retained evidence, and capture/publication failure. -Delete the superseded persisted receipt contract and replace consumer links with -the CLI report contract plus documented domain paths; do not create a new manifest -contract. Consumers check status, optionally decode the small report, then read -configured lane/auxiliary paths. Note partial stdout on delivery failure and valid -artifacts after post-publication failure. - -Add `docs/consumers/native-migration.md` covering old-to-new commands, -configuration/resources/catalog IDs, native retry intent, removed transient/repair -wrapper controls, workers/admission, cache reuse, and receipt traversal replacement. -Link it from canonical entry points. Exact old provider-attempt parity and automatic -cache conversion are not promised. Keep tagged release notes unchanged; preparing -or publishing a new release is outside this sprint. - -**Acceptance:** the complete D&D subprocess workflow uses `--input-file`, -`--output-dir`, and pipeline-level references as first-class inputs. Documentation -does not require removed sidecars, depend on debug layout as an API, claim cache -revalidation, or add new provisioning/retention restrictions. Artifact contracts -preserve their payload schemas and logical paths. - -**Validation:** exercise maintained subprocess examples with a fake provider and -compare resulting files/report. Check all references to the removed receipt, -old cache families, and legacy flags; retain necessary migration/history mentions. - -## Stage 41: Refresh Configuration, CLI, And LLM Internal Guides - -**Status:** Complete. Replaced the three internal guides with current native -ownership and source/test navigation. Configuration now explains the composed -definition, original typed result handoff, source contexts, detached runtime -conversion, complete-selection invocation reuse, focused inspection, and separate -catalog/preflight boundaries. CLI now explains lazy CommandKit construction, -standard suite wiring, registrars, supplied native overrides, ordered run -composition, and operation/capture/presentation outcomes. The retained LLM guide -describes direct native requests/results/continuation, fixed module inputs, -pipeline assessment, direct validator execution, prepared reuse, and native -observations without a replacement service or reporting ledger. - -Reviewed the named definitions/loaders, command composition/resolution, concrete -spell producer and combat-semantics validator, and their focused real-kit tests. -Removed obsolete adapter, scheduler, asset/schema registration, fingerprint, -profile-ledger, manual replay, and copied inspection descriptions. Public settings, -flags, defaults, state paths, and payloads link to canonical owners rather than -duplicated reference tables. All 85 local source/test/doc links and five pinned -upstream destinations/anchors resolve against repository/module sources. -Configuration, CLI, preflight, pipeline, spell extraction, and semantic-validator -package tests and vet pass, as does `git diff --check`. No production code or -operator contract changed; pipeline/state and module-specific guide rewrites -retain their next assigned stages. - -**Goal:** explain the actual native composition and execution boundary. - -**Inspect/change:** `docs/internal/{configuration,cli,llm}.md`; the final -configuration definition, composition-root loaders, command declarations, engine -construction, and native producer/validator call sites. - -**Work:** document one result handoff, source assembly, invocation reuse, standard -suite wiring, request overrides, native execution/continuation, and application -interpretation. Keep `llm.md` as an integration-focused explanation even though -its old package is removed. Link canonical operator and upstream contracts rather -than reproducing their fields, flags, or library mechanics. Remove old schemas/ -asset registration, provider-neutral adapters, scheduler, profile recording, and -inspection-conversion descriptions. - -**Acceptance:** a contributor can locate the current owner without being sent to -a deleted wrapper. Native execution types are explicit; pure domain types remain -separate. Volatile public details have one canonical owner. - -**Validation:** inspect the named packages and focused tests for every flow; -check cross-links and stale package references. - -## Stage 42: Refresh Pipeline And State Internal Guides - -**Status:** Complete. Rewrote both guides around current typed resolution, -materialization/preparation, operation-scoped complete assessment, native -continuation, strict/permissive disposition, and supported fallback. Documented -one ready-job pool, ordered barriers/results, cancellation draining, and separate -native backend admission. Handoffs distinguish usable qualified artifacts from -ordinary acceptance and propagate reuse exclusion only to actual dependents. - -State guidance now traces the retained initial prepared handle/hash, durable -typed/plan usability on hits without semantic reassessment, final acceptance -writes, best-effort cache I/O, independent publication, and one incremental -capture lifecycle with native observations. Removed retired checkpoint/resume, -separate pool, generic retry, reporting-sidecar, copied accounting, and heuristic -redaction flows. Public controls, paths, payloads, and handling link to canonical -owners; debug representation remains private. Inspected current runner/helper, -assessment, coordinator, handoff, cache, allocation/recorder, evidence, and CLI -publication code and focused regressions. All 84 local links/anchors resolve; -no current incoming anchors require repair. Pipeline, cache, recorder, bundle, -file I/O, and CLI package tests and vet pass; removal searches and -`git diff --check` pass. No runtime code/data or operator contract changed. -Module and D&D contributor recipes remain assigned to the next stage. - -**Goal:** explain acceptance, concurrency, recovery, and capture as implemented. - -**Inspect/change:** `docs/internal/{pipeline,state}.md`; typed helper/assessment, -worker coordinator, eligibility propagation, cache/store, recorder, and publication. - -**Work:** document initial/revised assessment, native semantic continuation, -strict/permissive terminal handling, supported fallback, ready-job workers, -generated-reference eligibility, accepted-artifact/plan reuse, and output/debug -ownership. Remove checkpoint fingerprints/resume, old chunk-plan canonicalization, -duplicate usage/debug wrappers, required manifest/receipt, and generic retry flows. -Link public state controls, artifact paths, and output contract definitions. - -**Acceptance:** current-flow descriptions distinguish native execution success, -domain acceptance, qualified fallback, reuse, and publication. No internal guide -promises a public debug layout or cache revalidation policy that does not exist. - -**Validation:** compare diagrams/prose with runner/cache/recorder and focused -tests; verify no stale state collaborator or operation ordering is carried forward. - -## Stage 43: Refresh Module And D&D Contributor Guides - -**Status:** Complete. Replaced obsolete correction-capability, private completion, -asset/schema registration, fingerprint, and reconciliation-service recipes with -the actual typed builder/spec interfaces and shared native producer path. -The extension guide now walks fixed input preparation, native initial/follow-up -requests, interpretation, supplied pipeline assessment, explicit supported -fallback, static resource dependencies, direct model validators, and deterministic -or legitimate no-model paths without synthetic requests or module-owned loops. - -D&D guidance retains all ten durable contract links and family identity, -source/evidence, contextual grounding, holder, combat-routing, and reconciliation -differences. It explains one-time windows/handles, fresh final-proposal application, -domain guidance, whole-source planning fallback, and the scene-description -construction limit without changing payloads. Updated the existing opt-in -combat-semantics evaluation procedure to native execution/capture/cache controls; -fake tests make no provider-quality claim. Walked the recipes against spell -extraction, item normalization, shared preparation/application, scene planning, -direct combat validation, registration, and native asset declarations. -All 86 local links/anchors and retained incoming anchors resolve. Asset, complete -module/integration, reconciliation, and pipeline package tests and vet pass; -removal searches and `git diff --check` pass. No production code, schemas, -evaluation corpus, or default-chain policy changed. Kit integration documentation -remains assigned to the next stage. - -**Goal:** future modules can follow the native architecture without copying drift. - -**Inspect/change:** `docs/internal/{modules,dnd}.md`; production registration, -representative spell producer, registry normalizers, scene chunker/validator, -native initial/correction assets, codecs, and shared domain helpers. - -**Work:** show how a module prepares fixed inputs, builds initial/correction native -requests, interprets results, supplies fallback, and receives pipeline assessment. -Document static dependency declarations and native prompt/schema ownership instead -of manifests/registration. Explain one-time normalization handles, fresh proposal -application, domain guidance, unchanged contextual-ID/source-reference rules, and -deterministic paths without synthetic requests. Link durable schemas and public -module configuration rather than repeating them. - -**Acceptance:** the extension recipe works for the actual typed interfaces and -retains artifact-family/domain boundaries. It requires no private completion -interface, schema registry, fingerprint, or module-owned retry engine. - -**Validation:** walk the recipe against one extraction and one normalization -implementation; inspect native asset paths/IDs and retained domain regression tests. - -## Stage 44: Align The Kit Integration Contracts - -**Status:** Complete. Updated ConfigKit and PromptKit integration contracts and -added CommandKit/ckconfig/ckprompt integration guidance for the actual selected -vanity-path releases. The guides describe one typed application result, native -contexts/focused views, captured engine/resources, direct requests/continuation, -prepared result reuse, independent admission, observations, and the fully installed -native command/reporting boundary. Removed old dual configuration engines, -catalog/adaptor APIs, latest-only replay, wrapper retry limits, scheduler/error -translations, copied inspection, and durable accounting claims. Public policy -and internals link to their canonical owners rather than copied upstream manuals. - -Compared current definition/loaders, suite declarations, reusable overrides, -preflight, producer/validator calls, and reporting with selected module API/docs. -Verified Go module release metadata and the separately tagged integration module -paths; notices match all five selected source hashes. All 48 local links and -20 tagged upstream destinations/anchors resolve against repository/module sources. -The one historical release link to an old integration heading remains unchanged -under release-history policy. Configuration, CLI, preflight, pipeline, and module -integration tests and vet pass, as do removal searches and `git diff --check`. -No dependency, runtime behavior, notice material, or published history changed. -Orientation/planning reconciliation and final acceptance remain next. - -**Goal:** describe Notarius's actual use of all three native libraries. - -**Inspect/change:** `docs/integrations/{pkg-configkit,pkg-promptkit}.md`; create -`docs/integrations/pkg-commandkit.md` for the implemented CLI integration. Inspect -selected public APIs, final module versions, source/engine loaders, standard suite -wiring, request execution, and reporting. - -**Work:** update supported integration/version references and Notarius's supplied -configuration, source contexts, runtime/request values, outcomes, and ownership. -Document the single application result/focused views, native prompt/schema -selection, continuation/admission, and standard CLI composition at the consumer -boundary. Link tagged upstream guides for native mechanics and canonical local -docs for product policy. Describe no unused future API, full copied upstream -manual, compatibility adapter, or private implementation as a public protocol. - -**Acceptance:** the integration docs agree with declared dependencies and current -calls; CommandKit/ckconfig/ckprompt adoption is complete, not deferred. Notices and -integration source links resolve to the intended components/revisions. - -**Validation:** inspect imports/call sites and local/upstream link destinations; -compare claims with Stage 37's assembled boundary tests. - -## Stage 45: Refresh Orientation And Reconcile Planning Status - -**Status:** Complete. README now uses the maintained minimal configuration and -transcript with native required input syntax and links both examples, current -contracts, kit integration, and migration guidance. Updated contributor routing -and the internal package map for the shared native load/engine, preflight, typed -producer/assessment, ready workers, accepted-result store, evidence, and recorder; -removed navigation to deleted adapter/reporting/checkpoint packages. - -Reconciled backlog wording for implemented compatibility checks, typed reference -projections, and concurrent lane completion while retaining human quality review, -larger reconciliation/context, distribution, and storage work separately. The -feature roadmap now distinguishes implemented choices/evidence from the still -pending final acceptance audit; retained its target/removal/acceptance scope and -historical baseline/comparison. Native library navigation is release-pinned. -Found and repaired one current Seriatim fixture link during the stale-name search. - -Both actual example pipelines pass direct offline preflight with credentials -unset; maintained minimal/complete/development example and shared-reference tests -pass using native provider fakes. All 149 local links/anchors and 16 selected -upstream destinations/anchors in the changed documents resolve; eleven dated -Audita comparison links remain classified historical evidence, not current owner -navigation. Current-doc/example removal searches leave only explicit migration -guidance; `git diff --check` passes. No code, schema, release, archive, or runtime -data changed. Stage 46 and full-goal acceptance remain unmarked/pending. - -**Goal:** entry points lead to current docs and preserve separate future scope. - -**Inspect/change:** `docs/internal/overview.md`, `docs/development.md`, `README.md`, -and `docs/roadmap/future.md`, plus status in this plan and the feature roadmap. - -**Work:** update package inventory/navigation for the actual composition root, -producer helper, cache, and recorder. Remove obsolete receipt/LLM-package reading -directions. Give README the shortest valid native `notarius run` quickstart and -links to the two examples, current reference docs, and migration guide. Link the -new kit integration contract where useful. Reconcile backlog items implemented -by this cutover without adding deferred features. Mark stages complete only from -actual evidence; leave final verification pending until Stage 46. Do not delete -the active plan/feature roadmap or prepare a release in this stage. - -**Acceptance:** orientation describes implemented behavior and routes to canonical -owners. Future scope stays in the backlog; historical ADR/release/archive records -are not rewritten to match new syntax. - -**Validation:** check quickstart/example paths and navigation, then search current -docs/examples for stale names while retaining meaningful migration/history mentions. - -## Stage 46: Verify The Complete Native Cutover - -**Status:** Complete. Audited every feature acceptance row against current -owning code, focused assertions, assembled/executable tests, and the checks below. -All earlier intermediate compilation/fixture blockers are closed: the full -default suite compiles and passes, including maintained examples and the concrete -module/validator/recovery tests previously recorded as blocked. - -Final checks pass with Go `1.27.0`, declared dependencies, and `GOWORK=off`: -`go test ./...`, `go vet ./...`, a fresh temporary `go build -trimpath` executable, -and `git diff --check`. Repeated focused race checks pass for mixed ready jobs, -backend admission/cancellation, shared native capture, recorder/allocation, and -assembled recovery/fallback. Twenty-five direct offline executable checks pass -with the OpenRouter credential unset: help/version, all three completions, -configuration scopes/sources/explanation/diff, native resources/requirements, -minimal/complete/development/overlay preflight, and native JSON/YAML values. -Provider-boundary and real subprocess tests supply success/fallback formats, -stream separation, required-input/alias behavior, and post-publication failures -without paid inference. - -The 252-file production inventory has fourteen native helper consumers (scene -planning, all ten extractors, three semantic normalizers), one direct native LLM -validator, and one composition-root engine constructor. Removed adapters, -schedulers, prompt/schema registries, generic retries, checkpoint/plan stores, -reporting sidecars, and copied accounting have no production consumers or tracked -legacy packages. Remaining test matches are explicit retired-input rejection, -unchanged legacy-data assertions, and deliberately selected native test budgets, -not pinned upstream defaults. Replaced the stale chunk-plan import-boundary fixture -with the current accepted-result store and removed its redundant special-case -guard; the existing generic framework boundary remains tested and unchanged. - -Compared schemas and D&D codec files byte-for-byte with the inspected baseline: -no differences. Reviewed module selection/build metadata and all eleven selected -dependency notices against their module-cache release hashes; no replacement -modules or old-path kits/catalog dependencies remain. Successor ADR changes alter -earlier status links only, preserving accepted bodies; releases/archives remain -historical. No output/cache/debug data was migrated or deleted. - -The read-only documentation check covers all 84 Markdown files, excluding literal -code-block examples from link interpretation. Current local destinations/anchors -resolve after fixing four stale D&D configuration anchors and the moved catalog -example link. Seven links in historical release notes intentionally refer to -retired checkout paths/headings; those records were preserved rather than -rewritten. Dated sibling comparison material remains evidence, not a current -implementation dependency. Selected upstream guide/source destinations were -checked against the exact module sources during the integration audit. - -Acceptance evidence was checked at these owners, not inferred from test names -or a successful build alone: - -| Feature acceptance row | Current evidence | -| --- | --- | -| Configuration/assets | [Typed definition/handoff](../../internal/core/config/native_settings_test.go), [main-relative source/peer/replacement cases](../../internal/cli/native_load_test.go), [operator schema execution](../../internal/cli/native_schema_load_test.go), and [every embedded declaration prepared](../../assets/package_test.go). | -| Commands/loaders | [Lazy native app/scopes/diff](../../internal/cli/application_test.go), [run inputs and shared roles](../../internal/cli/pipeline_run_test.go), [native overrides](../../internal/cli/execution_overrides_test.go), and [real executable routes](../../internal/cli/native_process_test.go). | -| Preflight/inspection | [Known/default/dynamic resources and partial profiles](../../internal/preflight/preflight_test.go) and [one-load offline commands without execution inputs/storage](../../internal/cli/pipeline_commands_test.go). | -| Domain workflow | [Spell pipeline contracts](../../internal/modules/integration/), [fresh typed normalization](../../internal/modules/dnd/normalize/itemregistry/native_execution_test.go), [candidate isolation/whole chains](../../internal/framework/pipeline/candidate_assessment_test.go), and [runner outcomes](../../internal/framework/pipeline/runner_operation_test.go). | -| Prompt/guidance contracts | [Native paired assets](../../assets/package_test.go), [correction defaults versus explicit selections/history](../../internal/framework/pipeline/producer_test.go), [complete ordered findings](../../internal/framework/pipeline/candidate_assessment_test.go), and [real combat judgment guidance](../../internal/modules/dnd/extract/scenedescriptions/native_execution_test.go). | -| Retry controls | [Presence/default/zero/precedence](../../internal/framework/pipeline/validation_policy_test.go), [loaded controls](../../internal/core/config/validation_policy_contract_test.go), [early/final-turn acceptance](../../internal/framework/pipeline/producer_test.go), and absent bundled repair counts in native declarations. | -| Non-LLM failure handling | [One deterministic production/error path](../../internal/framework/pipeline/runner_operation_test.go), direct generic/Seriatim modules, and [rejection of retired controls](../../internal/core/config/operational_document_test.go). | -| Continuation/failures | [Nil-error failed status, no prior recovery, native timeout/caller distinction](../../internal/framework/pipeline/producer_test.go), [unassessed undelivered/callback failures](../../internal/framework/pipeline/native_capture_test.go), and [direct validator failures](../../internal/modules/dnd/validate/scenedescriptions/combat_semantics/native_execution_test.go). | -| Strict/permissive outcomes | [Terminal handling](../../internal/framework/pipeline/producer_outcome.go), [runner counts/eligibility](../../internal/framework/pipeline/runner_operation_test.go), [whole-source plan fallback](../../internal/modules/dnd/chunk/scenes/chunker_test.go), and [final-only valid scene fallback/failure](../../internal/modules/dnd/extract/scenedescriptions/native_execution_test.go). | -| Admission/workers | [Mixed ready jobs, independent limits, serial/nested liveness, capacity modes, cancellation release](../../internal/framework/pipeline/ready_jobs_test.go), [stable ordering/draining](../../internal/framework/pipeline/runner_concurrency_test.go), and [nested same-backend assessment](../../internal/framework/pipeline/native_assessment_test.go). | -| Accepted-artifact cache | [Initial-key late applied-value writes, frozen handles, settings hits/content misses, terminal exclusions](../../internal/framework/pipeline/producer_cache_test.go), [typed usability and warning publication](../../internal/framework/pipeline/runner_cache_test.go), and [assembled recovery/actual dependents](../../internal/cli/recovery_contract_test.go). | -| Chunk-plan reuse | [Final plan key, current-source rematerialization, no semantic reassessment, changed-message partitions, and deterministic exclusion](../../internal/framework/pipeline/chunk_plan_cache_test.go). | -| Cache controls/migration | [Loaded roots](../../internal/core/config/operational_document_test.go), [resolved invocation precedence](../../internal/framework/pipeline/validation_policy_test.go), [raw atomic store/modes/legacy retention](../../internal/framework/artifactcache/filesystem_test.go), and [failed refresh/qualified lineage](../../internal/cli/recovery_contract_test.go). | -| Output/outcomes | [Domain-only encoder](../../internal/modules/generic/output/json/encoder_test.go), [native formats/counts/quiet/delivery/capture failures](../../internal/cli/run_result_command_test.go), [executable fake HTTP formats](../../internal/cli/native_process_test.go), and unchanged intrinsic schema/codec comparison. | -| Execution observations | [Once-only native enrollment, unknown usage, parent identity, result-less failures, and no hit executions](../../internal/framework/pipeline/native_capture_test.go) and [assembled native group facts](../../internal/cli/recovery_contract_test.go). | -| Debug controls/lifecycle | [Supplied empty/boolean selection](../../internal/cli/run_debug_test.go), [start-time relative latest/allocation failure](../../internal/core/debugbundle/bundle_test.go), [one incremental recorder/retained causes](../../internal/framework/debug/recorder_test.go), and [publication/capture ordering](../../internal/cli/run_result_command_test.go). | -| Debug evidence | [Full native snapshots/exchanges and hit/failure availability](../../internal/framework/pipeline/native_capture_test.go), [operation/normalization inputs](../../internal/framework/pipeline/operation_capture_test.go), [fixed mappings/proposals](../../internal/modules/dnd/normalize/itemregistry/native_execution_test.go), and [exact secret-aware projection without content rewriting](../../internal/cli/run_capture_test.go). | -| End-to-end migration | [Maintained minimal/complete/development workflows and explicit shared references](../../internal/cli/example_contract_test.go), [ordered failure/fix/reuse and fallback](../../internal/cli/recovery_contract_test.go), native executable checks, and current [migration guidance](../consumers/native-migration.md). | - -All required implementation, documentation, and verification work is complete. -No live-model quality claim, real credential, release, tag, push, new permanent -gate, or deferred backlog feature was needed or introduced. - -**Goal:** finish only when the full end state is supported by current evidence. - -**Inspect/change:** the assembled repository, feature roadmap acceptance matrix -and removal inventory, stage evidence, maintained examples, dependencies/notices, -and documentation links. Make final focused corrections where evidence exposes -an actual gap; do not invent another feature or compatibility layer. - -**Work:** verify every acceptance row against tests or direct inspection of its -owning boundary. Close compilation blockers recorded during intermediate stages. -Confirm all production LLM roles use the one engine/native path, domain artifacts -are unchanged, new standard commands/loaders share configuration, and no completed -decision remains deferred to a later CommandKit adoption. Confirm old adapters, -retry loops, global limits, schema registration, both legacy caches, required -reporting sidecars, and copied accounting have no current consumers. Review -dependency attribution and absence of permanent replacement modules. - -Run final development checks with declared dependencies and no sibling workspaces: - -```sh -GOWORK=off go test ./... -GOWORK=off go vet ./... -GOWORK=off go build ./cmd/notarius -git diff --check -``` - -Use a temporary binary destination if an ordinary build would leave an untracked -binary. Also run maintained example/preflight and representative native help/ -version/completion/format checks not already covered by the default suite. -Inspect removal matches in current Go/config/example files, rather than requiring -historical docs to contain none. Check local links with a small read-only checker -or manual verification if the repository has no link-check facility. - -**Acceptance:** final tests/vet/build pass; retained domain regressions and all -meaningful cross-boundary cases have coverage; examples and canonical docs agree -with the executable. Native timeout/cancellation, structural-failure status, -final-turn assessment, fallback disclosure/eligibility, cache hits, worker liveness, -and post-publication errors are verified. No live-model run, production key, -release publication, permanent new gate, or comprehensive performance benchmark -is required. Report any genuinely unverified result instead of declaring success. - -## Coverage Map - -| Feature-roadmap target | Primary stages | -| --- | --- | -| U1 dependency baseline and native handoff | 1–7, 35 | -| U2 embedded/native prompt and schema ownership | 4–5, 12–24, 35 | -| U3 typed modules and shared producer assessment | 8–12, 24 | -| U4 prompt pairs, guidance, complete revalidation | 9–23, 37 | -| U5 native retries and semantic-turn allowance | 3, 9–10, 13–24 | -| U6 strict/permissive outcomes and supported fallbacks | 3, 9–24, 27–28, 34, 37 | -| U7 workers and native backend admission | 3, 17, 25, 37 | -| U8 accepted-artifact and chunk-plan cache | 3, 11, 26–28, 37 | -| U9 native execution facts, artifact delivery, capture | 11, 29–31, 34, 37 | -| U10 complete native CLI and shared offline preflight | 5–7, 32–34, 36–37 | -| Successor decisions, canonical docs, migration, backlog | 38–45 | -| Entire acceptance matrix and removal inventory | 35, 37, 46 | - -No architectural decisions remain open for this implementation. Private API names, -debug filenames, and routine serialization choices stay within the selected -boundaries; they do not require another decision cycle or stronger contracts. diff --git a/docs/roadmap/promptkit.md b/docs/roadmap/promptkit.md deleted file mode 100644 index e40fa7c9..00000000 --- a/docs/roadmap/promptkit.md +++ /dev/null @@ -1,1514 +0,0 @@ -# Native PromptKit And CommandKit Migration - -## Purpose And Status - -Make Notarius fully native to PromptKit, ConfigKit, and CommandKit. Use upstream -configuration, execution, continuation, admission, inspection, commands, and -presentation directly. Retain the application work of producing, assessing, -publishing, and reusing typed artifacts. Historical duplication of library -behavior is not a requirement to preserve it. - -This is the feature roadmap for PromptKit, ConfigKit, CommandKit, -accepted-artifact caching, and artifact-first output. It owns the selected target, -decision status, removal inventory, acceptance criteria, and supporting evidence. -The earlier discussion documents are retired; Git history preserves the discussion. -The [implementation plan](implementation.md) owns the ordered, prompt-sized stages. -The native cutover, canonical documentation, and final whole-roadmap acceptance -audit are complete. The implementation plan records the owning evidence and -successful assembled checks; no live-model quality or release-publication claim -is implied. -Current behavior belongs in [Configuration](../config.md), [CLI Reference](../cli.md), -[Operations](../operations.md), and [native migration guidance](../consumers/native-migration.md). -Unrelated future work remains in the [separate backlog](future.md). - -Notarius predates many modern PromptKit capabilities: the library grew out of -mechanics shared by Notarius and related applications. Historical duplication -and slight behavioral differences do not express an intentional application -requirement. Evaluate remaining machinery against actual domain needs, and -consider upstream features when a reusable capability is missing. - -Complete one coordinated cutover for configuration, PromptKit resources, and the -CLI. Adopt native public conventions fully; preserving old CLI behavior is a -non-goal. Supply migration guidance and maintained examples instead of aliases, -parallel parsers, or compatibility wrappers for legacy behavior. CommandKit is -part of the main migration, not a later follow-up. Release preparation and -publication remain separate work. - -Adopting the kits together lets standard inspection and execution share the same -loaders and native facts. It avoids building temporary command and reporting -adapters while those boundaries are already being replaced. - -## Baseline And Dependency Target - -At Notarius revision `12eeb6369e40746b2c18c55397a289a1e1b5908d`, the application -used configuration version 5 and Go 1.27. Its dependencies included old-path -PromptKit v0.12.0 and ConfigKit v0.5.0, new-path ConfigKit v1.2.0, go-yaml -v4.0.0-rc.6, and two external PromptKit provider catalogs. - -The selected reviewed baseline is: - -| Module | Version | -| --- | --- | -| `go.maximumdirect.net/commandkit` | `v0.7.0` | -| `go.maximumdirect.net/ckconfig` | `v0.10.0` | -| `go.maximumdirect.net/ckprompt` | `v0.11.0` | -| `go.maximumdirect.net/configkit` | `v1.4.0` | -| `go.maximumdirect.net/promptkit` | `v0.24.0` | - -Keep Go's consumer minimum at 1.27.0. The reviewed set resolves stable -`go.yaml.in/yaml/v4` v4.0.0 and `github.com/santhosh-tekuri/jsonschema/v6` v6.0.3. -Published tags and dependency manifests were verified, and the modules resolved -together without local replacements in an isolated consumer module. Maintained -composed-CLI and offline-preparation examples and focused message-hash, -configuration-handoff, and schema tests passed without live provider calls. -Those checks established the upstream baseline; subsequent Notarius implementation -evidence and final acceptance are recorded in the plan. See the -[current integration boundary](../integrations/pkg-commandkit.md) and -[integration version guidance](https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/consumers/guide.md#module-selection) -and tagged [release notes][release], [migration guide][migration], -[format reference][formats], and [Go consumer guide][consumer]. - -U1 is resolved: PromptKit v0.24.0 supplies the prepared `MessageContentHash` -required by the cache, and ConfigKit v1.4.0 supplies typed result handoff and -focused inspection. Adopt these with the selected CommandKit integrations as -one baseline. The migration includes one application-owned load, -`promptkit`-rooted resources, native schema configuration, and the request API -changes described below; it is not just a dependency-version update. PromptKit's -v0.23.3 timeout and cancellation fixes remain included. Do not add a local -hash-policy fork or substitute `RenderedPromptHash` for message-content identity. -Neither a cross-backend global limit nor a comprehensive execution fingerprint -is required. Remove obsolete imports and catalog dependencies when unused; -avoid permanent `replace` directives and update [attribution](../oss/attribution.md) -under the [OSS policy](../policy/oss.md). - -Audita's reviewed v0.23.2 integration demonstrates native source assembly, -continuation, observations, and CommandKit wiring. Adopt those mechanisms -without copying its `RoleClient`, structured-output interface, or opaque exchange -wrapper. Its per-proposal repair branches and lack of checkpoints do not define -Notarius's artifact and recovery policies. - -## Target Architecture - -### Application Structure And Ownership - -| Part | Responsibility | -| --- | --- | -| Application composition | Load application and PromptKit settings together through ConfigKit; hand the completed result to one shared PromptKit engine for producers, validators, and continuation runs. | -| Artifact-family modules | Prepare domain inputs, select prompt IDs and response requirements, interpret responses, apply deterministic domain rules, and provide supported fallbacks. | -| Pipeline execution | Connect generation to configured validation and strict/permissive handling; manage ordered dependencies, typed handoffs, bounded workers, and stable result ordering. | -| PromptKit | Prepare and execute requests; maintain captured exchanges and conversations; enforce native retry, repair, turn, and admission budgets; report execution facts. | -| Application state and records | Publish artifacts, cache final accepted artifacts, provide a small completion summary, and capture optional debug evidence with pipeline context. | -| ConfigKit and CommandKit | ConfigKit owns configuration mechanics; CommandKit and ckconfig/ckprompt own standardized commands and presentation. Notarius supplies loaders, request builders, report content, and outcome mapping. | - -Use native PromptKit requests, results, exchanges, errors, and observers in -model-executing application code. Remove the private provider-neutral completion -API. Pure domain models, codecs, source-reference resolution, and deterministic -transformations need no PromptKit dependency. Application interfaces continue to -express artifact production, typed validation, output encoding, and state access. - -#### Module Execution And Assessment (U3 Resolved) - -Keep the existing typed stage boundaries and use native PromptKit values in -model-executing code. Place engine construction and shared loaders in focused -files in the `internal/cli` composition root. It constructs one native engine -for the run. Retire `internal/framework/llm` once its adapter, scheduler, asset -registry, and reporting responsibilities disappear; do not retain a package -solely to hold a small constructor or introduce another LLM service interface. - -Place one small shared producer helper alongside assessment in -`internal/framework/pipeline`. Connect modules to it through ordinary functions: - -1. The pipeline supplies an operation-scoped assessment callback capturing the - configured validator chain and acceptance criteria. It applies the selected - strict/permissive mode when no acceptable result is obtained. -2. The module prepares source material, references, and any normalization handle - mappings once. It builds the native initial request and supplies response - interpretation, correction-request, and supported fallback functions. -3. The helper receives those functions, the shared engine, and relevant - cache/debug collaborators. For cacheable work it prepares the initial request - and looks up the accepted-artifact cache; on a miss it executes the retained - request. Otherwise it uses ordinary native execution. -4. For each usable executed response, the helper invokes module interpretation and - pipeline assessment. Recognize native errors and failed validation results as - terminal failures before interpretation. When domain correction is appropriate, - it supplies the module's next request to native coordination. PromptKit owns - conversation state and turn iteration; the helper does not implement another - retry engine. -5. Return a typed application outcome. Cache only final accepted artifacts under - the initial request's key. A cache hit follows the existing direct-reuse policy - without model execution or renewed semantic assessment. - -Request builders return `promptkit.RunRequest`; interpreters consume native -results. Candidates, findings, and acceptance outcomes remain application-owned. -Keep operation-specific state in local variables or closures, without an -execution-plan language, replacement completion API, or copied conversation model. - -Use one pipeline assessment implementation for the initial candidate and every -correction, including the final permitted response. Do not repeat final validation -merely because the helper returns to the outer runner. Deterministic stages keep -their ordinary typed production/validation path without synthetic requests or -conversation machinery. LLM validators call the shared engine directly and return -domain findings; they do not automatically acquire a producer correction loop. - -Modules receive operation data and collaborators, not physical storage roots or -CommandKit reporters. Pure domain packages retain artifact models, codecs, -reference resolution, and deterministic rules. `internal/preflight` retains its -separate known-dependency checks. Exact argument grouping and helper signatures -can be refined against representative extraction and normalization during -implementation; ownership and placement are settled. U4 selects prompt roles and -guidance ownership below; U5 selects retry controls and the correction budget. -U6 selects strict-by-default handling and module-owned fallbacks below; -configuration precedence and fallback behavior are implemented. See -[Pipeline Internals](../internal/pipeline.md) and the plan's completed operation/ -module work and completed final acceptance evidence. - -Retain the fixed `input -> chunk -> extract -> merge -> normalize -> output` -shape, ordered steps and generated references, typed artifact-family boundaries, -and deterministic public ordering. Keep modules independent of physical output, -cache, and debug roots. Native adoption does not turn the pipeline into a general -workflow engine or redesign durable D&D artifacts. - -### Command Surface And Native Inputs - -Install the application `pipeline` family alongside ckconfig and ckprompt: - -| Command | Responsibility | -| --- | --- | -| `pipeline list` | Discover configured pipelines. | -| `pipeline show ` | Report resolved steps, lanes, modules, validators, and reference relationships. | -| `pipeline validate ` | Check composition and known prompt/profile dependencies offline. | -| `pipeline run --input-file PATH` | Execute the pipeline and publish artifacts. | -| `run --input-file PATH` | Native alias for `pipeline run`, with the same handler and inputs. | - -Use `commandkit.Alias("run", "pipeline run")`, not a forwarding command. Use -`notarius run` in quick starts and document both routes. Install the full selected -standard suite: native help/version; Bash, Zsh, and Fish completion; configuration -scopes, validation, show, sources, explain, and diff; PromptKit resource listings -and inspection for prompts, profiles, backends, namespaces, and collections; -requirements discovery/validation and prepared-request preview; and explicit -reachability, authentication, text, structured-output, and aggregate smoke tests. -Include upstream convenience aliases and useful completion hints. - -Use native typed declarations and supplied-value semantics. Preserve the -distinction between omission and explicit empty/false/zero values, then apply -each field's selected default or empty-value policy. Preserve native command-path, -operand, repeated-flag, and `--` parsing conventions. Check every occurrence of -a singleton flag and use the last successfully parsed value. - -| Concern | Selected input | -| --- | --- | -| Configuration | Native `--config`, `--overlay`, and supported application `--set POINTER=JSON`; scope named `app`. | -| Pipeline input | Required pipeline-ID operand and `--input-file PATH`. | -| Lanes | Repeatable `--lane ID`; omission selects the normal lane set. Commas are literal values, not separators. | -| Output | Application `--output-dir PATH`. | -| Cache | `--cache auto\|bypass\|refresh` overrides caching for the entire invocation, including chunk generation. Otherwise resolve module-binding `cache`, pipeline `cache`, then `auto`. | -| References | Repeatable `--reference SELECTOR=PATH` and `--without-reference SELECTOR`. | -| Presentation | Native `--format`, `--quiet`, `--verbose`, `--debug`, and `--progress`. | -| Capture destination | Application `--debug-dir PATH`, with the semantics below. | - -Keep the worker limit and semantic budgets in configuration, adjustable through -application `--set`. U5 selects `max_additional_turns`; U7 selects -`concurrency.workers` with a default of `64`. -Use ckprompt's reusable profile, session, and execution flags, including model, -temperature, timeout, reasoning effort, and transient retries, for run-wide -overrides across producers, validators, and continuations. Omission preserves -normal selection; explicitly -empty reasoning effort clears inheritance. Use native `--llm-profile` and -`--llm-session-id`, retiring the separate reasoning-clearing control. Native -transient retries remain distinct from semantic additional-turn budgets. -Individual prompt selection and named prompt inputs belong on standard PromptKit -commands; pipeline modules build their own requests. - -Notarius retains reference-selector expansion, specificity, required slots, and -external/generated-reference conflict checks. Pipeline reference selectors are -not individual-request `--llm-input-file` bindings. Remove custom dispatch, -assembled help, standard flag parsing, comma-separated `--only`, and bespoke -`--json` switches. Replace `config effective` with `config show` and `config -profile` with `promptkit show profile`. Old flags are migration evidence only. - -### Application Composition, Loaders, And Preflight - -One constructor registers application commands, the native alias, ckconfig's -`app` and `promptkit` scopes, ckprompt, and native help/version/completion/reporting. -It does not load configuration, build an engine, or initialize run storage. -Help, version, and completion work without application configuration. - -Share one application-owned configuration load and a separate engine-construction -step across application and standard commands: - -1. Select application configuration through explicit `--config`, then existing - environment/discovery policy. Embed `promptkit.Settings` under `yaml:"promptkit"` - in the application configuration and compose its ConfigKit definition with - `promptkit.Configure`. Assemble the native sources below before calling - `configkit.LoadLayered` once for that selection. Let ConfigKit apply overlays, - imports, inheritance, overrides, defaults, normalization, and structural - validation. Retain its native result and selected main-source context. Validate an - explicitly empty configuration path rather than falling through to discovery. -2. Pass the successful whole result to `promptkit.NewFromConfig`, with native - `RuntimeConfig` for runtime client/timeout/transient-retry options. Construction - captures the resolved resources without loading sources or running ConfigKit - callbacks again. Select `BundledOpenRouterUnderlay` before loading; this - constructor does not implicitly add the bundle. Across commands, `--config` - selects the application document. Request overrides do not mutate captured - engine configuration or the result's frozen inspection evidence. - -Use ConfigKit's native typed-value and subtree-inspection handoff. ckconfig's -application scope and ckprompt's focused `/promptkit` inspection share that same -result; use native adapters, including `FromConfigKitSubtree` where a focused -scope is needed. Keep native configuration paths, override capabilities, and -whole-load source/diagnostic evidence. A focused view is not a separately loaded -document. Remove separate PromptKit source loading, reparsing, and inspection -conversion rather than wrapping the new handoff in an application abstraction. - -Use small invocation-owned state through CommandKit's facilities to reuse loaded -results and constructed engines. A pipeline shares one engine across all roles. -`config diff` may load two distinct configurations within one invocation: keep -their selections separate, including overlays/overrides. No persistent loader -cache or configuration hash framework is needed. - -Reuse the application resolver for module/options checks, typed compatibility, -ordered dependencies, validator chains, and reference bindings. `pipeline list` -only enumerates configuration; `show` renders resolved composition rather than -building an engine to expand every profile. ConfigKit supplies loaded values and -native provenance/secret-aware inspection, not domain pipeline semantics. - -Centralize known prompt/profile dependency checks in `internal/preflight`. It -accepts the resolved pipeline, module declarations, and native engine. Check -every statically known dependency, including LLM validators and correction -prompts, plus applicable/default profiles. Partial profiles alone are not errors. -Share identifiers and selection logic with execution; no independent inventory, -asset registry, or synthetic request execution is needed. Dynamic selections -receive their final checks at runtime. Findings identify pipeline, step/lane, -stage/validator, and resource where applicable. - -`pipeline validate` and `run` call the same resolution/preflight functions; -handlers do not invoke other commands or parse their reports. Keep `config -validate` focused on configuration and remove its pipeline-specific mode. -Offline preflight reads configuration and prompt resources but needs no transcript, -reference contents, credential values, or provider calls. Concrete PromptKit -preparation/validation owns completed-request checks, including schemas and -credentials; provider smoke tests require explicit invocation. - -Preflight does not discover configuration, register modules, build engines, -initialize storage, or render CLI reports. Modules do not import it. Use native -request builders for standard commands and add application callbacks only where -needed, preserving supplied operator values. Output/cache/debug allocation -belongs to the run path, with debug timing selected below. - -### Configuration, Sources, And Prompt Assets - -Introduce application configuration version 6 as the only supported version. -Preserve unrelated discovery, pipeline structure, and resolution behavior. -ConfigKit continues to own loading and composition under -[ADR-0020](../adr/0020-delegate-configuration-mechanics-to-configkit.md). -Retire `promptkit.profile_file`, `profile_dir`, and `local_backend` in favor of -native companion sources; do not introduce a separate `promptkit.config_file`. -Retire `concurrency.total_llm` and revise retry/worker settings as described below. -U5 selects `max_additional_turns` and removes application structural-repair -controls; U7 selects the shared worker pool and retired global-limit migration. -U8 replaces the nested legacy cache settings with root-level `cache_directory` -and pipeline/module-binding `cache` modes. - -Use one source assembly in the shared ConfigKit load for runtime, preflight, -and inspection: - -1. PromptKit's bundled OpenRouter underlay. -2. Notarius's embedded native prompts and profiles under - `promptkit.collections.notarius`. -3. An optional, higher-priority operator underlay containing `prompts.yml` and - recursive `.yml`/`.yaml` files under `prompts/`, beside the main application - configuration actually selected by ConfigKit. - -These are ordered underlays of the single application load. The selected main -document and its imports, selected overlay, and explicit configuration overrides -participate through ConfigKit's native precedence. Do not introduce a second -PromptKit main document or apply overlays/overrides again during engine construction. - -Complete documents place native resources and `schema_root` beneath `promptkit`. -Use `SettingsInput` for wrapperless settings fragments and `CollectionInput` for -collection fragments; discovered fragments need an explicit destination because -discovery helpers return root-mounted inputs. Keep resource IDs and exact -references unchanged by the configuration path. Old root-level prompt/profile -lists are not consumed as PromptKit resources; do not translate them implicitly. - -Anchor companion discovery to that selected main source, whether explicit or -discovered. Configurations in the same directory share companions; there is no -working-directory fallback. The two companion locations are strict peers, so -filenames do not resolve conflicting operator records. Missing companions are -fine; loading errors do not silently fall back to embedded defaults. Preserve -native imports and source-relative resource paths. - -Native declarations become the sole prompt assembly mechanism. Keep the embedded -content tree and module ownership of prompt meaning, response requirements, and -domain interpretation. Remove prompt/fallback-profile registration in -`AssetRegistry`, per-module `PromptAssetManifest` plumbing, `ModulePromptFS`, -shared-name maps, synthetic `sharedassets` mounts, and manual prompt enumeration -for checkpoint fingerprints. The resolved schema design below also removes -schema registration and filesystem-flattening machinery. - -Convert maintained resources, examples, and fixtures to native identified lists, -`config:` source controls, and sibling inheritance (`extends: {id: parent}`). -Same-ID replacement supplies a complete record; distinct inherited variants can -customize an existing resource. Explain native profile-null/backend fallback -semantics. Notarius owns application version 6 through its ConfigKit definition; -PromptKit adds no resource-version rule. Retire the planned PromptKit version-2 -source header. Collection `release_version` is optional descriptive -metadata, not a selector or validation gate. Do not translate legacy resource -formats or synthesize compatibility aliases. - -Select prompts by ID alone; remove obsolete PromptKit version selectors and -related request/debug metadata. Retain current local resource IDs and use exact -qualified references, such as `notarius/dnd.spells` and -`notarius/dnd-extraction`. Keep the extraction profile's existing model/settings -and its reference to the bundled unqualified `openrouter` backend. Inherited -application profiles use sibling local IDs in the same collection. The bundled -inventory differs from the old external catalogs and selects no model for an -application request; document intentional replacements for removed catalog IDs. -The bundled backend supplies no concurrency limit. Under U7, Notarius injects no -backend concurrency or queue defaults; operators use native backend configuration. - -Preserve prompt prose, message ordering, roles, and cache boundaries where their -meaning carries forward. Review and deliberately revise initial/repair prompts -for accumulated conversation history; preserving every old byte or retry message -is not a migration requirement. Shared content should use native filesystem -contexts and imports rather than reconstructed virtual trees. - -#### Prompt Assets And Response Schemas (U2 Resolved) - -Use the existing root `assets` package's ordinary `embed.FS` directly for native -prompt sources and response schemas, with `fs.Sub` only where useful. Shared -message files use their actual paths within the native filesystem context. -Remove synthetic mounts, file copying, flattened schema namespaces, and module -asset manifests. Retire `readonlyfs` when its prompt-mounting and asset-flattening -consumers disappear; no replacement filesystem abstraction is needed. - -Declare response format, validation mode, and schema path in native prompt -definitions by default. Omit structural-repair attempt settings in bundled -prompts to inherit PromptKit's upstream default under U5. Modules select prompt -IDs, supply inputs, and interpret responses. U4 selects the same schema file for -each initial/correction pair. Execution, inspection, and prepared-request preview -then use the same declared contract without repeating schema selection in Go. -U4 selects prompt roles and contracts; exact prose is implementation work. -U5 selects budget ownership and defaults. - -Use a request-level native `*promptkit.OutputContractInput` only for a demonstrated -runtime-dependent contract. It replaces the whole configured contract, so supply -the intended format, validation mode, and schema together while leaving structural -repair at its native default by leaving `RepairAttempts` nil. `OutputContract` -remains the resolved type in native prepared details. Do not inject a repair count -merely to construct the override, or introduce a wrapper or per-call registry -solely to reproduce Audita's approach. Operator prompt replacements must still -produce responses the selected module can interpret; that module boundary does -not require duplicating each prompt's schema selection in Go. - -Declare the bundled collection's native `schema_root` in an embedded ConfigKit -source whose filesystem context points at the assets root. Bundled prompts use -`output.schema_path` relative to that root, for example -`dnd/spells/schemas/dnd_spells_llm.v1.json`. An operator can replace -`promptkit.collections.notarius.schema_root` with an external directory supplied -from the operator configuration's filesystem context. That replaces the whole -root for those schema-path lookups and must supply their selected schemas; -missing files do not fall back to the embedded root. Do not retain the removed -`WithSchemaFS`, `WithSchemaFile`, or `Config.SchemaDir` APIs or introduce an -application filesystem overlay. - -Support native prompt-owned inline `schema` and source-relative `schema_file` -selections as well, including operator prompts packaged with their own schemas. -Exactly one schema selection applies in JSON Schema mode. A nonnull collection -root takes precedence over `promptkit.schema_root`; an absent/null collection -root inherits it. Roots and file references retain the effective contributing -ConfigKit source's filesystem/base directory. There is no implicit working- -directory root. Request replacements use inline `Schema` or `SchemaPath` under -the selected prompt's applicable root; they do not inherit a configured -`schema_file` context. Native source composition owns these rules. - -Keep private response schemas as self-contained JSON documents. PromptKit owns -loading, compilation, structural validation, provider structured-output settings, -and structural repair. It loads and compiles schemas during preparation and -retains them in prepared execution. Remove Notarius's response-schema byte -registry, copied schema wrappers, defensive-copy layer, digest calculation, and -duplicate structural-validation machinery. Native prepared snapshots provide diagnostic -schema details; no schema fingerprint is added to the message-content cache key. - -Keep typed decoding, source/reference checks, reconciliation safety, semantic -validators, artifact codecs, and durable artifact schemas in Notarius. Response -schemas describe model output; artifact schemas describe published and consumed -domain artifacts. Preserve identifiers and versions belonging to actual artifact -contracts. Retire private schema keys, registration records, hashes, and copied -identity/version/name metadata whose only consumers were adapters, checkpoints, -or reporting. No independent schema metadata system replaces them. - -Native preparation is implemented and covered by asset/operator-schema tests -recorded in the plan. Final verification confirmed embedded and operator prompt -definitions, shared-file paths, schema selection from embedded and external -roots, source-relative/inline selection, and complete request-contract replacement -where actually used. Missing or invalid schemas follow native preparation -behavior. No new schema check requiring synthetic requests is added to offline -pipeline preflight. - -### Native Production, Continuation, And Assessment - -Start with one conversation per producer operation: a chunk-plan generation, -one lane's extraction of one chunk, or one lane's normalization pass. Use -complete replacement model responses. Normalization may continue to return a -reconciliation plan that deterministic code applies to produce the artifact. -Per-entity branching is not part of this migration. - -#### Prompt Roles And Correction Guidance (U4 Resolved) - -Give each correctable producer an initial native prompt and a domain-specific -correction prompt. Retain existing initial IDs and identify correction IDs -explicitly in the module, for example `notarius/dnd.spells.repair`. Preflight uses -those same declarations to check known prompt dependencies. Shared correction -wording can live in ordinary asset files; distinct domain/operation templates -supply the task and response expectations without a universal application-aware -correction protocol. - -The initial request supplies source material, references, and the task. Enduring -domain principles belong in its system instructions. The correction template -supplies a new user assignment with the current candidate's actionable findings; -PromptKit carries the preceding exchange. Do not rerender the original source or -manually replay prior responses. Review initial wording so later correction -assignments remain valid. Include additional application-derived context -only when needed to interpret the findings. Keep source/reference meanings and -normalization candidate handles fixed throughout the conversation. - -Use the same declared response schema for each initial/correction pair: - -| Role | Response contract | -| --- | --- | -| LLM chunk planning | Complete replacement chunk plan. | -| Extraction | Complete replacement extracted artifact candidate. | -| Normalization | Complete replacement reconciliation plan, applied afresh to the fixed original candidates. | -| LLM validation | The validator's assessment response; no separate semantic correction conversation by default. | - -No correction envelope, patch language, or special repair-result type is needed. -PromptKit retains structural validation and repair inside each native execution. -Notarius assesses the response returned by that public boundary, independently -of how many internal attempts produced it. - -Use the same native default profile for an initial/correction pair. Preserve -explicit application profile selections and run-wide execution overrides across -both requests. An operator may select a different default profile in a correction -prompt, subject to explicit request overrides. Validators retain their own -profile selection. Pass native selections and overrides without copying resolved -settings or adding a Notarius-wide repair-model configuration layer. - -The component that detects a defect owns its actionable correction guidance: - -- Each rejecting validator supplies guidance describing the defect and the - constraints a corrected answer must satisfy. A deterministic validator authors - it alongside the check; an LLM validator requests an actionable explanation in - its prompt and returns it through the application validation contract. -- Preserve the explicit `CorrectionGuidance` boundary separately from operator - diagnostics. Guidance need not prescribe an exact replacement, and validators - do not construct the complete correction prompt. -- Module interpretation and normalization safety checks supply their own guidance - when defects arise before an artifact candidate can be materialized. -- The pipeline collects guidance in configured validator order. The module's - domain-specific template renders it into the correction assignment. Neither - the pipeline helper nor the template duplicates validators' defect-detection - logic. Reason codes and operator messages are not automatically instructions. - -Assess the same candidate through every applicable configured validator before -requesting correction. A rejection does not short-circuit later validators; -collect all actionable findings they return. Reassess every revised candidate -through the complete applicable chain, including earlier approvals and the final -permitted response. The existing validator-owned guidance and ordered-chain -semantics remain useful; replace the hard-coded `CorrectionRequest` message -builder with native template rendering and retain useful ordered aggregation. - -Applicability requires usable input: if interpretation cannot construct a typed -candidate, its module supplies guidance and artifact validators cannot yet run. -Cancellation or fatal execution failures can interrupt assessment. An unavailable -validator assessment is a separate outcome, not a detected defect or an instruction -to change the artifact. Do not convert provider errors into correction guidance; -apply U6's strict/permissive handling when no acceptable result is obtained. - -Ownership, response contracts, and default profile behavior are resolved. Refine -exact template prose, correction IDs, and necessary operation-specific inputs -within implemented module boundaries. The plan records producer/prompt/assessment -coverage and successful final verification. U5 selects semantic budgets and U6 -selects strict-by-default terminal handling and fallbacks. - -#### Execution And Assessment Flow - -The application flow is: - -1. Prepare source context, references, and candidate-handle mappings once. Build - the native initial request. For cacheable work, prepare and check the cache - before execution as described below; execute the retained request on a miss. - Otherwise use ordinary `Engine.Run`. Capture an exchange when continuation - or debug inspection needs one. -2. Check the native error and public validation status before treating a response - as usable. PromptKit may return `ValidationFailed` with a nil Go error after - exhausting structural repair; that is a terminal failure, not a candidate for - application structural repair or cache storage. Decode a usable response and - apply module interpretation and safety checks. A module can supply actionable - findings before it has a materialized candidate. Assess each candidate through - all applicable configured validators and collect their actionable guidance - before requesting correction, as selected in U4. -3. For actionable correction, call `Engine.Coordinate` with the captured exchange - and the resolved `max_additional_turns` as native `AdditionalTurns`. Its callback - selects a repair prompt and supplies findings; PromptKit retains the initial - exchange and subsequent turns. -4. Apply the same native-result check, interpretation, and assessment to every - delivered revised response, rebuilding from the fixed original inputs. Run - the complete applicable validation chain even when a revision produces an - unchanged candidate, including validators that previously approved. Assess - the final permitted response before declaring exhaustion. -5. Stop on acceptance or a terminal condition. If no acceptable result is obtained, - fail in strict mode or produce the module's supported fallback in permissive - mode. Native `stopped` and `exhausted` outcomes do not themselves mean application - acceptance. Cancellation stops work without constructing a fallback. - -Accumulated history increases context use; U5's allowance bounds additional -semantic correction while stopping as soon as the candidate is accepted. - -Remove `single_response_v1`, raw-response `ModelCandidate`/`SemanticCorrection` -transport, manual assistant/user replay, and correction-protocol capability -checks. Replace `runProducerAttempts`' LLM retry loop with native coordination and -application assessment. Adapt `NormalizeRetry` and `producerRetryDirective` to -findings and explicit fallback information. Retain useful ordered collection and -deduplication of actionable findings, rendered through a native repair prompt. -Reason codes and operator messages are not automatically model instructions. - -#### Retry Controls And Correction Budget (U5 Resolved) - -Delegate transport retries and structural repair to PromptKit. Notarius selects -only the allowance for application-directed semantic correction: - -| Concern | Control and selected default | -| --- | --- | -| Temporary provider failures | Native PromptKit transient retries; inherit the upstream default when omitted and preserve native execution overrides. | -| Structurally invalid model output | Native output-contract policy; bundled prompts and application requests inherit PromptKit's upstream structural-repair default. | -| Actionable domain defects | Application `max_additional_turns`, passed directly to native `AdditionalTurns`; application default `2`. | - -`max_additional_turns` is a nonnegative maximum, excluding the initial producer -response. Resolve producer-binding override, then pipeline override, then the -application default of `2`. Omission inherits; explicit `0` disables semantic -correction. It applies to LLM-backed producers. Validators do not receive a -separate semantic continuation budget, and deterministic stages do not acquire -conversation machinery. Keep the field in configuration, adjustable through the -existing application `--set` mechanism, without another dedicated CLI flag. - -For example, a producer binding can override its pipeline allowance: - -```yaml -extract: - module: dnd/spells - max_additional_turns: 2 -``` - -Two additional turns are an allowance, not mandatory executions. An accepted -initial candidate needs none; an accepted first correction needs only one. A -second turn is available when actionable defects remain and continuation is -appropriate. Assess every delivered response, including the final permitted turn, -and stop on acceptance or a terminal condition. PromptKit enforces the turn -limit; do not add an application counter or retry loop. Internal transport -attempts and structural repairs do not consume this semantic-turn allowance. -The default is an application policy choice, not a measured claim about how -frequently corrections will be needed. - -For structural repair, omit `output.repair_attempts` from all bundled initial, -correction, and validator definitions. Remove Notarius's pipeline/binding/ -validator `structured_output_repair_attempts` fields and resolution plumbing. -Do not replace them with a hard-coded default, including zero, in request -construction or a native output-contract override. Explicit operator -customization remains available through native PromptKit prompt configuration. - -PromptKit v0.24.0 defaults omitted/null budgets to one structural repair for -`basic`, `json`, and `json_schema`, and zero for `none`. These are upstream -defaults, not Notarius-pinned counts; Notarius deliberately inherits future -changes. Nil request budgets inherit the native default, while explicit zero -disables repair. Notarius still declares response format, -validation mode, and schema; PromptKit owns the mechanics of obtaining a valid -response. At the public boundary, handle a usable result or terminal failure, -including a returned result reporting `ValidationFailed` with a nil Go error. -Apply that boundary to producers, continuations, and LLM validators; consume the -native terminal status without inspecting or reproducing internal attempts. -Do not restart the operation to extend PromptKit's exhausted repair budget. - -Remove blanket outer LLM retries, including repeated LLM validator invocations in -`executeValidationChain`. Keep ordered candidate assessment and distinguish a -validator execution failure from a domain rejection. Use native `OperationTimeout` -when that operation scope is needed, without adding an arbitrary default or -replacing broader application deadlines. Cancellation stops work. - -Remove generic retries for non-LLM operations as well. Parsing, chunk -materialization, deterministic merging, normalization-plan application, and -encoding run without an outer attempt loop. Repeating those operations against -unchanged inputs ordinarily cannot correct a defect. Return operation errors -through normal pipeline failure handling; apply the selected terminal policy to -a rejected deterministic candidate instead of regenerating the identical value. -Cancellation stops work. - -A demonstrated transient I/O recovery need belongs with that specific operation, -where repeatability and failure conditions are understood. Do not retain a generic -stage retry setting or framework in anticipation of filesystem or future remote- -storage failures. After a fix, an operator can rerun the pipeline using the -accepted-artifact cache for successful predecessors while cheap deterministic -preparation runs again. - -Retire producer and validator `retries` fields entirely, together with the generic -outer attempt loop. Migration guidance maps intent: a producer allowance intended -for semantic correction becomes `max_additional_turns`; native transport controls -and prompt-owned structural customization cover their own concerns. Remove -non-LLM retry settings without a replacement generic allowance. Do not promise -equivalent total provider-attempt counts. U5 is resolved; U6 selects strict-by-default -handling and module-owned fallbacks. Native producer/assessment integration and -removal of outer retries are implemented; see the plan's completion evidence. - -#### Terminal Outcomes And Module Fallbacks (U6 Resolved) - -Use two modes when an operation does not obtain a successful, validated result: - -| Mode | Terminal behavior | -| --- | --- | -| Strict | Require a successful, validated result; otherwise fail the pipeline. | -| Permissive | Produce the module's supported fallback, record a warning, and continue with that artifact. | - -The pipeline selects the mode; the module owns the fallback algorithm. Replace -the failure-category policy object and separate `fail_run`, `reject_output`, and -`warn_continue` choices with this selection. An empty artifact is a supported -fallback where the domain permits it; no separate omit-output action is needed. -The application default is strict for every role, including normalization. -Modules do not declare their own defaults. Permissive handling requires an -explicit configuration selection; normalization has no automatic exception. - -Use `mode: strict` or `mode: permissive` at pipeline level, with an optional -`mode` override on each configured producer module binding. Resolve the module -binding first, then the pipeline setting, then the application default `strict`. -Omission inherits; either explicit value overrides the broader setting. A strict -pipeline can therefore permit fallback for one module, and a permissive pipeline -can require strict handling for an essential module. This is binding configuration, -not a default declared by the module implementation. Validators report findings; -the producer's resolved mode governs disposition. Use ordinary configuration and -the existing `--set` mechanism, without a dedicated mode flag. - -Attempt application-directed correction when appropriate within the selected -budget before terminal handling. Neither mode extends PromptKit's native retry -or repair budgets or adds retries to deterministic operations. Preserve the -distinction between a detected defect and an unavailable assessment for correction -guidance and diagnostics; neither establishes ordinary acceptance. Cancellation -remains cancellation and does not initiate fallback work. If a supported fallback -cannot be constructed, fail execution. - -Fallbacks must satisfy the structural artifact contract and invariants required -by downstream stages. They need not satisfy every ordinary semantic acceptance -criterion whose failure led to fallback. Do not rerun the complete ordinary -validator chain as a condition of using the fallback or claim normal validation -passed. Keep the necessary checks with module fallback construction; no separate -configurable fallback-validation framework is needed. - -Use these domain-specific fallbacks: - -| Module | Fallback direction | -| --- | --- | -| D&D spell extraction | A valid empty spell artifact, so that the failed chunk contributes no extracted spells. | -| D&D transcript chunking | One chunk containing the entire transcript, preserving source order and references. This is the default fallback; no deterministic slicing machinery or sizing policy is needed for this migration. | -| Normalization | Apply only safe consolidation groups from the final usable response afresh to the fixed originals, retaining everything else. If that response is unusable, use the deterministic merge of the input chunk artifacts. Reuse an already available merged input. | - -Keep source grounding, reference resolution, candidate isolation, and -reconciliation safety checks. Unknown or overlapping handles remain invalid; -partial normalization applies only compatible safe groups and leaves other -original candidates separate. Use the final response and fixed original inputs, -without recovering an earlier candidate or combining proposals across turns. -Concrete module algorithms must make their supported fallback and required -invariants explicit. Refine module-specific construction without introducing a -generic fallback language. Whole-transcript chunking deliberately favors simple -machinery for a rarely expected fallback; it does not promise bounded downstream -request size. - -Fallbacks may be published and consumed downstream, but they and dependent results -affected by their incompleteness are ineligible for accepted-artifact cache lookup -or storage. Carry the small application outcome facts needed to skip that reuse; -do not add dependency fingerprints or a provenance ledger. Fully validated, -unaffected predecessors remain reusable, allowing later runs to retry work that -previously needed fallback. - -Record why fallback was used and expose that qualification in the ordinary -completion report, even when diagnostic verbosity hides individual warnings. -Use the same application outcome for reporting and optional debug detail. Keep -conversation state in process, outside ordinary artifact/cache payloads; -explicitly requested detailed debug is separate. U6 and module-specific -construction are implemented; the plan records terminal-mode and domain coverage. - -### Admission And Pipeline Workers (U7 Resolved) - -Remove `Scheduler`, `scheduledClient`, and custom global queue/permit plumbing. -Retire the separate cross-backend ceiling and the default/maximum extraction -worker coupling to `concurrency.total_llm`. Use one shared pool for ready pipeline -jobs, configured by the positive integer `concurrency.workers`, default `64`. -Omission uses the default; explicit `1` provides serial job execution. Keep this -in application configuration, adjustable through `--set`, without a dedicated -CLI flag or separate extraction/lane worker settings. - -The pool serves both kinds of work: - -| Job | Work covered by one worker slot | -| --- | --- | -| Extraction | One chunk-and-lane extraction, including its validation and semantic correction. | -| Lane completion | A ready lane's merge and normalization, including their validation and correction. | - -The limit counts total active jobs across both kinds, with no separate reserved -capacity. The existing pipeline coordinator tracks dependencies and makes lane -completion ready after that lane's extraction finishes. A worker never holds a -slot while waiting for prerequisite jobs. Validators and correction turns execute -within the current job instead of acquiring another worker slot. Preserve stable -public result ordering regardless of completion order. Input preparation, chunking, -and final publication retain their sequential placement around lane execution. -This remains execution of the fixed pipeline, not a general workflow scheduler. - -Count application jobs, not OS threads or all process goroutines. Do not derive -the default from CPU count or `GOMAXPROCS`; model-backed jobs often wait for I/O -or native admission. The pool bounds active application state and preparation as -well as model-backed work. `64` supplies practical headroom for backend utilization -while remaining well below the reviewed native default queue capacity; it is not -a benchmark-derived optimum or a guarantee of backend saturation. Operators may -tune it for workload and host resources independently of backend settings. - -Use one shared native engine across all roles. Native capacity belongs to the -engine and backend ID: profiles selecting the same ID share its pool; different -backend IDs or separate engines do not automatically share limits, even for the -same endpoint. Endpoint-only profiles and backends without a positive limit -remain unrestricted by native backend admission. - -Inject no Notarius-owned backend concurrency or queue defaults. Preserve native -operator configuration and inherit upstream defaults rather than deriving backend -settings from the worker limit. In the reviewed PromptKit baseline, omitted/zero -backend concurrency is unlimited and has no queue; with positive concurrency, -omitted queue capacity defaults to `1024`. Explicit queue capacity, including zero, -uses native semantics. These are upstream defaults, not values pinned by Notarius. -With an unlimited backend, active jobs can generate concurrent calls up to the -application's workload; the worker default does not imply a separate LLM ceiling. - -For limited backends, the reviewed baseline bounds active model attempts by -`concurrency_limit` and admitted operations by that limit plus `queue_capacity`. -Ordinary execution admission spans preparation, retries, and repairs; split -`PrepareExecution` itself reserves no capacity, with admission occurring during -`RunPrepared`. Active permits are released during retry waits. -A completed continuation turn releases admission before assessment, allowing a -nested validator to use the same backend. Configure native limits at the intended -capacity boundary. Worker counts need not be less than backend concurrency; -native admission can queue waiting operations. - -Handle `ErrCapacityExceeded` through U6's existing strict/permissive policy: - -- A producer admission failure fails in strict mode or uses the module's - supported fallback in permissive mode. -- A validator admission failure means assessment is unavailable; apply the - producing module's resolved mode. Do not turn it into a semantic rejection - or a correction instruction. -- Cancellation stops work and does not initiate fallback. - -Preserve the error and backend identity in diagnostics. Add no application retry, -backoff, or second admission queue. A full native queue rejects admission without -calling the model; the large default queue makes this unusual with modest worker -counts, but explicit operator limits remain authoritative. Use ordinary fallback -reporting and cache exclusions when permissive handling continues. - -Aggregate concurrency across independent backends may exceed the former global -ceiling. That change is intentional. Bounded pipeline workers are application -work controls, not a substitute cross-backend LLM ceiling. - -Retire `concurrency.stage_workers.extract` along with `concurrency.total_llm`. -Migration guidance selects `concurrency.workers` independently and configures -each backend natively. Existing explicit extraction-worker values can inform the -new shared limit, but its scope now includes lane completion. Do not automatically -translate a cross-backend ceiling into per-backend limits or preserve worker -inheritance/caps from that field. Provide a native limited-backend example with -usable queue capacity, without requiring operators to override the upstream -queue default. U7 is implemented with focused worker/admission and assembled -coverage and final acceptance recorded in the plan. - -### Accepted-Artifact Cache (U8 Resolved) - -Replace both the checkpoint subsystem and the separate source-keyed chunk-plan -store with one cache around the complete Notarius producer operation. Its value -is an accepted answer to the original task, including an accepted chunk plan for -LLM-backed chunk generation. This avoids repeating expensive model work after a -downstream failure without caching every cheap deterministic pipeline step. - -```text -Prepare initial request -> obtain MessageContentHash -> look up /.json - Hit: decode the accepted artifact and continue the pipeline - Miss: execute the retained request and complete the Notarius operation - -> interpret, validate, and perform application-directed corrections - -> on final acceptance, store the eligible artifact under the INITIAL hash -``` - -Use the upstream message-content hash of ordered message roles and exact text. -Include message boundaries and order, excluding model/settings, session/execution IDs, -timing, provider cache hints, and other non-content metadata. Add no stage, -prompt-ID, library-version, whole-configuration, schema-policy, or implementation -fingerprint. Actual input or schema text rendered into messages participates as -message content; metadata outside those messages does not. - -| Change | Intended reuse behavior | -| --- | --- | -| Fix a downstream stage | Earlier artifacts remain reusable when their initial messages match. | -| Change initial message text, roles, order, or boundaries | Cache miss. | -| Change a prompt ID, model, or settings without changing messages | Cache hit remains possible. | -| Change deterministic interpretation or validation | Existing accepted artifacts remain reusable; explicit bypass/refresh applies the new policy. | - -Call `PrepareExecution` before model execution and obtain the key from the returned -handle's `Details().MessageContentHash`. Treat this native value as -opaque and filename-safe. Preserve the prepared snapshot for execution on a miss -using `RunPrepared`, and discard the handle on a hit. `OnPrepared` execution hooks -alone cannot short-circuit execution. `RenderedPromptHash` includes session -identity and cache hints; it is not the cache key. U1's reviewed release supplies -the required API, with no local hash implementation or extra key dimensions. - -Keep the initial key throughout corrections. Cache the final Notarius artifact -through its existing codec, not an intermediate response, `RunResult`, exchange, -or prepared handle. For normalization, save the accepted normalized artifact -after applying the reconciliation plan, not the proposed plan itself. Save only -after the entire Notarius operation succeeds, including domain validation and -any application-directed corrections. Failed, rejected, and validation-incomplete -work is not cacheable. U6's fallbacks and dependent results affected by their -incompleteness are ineligible for both lookup and storage, even if the downstream -operation itself validates successfully. An unrelated accepted predecessor remains -reusable. This requires a small eligibility fact, not a dependency fingerprint graph. - -A hit loads the accepted artifact directly, with ordinary decoding/usability -checks; absent or unusable entries are misses. It makes no model calls, rebuilds -no result/exchange, replays no corrections, and performs no new semantic -validation merely because the artifact was cached. It makes no claim of acceptance -under changed validation policy and contributes no historical token usage to -current execution totals. - -For an LLM chunker, cache the final accepted plan after materialization, configured -validation, and any application-directed corrections succeed. On a hit, decode -the plan and deterministically materialize chunks from the current source. -Retain ordinary structural and source-reference usability checks; an unusable -plan is a miss. Do not rerun semantic validators merely because the plan was -cached. The plan remains a domain result, without the old provenance envelope or -a cached copy of the full transcript. Deterministic chunkers execute normally; -do not manufacture a PromptKit request or a second hash policy to cache them. - -Retire the guarantee of one canonical partition per source regardless of the -configured chunking request. Matching initial messages reuse the same accepted -plan; changed prompt/reference/input text can produce a miss and a different -partition. Model/settings-only changes can still hit under the shared hash -policy. Source identity alone no longer selects a plan. This deliberately -supersedes the policy in [ADR-0005](../adr/0005-cache-canonical-chunk-plans-by-source.md) -as the migration lands; preserve the accepted ADR and record a successor decision. -Chunk-plan semantics and deterministic materialization remain application-owned; -their storage and reuse use the same mechanism as other accepted results. - -PromptKit owns everything inside each native request, including structural -validation, transport retries, and repairs. Notarius consumes its public response -or terminal failure and owns domain acceptance. Internal PromptKit attempts do -not determine cache eligibility. Native diagnostic evidence does not transfer -that ownership. No cached-response injection, replay API, exchange restoration, -or PromptKit-owned write-through cache is requested. - -#### Cache Configuration And Reuse Controls - -Use root-level `cache_directory` for the shared cache root. Omission or an empty -value selects `os.UserCacheDir()/notarius`: normally `~/.cache/notarius` on Linux, -respecting `XDG_CACHE_HOME`. An explicit directory is the exact root, independent -of output/debug roots and run IDs. Document `/var/cache/notarius` as the convention -for an explicitly configured system-wide installation. Directory selection does -not enable or disable caching; modes govern cache I/O. - -Use `cache: auto|bypass|refresh` at pipeline level, with optional module-binding -overrides. Resolve an explicitly supplied `--cache` first, then module-binding -`cache`, then pipeline `cache`, then default `auto`. Omission inherits. An explicit -CLI value governs all cacheable work in the invocation, including chunk generation; -without it, module overrides take precedence over the pipeline setting. Ordinary -configuration and `--set` can select more specific bindings. No independent -recording switch, separate chunk-cache control, or per-step recomputation selector -is needed. - -| Mode | Read existing entries | Store newly accepted results | -| --- | --- | --- | -| `auto` (default) | Yes | Yes | -| `bypass` | No | No | -| `refresh` | No | Yes | - -Modes never override the accepted-result eligibility rules. `refresh` replaces an -entry only after a newly accepted result is obtained; failure or fallback leaves -an existing entry untouched. `bypass` performs no cache I/O or directory creation. -Rerunning with default `auto` reuses successful predecessors and executes previously -unsuccessful work without a separate resume request. - -Use a small filesystem lookup/write facility and ordinary temporary-file/rename -publication. `/.json` contains the accepted artifact or -chunk plan, with no required manifest, acceptance ledger, historical provenance, -or versioned checkpoint envelope. Lookup does not require original output or -debug directories. - -Missing entries are ordinary misses. Unreadable, malformed, or unusable entries -are misses with useful diagnostics. Cache write failures warn and preserve the -successful application result; do not rerun model work to compensate for a failed -cache write. Cache I/O failures do not fail otherwise successful pipeline work. -Invalid configuration still follows ordinary configuration-error handling. -No expiration policy, garbage collector, or cache migration command is required. - -#### Migration From Both Legacy Caches - -Start the unified cache empty and populate it through ordinary execution. Ignore -legacy checkpoint and source-keyed chunk-plan records; do not import, reinterpret, -rewrite, or automatically delete them, even when their directories are beneath -the new root. They do not supply the new message-content keys. Operators may -remove obsolete directories when convenient. - -Retire `cache.checkpoints` and `cache.chunk_plans`, including their separate -directories and checkpoint recording switch. Remove `--resume`, -`--recompute-step`, and the separate chunk-cache flag, including the previously -planned `--chunk-cache`. Release guidance maps those intentions to root-level -`cache_directory`, pipeline/module `cache`, and invocation-wide `--cache`. -Explain automatic reuse, first-run population, and the move from a canonical -source partition to request-content-based plan reuse. - -Remove checkpoint identities, dependency fingerprints, invocation-wide barriers, -recovery bookkeeping, and the separate canonical-plan lookup/publication policy. -Retain domain plan interpretation and materialization without retaining a second -cache mechanism. U8 is implemented using U1's selected upstream hash capability; -store, producer/plan, and assembled recovery coverage is recorded in the plan. - -### Inspection And Execution Bookkeeping - -Delegate configuration/resource inspection, requirements, concrete preview and -validation, and explicit smoke diagnostics to ckconfig/ckprompt through the shared -loaders. Remove effective-YAML parsing, the profile-inspector wrapper, and copied -effective-parameter types. A configured profile may be partial; use native -prepared/result details for the actual execution target, including overrides. -Standard commands retain their native reports, support/applicability distinctions, -and failure behavior. Smoke status zero alone does not prove a check passed. - -For ordinary `Run` and coordinated turns, use `OnPrepared` to retain actual -request details for diagnostics. For cacheable operations, retain the already -available `PreparedExecution.Details()` before discarding or executing the handle. -That snapshot is available on a cache hit even though no execution or `OnPrepared` -callback occurs. Debug capture must not trigger another preparation or model call. -Native observations provide terminal facts even without a returned `RunResult`. -`OnPrepared` is absent on preparation/admission failures and is not a completion -record. Keep synchronous callbacks lightweight, compose capture/accounting -observers, and synchronize shared state where operations overlap. `OnPrepared` -runs while admission is held; persist retained evidence at normal operation -boundaries. - -Use native `RunID`/`ParentRunID` with application step/lane/chunk/validator scope. -Remove `LLMExecutionSummary`, duplicate usage types, conversions, cloning, -independent accumulators, debug completion wrappers, synthetic call IDs, and -adapter profile-recording/forwarding. Native snapshots serve actual-target -inspection; do not reconstruct a durable profile inventory. - -Use one native `ExecutionGroup` per invocation for diagnostic totals. Explicitly -enroll every actual initial, continuation, and nested LLM validator execution once, -with application step/lane/chunk/validator context attached to observations. -Use native aggregation directly, without application-maintained totals or a -hierarchy of overlapping groups. Preserve terminal coverage and unknown-versus-zero -usage. Cache hits create no execution and add no historical usage to current totals. -Concrete observer composition, enrollment plumbing, and context attachment remain -implementation choices within this contract. These facts support diagnostics and -capture, not a mandatory consumer execution ledger. Notarius's application outcome -supplies publication and fallback qualifications for the completion summary; -assessment, failure, and reuse details support optional diagnostics and debug -capture without independently maintained reporting state. - -### Artifact Delivery And Completion Output - -Artifacts are the primary deliverable: publish accepted domain artifacts and -permitted module fallbacks as files in the output directory. Preserve intrinsic -domain content and artifact schemas. -CLI `--format` controls the completion report, not payload encoding. Consumers -need no run manifest, metadata bundle, persisted receipt, or execution history to -use the artifacts. Retire reporting-only metadata and mandatory manifests instead -of rebuilding them with native values. - -Preserve the existing run-specific destination and domain artifact paths: - -```text -// - lanes/.json - chunk-map.json # when explicitly enabled and available - evidence-context.json # when explicitly enabled and available -``` - -Keep the existing lane filename mapping and intrinsic payload schemas. Remove -the mandatory `index.json`, `manifest.json`, and reporting-only `rejected.json`, -`warnings.json`, and `diagnostics.json` sidecars. Their relevant diagnostic content -belongs in ordinary diagnostics and optional debug capture; retain intrinsic domain -content in the artifacts themselves. `output_directory` identifies the absolute -run-specific directory, not the configured output root. - -Consumers check process status, optionally decode the small completion report, and -read documented artifact paths beneath the published directory. For configurable -pipelines, the pipeline configuration identifies expected lanes. Migration guidance -replaces receipt/index traversal with those paths, without introducing a replacement -discovery manifest or a payload-schema change. - -Submit one small structured value through `Reporter.Report`: - -| Field | Meaning | -| --- | --- | -| `pipeline` | Selected pipeline identifier. | -| `artifact_count` | Deliverable artifacts, including reused artifacts; excludes intermediates, cache entries, and debug files. | -| `output_directory` | Absolute published artifact directory. | -| `debug_directory` | Optional absolute completed run-specific capture directory; omitted when capture is disabled. | -| `fallback_count` | Number of producer operations that emitted a fallback; omitted when zero. | - -Supply a native text template for the ordinary successful case: - -```text -Success: dnd-session produced 6 artifacts in /work/results/run-123 -``` - -When capture is available, append `Debug capture: `. Provide the -path rather than mirroring the bundle. JSON/YAML render the same structured -value through native serializers. A plain string would render natively as text -but lose named fields in machine output; the template gives both presentations -without parallel encoders. No report-version, run-ID, token, profile, warning-count, -or validation-history collection is required. - -When a completed run used a fallback, qualify the human summary and structured -value with that fact, even if native diagnostic verbosity suppresses individual -warnings. Count each producer operation that actually emitted a fallback once; -do not count rejected attempts, individual validator failures, or downstream -artifacts affected by that fallback. JSON/YAML include `fallback_count` only when -positive. The native text template can render: - -```text -Success with fallback: dnd-session produced 6 artifacts in /work/results/run-123 -2 operations used fallback. -``` - -Do not present the run as if all operations obtained normally validated results. -Detailed reasons belong in optional diagnostics/debug capture, without per-producer -histories in the completion report. -Quiet mode continues to suppress stdout. - -Wire results to stdout and diagnostics to stderr: - -| Mode | Stdout | Stderr | -| --- | --- | --- | -| Ordinary | Brief completion report. | Failure explanation when needed. | -| `--verbose` | Same completion report. | Native information, warnings, and failures. | -| `--quiet` | Nothing. | Failure explanation when needed. | -| `--debug` | Completion report with capture path when enabled. | Native diagnostics and default progress. | - -Use native mode ordering (last enabled mode wins), explicit progress selection, -and quiet suppression. Quiet still permits failure explanations and does not -prevent rendering errors from failing the invocation. `--format json --verbose` -keeps stdout valid report JSON. Standard configuration/PromptKit commands retain -their own native values, with no universal Notarius envelope. Artifact-quality -advisories retain their domain classification. - -Optional saved reports use shell redirection, without a report-file flag or -automatic duplicate persistence: - -```sh -notarius run dnd-session --input-file transcript.json --format json > run-report.json -``` - -Publish artifacts and finish requested capture before submitting a successful -completion report. Use native outcomes: status 0 for command completion, 1 for -operational/cancellation/publication/presentation failure, 2 for invalid invocation -or identifiable invalid configuration, and 3 for native `config diff --check` -differences. Preserve ckconfig's structured classification and underlying causes; -not every preflight failure is invalid input. A permissive run that completes with -supported fallbacks succeeds with an explicit fallback qualification. Strict-mode -failure, inability to construct a supported fallback, and other terminal pipeline -failures fail the command. - -On failure, supply a useful native explanation and nonzero status, without a new -structured error-envelope protocol. `--format` does not serialize diagnostics. -Retain CommandKit's separation of operation causes from output/diagnostic delivery -failures. If publication succeeded before capture/report delivery failed, identify -the published path and available debug location. Published artifacts remain valid; -do not rerun model work or rewrite execution history. Subprocess callers check -status before decoding a successful report; failed writes may leave partial stdout. - -### Debug Capture And Optional Diagnostics - -Capture the **most convenient** debug representation, rather than optimizing -for conciseness, exhaustive completeness, or strict deduplication. Native snapshots -and readable views may repeat evidence. Reuse application decisions and native -execution facts instead of maintaining a second ledger or transcript model. -Debug formats are diagnostic data, not stable consumer contracts. - -#### Controls And Location - -Bind an application boolean to native `--debug`. Capture requires that boolean -and a nonempty effective directory; a directory alone does not enable capture. -Select `--debug-dir PATH` over `debug.directory` over `/tmp/notarius-debug`. -Resolve nonempty configuration-relative paths against their source and CLI-relative -paths against the invocation directory. Recognize explicit empty values first: -`--debug-dir ""` or `debug.directory: ""` disables capture without fallback. -A nonempty CLI override can re-enable capture over an empty configured root. -An unusable nonempty destination fails requested capture rather than selecting -another directory. Remove the configuration-relative `./notarius-debug` default. - -Capture selection is independent of the final reporting mode. `--debug --quiet` -captures while suppressing stdout; `--quiet --debug` selects debug verbosity and -capture. `--debug=false` disables capture through native binding semantics. -Empty-directory debug retains native verbosity. No separate capture flag or -capture-depth switch is needed. - -#### Lifecycle And Failure Handling - -Allocate the run bundle after application configuration and run identity are -available, before engine construction and pipeline preflight. Argument/config-load -failures may have no bundle. Keep allocation out of shared loaders. - -Create `//` and update `/latest` to a relative -symlink to that run at session start. It points to the newly started session even -if that session fails. Do not retarget at completion; concurrent starts use the -last successful link update. Runs without capture leave it untouched. - -One application recorder accepts initial context, native evidence, and application -decisions, then finalizes once with pipeline/publication/capture outcomes. Persist -as operations advance, without holding the whole pipeline's content in memory. -Keep debug separate from output/cache and do not automatically delete bundles. - -Requested capture is required work: allocation/link failures stop before model -work; recording failures surface at the next normal application boundary; -finalization failure fails the command. Observer callbacks retain errors for -boundary checks because they cannot return errors. An existing pipeline failure -remains primary, with capture failures reported additionally and causes retained. -Already-written data and published artifacts remain available. Capture failures -never cause model retries. Finalize before submitting the success report. - -#### Inventory And Depth - -Replace separate summary/trace categories with a compact run overview, -configuration/composition snapshots, parsed source, and per-operation records. -Organize operation directories by meaningful pipeline scope, with native execution -IDs within those operations. Provide a readable run overview and clearly named -structured snapshots and text views. `latest` is the selected link name; exact -debug filenames, directory nesting, and serialized application fields are explicitly -delegated to implementation. Evaluate them by ease of investigating an operation; -do not freeze them as a public format or introduce another debug framework. - -| Capture | Selected contents and depth | -| --- | --- | -| Run overview | Application version, run/pipeline identity, timing, outcome, publication location, effective invocation selections/overrides, and navigation. Terminal facts come from the same application outcome used by the CLI. | -| Configuration and composition | Native application and PromptKit effective snapshots, normally once each, plus resolved steps, lanes, modules, validators, and references. | -| Parsed source | Generic input document with unit identities and content, independent of the original input file. If parsing fails, retain the input location and available diagnostics. | -| Operation inputs | Actual source/chunk and reference content, incoming artifacts, prompt bindings/variables, and fixed original normalization candidates/handle mappings. Include directly even when duplicated elsewhere. | -| Prepared requests | Native `PreparedRun`, including settings, messages, and exposed output-contract/schema details when available. | -| Execution events | Native per-execution events in occurrence order, including attempts/retries/repairs, timing, usage availability, and terminal metadata. | -| Results and responses | Native `RunResult` plus exact `RawOutput` in a separate editor-readable text file. | -| Conversations | Full `Exchange.Details()` for every application-visible turn, including rejected turns, plus simple readable role-labeled messages. Repeated prefixes are acceptable. Use prepared messages when no completed exchange exists and label the representation. | -| Interpretation and assessment | Candidate via its codec before acceptance; decode/safety failures; each validator's findings and correction guidance; application disposition, any fallback artifact and reason, and links to LLM validator executions. | -| Normalization and corrections | Proposed reconciliation plan and applied candidate; constructed correction inputs even if the next preparation fails. | -| Deterministic stages and reuse | Chunk plans/materialized chunks, merged/normalized artifacts, and the loaded cached artifact with the current hit decision. No reconstructed history or fabricated execution on reuse. | - -Use indented JSON for native structured snapshots and existing domain codecs. -Retain selected content in full, with no arbitrary truncation. Enable exchange -capture where debug inspection requires it, including single-turn calls; -continuation's in-process exchange needs remain independent of durable capture. -Keep operation inputs self-contained enough for direct investigation without -requiring original inputs, a cache entry, or reconstruction of deltas. Do not -archive every original file or serialize the pipeline's entire internal state. - -On failure, retain available evidence, where execution stopped, errors, and -application disposition. Preparation/admission failure may have no `OnPrepared`; -operational failure may have no `RunResult`. Capture a coordination `Undelivered` -response as unassessed. Native internal-attempt metadata is useful but intermediate -provider response bodies are not required. No provider wrapper or complete -provider-transcript guarantee is part of this design. - -#### Data Handling And Diagnostic Commands - -Operators secure debug data as they secure ordinary application inputs and -outputs, for example with filesystem permissions. Debug contains internal data; -no separate security/retention model or sanitized-bundle promise is required. -Do not directly print/serialize ConfigKit-marked secrets, and preserve native -credential omissions through secret-aware inspection. Do not heuristically scan, -scrub, or rewrite prompts, source material, model output, or arbitrary strings. - -Inspection/diagnostic commands use native reports and do not automatically create -pipeline debug bundles. Operators may redirect those reports. Neither debug mode -nor capture initiates provider probes. Explicit smoke checks retain native -support, applicability, and outcome distinctions. - -## Decision Status - -U1–U10 are resolved. There are no remaining open architectural or dependency -choices in this roadmap. Native loading/execution/commands, accepted-result reuse, -artifact-first reporting, capture, examples, and documentation are implemented. -Observer wiring, private debug representations, and routine API placement are -concrete implementation choices, not deferred architecture. The completed assembled -acceptance/removal/link audit is recorded in the [implementation plan](implementation.md). -IDs retain continuity with the earlier roadmap. - -**U1 resolved:** The [reviewed dependency set](#baseline-and-dependency-target) -selects PromptKit v0.24.0, ConfigKit v1.4.0, ckconfig v0.10.0, ckprompt v0.11.0, -and CommandKit v0.7.0. Native `MessageContentHash` satisfies the cache requirement; -typed result handoff and focused inspection enable one application-owned load. -The native resource layout, schema selections, and `OutputContractInput` -API are implemented. Upstream and focused assembled checks passed as recorded -in the plan, including successful final full acceptance. - -**U2 resolved:** Ordinary embedded assets, prompt-declared output contracts, -native schema-source selection, and removal of duplicate schema/filesystem -machinery are selected in [the schema design](#prompt-assets-and-response-schemas-u2-resolved). -The native assets and schema selection are implemented with preparation and -operator-contract coverage in the plan. U4 selects prompt roles/contracts, and U5 -selects native repair delegation and the application correction allowance. - -**U3 resolved:** Typed stage boundaries, composition-root engine construction, -and the shared pipeline producer helper with module functions and a pipeline -assessment callback are selected in [the execution design](#module-execution-and-assessment-u3-resolved). -The typed operation/helper signatures are implemented within that ownership. -U4/U5 select prompt/guidance and retry behavior; U6 selects terminal -handling, configuration precedence, and module-owned fallbacks. - -**U4 resolved:** Domain-specific initial/correction prompt pairs, shared response -schemas, default profile behavior, validator-owned guidance, and complete -applicable-chain assessment are selected in -[the prompt and guidance design](#prompt-roles-and-correction-guidance-u4-resolved). -Native prompt pairs and operation-specific inputs are implemented within that -ownership. U5 selects budgets; U6 selects strict/permissive handling. - -**U5 resolved:** `max_additional_turns` defaults to `2`, with binding over -pipeline precedence and explicit zero disabling correction. Native transient and -structural-repair defaults remain upstream-owned; bundled prompts and application -requests do not pin structural repair. The [retry-control design](#retry-controls-and-correction-budget-u5-resolved) -also removes generic non-LLM retries, all producer/validator `retries` fields, -and the generic outer attempt loop. Any demonstrated transient I/O recovery stays -with its specific operation. U6 selects terminal handling. - -**U6 resolved:** [Strict/permissive handling and module-owned fallbacks](#terminal-outcomes-and-module-fallbacks-u6-resolved) -replace failure-category policies. Fallbacks satisfy artifact/downstream invariants -without asserting ordinary semantic acceptance; use only the final response and -fixed originals. Fallback and affected downstream work remain outside accepted- -artifact caching, and completion reports disclose fallback use. Strict is the -application default, including normalization; there are no module-declared -defaults. `mode` resolves module binding, then pipeline, then that default. -Chunking falls back to one whole-transcript chunk; spell extraction uses its empty -artifact, and normalization uses safe final-response groups or merged input. -Module-owned construction and terminal-mode coverage are implemented; see the -plan and [D&D guide](../internal/dnd.md#planning-scene-validation-and-fallback-limits). - -**U7 resolved:** [One pipeline worker pool and native admission](#admission-and-pipeline-workers-u7-resolved) -replace the separate worker pools and application LLM scheduler. -`concurrency.workers` defaults to `64` and bounds ready extraction/lane completion -jobs together, independently of CPU count and backend settings. PromptKit owns -backend capacity with no injected application defaults; admission failures use -U6's mode without extra retries or queues. Retire `total_llm` and the old extraction -worker field with explicit migration guidance. - -**U8 resolved:** [The unified accepted-result cache](#accepted-artifact-cache-u8-resolved) -includes LLM chunk plans and replaces both legacy caches. Root-level -`cache_directory` selects a per-user default or an explicit root, with -`/var/cache/notarius` documented for system-wide installation. Mode precedence is -explicit `--cache`, module-binding `cache`, pipeline `cache`, then `auto`. -`auto` reads/writes, `bypass` does neither, and `refresh` writes newly accepted -results without reading. Cache I/O is best effort; legacy records remain untouched -and are not imported. U1 supplies the selected native hash API. - -**U9 resolved:** [Execution bookkeeping](#inspection-and-execution-bookkeeping) -uses one native execution group per invocation, with all actual initial, -continuation, and LLM validator executions enrolled once and application scope -attached. [Completion output](#artifact-delivery-and-completion-output) adds only -optional positive `fallback_count`, counting producer operations that emitted -fallbacks. Preserve run-specific domain artifact paths and schemas; retire index, -manifest, and reporting-only sidecars and migrate consumers to documented paths. -Observer wiring and [debug filenames/layout](#inventory-and-depth) are explicitly -implementation choices, not unresolved architecture or stable debug contracts. -Implemented native observations/capture/publication and reporting tests are -recorded with successful final acceptance in the plan. - -**U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight, -output, and debug policy are implemented as selected above. -CommandKit belongs to the coordinated cutover. CLI compatibility is a non-goal; -existing artifact schemas remain domain contracts. Any actual artifact-format -change discovered during implementation needs an explicit compatibility decision, -not an assumption that native CLI adoption authorizes it. - -## Implementation Planning - -The [implementation plan](implementation.md) owns execution order, repository -touchpoints, stage acceptance criteria, and validation commands. This roadmap -defines the complete target and acceptance criteria. Release preparation and -publication remain separate work. - -## Removal Inventory - -| Retire or replace | Retained application responsibility | -| --- | --- | -| Separate application/PromptKit loading, duplicated source interpretation and inspection conversion | One composed ConfigKit load and native result handoff to engine construction and focused inspection. | -| `StructuredLLMClient`, private request/response types, adapter conversions, emptied `internal/framework/llm` package | Composition-root engine construction, native module request/interpretation functions, and pipeline assessment/helper. | -| `AssetRegistry`, `PromptAssetManifest`, `ModulePromptFS`, synthetic mounts, flattened schema namespaces, `readonlyfs`, manual fingerprint enumeration | Ordinary embedded content, native source assembly, and module meaning. | -| Private response-schema registry/wrappers, byte copies, digest and reporting-only identity metadata, duplicate structural validation, old schema constructor options | Native prompt output contracts, schema selections and roots; application interpretation and durable artifact contracts. | -| `single_response_v1`, raw `ModelCandidate`/`SemanticCorrection`, manual replay, correction capability checks | Typed candidates, actionable findings, native exchanges, assessment. | -| Generic `runProducerAttempts` outer loop, repeated validator executions in `executeValidationChain`, all producer/validator `retries` fields, module rerun directives, hard-coded `CorrectionRequest` message builder | Ordinary deterministic execution, native LLM execution/continuation, ordered validators, validator/module-owned guidance, native domain correction templates, and domain acceptance/rejection/fallback. | -| Application `structured_output_repair_attempts` fields/resolution and explicit bundled `output.repair_attempts` | Native upstream structural-repair default, with operator customization through native prompt configuration. | -| Failure-category `validation_policy` fields and separate `fail_run`/`reject_output`/`warn_continue` actions | Strict/permissive selection, module-owned fallback construction, and explicit fallback outcomes. | -| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, `concurrency.stage_workers.extract`, separate extraction/lane worker pools | One ready-job pool controlled by `concurrency.workers`, default `64`, and native backend admission. | -| Checkpoint stage/dependency/configuration fingerprints, invocation-wide barriers, manifests, acceptance/provenance envelopes | Small accepted-artifact cache using the upstream message hash, ordinary load checks, explicit controls. | -| Separate source-keyed chunk-plan store, canonical-plan provenance/publication policy, nested legacy cache configuration, `--resume`, `--recompute-step`, separate chunk-cache flag | Unified message-content lookup/storage, domain chunk-plan interpretation/materialization, `cache_directory`, pipeline/module `cache`, and invocation-wide `--cache`. | -| Inspection wrappers, copied effective parameters, usage summaries/conversions/accumulators, debug completion wrapper, synthetic IDs, profile recorder | Native facts with application scope for diagnostics. | -| Custom dispatch/help/version/parsing, command-specific loading, copied inspection, legacy aliases/flags, parallel encoders and verbosity handling | Native declarations/loaders, pipeline handlers, domain input checks. | -| Mandatory `index.json`/`manifest.json`, reporting-only `rejected.json`/`warnings.json`/`diagnostics.json`, extensive receipts and durable execution ledgers | Preserved domain artifact paths and schemas, one small native completion report with optional `fallback_count`, ordinary diagnostics, and optional debug evidence. | -| Summary/trace hierarchy, heuristic debug-content scrubbing, custom transcript/deduplication machinery | Convenient run/operation capture and native secret/credential omissions. | - -## Acceptance And Verification - -Use real CommandKit, ConfigKit, and PromptKit in representative assembled flows -with a fake at the provider boundary. Keep the suite deterministic, offline, and -independent of real credentials. Test domain interpretation/assessment at their -package boundaries; use subprocess tests for streams, exit status, alias wiring, -and representative machine output. Do not duplicate upstream parser/renderer/ -retry tests or freeze complete help text and native debug snapshots. - -| Area | Required evidence | -| --- | --- | -| Configuration/assets | Application version 6 and resources beneath `promptkit` work without a separate PromptKit resource-version gate or legacy translation; companions/native imports retain main-source context; peer conflicts fail; same-ID replacement, real shared-file paths, prompt-declared contracts, embedded/external roots and source-relative/inline schemas work. Root replacement has no missing-file fallback. Necessary `OutputContractInput` overrides replace the complete contract. | -| Commands/loaders | One composed ConfigKit load supplies engine construction and native inspection, including explicit bundled resources, source evidence, and focused paths; no duplicate PromptKit load. Required operands, native alias, supplied-value distinctions, references, and run-wide overrides reach intended handlers/roles; help/version/completion need no config; diff selections remain distinct. | -| Preflight/inspection | Same definitions across execution and inspection; missing statically selected resources fail with context; partial profiles alone pass; no provider calls, transcript, or credential requirement for offline preflight. | -| Domain workflow | Representative extraction and normalization preserve source/reference identity, fixed handles, safe plan application, ordered validators, artifact schemas, and stable output ordering; one assessor checks initial/corrected candidates without an extra final pass; deterministic stages and LLM validators need no producer conversation loop. | -| Prompt/guidance contracts | Domain-specific initial/correction pairs use the same response schema and default profile; explicit profile/execution overrides reach both; operator correction-profile defaults work when not overridden; all applicable validators assess the same candidate despite earlier rejection and reassess each delivered revision, including an unchanged candidate; their actionable findings reach the correction template; module interpretation defects and unavailable assessments remain distinct. | -| Retry controls | `max_additional_turns` resolves binding, pipeline, then application default `2`; explicit zero disables correction; acceptance stops early and the final allowed turn is assessed. Bundled declarations/application overrides leave structural repair unspecified, native operator customization remains effective, and tests do not freeze upstream default attempt counts. | -| Non-LLM failure handling | Deterministic errors propagate without generic retries; rejected deterministic candidates reach terminal policy without regeneration; cancellation stops work. Producer/validator `retries` fields and the generic outer attempt loop are retired. | -| Continuation/failures | Accumulated history, full revalidation and final-turn assessment; native failed validation with a nil Go error reaches terminal handling before domain interpretation or caching in producer, continuation, and validator paths. No blanket retries beyond native exhaustion; validator failure differs from rejection; no earlier-candidate recovery or cross-turn proposal union; cancellation stops without fallback work. | -| Strict/permissive outcomes | `mode` resolves module binding, pipeline, then strict; omission inherits and either explicit value overrides. Strict is the default for every role, including normalization, without module-specific defaults. Strict failure when no acceptable result is obtained; permissive handling produces module fallback with required artifact/downstream invariants, without rerunning the ordinary full validation chain or claiming normal acceptance. Chunking fallback is one whole-transcript chunk preserving source order/references. Fallback construction failure fails execution; successful fallback use is disclosed in human/machine completion reports. | -| Admission/workers | One shared extraction/lane completion pool obeys `concurrency.workers`, default `64`; explicit `1` runs serially without dependency or nested-validator deadlock. Only ready jobs occupy slots; result ordering remains stable. Backend limits remain independent with no application-injected defaults. Producer/validator admission failures use U6's mode, preserve useful diagnostics, and add no retries; cancellation releases capacity. No separate global LLM ceiling. | -| Accepted-artifact cache | Downstream recovery; initial-message keys; corrected final artifact stored late under the initial key; changed messages miss; settings/ID-only changes can hit; no model/revalidation on hits; unusable entries miss. Failed/rejected/incomplete work is not stored; fallbacks and affected downstream work skip lookup/storage while unaffected accepted predecessors remain reusable; no required history. | -| Chunk-plan reuse | Final accepted plan stored under the initial request hash after materialization/validation/correction; hits rematerialize with structural/source-reference usability checks and no renewed semantic validation. Changed messages can select a new partition; deterministic chunkers run without synthetic cache requests. | -| Cache controls and migration | Root-level directory default/explicit selection; mode precedence CLI, module binding, pipeline, then `auto`. `auto` reads/writes; `bypass` does no I/O or directory creation; `refresh` only replaces entries after new acceptance. Unusable entries miss; cache I/O failures preserve successful pipeline results. Both legacy stores remain untouched and unread; no conversion or separate canonical-plan mechanism. | -| Output/outcomes | Preserved run-specific domain paths, lane filename mapping, and payload schemas; optional chunk-map/evidence-context publication remains. No index/manifest/reporting sidecars. Native text/JSON/YAML summary includes positive `fallback_count` and omits zero; count emitting producer operations, not attempts, validator failures, or affected derivatives. Quiet suppression, verbose stderr, debug path, and useful nonzero failures hold. Publication survives later capture/report failure without model reruns. | -| Execution observations | One native invocation group enrolls actual initial/continuation/LLM validator executions once, with application scope; native totals preserve unknown-versus-zero usage without application accumulators or overlapping-group aggregation. Result-less failures remain observable; cache hits add no executions/history; native execution success remains distinct from acceptance. | -| Debug controls/lifecycle | Default/CLI/config/explicit-empty paths and native mode ordering; quiet capture; allocation before engine/preflight; start-time relative `latest`, no completion retarget; disabled runs leave it untouched; capture failures preserve primary causes and publication facts. | -| Debug evidence | Available input/source/native snapshots, readable text, all visible turns, assessments, normalization and deterministic outputs, cache hits, and failed-run evidence. Cache hits retain prepared details without an execution event, duplicate preparation, or model call; no invented provider transcript. ConfigKit secrets/native credentials omitted, no heuristic content scrubbing. | -| End-to-end migration | Maintained minimal and complete D&D workflows, session routing, reference behavior, raw artifacts, reports, and migration examples work through the new executable. No postponed CommandKit adoption. | - -Run focused checks during implementation and the repository-wide checks for the -completed cutover under the [testing policy](../policy/testing.md): - -```sh -go test ./... -go vet ./... -go build ./cmd/notarius -``` - -Check maintained examples, dependencies, attribution, documentation links, and -removal coverage. Live provider smoke and human model-quality evaluation remain -optional, outside the default suite. These migration checks add no permanent -release gates. Documentation completion does not claim that implementation -acceptance checks have passed. - -## Migration Guidance And Documentation - -Provide practical release guidance for operators and subprocess callers: - -- New commands/flags, native `run` alias, `--format`, stdout/stderr, outcomes, and - saving optional reports by redirection. No legacy CLI compatibility promise. -- Application version 6 and native resources beneath `promptkit`, embedded-only - versus companion examples in appropriate directories, imports, qualified IDs, - replacement/inheritance, native local backends, and removed catalogs/version - selectors. Explain native schema selections and collection/application roots, - external root replacement, - and the removal of a separate PromptKit source-version requirement. -- Shared `concurrency.workers` default `64`, ready-job scope, explicit serial - execution, native backend concurrency/queue settings, and removal of the global - limit and old extraction-worker field without automatic backend translation. - Explain strict/permissive handling of producer and validator admission failures. -- `max_additional_turns` default/precedence/zero behavior, and intent-based - migration from LLM `retries`. - Explain retirement of all producer/validator `retries` fields and generic non-LLM - retries, upstream-owned structural defaults, removal of application repair - controls and bundled counts, native operator customization, accumulated-history - costs, the universal strict default, pipeline `mode` with module-binding overrides, - and each module's fallback behavior and downstream guarantees, including the - whole-transcript chunking fallback. -- Unified accepted-artifact and chunk-plan caching, message-based misses, - settings-only reuse, root-level `cache_directory` and the per-user/system-wide - conventions, pipeline/module `cache`, and invocation-wide `--cache` precedence. - Explain refresh/bypass, best-effort I/O, fallback/affected-work exclusion, removal - of both legacy mechanisms without importing/deleting records, and retirement of - canonical-source partition reuse. -- Artifact delivery through preserved run-specific domain paths and lane filename - mapping, without index/manifest/reporting sidecars or receipt traversal. Document - `fallback_count` and its omission/counting rules; consumers use the published - directory and pipeline-defined lanes. Preserve intrinsic artifact schemas and - document any explicitly selected format change. -- Debug root/default/empty behavior, `latest`, native mode ordering, failure - behavior, and ordinary operator responsibility for application data. - -Update canonical [configuration](../config.md), [CLI](../cli.md), -[operations](../operations.md), [internal overview](../internal/overview.md), -[LLM](../internal/llm.md), [pipeline](../internal/pipeline.md), -[modules](../internal/modules.md), [state](../internal/state.md), and affected -integration/consumer guides as behavior lands. Remove superseded manifest/receipt -contracts then; keep current-behavior documentation accurate until implementation. -Update [architecture policy](../policy/architecture.md) for native dependencies -and changed scheduler, retry, asset, cache, and reporting boundaries. - -Record significant changes in successor ADRs, including -[asset assembly](../adr/0011-centralize-llm-assets.md), -[validation retries](../adr/0014-feedback-aware-validation-retries.md), -[module retries](../adr/0016-feedback-aware-module-requested-retries.md), -[PromptKit configuration](../adr/0019-adopt-native-promptkit-configuration.md), -the [canonical chunk-plan policy](../adr/0005-cache-canonical-chunk-plans-by-source.md), -admission, caching, and CLI/reporting ownership. Preserve accepted decision text -and update supersession status under [documentation policy](../policy/documentation.md#architecture-decision-records). -This roadmap owns the target and decision rationale; the implementation plan -owns execution stages and their status. Git history retains earlier discussions. - -## Evidence And Reading - -These sources support the selected design, not a claim of completed Notarius -implementation. Sibling links assume the local repositories are available; the -tagged PromptKit links below preserve the reviewed baseline. - -### Native Library Guidance - -- [CommandKit consumer guide](https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/consumers/guide.md) and - [CLI conventions](https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md): module selection, native - aliases/inputs, shared loaders, reporting modes, outcomes, and delivery failures. -- [ckconfig](https://forge.maximumdirect.net/base-layer/commandkit/src/tag/integrations%2Fckconfig%2Fv0.10.0/integrations/ckconfig/README.md) and - [ckprompt](https://forge.maximumdirect.net/base-layer/commandkit/src/tag/integrations%2Fckprompt%2Fv0.11.0/integrations/ckprompt/README.md): standard suites - supplied with application loaders and request builders. -- [ConfigKit v1.4.0 release notes][configkit-release]: typed result handoff and - focused inspection over the original load's provenance and resource contexts. -- [ckconfig v0.10.0][ckconfig-release] and [ckprompt v0.11.0][ckprompt-release]: - the selected integration compatibility and API changes. -- [Shared-loading guidance][consumer-loading] and the - [composed CLI example](https://forge.maximumdirect.net/base-layer/commandkit/src/tag/integrations%2Fckprompt%2Fv0.11.0/integrations/ckprompt/examples/composed/main.go): - composed rules, explicit bundled resources, one application result, native - engine construction, and focused standard scopes. -- [PromptKit consumer guide][consumer] and - [capacity behavior](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/internal/capacity.md): native - preparation/execution, continuation, observations, and admission. - -### Audita Comparison - -The comparison was completed on 2026-11-18 against Audita revision -`da5b8913d1206a8e8f0a784b2322760385548a60`, using PromptKit v0.23.2, -ConfigKit v1.2.1, CommandKit v0.7.0, ckconfig v0.9.0, and ckprompt v0.10.0. -The observations below concern that revision; linked sibling files may evolve. - -| Evidence | Lesson carried into this roadmap | -| --- | --- | -| [Engine assembly](../../../audita/internal/framework/llm/runtime.go), [source tests](../../../audita/internal/framework/llm/engine_sources_test.go), and [assembly failure tests](../../../audita/internal/framework/llm/runtime_assembly_test.go) | One captured engine, native underlays and main-source-relative companions, whole-record replacement, preserved file contexts, strict peer conflicts, and no silent fallback on operator-source errors. | -| [Adapter tests](../../../audita/internal/framework/llm/adapter_test.go) | Ordinary embedded schemas and complete per-request output contracts work with operator prompt replacement. U2 chooses prompt-declared defaults for Notarius; Audita's schema registry and per-call overrides are not requirements to copy. | -| [Native coordination bridge](../../../audita/internal/framework/llm/coordinate.go), [repair flow](../../../audita/internal/framework/runner/repair.go), [conversation tests](../../../audita/internal/framework/llm/conversation_test.go), and [repair tests](../../../audita/internal/framework/runner/repair_test.go) | Captured exchanges, accumulated history, complete candidate revalidation, final-turn assessment, and canceled/undelivered response handling. | -| [Prompt guidance](../../../audita/docs/internal/prompts.md) | Durable system instructions, an initial user task, and subsequent repair assignments suit native continuation. | -| [Observations](../../../audita/internal/framework/llm/observations.go) and [CLI composition](../../../audita/internal/cli/app_wiring.go) | Native execution totals with application scope and explicit nested-validator enrollment; standard CommandKit suites using shared loaders. | - -Audita repairs individual transcript corrections in independent branches; -Notarius's selected unit is a whole producer operation and typed artifact. -Audita's wrappers, response-schema registry, per-proposal policies, and absence -of a cache do not define Notarius requirements. - -At the reviewed revision, focused LLM, prompt/schema, runner, and proposal-generation -tests passed, together with subprocess checks for correction conversations, -cancellation during repair, and repair-disabled/output selections using a local -fake provider. These historical checks do not establish live-model quality or -completion of Notarius's migration acceptance checks. - -[release]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/releases/v0.24.0.md -[migration]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/configuration-migration.md -[formats]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/formats.md -[consumer]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/pkg-promptkit.md -[consumer-loading]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/pkg-promptkit.md#reuse-an-application-owned-load -[configkit-release]: https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.4.0/docs/releases/v1.4.0.md -[ckconfig-release]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/integrations%2Fckconfig%2Fv0.10.0/integrations/ckconfig/docs/releases/v0.10.0.md -[ckprompt-release]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/integrations%2Fckprompt%2Fv0.11.0/integrations/ckprompt/docs/releases/v0.11.0.md diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 283d5684..00000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,607 +0,0 @@ -# Native Migration Review Remediation - -## Scope And Baseline - -Review date: 2026-11-22. - -Reviewed commit: `75444b8ef300ca8efee57d7a2271d21ce258b49c`. - -Integration follow-up baseline: `fd134e85e54c6e0f74aab5c77df5bada7c1b1e6a`. -The follow-up baseline adds only the initial review findings; application code -is unchanged between these revisions. - -Algorithmic follow-up baseline: `fdaf39162b46b528fddcd8a4cad18a09e1daf661`. -Only this findings document changed after the application baseline above. - -This review followed the 46 stages in [the implementation plan](implementation.md) -against the [native migration target](promptkit.md), current owning code, -focused tests, maintained examples, and canonical documentation. The migration -is substantially complete. Findings 1–2 come from the initial review. Findings -3–5 come from an integration pass. Findings 6–9 and the complexity note on -Finding 1 come from a stage-ordered algorithmic pass. The latter are refactoring -opportunities for consideration, not measured production bottlenecks or new -performance guarantees. None calls for another compatibility layer or application -execution abstraction. The evidence below describes the reviewed baselines; -per-finding completion markers record the subsequent remediation work. - -## Finding 1: Retain Genuine Native Timeouts During Concurrent Error Selection - -**Status: Complete.** Operational deadlines participate in linear minimum -selection; sibling cancellation noise and authoritative caller cancellation are -preserved. Pipeline tests and race checks pass. - -Priority: P2. Relevant stages: 10, 25, 37, and 46. - -### Evidence And Impact - -[`selectRunError`](../../internal/framework/pipeline/runner_concurrent.go), -lines 398–428 at the reviewed revision, suppresses every error matching -`context.Canceled` or `context.DeadlineExceeded` when any other failure is -available. This retained cancellation filter does not distinguish a canceled -sibling from an independently failed native operation. - -A native PromptKit operation timeout matches both -`promptkit.ErrOperationTimeout` and `context.DeadlineExceeded`, even when the -caller context remains active. Consequently, a genuine timeout in an earlier -stage/lane/chunk scope is discarded in favor of a later non-context error, -rather than participating in the runner's stable scope ordering. The returned -cause then loses that native timeout identity. - -The producer helper itself handles this distinction correctly; the gap is at -the concurrent runner's final selection boundary. A temporary review test used -the real PromptKit engine with a fake provider, obtained an actual native -operation timeout with an active caller, and supplied it as the earlier of two -scoped failures. The selector returned the later error instead. Existing helper -timeout coverage does not exercise this aggregation boundary. - -### Required Change - -Refine error selection so genuine operational deadline/timeout failures remain -eligible for the existing stable scope ordering. Distinguish them from -runner-induced sibling cancellation using the relevant context state and native -causes. Preserve authoritative caller cancellation and suppression of derivative -cancellation noise. An active caller's deadline-shaped operation failure must -not be discarded merely because it matches `context.DeadlineExceeded`. - -Keep this a small runner correction; no new timeout controls, retry loop, or -application error taxonomy is needed. - -### Acceptance - -- With an active caller, an earlier-scoped native operation timeout and a later - ordinary failure select the timeout and retain its native `errors.Is` identity. -- A non-caller provider deadline also remains a genuine operational failure. -- Caller cancellation/deadline and runner-induced sibling cancellation retain - their intended precedence/noise handling. -- Ordinary multi-failure ordering and worker cancellation/draining remain intact. -- Add focused behavioral coverage at the runner error-selection boundary, using - the existing native timeout fixture where useful. Run the pipeline tests and - their concurrency/race checks. - -### Algorithmic Follow-up: Select A Minimum Instead Of Sorting - -With `E` collected errors, the current selector comparison-sorts all eligible -records by `(stage, lane, chunk)` solely to return the first. It performs -`O(E log E)` comparisons; stable sorting can also incur additional superlinear -data movement. After eligibility classification, a minimum scan takes `O(E)` -comparisons and constant auxiliary space; error-chain classification costs remain -separate. Keep the first occurrence on equal keys, matching stable-sort behavior. -This can accompany the correctness -fix without changing error precedence or cancellation semantics. Its practical -impact is smaller than the document/cache indexing findings below. - -## Finding 2: Narrow The Documented Empty-Validator-Chain Guarantee - -**Status: Complete.** Architecture, configuration, pipeline internals, and the -active plan now distinguish intrinsic checks from configurable citation checks. -Documentation links and whitespace were checked; runtime behavior is unchanged. - -Priority: P2. Relevant stages: 11, 38, 39, and 42. - -### Evidence And Impact - -Three current documents make source checking sound unconditional: - -- [Architecture policy](../policy/architecture.md), lines 229–232: intrinsic - codec/source correctness still applies with an empty chain. -- [Configuration reference](../config.md), lines 350–356: intrinsic typed - codec/source checks remain with an empty configured chain. -- [Pipeline internals](../internal/pipeline.md), lines 75–77: an empty chain - cannot remove intrinsic source-addressing checks. - -Intrinsic durable-codec validation does remain. However, ordinary artifact -checks that cited units exist in the current source or lie within an extraction -chunk are supplied by the configured source-reference validators; final artifact -serialization does not independently perform those checks. See -[`serializeArtifact`](../../internal/framework/pipeline/runner_typed.go) and -the configured [candidate assessor](../../internal/framework/pipeline/candidate_assessment.go). - -A temporary offline native-CLI probe configured `dnd/spells` extraction and -`noop` normalization with `validators: []`, disabled cache use, and supplied a -schema-valid spell citation to unit 999 against a transcript containing only -unit 1. The command succeeded and published that citation. This does not show a -failure of the default validator chains: the probe explicitly replaced them. -It demonstrates that the broad documented guarantee is stronger than the -implemented configurable validation boundary. - -### Required Change - -Align the three documents with the actual, selected boundary. Distinguish: - -- intrinsic durable-codec/schema validation; -- source-document validation and chunk-plan materialization; -- checks a particular module necessarily performs while interpreting, - constructing fallback, or decoding a cache hit; and -- artifact citation existence/containment checks supplied by configured - validators. - -State that an explicit empty/replacement chain can omit those ordinary artifact -source-reference checks. Link to the canonical default-chain/configuration -description rather than repeating detailed validator inventories. - -Preserve operator replacement of validator chains. Do not add a mandatory -artifact validator, prohibit empty chains, or make every codec consult the -source document merely to satisfy the stronger wording. Those would expand the -requested contract. The Stage 11 completion summary's claim of mandatory final -codec/source checks should be qualified consistently while the active plan is -retained. - -### Acceptance - -- The architecture, configuration reference, pipeline guide, and active plan - distinguish unconditional codec checks from configurable citation validation. -- They do not imply that disabling validators disables source parsing, - materialization, or necessary module/fallback/cache construction checks. -- Existing default chains, explicit chain replacement, and runtime behavior - remain unchanged. -- Review related current documentation for the same overbroad claim. No new - prose-snapshot or mandatory-source-validation test is needed for this - documentation correction. - -## Finding 3: Rebind Cached Scene Identity To The Current Chunk Plan - -**Status: Complete.** Scene hits require one exact current-range scene and -reattach its current ordinal identity without model work or reassessment. -Expansion/collapse integration regressions and scene-module tests pass. - -Priority: P1. Relevant stages: 11, 23, 27–29, and 37. - -### Evidence And Impact - -The new message-content cache interacts incorrectly with the existing -position-derived scene identity: - -- [`MaterializeChunkPlan`](../../internal/core/source/chunk_plan.go), lines - 169–179, renders only source units into chunk content, while assigning - `chunk-000001`, etc. from the range's position in the plan. -- [Scene extraction](../../internal/modules/dnd/extract/scenedescriptions/extractor.go) - deterministically attaches the current chunk ID and range in `mapResponse`. - The model does not supply this identity, and the ID is not rendered into the - initial scene-description messages. -- [`artifactProducerCache`](../../internal/framework/pipeline/runner_cache.go), - lines 21–37, decodes a hit and checks cited source units, but does not rebind - its scene identity. The hit bypasses extraction interpretation and assessment. - -Changing an earlier scene boundary can therefore renumber an unchanged later -scene without changing that later scene's prompt hash. Its old accepted artifact -is a legitimate content-key hit, but its stored scene ID belongs to the old plan. - -A temporary native-CLI integration probe used real scene planning, extraction, -merging, normalization, default validators, and a filesystem cache. The chunk -binding used `cache: refresh`; scene extraction used ordinary `auto`. The fake -provider returned these two valid plans for the same three-unit transcript: - -1. `[1–2], [3]`, followed by `[1], [2], [3]`: the second run failed normalization - with `scene ID "chunk-000002" has conflicting records`. -2. The reverse change: the second run succeeded but published IDs - `chunk-000001` and `chunk-000003` for a two-chunk plan. The last ID should have - been `chunk-000002`. - -Both cases pass with invocation-wide cache bypass. The unchanged suffix was -reused in the failing cases, as confirmed by provider-call counts. This is not -an invalid generated plan or intentionally skipped semantic revalidation. - -The stale ID also conflicts with the existing [scene registry's exact match](../../internal/modules/dnd/scenedescriptions/registry/registry.go): -combat/enemy consumers require both the current chunk ID and range, so a stale -scene can become unavailable classification and suppress downstream extraction. - -### Required Change - -Add the missing current-operation adaptation at the scene cache-hit boundary. -For a usable single-scene artifact describing the exact current range, -deterministically reattach the current chunk identity while retaining its accepted -kind/title/summary. If a cached artifact cannot be safely adapted, treat it as an -unusable miss before returning it as accepted work. - -Keep domain-specific adaptation with the scene module, using the supplied typed -cache collaboration or a small extension to that boundary if needed. Preserve -the content-only native cache key and no-semantic-reassessment policy. Do not -render an internal scene ID into the prompt, add a partition fingerprint/key -dimension, restore canonical source-only chunking, or ask the model to copy IDs. -Review other current cached artifacts for equivalent operation-derived fields; -do not invent a generic identity framework for families that do not need one. - -### Acceptance - -- Repartitioning earlier units can move an unchanged later scene to a different - chunk ordinal without duplicate or stale scene IDs in the published artifact. -- Current scene IDs/ranges match the chunk map and remain usable by the existing - combat/enemy eligibility resolver. -- The unchanged scene can reuse its accepted semantic content without another - model call or validator assessment when deterministic adaptation is safe. -- Wrong-range, multi-scene, or otherwise unusable entries are not silently - rebound into valid-looking scenes; they miss normally. -- Add assembled two-run expansion/collapse coverage using the existing native - fake-provider/cache setup, and retain ordinary same-plan hit coverage. - -## Finding 4: Bound Framework-Generated Fallback Diagnostic Messages - -**Status: Complete.** Framework fallback/cache samples share bounded UTF-8 -rendering without changing full outcome evidence or module diagnostic checks. -Multibyte/invalid-UTF-8 regressions and a typed spell CLI fallback probe pass. - -Priority: P2. Relevant stages: 9, 11, 24, 29, and 34. - -### Evidence And Impact - -[`FinishProducerFailure`](../../internal/framework/pipeline/producer_outcome.go) -retains the complete failure text in `ProducerOutcome.FallbackReason` after a -supported permissive fallback is successfully constructed. -[`operationDiagnosticGroups`](../../internal/framework/pipeline/runner_operation.go), -lines 51–58, copies that text unchanged into a framework-generated warning sample. -The existing [diagnostic contract](../../internal/framework/contracts/diagnostics.go) -requires valid UTF-8 and a maximum sample-message size of 4096 bytes. - -Native errors and joined assessment errors are not bounded diagnostic samples. -Even individually bounded domain validator messages can exceed the sample limit -after joining several findings and adding validator context. The new fallback -bridge thus creates invalid diagnostics from otherwise valid operation outcomes. - -A temporary offline native-CLI probe ran a minimal spell pipeline in permissive -mode with a fake provider returning a long ordinary error. The valid empty spell -fallback was constructed, but the run then failed with -`producer diagnostic 0: diagnostic sample 0: diagnostic sample message exceeds maximum length`. -The failure occurs even with ordinary visibility hiding optional warnings. - -### Required Change - -Render the framework's fallback warning into a bounded, valid-UTF-8 sample before -promoting it. Reuse or factor the small bounding behavior already present in -[`producerCacheObserver`](../../internal/framework/pipeline/runner_cache.go) -rather than adding a second diagnostic system. - -Bound only the generated diagnostic sample, leaving the original error/outcome -text and requested evidence unchanged. This is operator diagnostic presentation, -not model correction guidance. Preserve validation of module-supplied diagnostics; -do not raise/remove the existing cap or suppress genuine malformed-module -contracts to work around the framework's own unbounded sample. - -### Acceptance - -- A supported permissive fallback still publishes qualified output when its - native or aggregated-assessment cause exceeds the diagnostic sample limit. -- The generated warning is nonblank, bounded, and valid UTF-8, including at a - multibyte truncation boundary. -- Strict handling, caller cancellation, fallback counting/reuse exclusion, - full requested evidence, and module-diagnostic validation remain unchanged. -- Add focused fallback-to-diagnostic integration coverage, including a real - typed producer path with a fake long native error. No model retry is added. - -## Finding 5: Remove The Disconnected Legacy LLM Error Contract - -**Status: Complete.** Removed the unused legacy error taxonomy and its self-test; -current execution retains native PromptKit errors and application assessment/ -capture errors. Contracts, pipeline, CLI, and module integration tests pass. - -Priority: P3. Relevant stages: 8, 24, 35, and 44. - -### Evidence And Impact - -At the reviewed baseline, `internal/framework/contracts/errors.go` still -defines the pre-migration `ErrInvalidStructuredOutput`, `ErrLLMCapacityExceeded`, -`ErrLLMGeneration`, `LLMGenerationError`, constructors, and copied failure/attempt -metadata. Current producers and validators retain native PromptKit errors instead. -Graph/code searches find this old taxonomy only in that file and its -self-test (`internal/framework/contracts/errors_test.go`), not in an active -application consumer. - -This is leftover contract surface from the removed adapter, not a second active -execution path. It nevertheless presents obsolete sentinels and metadata accessors -as available framework conventions, although matching actual native failures -against them no longer works. Stage 35's hard-cutover cleanup is incomplete here. - -### Required Change And Acceptance - -Remove the unused legacy error definitions and their self-test. Keep native -`errors.Is`/`errors.As` behavior and the separate, actively used application -assessment/capture error types. Do not bridge native errors into these retired -sentinels or add aliases for compatibility. Verify no current callers remain and -run contracts, pipeline, CLI, and module integration tests after removal. - -## Finding 6: Use Indexed Source Validation In The Chunk-Plan Hot Path - -**Status: Complete.** Document validation builds one unit-position map, and plan -materialization reuses checked endpoints without rescanning or duplicate -framework validation. Source, chunk-map, and pipeline tests cover retained -ordering, ownership, malformed-input behavior, and nonmonotonic IDs. - -Priority: P2. Relevant stages: 16, 24, 28–29, and 37. - -### Evidence And Complexity - -The supporting [source validation code](../../internal/core/source/validation.go) -predates this migration, but the new chunk-plan assessment/cache path repeatedly -uses it. `ValidateDocument`, lines 95–101, validates each unit's self-reference -with `ValidateRef`. That helper resolves both endpoints through the linear -`UnitIndex` scan. For a valid document of `N` units, the second validation pass -alone makes `2 * (1 + ... + N) = N(N+1)` unit-ID comparisons: `Theta(N^2)` work. - -[`ValidateChunkPlan` and `MaterializeChunkPlan`](../../internal/core/source/chunk_plan.go) -also repeatedly scan the source for each of `C` ranges. A materialization thus -performs `O(N^2 + CN)` source-position work in addition to field checking, -annotation processing, metadata ownership, and payload encoding. -The pipeline's -[`validateAndMaterializeChunkPlan`](../../internal/framework/pipeline/chunk_validation.go) -validates once explicitly and then calls a materializer that validates again. -The [new plan path](../../internal/framework/pipeline/runner_chunk_plan.go) -invokes this during assessment, cache encode/decode, and final delivery. Cached -plans therefore retain this deterministic cost even without inference. - -### Potential Refactoring - -Use the existing immutable `source.DocumentIndex` for bulk endpoint lookups. -Build/reuse a unit-position map while validating the document, then check its -self-references in linear expected time. Share resolved positions through a -small private validation/materialization helper so materialization does not -rescan endpoints or repeat the same validation unnecessarily. Keep standalone -public validation/materialization entry points correctly validating their inputs. - -The attainable position-lookup cost is `O(N + C)` expected work with `O(N)` -index space. Field/payload inspection, annotation canonicalization, metadata -ownership, and JSON encoding remain separate costs. This does not promise linear -total output size: permitted overlapping generic ranges can themselves require -superlinear copied output. Repeated materialization of accepted plans can be -reduced where already-owned checked chunks are available, without retaining a -whole-run ledger or removing necessary current-source checks. - -### Preserve And Verify - -Preserve document-order semantics for nonconsecutive/nonmonotonic unit IDs, -duplicate/missing/wrong-source/reversed-reference rejection, annotation handling, -deep ownership, and deterministic chunk IDs. Generic plans still permit their -existing gaps/overlaps; D&D coverage rules remain with scene interpretation and -configured validators. No stricter plan format or new size limit is needed. - -Use the existing source validation/materialization, chunk-plan operation/cache, -and chunk-map tests. Add only missing behavioral regressions; source inspection -or optional synthetic operation counts can verify the scaling change without -wall-clock thresholds or production data. - -## Finding 7: Share A Lazy Source Index Across Artifact Cache Decoders - -**Status: Complete.** Run-scoped decoders share a concurrency-safe lazy immutable -index, acquired only when a decoded hit has evidence to check. Concurrent-hit -runner coverage and the pipeline race suite pass; cache modes and usability -checks remain unchanged. - -Priority: P2. Relevant stages: 11, 21–22, and 27–28. - -### Evidence And Complexity - -[`artifactProducerCache`](../../internal/framework/pipeline/runner_cache.go), -line 17, eagerly creates a full `DocumentIndex` for each model-backed operation -whenever a store collaborator exists. It does so before the producer helper -checks bypass/refresh mode, incoming reuse eligibility, or whether a module takes -a legitimate no-model branch. Cold misses also never use the resulting index. - -For `J` such operations against `N` source units, this adds `Theta(JN)` index -construction work. With a fixed number of lanes and chunks proportional to -transcript length, that cache-specific term becomes quadratic in `N`. -Concurrent operations can retain `O(min(J,W)N)` cache index entries while waiting -for model work, where `W` is the active worker bound. This is separate from -codec decoding and domain/validator input ownership costs. - -### Potential Refactoring - -Supply the cache decoders with one run-scoped, immutable source-position index. -Acquire it lazily only when a decoded hit actually needs evidence checks, using -ordinary concurrency-safe initialization. The cache-specific work becomes -`O(N + R)` for all hits, where `R` is the total number of checked cached source -references, with `O(N)` shared index space. Runs with no such checks need not -construct the cache index at all. - -This is a small in-memory collaborator, not another persistent cache or source -identity framework. Keep the native message-content key, modes, hit validation, -fallback eligibility, and no-semantic-reassessment policy. Finding 3's scene -identity adaptation is a separate usability concern; shared indexing must not -replace it. Other intentional domain/validator snapshots need not be weakened. - -### Preserve And Verify - -Retain wrong-source/missing/reversed citation rejection on hits, normal misses, -refresh/bypass/ineligible/no-model behavior, and canceled-hit delivery. Exercise -concurrent hits with the existing cache/runner tests and race checks. Do not add -a public index-build-count contract or a timing-based release gate. - -## Finding 8: Index Debug Sensitivity Paths Once - -**Status: Complete.** A single historical-path traversal groups marked suffixes -by exact native resource prefix before resource association. Capture regressions -and race checks pass, including mixed root/collection resources, escaped keys, -nested arrays, and native snapshot ownership. - -Priority: P3. Relevant stages: 30–31 and 33. - -### Evidence And Complexity - -[`newRunEvidenceRecorder`](../../internal/cli/run_capture.go), lines 64–91, -collects all `H` historical configuration paths, then scans that full list for -each of `P` profiles/backends across root and collection resources. Prefix -matching therefore performs `O(PH)` work, plus path-string processing. With a -roughly fixed number of fields per resource, `H` grows with `P`, yielding a -quadratic resource-count term. - -The caller correctly constructs this recorder only when capture is enabled. -This finding concerns debug startup for large native configurations, not an -unconditional cost on ordinary runs. - -### Potential Refactoring - -Walk metadata paths once and group relevant sensitive suffixes by their exact -resource prefix. Then associate those groups with root/qualified resource IDs in -one resource traversal. This takes `O(H + P)` path/resource visits, plus processing -the path characters and retained masks, rather than rescanning every unrelated -configuration path for every resource. - -Retain native sensitivity classification, correct JSON-pointer segment unescaping, -root/collection distinctions, nested maps/arrays, and nonmutation of native -snapshots. Do not replace exact metadata with credential-like string searches or -broaden omission policy. Use the existing configuration/native capture tests; -add a mixed-resource case only if needed to verify equivalent mask routing. - -## Finding 9: Traverse Dense Chunk Results Without Sorting Their Keys - -**Status: Complete.** Lane assembly traverses the materializer's dense ordinals -directly, retaining artifact/diagnostic order, fallback counts, and reuse -eligibility without allocating or sorting a temporary key slice. Pipeline -concurrency/ready-job tests and race checks pass. - -Priority: P3. Relevant stages: 11, 25, and 37. - -### Evidence And Complexity - -[`finalizeLaneExtract`](../../internal/framework/pipeline/runner_concurrent.go), -lines 333–349, copies result-map keys to a new slice and sorts them before -assembling the lane. For `C` chunks this costs `O(C log C)` comparisons and -`O(C)` temporary key storage per lane. - -The runner's chunks are already assigned contiguous zero-based indexes by -`MaterializeChunkPlan`; successful finalization occurs only after all of that -lane's jobs return. The required order is therefore known, not arbitrary. - -### Potential Refactoring And Verification - -Traverse the known ordinal range directly for `O(C)` expected map-lookup work -and no temporary key slice, or use an indexed result slice if it simplifies -storage. Keep all artifact/diagnostic ordering, fallback counts, eligibility, -failure handling, cancellation draining, and ready-job scheduling unchanged. -No new chunker restriction is required: dense ordinals are already produced by -the existing materializer, regardless of transcript unit-ID spacing. - -The current runner concurrency and ready-job tests cover out-of-order completion -and public result order. Reuse those checks; do not couple tests to a particular -map/slice representation or add another scheduler. - -## Review Coverage And Verification - -Owning boundaries were inspected in implementation order: - -| Stages | Reviewed boundary | -| --- | --- | -| 1–7 | Declared dependencies/notices, native resources/schemas, one ConfigKit result and engine, CommandKit/ckconfig/ckprompt construction and supplied values. | -| 8–11 | Typed production/assessment, full ordered validator chains, native continuation and terminal disposition, runner handoffs and eligibility. | -| 12–24 | All ten D&D extractors, three semantic registry normalizers, scene planning, direct combat-semantic validation, deterministic paths and registration. | -| 25–28 | Ready-job workers, native admission, accepted-artifact cache, prepared handles/initial keys, current-source cache usability and chunk-plan materialization. | -| 29–34 | Domain-only publication, incremental requested capture, offline preflight/inspection, run composition, native reports and failure outcomes. | -| 35–37 | Removal inventory, maintained examples, assembled recovery and subprocess coverage. | -| 38–46 | Successor ADRs and historical-body preservation, canonical references, operations/consumer/migration/internal/integration guides, orientation and final acceptance evidence. | - -Checks completed during the initial review: - -- `GOWORK=off GOPROXY=off go test ./...` passed. -- `GOWORK=off GOPROXY=off go vet ./...` passed. -- A fresh `GOWORK=off GOPROXY=off go build` to a temporary binary passed. -- `GOWORK=off GOPROXY=off go test -race ./internal/framework/pipeline - ./internal/cli` passed. -- Offline executable checks passed for version, minimal/complete pipeline - preflight, and native prompt requirements; native diff check returned its - expected differences status (3). -- The migration diff contains no changes to model-facing JSON schemas or D&D - intrinsic codec schema files. Historical ADR decision bodies are retained; - supersession edits are confined to status descriptions. -- Temporary Go-overlay probes reproduced Findings 1–2 without modifying - repository code or tests. Their deliberately failing assertions are review - evidence, separate from the passing default suite. - -The integration follow-up inspected the stage boundaries above against the -retained source/chunk identity, domain codecs/validators, generated references, -diagnostic limits, physical publication, and subprocess contracts. It reused -the earlier full-suite/vet/build/race results only after confirming that the -application code and dependencies were unchanged. Additional checks were: - -- `GOWORK=off GOPROXY=off go test ./internal/core/config - ./internal/framework/pipeline ./internal/framework/semanticreconcile - ./internal/modules/integration ./internal/cli` passed. -- Real native-CLI/fake-provider Go-overlay probes reproduced both directions of - scene repartitioning in Finding 3. Both corresponding bypass controls passed. -- A native-CLI/fake-provider overlay probe reproduced the long-error permissive - fallback failure in Finding 4. -- Graph/code searches confirmed that Finding 5's legacy taxonomy has no active - callers outside its own definition/self-test. - -These probes live only in a temporary review directory. No production code, -test, configuration, schema, or existing canonical documentation was changed. - -The algorithmic pass followed the same stage order, inspecting actual loop/call -costs rather than treating nested-loop graph metadata as proof of a bottleneck: - -- Stages 1–7: native loading/delegation, bounded invocation-selection reuse, - resource/input construction, and absence of a second application load. -- Stages 8–11: candidate serialization/isolation, complete-chain traversal, - native helper control flow, and runner/handoff state. -- Stages 12–24: fixed extraction mappings, bounded reconciliation preparation, - interval coalescing, indexed duplicate/group application, direct validators, - deterministic paths, and the source/materialization costs in Finding 6. -- Stages 25–29: constant-work ready-job dispatch, result/error ordering, - filesystem byte traversal, source-index lifetime, and reuse/auxiliary assembly. -- Stages 30–34: incremental capture, sensitivity-path routing, preflight, - command composition, diagnostics, and publication. -- Stages 35–46: retained/removal boundaries, example/assembled coverage, and - documentation consistency with the algorithms and selected policies above. - -Explicitly selected behavior was not reopened: complete validation on every -candidate, fresh normalization application to fixed originals, native growing -history, content-only cache identity, no semantic reassessment on hits, the ready- -job scheduling policy, and self-contained debug records even when content repeats. -The earlier expressly recommended per-handoff output index was also left alone. -Sorting genuinely unordered citations/intervals and collision-safe indexed domain -grouping are not findings merely because they contain loops or comparisons. - -Complexity claims in Findings 6–9 follow from the current source and call sites; -they are not throughput estimates. Position-map costs use ordinary expected -hash-map complexity; retained field/payload processing is accounted for separately. -The existing -offline suites for `internal/core/source`, `internal/framework/pipeline`, -`internal/framework/artifactcache`, `internal/framework/semanticreconcile`, and -`internal/cli` passed again. No production workload or performance data was needed, -and no new benchmark, timing assertion, or permanent gate was added. - -No live provider inference was performed. This review does not establish -production model quality or performance. Intentional changes such as the hard -configuration/CLI cutover, native retry ownership, content-only cache identity, -no semantic reassessment on hits, explicit permissive fallback, and native -diagnostic visibility are part of the agreed target, not regressions. - -## Remediation Completion - -All nine findings are addressed in focused commits, with completion markers at -their owning findings above. Final verification after the code changes passed: - -- `GOWORK=off GOPROXY=off go test ./...`. -- `GOWORK=off GOPROXY=off go vet ./...`. -- A fresh `GOWORK=off GOPROXY=off go build` to a temporary binary. -- Race tests for `internal/framework/pipeline`, `internal/framework/chunkmap`, - `internal/core/source`, `internal/cli`, and - `internal/modules/dnd/extract/scenedescriptions`. -- Executable `version`, offline pipeline validation of both maintained D&D - examples, and native spell prompt-requirements inspection. -- `git diff --check`, this document's local link destinations, and the affected - configuration-reference anchors. - -The scene-cache regression covers both repartitioning directions, bypass -controls, current IDs, actual reuse without reassessment, and unusable hits. -The fallback regression exercises a real typed spell pipeline with a fake long -provider failure; focused checks retain full causes and module diagnostics. -Source/cache/capture regressions cover nonmonotonic unit order, concurrent hits, -and exact sensitivity routing without native snapshot mutation. No production -inference, new permanent validation gate, or performance guarantee was added.