diff --git a/docs/roadmap/commandkit-notepad.md b/docs/roadmap/commandkit-notepad.md deleted file mode 100644 index d06e37b..0000000 --- a/docs/roadmap/commandkit-notepad.md +++ /dev/null @@ -1,767 +0,0 @@ -# CommandKit migration notepad - -Status: consolidated into the [CommandKit adoption roadmap](commandkit.md). -Started 2026-11-19. - -Narratio will become a CommandKit-native application. This notepad collects the -agreed direction and discussion history. The roadmap owns the consolidated -target and required work; this notepad is not an implementation plan or a -description of current Narratio behavior. - -## Agreed direction - -### Full native cutover - -Adopt CommandKit's public-facing conventions and recommendations to the fullest -extent. Backward compatibility with Narratio's existing CLI conventions is a -non-goal. Provide migration guidance in release documentation so operators can -update their commands and scripts. - -Use native command declarations, typed inputs, routing, help, version, -reporting, and outcome handling. Adopt upstream parsing and presentation -semantics rather than building compatibility parsers, legacy output modes, or -status translations to reproduce the existing CLI. - -Consequently, differences in repeated-flag handling, option placement, help, -version output, configuration comparison syntax, and exit classification are -migration topics to explain, not legacy behavior to preserve. Existing command -names and shortcuts may survive when they remain useful domain operations; -their historical existence alone does not require retaining them. - -### Modules and ownership - -Plan adoption of CommandKit core and its ConfigKit integration, `ckconfig`. -Use the standard inspection capabilities where they apply to Narratio's -configuration. Generated help and shell completion belong in the direction -for the cutover. - -ConfigKit owns configuration policy and the lifecycle through the final, -fully resolved configuration handoff. Narratio supplies its configuration -schema, domain defaults, validation, and genuine configuration-resolution -rules through ConfigKit's supported extension points. Those callbacks operate -within ConfigKit's lifecycle and provenance model. - -Narratio owns interpreting the handed-off configuration and using it for -pipeline orchestration, artifact selection, manifests, resume, publishing, -restore, and cleanup. CommandKit owns CLI mechanics and presentation; -`ckconfig` exposes ConfigKit's inspection features through native adapters. -Narratio provides neutral command declarations, selection, and acquisition -wiring rather than an intervening configuration or inspection policy layer. - -Keep `ckprompt` and direct PromptKit engine ownership outside this migration. -Narratio delegates prompt execution to Scriptorium. Prompt-requirements -inspection and early pipeline dependency checks remain separate feature work -using an eventual Scriptorium external contract. CommandKit adoption can -proceed independently. - -### Configuration handoff and domain interpretation - -The completed ConfigKit result is authoritative configuration. Consume it -without post-handoff mutation. Any genuine configuration defaulting, -normalization, or resolution must complete within ConfigKit's supported -lifecycle before handoff, with native evidence. If a required capability is -missing, identify the specific upstream gap instead of implementing a second -configuration system in Narratio. - -Domain interpretation produces separate domain objects. For example, Narratio -can interpret artifact-family declarations and a party roster to produce a -concrete artifact catalog, execution plan, and publication actions. It must -not write those products back into the resolved configuration or present them -as a replacement effective configuration. Domain derivation remains Narratio's -responsibility; it does not need to be moved into configuration callbacks merely -to appear in standard inspection output. - -Standard inspection uses the genuine ConfigKit result through native -`ckconfig` adapters. Do not construct a separate effective inspection document, -combine it with another result's authoring evidence, or add custom path or -provenance reconciliation. Native inspection and override semantics are -upstream contracts that Narratio exposes neutrally. - -### Command organization - -Use one general execution command and command families for artifacts, session -operations, and configuration: - -```text -narratio -├── run SESSION -├── artifacts -│ ├── list -│ └── regenerate SESSION -├── session -│ ├── init SESSION -│ ├── plan SESSION -│ ├── validate SESSION -│ ├── status SESSION -│ ├── restore SESSION -│ ├── clean [SESSION | --all] -│ ├── artifacts SESSION -│ └── locks -│ ├── list SESSION -│ ├── add SESSION SOURCE -│ └── remove SESSION SOURCE -├── config -│ ├── scopes -│ ├── validate -│ ├── show -│ ├── sources -│ ├── explain -│ └── diff -├── version -├── completion -└── help -``` - -`run SESSION` selects the full pipeline by default. `--from` and `--through` -select an inclusive contiguous range. Add `--stage` for one stage, mutually -exclusive with the range selectors; it selects the same stage as both endpoints. -`--force` independently requests forced execution. `session plan` accepts the -same execution-selection options, including stage/range, force, and artifacts. - -Use positional session IDs on session-oriented commands and remove their -duplicate `--session-id` spelling. Campaign and configuration selection remain -flags, with spelling defined below. - -Remove `run-stage`, `analyze`, and `publish`. Single-stage execution uses -`run --stage`; forced regeneration adds `--force` explicitly. Move top-level -`clean` to `session clean`. Use the explicit `session locks list` operation so -the bare locks group displays native help. - -Use `artifacts regenerate SESSION` for forced extraction through analysis -without publication, sharing execution logic with `run`. It regenerates the -complete configured extraction bundle. `--artifacts` selects analysis targets -and their required prerequisites, not extraction lanes. - -Use `artifacts list` for configured artifact definitions, producers, and -dependencies, including generated family members from the selected pipeline, -campaign, and party context. Bare `artifacts` displays native group help. -`session artifacts SESSION` retains session-specific availability and -publication reporting. - -Narratio owns the `artifacts` family; `ckconfig` owns the standard `config` -family. Replace the proposed `config artifacts` and top-level -`regenerate-artifacts` spellings without compatibility aliases. - -### Configuration scopes and views - -Expose four scopes corresponding to Narratio's configuration domains. Use `app` -as the native default scope. - -| Scope | Native ConfigKit configuration result | -| --- | --- | -| `app` | Resolved application/pipeline configuration | -| `campaign` | The selected campaign's settings | -| `session` | The selected session's settings, with applicable configuration defaults resolved within ConfigKit's lifecycle | -| `party` | The party configuration selected by `--party-file` or the campaign default | - -Expose standard `show`, `diff`, `sources`, `explain`, and `validate` behavior -through native `ckconfig` adapters for each scope's ConfigKit result. Effective -values, provenance, history, validation findings, and path behavior come from -that result and the upstream integration. Narratio supplies domain validation -through the supported ConfigKit lifecycle, not a separate inspection policy. - -Remove the earlier requirement that app show/diff include expanded artifacts -and generated publish rules. They display and compare whatever the genuine -ConfigKit result contains. Domain products derived after handoff are not added -to standard inspection, and differences between those products are not added -to native configuration comparisons. A separate raw-versus-resolved mode is -not part of the design. - -Use `artifacts list` as a Narratio-owned domain catalog of concrete artifact -definitions and dependencies derived from the handed-off configuration and -selected party. Planning commands describe derived execution and publication -work. These domain views do not redefine ConfigKit's effective configuration. -`session artifacts SESSION` reports session-specific availability and -publication state. Future input discovery can extend the domain catalog -separately from this migration. - -Acquire only the context needed for the selected scope's native configuration -load. Applicable campaign defaults for session configuration resolve before -handoff. Standard app inspection does not acquire campaign/party context solely -to expand runtime artifacts. Domain catalog and planning operations acquire -the additional context needed for domain work. Campaign inspection does not -require a session. Scope discovery and help do not load configuration. Report -missing required context explicitly rather than presenting an incomplete load -as a complete configuration result. - -Retain CommandKit's operation-specific `--scope all` behavior. Do not silently -omit a selected scope whose required context is missing. - -Comparisons use independently resolved left and right selections through native -selectors and their `--against-*` counterparts. Permit comparisons across -overlays, configuration files, and campaigns; the existing same-campaign -restriction is not retained. - -### Session acquisition and selectors - -Inspection and execution use the same session-selection rules. Load an explicit -session file exactly when supplied; otherwise use normal local discovery, then -configured remote discovery when no local session exists and its identity is -known. Missing or invalid explicit files are errors. Invalid content, access -failures, and identity mismatches do not trigger fallback to another source. - -Remote discovery prefers the authored session-root configuration and falls back -to the committed prepared snapshot only when the authored object is absent. -Configuration inspection permits these network reads without requiring a -separate remote opt-in. It creates no local runtime state: no workspace -initialization, restored files, execution locks, or manifest changes. Source -inspection identifies local, remote-authored, and committed-snapshot origins. -Forced-local or forced-remote selection is not part of the initial design. - -| Selector | Meaning | -| --- | --- | -| `--config PATH` | Pipeline configuration root for every scope | -| `--overlay NAME` | Pipeline overlay | -| `--campaign ID` | Campaign identity | -| `--campaign-file PATH` | Explicit campaign configuration file | -| `--session ID` | Session identity on configuration inspection commands | -| `--session-file PATH` | Explicit session configuration file | -| `--party-file PATH` | Explicit party roster file in place of the campaign default | -| `--secrets-dir PATH` | Credential-bootstrap directory selection | - -Changing scope does not change the meaning of `--config`. Session-oriented -commands retain their positional session identity and can also select an -explicit source using `--session-file`. - -Identity and file selectors are complementary: the file selects the source and -an accompanying ID asserts the intended identity. Check that assertion against -the effective configuration after loading and resolution; mismatches fail -clearly. -For inspection, the file can supply identity when no identity selector is -given; session-oriented commands retain their required positional identity. - -The campaign supplies the default party file. `--party-file` selects an alternate -roster for the invocation without editing the campaign or its usual roster. -Resolve CLI-relative paths from the invocation directory and load an explicit -file exactly; a missing or invalid selected file fails rather than falling back -to the campaign default. Apply the same selection throughout inspection, -players derivation, artifact expansion, planning, and execution. Source evidence -identifies the selected roster and its imports. - -Comparisons expose `--against-campaign`, `--against-campaign-file`, -`--against-session`, `--against-session-file`, and `--against-party-file` -alongside CommandKit's native right-side configuration selectors. Omitted -right-side selectors use normal defaults rather than inheriting left-side -values, including each side's campaign-default party file. - -### Configuration inspection capabilities - -Enable the full applicable native inspection capabilities across the four -configuration scopes: - -- path-selected show using JSON Pointer syntax; -- explanation and path discovery, including available history and provenance; -- detailed validation findings with native severity and code filters; -- detailed comparisons with before/after values and presence information; and -- native `diff --check`, including status `3` when differences exist. - -These capabilities use upstream operations and evidence from the scope's native -result. Filtering findings changes presentation, not validation success. -ConfigKit owns the relationship between effective, historical, explainable, -and overridable paths; Narratio does not add a writable-path registry, -reinterpret explicitness, or manufacture provenance for domain products. - -### Native inspection overrides; execution overrides deferred - -Enable native `ckconfig` field overrides only for the `app` inspection scope. -Campaign, session, and party scopes retain their inspection capabilities; -alternate inputs use `--campaign-file`, `--session-file`, and `--party-file`. - -Defer field overrides on Narratio's execution and planning commands until -CommandKit provides a reusable binding. Do not build a local assignment parser, -flag binding, or equivalent machinery. Operators use configuration files and -overlays to select execution settings. This optional convenience is not a -dependency of the migration. - -Within standard inspection, `ckconfig` supplies the `POINTER=JSON` binding and -ConfigKit applies its native override mechanism. Do not introduce additional -protected-field lists, custom merge semantics, or an override acceptance policy. - -Use these flag spellings: - -| Flag | Target | -| --- | --- | -| `--set POINTER=JSON` | App/pipeline configuration on native app inspection only | -| `--against-set POINTER=JSON` | Right-side app/pipeline configuration on app comparisons | - -Declare native override support only for `app`. Other inspection scopes do not -accept `--set`; native `--scope all` also rejects overrides because not every -selected scope supports them. Aggregate inspection without overrides remains -available as supported by each operation. Do not add scope-specific `--set-*` -or `--against-set-*` aliases, cross-scope override dispatch, or scope prefixes -inside JSON Pointers. - -The additional convenience of modifying a discovered input without preparing a -file does not justify field overrides for the other domains in this migration. -File selection can supply the same effective domain values, although its -authoring and provenance differ from late CLI overrides. This decision removes -the need to compose overrides across supporting domains or reconcile generic -and explicitly targeted override flags. - -ConfigKit applies overrides after typed defaults and decoding, before its -resolution and semantic validation callbacks. Preserve native explicit empty, -false, zero, and null values, mapping merge and whole-sequence replacement, and -duplicate/overlapping-path rejection. Earlier parsing and decoding failures -remain errors. CommandKit's last-value convention for singleton flags does not -replace ConfigKit's conflict rules for override records. - -Override paths and acceptance follow ConfigKit's native input semantics for -application configuration. Standard inspection does not promise that every -displayed path accepts an override. Narratio passes explicit assignments to -ConfigKit without translating runtime catalog paths into input paths or -patching domain products after handoff. - -Inspection overrides leave source files unchanged and do not carry into later -planning or execution. Run metadata and prepared configuration record settings -selected through files and overlays using established secret-safe -representations. Reuse remains based on semantic evidence. - -### Identity assertions and credential overrides - -Narratio checks the effective campaign/session identity against any explicitly -asserted identity after loading and resolution so an operation targeting one -session cannot silently use another session's workspace, lock, manifest, or -remote destination. This is an ordinary domain consistency check, independent -of the selected source. Alternate campaign and session values come from file -selection rather than field overrides. Native app inspection overrides retain -upstream acceptance without a protected-field list. - -When no identity is asserted, such as inspection of an explicit session file -without `--session`, the effective document supplies its identity. -Session-oriented commands retain their required positional assertion. - -Credential-related app fields accept native inspection overrides and use -ConfigKit's ordinary typed-secret handling and normal configuration validation. Safe -inspection, diagnostics, and persistence remain separate output -responsibilities; they do not justify additional restrictions on which app -fields may be overridden. - -### Reports and machine-readable output - -Use CommandKit's native reporting system throughout. Produce one structured -result per invocation where the command has a meaningful result, keeping -progress and diagnostics separate. Standard ckconfig commands retain their -native report shapes; Narratio defines report values for its domain operations. - -| Command family | Report content | -| --- | --- | -| `run`, `artifacts regenerate` | Campaign/session identity, selected stages, execution or reuse outcomes, generated artifacts, and completion status | -| `session plan` | Selected stages, planned execution/reuse decisions and reasons, artifact targets, and prerequisites | -| `session status` | Local stage state and available remote publication information | -| Artifact catalogs | Source identities, producers, dependencies, and session-specific availability/publication where applicable | -| `session validate` | Overall outcome and individual findings | -| `session restore`, `session clean` | Planned or completed actions, conflicts, and dry-run status | -| Session initialization and lock mutations | Concise confirmation identifying the affected session, destination, or lock | - -Support native `--format text|yaml|json` conventions. Start with native automatic -presentation, including YAML for structured values. Add a custom text template -only when representative output demonstrates a substantial readability benefit. -The template presents the same report value rather than a separate information -model; recreating the existing handwritten text output is not a goal. - -Long-running operations emit progress through the diagnostic channel and their -final report through the result channel, allowing JSON results to be captured -without progress mixed into the document. A streaming JSON event protocol is -deferred as a separate interface. - -When execution stops after meaningful domain work, report what completed and -where work stopped alongside the failed command outcome. Preserve the success -of earlier durable work. Failures before meaningful domain work, such as invalid -arguments, need only the native error; do not manufacture an empty run report. -Report-delivery failure follows CommandKit's native handling and does not undo -completed pipeline work. - -Define public report values separately from internal structs and durable -manifests. Expose information useful to operators and callers without making -the entire internal state or manifest schema the CLI contract. Document -application-owned report fields as a public interface. An additional schema -version mechanism is not part of the initial design; review native report -changes during dependency upgrades. - -`--format` controls the command report, not the format of generated artifacts. -For example, session initialization still writes its YAML configuration when -the confirmation report uses JSON. - -### Diagnostics and invocation lifecycle - -Adopt CommandKit's native visibility rules. Ordinary mode shows results and -failures; warnings and information require verbose/debug mode. Progress requires -`--progress` or debug mode. Quiet suppresses successful results and progress -while retaining failures. Conditions that materially affect the result also -belong in its structured report rather than depending solely on warning -visibility. - -Classify Narratio's messages by their purpose: - -| Category | Examples | -| --- | --- | -| Result | Final execution summary, plan, status, or configuration report | -| Failure | Failed stage, invalid configuration, or unavailable required input | -| Warning | Recoverable problem worth investigating | -| Information | Configuration selection or successful credential-bootstrap summary | -| Progress | Stage started/completed, artifact completed, or prior work reused | -| Debug | Additional safe acquisition or execution detail | - -CommandKit owns console presentation: results go to stdout and diagnostics and -progress go to stderr. Domain operations return results and errors without -independently printing duplicate terminal failures at several layers. Preserve -downstream stdout/stderr in existing captured diagnostic logs; console output -provides concise context and useful log locations instead of automatically -replaying downstream output. - -Durable logs and checkpoints retain their own policies. Quiet does not disable -logging or persistence. Debug increases diagnostic visibility without -automatically enabling downstream prompt-rendering/debug artifacts. Reporting -modes do not change pipeline selection, execution, or retention. - -Keep startup lightweight and acquire dependencies only when needed: - -1. Parse the command and validate syntax and input combinations. -2. Bootstrap credentials when the operation needs configuration. -3. Load the required configuration through ConfigKit's completed lifecycle, - then derive the domain context needed by the operation. -4. Acquire execution resources and perform the operation. -5. Finalize domain state, release resources, present the result, and return the - outcome. - -Help, version, completion, and scope discovery do not bootstrap secrets, load -configuration, contact storage, or initialize stage adapters. Configuration- -dependent validation happens after loading; syntactic errors fail before it. -Session inspection may perform the already-agreed remote reads when needed. - -The process boundary supplies an invocation context canceled by Ctrl-C or -termination. Propagate that context through CommandKit and Narratio operations. -On cancellation, stop scheduling further work, cancel active operations, and -let existing adapters perform their resource/subprocess shutdown. Attempt the -appropriate terminal manifest update and release held locks, preserve completed -checkpoints, and report meaningful partial results. Report persistence or -shutdown errors alongside the originating cancellation. Cancellation does not -invoke the pipeline cleanup stage or delete resumable work. Abrupt process -death is not guaranteed to permit finalization. - -Use native outcome classification: success is `0`; operational failure, -cancellation, and output failure use `1`; invalid invocation or identifiable -configuration input uses `2`; `diff --check` uses `3` when differences exist. -Use public configuration-error classification support rather than matching -error strings. Failure explanations identify the affected operation and useful -next steps while retaining underlying error identity for programmatic access. - -Include native completion for command paths, flags, static stage/scope values, -and appropriate file/directory selectors. Dynamic campaign and artifact -suggestions requiring configuration loading or storage access remain deferred. - -### Delivery and migration guidance - -Deliver one complete native CLI cutover through a single end-to-end -implementation sprint. Optimize the work sequence and commits for efficiently -reaching the intended final state. Intermediate commits may leave the overall -application incomplete, incoherent, or noncompiling; they are not independently -shippable units and do not require standalone application validation. Do not -introduce temporary compatibility machinery merely to keep each intermediate -commit working. Report intermediate status accurately, use focused checks when -useful, and require a coherent, compilable, fully validated final application. - -Useful work boundaries are core invocation/domain commands; configuration -handoff, native scopes and inspection overrides, and file selection; -reports, diagnostics, cancellation, and completion; and integration -verification, removal of superseded CLI machinery, and documentation. These -are organizational boundaries, not mandatory buildable milestones or a -supported dual interface. The subsequent implementation plan determines the -most efficient order and dependencies. - -Use CommandKit v0.7.0 and ckconfig v0.9.0 as the initial roadmap baseline. -Review relevant public contract changes before deliberately selecting a later -release. Before finalizing implementation details, verify native ConfigKit -loader and `ckconfig` adapter integration and composition of independently owned -domain and standard configuration command families. Address required upstream -capability gaps upstream rather than compensating with custom effective views, -provenance reconciliation, or replacement configuration behavior. - -Remove the current post-load configuration mutation and the split between -runtime inspection snapshots and authoring evidence. Separate domain products -from the authoritative configuration handoff while preserving their intended -processing behavior. Do not retain those inspection projections merely to -reproduce existing output. Safe persisted configuration and domain artifacts -remain useful records, but do not replace the result used by native inspection. - -Retain Narratio's stage sequencing, resume authority, publication semantics, -restore safety, and cleanup rules. Changes required for the agreed ownership -boundary, inspection, reporting, and cancellation behavior are in scope. -Workspace reorganization, downstream-tool adoption, and Scriptorium requirements -inspection remain separate work. - -Validate the integration through representative behavioral acceptance cases: - -- Native arguments and selectors reach the intended Narratio operation. -- Standard configuration commands expose genuine ConfigKit results through - native adapters, with upstream value, provenance, and path semantics. -- Domain catalog and plan generation consume handed-off configuration without - modifying it or replacing the native inspection result. -- App inspection, planning, and execution use the same configuration loading - semantics for equivalent file and overlay selections. Inspection overrides - apply only to the inspection invocation. -- Explicit party-file selection consistently feeds roster inspection, derived - players, artifact expansion, planning, and execution; comparison sides remain - independent. -- Field overrides are limited to native app inspection. Other scopes and - aggregate inspection retain native behavior when overrides are unsupported; - execution and planning commands expose no generic `--set` binding. -- Reports remain separate from diagnostics and preserve meaningful partial - results. -- Help and completion avoid configuration and service acquisition. -- Cancellation reaches active work, preserves completed checkpoints, and - permits terminal persistence and lock release. -- Existing consequential pipeline behavior remains protected. - -Use real CommandKit and ConfigKit with controlled external boundaries. Update -or retire tests enforcing superseded CLI conventions while retaining domain -behavior coverage. Final verification uses the repository's established checks -without requiring live services or model calls; do not duplicate upstream -generic parser or configuration tests. Dependency attribution is part of the -adoption work. - -Provide workflow-oriented release migration guidance: an old-to-new command -table plus explanations of removed shortcuts, identity/file selectors including -party-file selection, inspection-only app overrides, native flag repetition and -placement, native configuration inspection and its distinction from domain -catalogs and plans, configuration comparisons, structured output, diagnostic -visibility, exit classifications, and cancellation. Identify affected examples, -operator procedures, scripts, and downstream callers. -Update canonical documentation and maintained examples as implementation lands; -release notes summarize and link to those contracts rather than duplicate the -command reference. - -Select the Narratio release number under the release policy once the final -compatibility impact is known. No version or publication action is selected by -these planning decisions. Consolidate the agreed target behavior, boundaries, -migration impact, and acceptance criteria into a feature roadmap before preparing -the detailed implementation plan. - -## Reviewed upstream baseline - -The initial review covered these released modules: - -| Module | Version | Relevant capability | -| --- | --- | --- | -| `go.maximumdirect.net/commandkit` | v0.7.0 | Typed commands and inputs, presence tracking, aliases, help/version, completion, reporting, and outcomes | -| `go.maximumdirect.net/ckconfig` | v0.9.0 | Configuration scopes, validation, safe display, sources, explanation, comparison, and optional overrides | -| `go.maximumdirect.net/ckprompt` | v0.10.0 | PromptKit inspection and request tooling; reviewed for context, outside adoption scope | - -Core v0.7.0 and ckconfig v0.9.0 require Go 1.27.0; ckconfig selects ConfigKit -v1.2.1. Narratio already declares those Go and ConfigKit versions. CommandKit -identifies itself as alpha. The core and ckconfig versions above are the initial -roadmap baseline; revisit these references if that baseline deliberately changes. - -Upstream references: - -- [Consumer guide and module selection](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/d330d7637f2e7507eca931a899397222c82bc892/docs/consumers/guide.md) -- [Core CLI conventions](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/b57bfe176cff05a892e6cb07ebb5f350239e55d4/docs/cli.md) -- [ConfigKit integration public contracts](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/9962368f0c1ea1f6ad010638e94621091105d745/integrations/ckconfig/contracts.go) - -Narratio's existing [CLI reference](../cli.md) and -[configuration internals](../internal/configuration.md) provide the current -baseline. They remain current-behavior documents until the migration lands. - -## Open decisions - -No unresolved discussion decisions remain. Execution and planning field -overrides are deferred until CommandKit provides a reusable binding; Narratio -will not implement its own. Native app inspection overrides remain in scope. - -Consumer probes against the reviewed releases confirmed native inspection, -app-only override capabilities, independent comparisons, and lazy acquisition. -The inability to extend the generated `config` group is resolved by the -independent `artifacts` family. Detailed implementation sequencing remains -for the subsequent plan; release version selection remains deferred. - -The configuration ownership boundary is recorded in -[ADR 0003](../adr/0003-assign-configuration-ownership.md). The consolidated -[roadmap](commandkit.md) is ready for implementation planning. - -## Discussion and decision record - -### 2026-11-19 — Initial direction - -Accepted the direction of adopting core and ckconfig while keeping direct -PromptKit integration and Scriptorium requirements work separate. The initial -review identified command design, configuration views, reporting, and lifecycle -as the main discussion areas. - -The user established full CommandKit-native cutover as the guiding decision: -adopt upstream public conventions and recommendations to the fullest extent, -provide release migration guidance, and treat existing CLI compatibility as a -non-goal. This settles the earlier question of whether to preserve legacy CLI -conventions. - -### 2026-11-19 — Command organization agreed - -Accepted the proposed command tree, `run --stage` selector, positional session -IDs, shared selection syntax for planning, explicit lock listing, and movement -of cleanup under `session`. Removed the separate single-stage and implicitly -forced analysis/publication commands from the target CLI. Retained -`regenerate-artifacts` because it expresses a useful multi-stage regeneration -workflow. Configuration scopes and views are the next discussion topic. - -The later artifacts-family decision supersedes the regeneration and catalog -command locations recorded here. - -### 2026-11-19 — Configuration scopes and views agreed - -The later configuration-ownership revision supersedes the expanded app -inspection view recorded below. The four scopes remain, with standard -inspection exposing their native ConfigKit results. - -Accepted `app`, `campaign`, `session`, and `party` scopes, with `app` as default. -Show/diff exposes effective values and source/explanation operations use native -provenance. Concrete artifacts remain visible in the effective pipeline and the -separate configuration catalog; session artifact inspection owns availability. -Accepted context-dependent acquisition, explicit missing-context errors, native -aggregate-scope behavior, and independently resolved comparisons across -campaigns. Session acquisition policy and selector spelling are next. - -### 2026-11-19 — Session acquisition and selectors agreed - -Accepted shared inspection/execution discovery with automatic remote fallback, -read-only acquisition without local runtime state, and explicit source -provenance. Accepted distinct identity/file selectors, identity assertions when -both are supplied, campaign-owned party selection, and independent right-side -comparison selectors. Optional configuration capabilities and overrides are -the next discussion topic. - -The later override-scope revision supersedes campaign-only party selection by -adding an explicit party-file selector while retaining the campaign default. - -### 2026-11-19 — Configuration capabilities and overrides agreed - -The cross-domain override decision below is superseded by the later -override-scope revision. Inspection capabilities and native app override -semantics remain accepted. The subsequent execution-override deferral supersedes -the inspection/planning/execution parity requirement recorded here. - -Accepted full applicable inspection capabilities and native overrides across -all four configuration domains, with parity across inspection, planning, and -execution. Selected verb-first `--set-app`, `--set-campaign`, `--set-session`, -and `--set-party` spellings and corresponding `--against-set-*` comparison -flags, alongside native `--set` and `--against-set`. - -Clarified that overrides have no Narratio-specific protected fields or custom -merge semantics. Identity assertions are checked against effective values after -native overrides and resolution; they do not make identity fields immutable. -Credential overrides use native typed-secret handling. Safe output and -persistence do not impose additional override restrictions. - -### 2026-11-19 — Reporting agreed - -Accepted native structured reporting with one meaningful result document per -invocation, separate diagnostics/progress, and native ckconfig report shapes. -Custom text templates require a demonstrated readability benefit. Domain -reports describe partial execution on failure without undoing durable work; -early failures need no empty report. Report models remain separate from -manifests and generated artifact formats, with documented public fields and no -additional schema version mechanism initially. Streaming JSON events remain -deferred. Diagnostics and invocation lifecycle are the next discussion topic. - -### 2026-11-19 — Diagnostics and invocation lifecycle agreed - -Accepted native diagnostic categories, visibility, and outcome classification, -with CommandKit owning console presentation separately from durable logs. -Startup acquires only the resources needed by the selected operation. Process -signals cancel the invocation context, preserving completed checkpoints and -allowing ordinary terminal persistence and lock release without triggering -pipeline cleanup. Accepted static and filesystem completion, with dynamic -campaign/artifact suggestions deferred. Delivery and migration guidance remain -open. - -### 2026-11-19 — Delivery and migration agreed - -Accepted one complete native cutover, the reviewed release baseline, explicit -public API fit checks, domain-behavior preservation, focused integration -verification, and workflow-oriented migration guidance. Release version -selection and publication remain separate from planning. - -The user specified a single end-to-end implementation sprint. Intermediate -commits need not leave the overall application coherent or compilable; their -purpose is to support efficient progress toward the final state. Coherence, -compilation, and full validation are required at completion, without imposing -independently shippable or fully validated intermediate milestones. All -currently discussed decision areas are settled for roadmap consolidation. - -### 2026-11-19 — Override scope simplified and party-file selection added - -Revisited the need for field overrides outside application configuration. -Accepted native `--set` and `--against-set` only for `app`, retaining inspection -of all four scopes and using file selectors for alternate campaign, session, -and party inputs. Removed the proposed scope-specific override flag families -and their cross-domain composition requirements. Native aggregate inspection -rejects overrides when the selected scopes do not all support them. - -Added `--party-file` and independent `--against-party-file` selection. The -campaign continues to supply the default roster; an explicit file supplies the -invocation's roster consistently across inspection, derivation, planning, and -execution. - -No identified workflow requires non-app field overrides. Their convenience does -not justify the additional interface and integration machinery. The intent is -to meaningfully reduce complexity while retaining native app override behavior -and ordinary file-backed configuration composition. This revision supersedes -the earlier all-domain override and campaign-only party-selection decisions. - -The later execution-override decision limits field overrides to native app -inspection until a reusable upstream binding is available. - -### 2026-11-19 — Configuration ownership and native inspection clarified - -Accepted ConfigKit's completed result as authoritative configuration, with -Narratio's schema, defaults, validation, and genuine resolution rules supplied -through ConfigKit's supported lifecycle. Narratio consumes the handoff without -mutation and produces separate domain catalogs, plans, and execution objects. -CommandKit's `ckconfig` integration neutrally exposes the native ConfigKit -features; Narratio does not reinterpret or improve their inspection semantics. - -Superseded the requirement that app show/diff contain expanded artifacts and -generated publish rules. Standard inspection uses native results and adapters; -domain catalogs and planning commands expose the work derived from those -results. Remove post-load configuration mutation and separate inspection -snapshots rather than reconcile them with retained authoring evidence. Native -path and override semantics remain upstream contracts, not a Narratio design -problem. - -The user identified the configuration/inspection split as a boundary violation. -The agreed correction is to restore ownership, not add machinery to make -different representations appear consistent. A potential ADR recording this -boundary is the next discussion topic. - -### 2026-11-19 — Artifacts command family agreed - -Accepted `artifacts list` for the configured domain catalog and -`artifacts regenerate SESSION` for forced extraction through analysis. -Bare `artifacts` displays help. Session-specific availability and -publication remain under `session artifacts SESSION`. - -These commands replace `config artifacts` and top-level `regenerate-artifacts` -without aliases. Narratio owns the complete artifacts family, avoiding the -native configuration group's extension gap and keeping domain reports separate -from configuration inspection. The execution override binding remains the next -discussion topic. - -### 2026-11-19 — Execution overrides deferred - -Deferred field overrides on execution and planning commands until CommandKit -provides a reusable binding. Narratio will not build the proposed local binding -or other override machinery. Files and overlays provide execution settings; -native `ckconfig` overrides remain available for app inspection only. - -Execution overrides are an optional operator convenience, not a core workflow -requirement or a migration dependency. This settles the remaining binding -decision and supersedes the earlier override-parity requirement. - -### Continuing the discussion - -For each topic, record the question, the agreed outcome, its rationale, and any -remaining dependency or follow-up. Date decisions and update the corresponding -open topic so settled issues are not repeatedly reopened. Distinguish proposals -from accepted decisions. - -Once the relevant decisions are settled, consolidate the target behavior, scope, -compatibility impact, and acceptance criteria into the feature roadmap. Keep -implementation sequencing for the subsequent plan. diff --git a/docs/roadmap/commandkit.md b/docs/roadmap/commandkit.md deleted file mode 100644 index cef6df6..0000000 --- a/docs/roadmap/commandkit.md +++ /dev/null @@ -1,383 +0,0 @@ -# CommandKit Adoption - -Status: implemented; all stages in the [implementation plan](implementation.md) -are complete. - -This roadmap consolidates the [migration discussion](commandkit-notepad.md). -It defines the intended end state and required work. The -[implementation plan](implementation.md) defines the ordered implementation -stages. - -## Objective And Scope - -Make Narratio a CommandKit-native application, adopting upstream command, -configuration-inspection, reporting, and invocation conventions. Deliver one -complete cutover. Backward compatibility with the existing CLI is a non-goal; -release documentation will explain the migration. - -Adopt `go.maximumdirect.net/commandkit` v0.7.0 and -`go.maximumdirect.net/ckconfig` v0.9.0 as the initial baseline. They fit Narratio's -existing ConfigKit v1.2.1 and Go 1.27.0 minimum. Review relevant public contract -changes before deliberately selecting later releases. - -Preserve consequential pipeline behavior: canonical stage sequencing, semantic -resume evidence, publication boundaries, restore safety, and cleanup rules. -Changes needed for the agreed configuration ownership, CLI, reports, and -cancellation behavior are in scope. - -## Current Baseline - -The [CLI reference](../cli.md), [configuration internals](../internal/configuration.md), -and [internal overview](../internal/overview.md) describe the implementation. -The migration starts from these differences: - -- `internal/app` owns handwritten routing, flag parsing, help, output, and exit - handling. CommandKit and ckconfig are not dependencies yet. -- ConfigKit already loads configuration, but subsequent campaign/session - composition and artifact-family expansion modify runtime configuration. - Expansion adds concrete artifacts and publish rules to the pipeline object. -- Show and diff use a safe final-domain projection frozen as a separate native - result; provenance comes from retained authoring results. App inspection - acquires campaign/party context for expansion, and comparisons require the - same campaign. -- Session selection and operations exist, but their CLI wiring needs the agreed - identity/file distinction, native scopes, and explicit party-file selection. -- The command dispatcher starts with `context.Background()`. The runner releases - locks on failure, but stage failures return no run summary and bypass its - successful finalization path. Signal propagation, partial reporting, and - terminal persistence need coordinated changes. - -Consumer probes against the selected upstream releases confirmed native scope -inspection, app-only overrides, independent comparisons, and lazy acquisition. -Two upstream limits are already accommodated: the generated `config` group -cannot accept Narratio subcommands, and execution commands have no reusable -override binding. The design uses a separate `artifacts` family and defers -execution overrides. Neither limitation blocks adoption. - -## Target Behavior - -### Ownership And Configuration Handoff - -Follow [ADR 0003](../adr/0003-assign-configuration-ownership.md): ConfigKit owns -configuration through the completed handoff; Narratio consumes it without -mutation and derives domain objects; CommandKit and ckconfig own CLI mechanics -and native inspection. - -Supply schemas, defaults, configuration validation, and genuine resolution -rules through ConfigKit's supported lifecycle. Applicable campaign defaults for -session configuration complete there. Native inspection uses the actual result, -including its values, provenance, history, and validation evidence. - -Derive party/player projections, concrete artifact catalogs, execution plans, -and publication actions as separate domain products. Keep their validation in -the appropriate domain operation. Do not put these products back into the -configuration, manufacture inspection evidence for them, or move domain work -into configuration callbacks merely to include it in `config show`. - -Safe persisted configuration and domain records may continue serving preparation -and resume. They do not replace the native inspection result. Resume remains -based on result-affecting semantic evidence, not configuration provenance or a -blanket effective-digest cache key. - -### Command Organization - -```text -narratio -├── run SESSION -├── artifacts -│ ├── list -│ └── regenerate SESSION -├── session -│ ├── init SESSION -│ ├── plan SESSION -│ ├── validate SESSION -│ ├── status SESSION -│ ├── restore SESSION -│ ├── clean [SESSION | --all] -│ ├── artifacts SESSION -│ └── locks -│ ├── list SESSION -│ ├── add SESSION SOURCE -│ └── remove SESSION SOURCE -├── config -│ ├── scopes -│ ├── validate -│ ├── show -│ ├── sources -│ ├── explain -│ └── diff -├── version -├── completion -└── help -``` - -`run SESSION` selects the full pipeline by default. `--from` and `--through` -select an inclusive contiguous range. `--stage` selects one stage and is mutually -exclusive with range selectors. `--force` is independent. `session plan` accepts -the same stage/range, force, and artifact selections. - -`artifacts regenerate SESSION` forces extraction through analysis without -publication, sharing execution logic with `run`. It regenerates the complete -configured extraction bundle; `--artifacts` selects analysis targets and their -prerequisites, not extraction lanes. - -`artifacts list` reports configured definitions, producers, and dependencies, -including family members derived from the selected pipeline, campaign, and -party. `session artifacts SESSION` reports availability and publication state. -Planning exposes derived execution and publication work. These are domain views, -separate from native configuration inspection. - -Remove `run-stage`, `analyze`, and `publish`; single-stage work uses -`run --stage`, adding `--force` when requested. Move top-level `clean` to -`session clean` and `regenerate-artifacts` to `artifacts regenerate`. -Replace `config artifacts` with `artifacts list`. Use explicit `session locks list`. -Bare command groups display native help. Do not retain compatibility aliases or -duplicate `--session-id` flags for positional session arguments. - -### Configuration Scopes And Selection - -Expose four native ckconfig scopes, with `app` as the default: - -| Scope | Configuration inspected | -| --- | --- | -| `app` | Resolved application/pipeline configuration | -| `campaign` | Selected campaign settings | -| `session` | Selected session settings, including applicable configuration defaults | -| `party` | Party configuration selected explicitly or through the campaign default | - -Enable applicable native show, path selection/discovery, sources, explanation -and history, validation findings and filters, and detailed comparison. Filtering -findings does not change validation success. Retain native per-operation -`--scope all` behavior, reporting missing required context rather than silently -omitting scopes. There is no additional raw/resolved view or Narratio-owned -path/provenance reconciliation. - -Acquire only the context required by the selected scope. App inspection does -not acquire campaign/party context solely for artifact expansion. Campaign -inspection does not require a session. Catalog and planning commands acquire -the context needed for their domain work. - -| Selector | Meaning | -| --- | --- | -| `--config PATH` | Pipeline configuration root, regardless of scope | -| `--overlay NAME` | Pipeline overlay | -| `--campaign ID` | Campaign identity | -| `--campaign-file PATH` | Explicit campaign file | -| `--session ID` | Session identity for configuration inspection | -| `--session-file PATH` | Explicit session file | -| `--party-file PATH` | Explicit party file in place of the campaign default | -| `--secrets-dir PATH` | Credential-bootstrap directory | - -Session-oriented domain commands use positional session identity and may also -select `--session-file`. A file selects the source; an explicit identity asserts -the intended effective identity after resolution. Inspection may obtain identity -from the file when no identity is asserted. Mismatches fail clearly. - -Resolve explicit CLI-relative paths from the invocation directory. An explicit -party file supplies the roster consistently for inspection, players, artifact -expansion, planning, and execution; missing or invalid explicit files fail -without falling back to the campaign default. Native source evidence identifies -the selected file and its imports. - -Diff resolves both sides independently, permitting different files, overlays, -and campaigns. Add `--against-campaign`, `--against-campaign-file`, -`--against-session`, `--against-session-file`, and `--against-party-file` alongside -native `--against-config` and `--against-overlay`. Omitted right-side selectors -use normal defaults, including that side's campaign-default roster; they do not -inherit left-side values. - -### Session Acquisition - -Share selection rules between inspection and execution: load an explicit file -exactly; otherwise use normal local discovery, then remote discovery when no -local session exists and identity is known. Invalid content, access failures, -and identity mismatches do not trigger fallback. Remote discovery prefers the -authored session-root configuration, using the committed prepared snapshot only -when the authored object is absent. - -Inspection may perform these network reads without a separate opt-in. It does -not initialize a workspace, restore files, acquire execution locks, or change -manifests. Source inspection distinguishes local, remote-authored, and committed -snapshot origins. No forced-local or forced-remote selector is needed initially. - -### Overrides - -Enable native `--set POINTER=JSON` only for app inspection and `--against-set` -for its comparison side. ConfigKit owns assignment acceptance, ordering, merge, -conflict, path, and typed-secret semantics. Do not add protected-field lists, -translate domain catalog paths, or promise that every displayed path is -overridable. Normal configuration validation and secret-safe output still apply. - -Campaign, session, and party use file selectors rather than field overrides. -Native aggregate inspection rejects overrides because not every scope supports -them; aggregate operations without overrides retain upstream behavior. Do not -add scope-specific override flags or cross-scope dispatch. - -Inspection overrides affect that invocation only. Defer field overrides on -execution and planning until CommandKit supplies a reusable binding. Narratio -will not build its own parser, binding, or equivalent machinery. Files and -overlays select execution settings; this convenience is not an adoption -dependency. - -### Reports And Diagnostics - -Use native reporting and ckconfig report shapes. Domain operations return one -structured result where meaningful, with public report values separate from -internal state and manifests: - -| Operation | Useful report content | -| --- | --- | -| Run/regenerate | Identities, selected stages, execution/reuse, artifacts, completion | -| Plan | Execution/reuse decisions and reasons, targets, prerequisites | -| Status | Local stage state and available remote publication information | -| Artifact inspection | Source identities, producers, dependencies; availability/publication for session views | -| Session validation | Overall outcome and findings | -| Restore/clean | Planned/completed actions, conflicts, dry-run status | -| Initialization/lock mutations | Concise confirmation of the affected session, destination, or lock | - -Use native automatic presentation, including YAML for structured values, and -`--format text|yaml|json`. Add text templates only where representative output -shows a substantial readability benefit, presenting the same report data. -Document public report fields without adding a schema-version mechanism. -Report format does not change generated artifact formats, including session YAML. - -CommandKit owns console presentation: results on stdout, diagnostics/progress -on stderr. Ordinary mode shows results and failures; warnings/information require -verbose/debug; progress requires `--progress` or debug. Quiet suppresses successful -results and progress while preserving failures. Material conditions also appear -in the result, independent of diagnostic visibility. - -Emit Progress for stage transitions and useful finer-grained activity, including -download starts/completions, individual speaker and artifact completions, and -reuse of prior work. These events are available with `--progress` alone and use -native streamed, unprefixed lines. Info supplies context: selected sources, -destinations, configuration, and reasons for decisions. For example, downloading -a speaker's audio is Progress; identifying the selected audio source is Info. - -Keep the categories independent. `--verbose` enables informational messages and -warnings without enabling progress; `--progress` enables progress without -enabling informational messages or warnings. With both flags, stderr carries -both categories while stdout retains the final report. Do not introduce a -separate verbose-progress level or require both flags for finer-grained events. - -Keep downstream streams in existing captured logs and provide concise console -context and log locations without automatic replay. Quiet does not disable logs -or persistence; debug does not automatically enable downstream debug artifacts. -Visibility options do not change pipeline selection, execution, or retention. - -Failures after meaningful work return partial results alongside the failed -outcome, preserving earlier durable successes. Early input failures need no -empty report. Native report-delivery failures do not undo completed work. - -### Invocation Lifecycle - -Parse and check input combinations before credential bootstrap. Load required -configuration, derive domain context, then acquire execution resources. Finalize -state and release resources before completing the invocation. Configuration- -dependent checks occur after loading. Help, version, completion, and scope -discovery do not bootstrap secrets, load configuration, contact storage, or -initialize adapters. - -The process boundary supplies a context canceled by Ctrl-C or termination. -Propagate it to active operations and existing adapter shutdown paths; stop -scheduling further work. Attempt appropriate terminal manifest persistence and -lock release, preserve completed checkpoints, and report meaningful partial -results. Retain persistence or shutdown errors alongside the original failure. -Cancellation does not run the cleanup stage or delete resumable work. Abrupt -process death does not guarantee finalization. - -Use native outcomes: `0` success; `1` operational failure, cancellation, or output -failure; `2` invalid invocation or identifiable configuration input; `3` changed -`diff --check`. Use public error classification and retain underlying error -identity, avoiding error-string matching and duplicate terminal failures. - -Provide native completion for command paths, flags, static stage/scope choices, -and applicable file/directory selectors. - -## Required Work - -The required changes span these areas. - -1. **Replace the command boundary.** Add the selected dependencies and native - application/command declarations. Wire typed inputs to domain operations, - introduce the artifacts family and revised selectors, and remove superseded - routing, parsers, help, output, and shortcuts. Retain shared bounded execution - for run, planning, and regeneration where applicable. -2. **Correct the configuration handoff.** Review post-load composition: move - genuine configuration defaults/resolution into supported ConfigKit callbacks; - derive catalogs, player projections, and publication actions separately. - Adapt planning, stages, adapters, prepared records, and semantic evidence to - consume those products without mutating completed configuration. Remove - replacement inspection snapshots and their separate evidence path while - retaining records needed for preparation and resume. -3. **Integrate native scopes and acquisition.** Expose genuine results through - `ckconfig.FromConfigKit`, register the four scopes and their selectors, and - enable app-only inspection overrides. Share local/remote selection with - execution, add party-file selection, and remove the same-campaign diff - restriction. Keep each comparison side independent and acquisition lazy. -4. **Integrate presentation and lifecycle.** Define domain report values, route - diagnostics and finer-grained progress through CommandKit, and replace ad hoc - status handling. Propagate signal cancellation and adapt runner - failure/finalization paths to preserve partial results and attempt terminal - persistence. Add native completion. -5. **Complete integration and migration.** Update behavioral tests, remove - obsolete CLI machinery and expectations, review dependency attribution, and - update canonical documentation, examples, and affected scripts with the - implementation. Prepare workflow-oriented release migration guidance. - -## Verification And Completion - -Use real CommandKit and ConfigKit with controlled external boundaries. Cover -the integration risks without duplicating upstream parser/configuration tests: - -- Native commands and selectors invoke the intended operation; stage/range, - force, analysis-target selection, and complete-bundle regeneration retain - their agreed meanings. -- Native inspection uses genuine results and evidence. Catalog/plan derivation - does not mutate them. Equivalent files and overlays have the same loading - semantics across inspection, planning, and execution; inspection overrides - remain invocation-local and unavailable on execution commands. -- Required context, identity assertions, exact-file errors, local/remote - fallback, and source labels behave consistently without inspection creating - runtime state. Party selection reaches each consumer, and diff sides remain - independent. App-only and aggregate override behavior follows ckconfig. -- Structured results remain separate from diagnostics, retain meaningful partial - work, and use native outcomes. Representative finer-grained progress events - appear with `--progress` alone; informational context follows verbose mode, - and combining the flags exposes both without changing stdout reports. -- Lazy commands avoid configuration/services; cancellation reaches active work, - preserves checkpoints, and permits terminal persistence and lock release, - including their failure reporting. -- Existing consequential stage, resume, publication, restore, and cleanup - behavior remains protected. Safe inspection, diagnostics, and persistence - preserve the established secret-handling boundary. - -Update or retire tests enforcing replaced CLI conventions while retaining useful -domain coverage. Default verification remains offline and independent of live -services, real credentials, and paid model calls. Run the repository's established -final checks from [Development](../development.md#validation): tests, race-enabled -tests, vet, and build, including documentation and maintained-example checks. - -## Migration And Deferred Work - -Release guidance will include an old-to-new command table and explain identity -versus file selectors, party-file selection, inspection-only overrides, native -flag repetition/placement, inspection versus domain views, comparison defaults, -structured output, diagnostic visibility, exit classifications, and cancellation. -Update the [CLI](../cli.md), [configuration](../config.md), -[operations](../operations.md), relevant internal/integration documentation, -and [maintained examples](../../examples/README.md) when implementation lands. -Release notes summarize the impact and link to those canonical contracts. -Select the release number under [release policy](../policy/release.md) once the -final compatibility impact is known; this roadmap does not authorize publication. - -Execution/planning overrides await a reusable upstream binding. Dynamic -campaign/artifact completion and streaming JSON events are deferred. Direct -ckprompt/PromptKit ownership, downstream-tool adoption, workspace reorganization, -and Scriptorium-backed input discovery and early dependency checks remain -separate work. The catalog exposes Narratio's configured knowledge; this -migration does not promise to discover prompt requirements or prove that every -configured pipeline can complete. - -No significant design decisions remain open. Release selection remains separate -from implementation. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index fbc9141..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,895 +0,0 @@ -# CommandKit adoption implementation plan - -## Scope and authority - -Implement the agreed target in [CommandKit adoption](commandkit.md), following -[ADR 0003](../adr/0003-assign-configuration-ownership.md). This plan is for a -Luna coding agent implementing the stages in sequential order during one -end-to-end sprint. Each stage is one focused implementation prompt. - -The inspected baseline is commit `bcbfc0f`. Narratio already uses ConfigKit -v1.2.1 and declares Go 1.27.0. Its handwritten command boundary, post-load -configuration mutations, replacement inspection snapshots, and successful-only -run summaries are the migration inputs. - -Select these released modules directly: - -| Module | Version | Upstream source commit | -| --- | --- | --- | -| `go.maximumdirect.net/commandkit` | v0.7.0 | `b57bfe176cff05a892e6cb07ebb5f350239e55d4` | -| `go.maximumdirect.net/ckconfig` | v0.9.0 | `9962368f0c1ea1f6ad010638e94621091105d745` | -| `go.maximumdirect.net/configkit` | v1.2.1, retained | `51ef3144158db1a09fabdbd1c7ae59908c84923b` | - -Use the pinned [CommandKit consumer guide](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/b57bfe176cff05a892e6cb07ebb5f350239e55d4/docs/consumers/guide.md), -[CLI contract](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/b57bfe176cff05a892e6cb07ebb5f350239e55d4/docs/cli.md), -[ckconfig contracts](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/9962368f0c1ea1f6ad010638e94621091105d745/integrations/ckconfig/contracts.go), -and [ConfigKit consumer guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md) -as upstream authorities. The ckconfig release's own module metadata and -[release note](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/9962368f0c1ea1f6ad010638e94621091105d745/integrations/ckconfig/docs/releases/v0.9.0.md) -confirm the selected pairing; the earlier core guide's integration-version table -predates that coordinated release. Read the released public declarations when -details are needed. Sibling checkouts are optional reference material, not -build or test prerequisites. - -## Working rules - -- Start with [Development](../development.md), [Agent Policy](../policy/agents.md), - and the roadmap. Read applicable architecture, testing, documentation, OSS, - and release policies through the development guide. Reuse unchanged policies - already read. Prefer the codebase graph for code discovery; confirm current - source when indexed locations are stale. -- Intermediate stages may be incomplete, noncompiling, or have tests awaiting - later stages. Record that accurately. Do not build compatibility layers, - parallel dispatchers, replacement configuration views, or temporary no-op - operations merely to keep intermediate commits green. -- Commit each completed nonempty unit using a concise plain-English message. - Add focused tests with their owning behavior. Use real CommandKit and - ConfigKit; replace only external boundaries, clocks, randomness, and other - sources of nondeterminism where needed. Keep default tests offline and - independent of installed downstream applications, credentials, and paid APIs. -- Preserve consequential stage ordering, invalidation, checkpoint reuse, - publication, remote locks, restore confinement, workspace modes, and cleanup - behavior. This CLI cutover does not authorize workspace reorganization, - downstream adapter upgrades, session migration machinery, or new retention - rules. Keep the pending Audita and workspace roadmaps separate. -- Preserve existing operation-specific options unless the roadmap replaces - them. Adopt upstream flag placement, repetition, help, reporting, and outcome - behavior; retire tests enforcing superseded parser conventions. Avoid exact - formatting or error-wording promises unless the contract actually owns them. -- Use concise, affirmative documentation. Keep current contracts in their - canonical owners and proposed implementation status here. The migration - discussion is historical context; the consolidated roadmap owns decisions. - -## Implementation boundaries - -Use a small number of concrete data boundaries, adapting the existing owners: - -- **Loaded configuration:** retain actual typed ConfigKit results for pipeline, - campaign, session, and party. Complete configuration defaults, normalization, - and validation inside supported load callbacks. Consume completed values - without mutation; no defensive-copy framework or new public YAML schema is - required to express this ownership. -- **Derived domain context:** hold the selected roster, players projection, - concrete artifacts, family origins, resolved dependencies, and publication - actions separately. Existing `config.Config`, `stage.Env`, and - `artifacts.EffectiveArtifactSet` can carry or reference these products. - Keep CommandKit types at the application/process boundary. -- **Invocation selection:** retain explicit presence for identity, file, overlay, - and secrets selectors. Share acquisition between native inspection and domain - commands, with invocation-owned state and independent comparison inputs. -- **Operation results:** domain operations return useful values and errors; - application handlers present public report DTOs with CommandKit. Use stable - JSON field tags for YAML/JSON agreement and keep manifests, credentials, and - internal dependency containers out of public reports. - -Use native automatic report presentation, including YAML for structured values, -and `--format text|yaml|json`. Add a text template only when representative -output demonstrates a substantial readability benefit, using the same report -data. Report format remains independent of generated artifact formats. - -These are responsibility boundaries, not a requirement to create four new -packages or a generic command, configuration, or event framework. - -## Stage 1: Install the native application boundary - -**Completed.** Installed the pinned native modules, caller-context/stream entry -point, native help/version, supplied-value selector declarations, and outcome -classification with preserved causes. Native boundary tests, release-script -checks, documentation checks, and repository-wide compilation pass with Go -1.27.4. Domain command invocation tests await their scheduled native handlers; -the handwritten dispatcher is no longer an execution path. - -Read: roadmap command organization and invocation lifecycle; upstream consumer -guide sections on core commands and invocation reporting; `internal/app/commands.go`, -`command_dependencies.go`, `version.go`, and `cmd/narratio/main.go`. - -1. Add the selected CommandKit and ckconfig requirements, retaining ConfigKit - v1.2.1 and the existing Go minimum. Use released modules, without local - filesystem replacements or an upstream workspace dependency. -2. Establish the native application construction and execution entry point in - `internal/app`. Supply Narratio's existing build version through - `commandkit.Identity`; use native help and version. Accept a caller context - and caller-owned streams, and return the native outcome to the process - boundary. Update the version-checking consumer in release tests/scripts for - the native version representation when its implementation changes. -3. Establish typed selection inputs and reusable native flag declarations using - `Supplied` presence. Domain checks on combinations belong in `CheckInput` or - the appropriate post-load operation, not token scanning. Other stages add - the actual command handlers; no second legacy dispatch path is needed. -4. Establish a small application outcome adapter: public ckconfig classification - for configuration errors, usage outcomes for identifiable Narratio input - errors, and operational outcomes otherwise. Preserve wrapped/joined causes - and operational precedence. Present terminal failures once. - -Check: native root/group help and version can run with deliberately unusable -configuration and external dependencies; invalid invocation returns status 2. -Construction itself performs no credential bootstrap or acquisition. - -## Stage 2: Retain authoritative pipeline and campaign load results - -**Completed.** Pipeline and campaign loaders expose their actual typed native -results, retaining structured load failures and recorded resolution evidence. -Campaign configuration checks now run in its definition lifecycle; resource -existence remains a domain concern. Removed duplicated pipeline metadata and -post-load dependency merging. Native result/import/overlay checks and builds -pass with Go 1.27.4. Two existing dependency tests await the separate concrete -catalog in Stage 4; their expectations remain intact. - -Read: roadmap ownership; `internal/config/pipeline_configkit.go`, -`document_configkit.go`, `load.go`, `effective_digest.go`, and the upstream -ConfigKit sections on recorded application resolution and inspection. - -1. Expose retained pipeline and campaign ConfigKit results through typed loaded - context/accessors suitable for `ckconfig.FromConfigKit`. Keep load errors and - native diagnostics available rather than flattening them into text. -2. Move genuine pipeline defaults and normalization currently performed after - `configkit.Load` into its definition lifecycle, recording changes with native - helpers. Audit `finalizeLoadedPipeline` and dependency/default handling; - concrete family expansion and campaign-dependent work stay outside it. -3. Put configuration-owned campaign checks in the campaign definition's - validation callback. Keep input-file existence, remote readiness, and - artifact execution checks with their domain operations. -4. Retain source-relative resource semantics using native metadata during - resolution. Do not add import/source-authoring acceptance rules. Remove - metadata bookkeeping rendered redundant by the retained results; later stages - remove the replacement inspection path after its consumers move. - -Check: loaded values and native show/explain agree for defaults, imports, -overlays, and explicit values. Loading an app pipeline does not require a party -or mutate its completed result to make derived artifacts appear. - -## Stage 3: Complete session defaults within ConfigKit's lifecycle - -**Completed.** Session loads receive selected-campaign defaults and record them -before handoff, retaining their actual native result and source inventory. -Concrete-session checks run in definition validation; typed identity assertions -remain explicit. Stable input references are derived without mutating loaded -values, and standalone rendering preserves explicit optional-input clearing. -Session/default/import/discovery, preparation, initialization, native boundary, -and build checks pass with Go 1.27.4. The two catalog dependency tests noted in -Stage 2 remain pending Stage 4. - -Read: `internal/config/document_configkit.go`, `load.go` and -`mergeCampaignSession`, `document_render.go`, campaign/session validation, -and ConfigKit's resolution recording and presence contracts. - -1. Pass the selected campaign's applicable configuration defaults into the - session definition before loading finishes. Resolve effective campaign - identity and stable session input defaults there, using presence and domain - rules rather than a post-load zero-value merge. Preserve supported explicit - empty/null/false/zero semantics and validate effective values normally. -2. Record defaults and normalization with the native helpers and truthful - campaign context. Keep the session's actual load as its inspection result; - do not simulate a combined source inventory or manufacture a second result. -3. Move concrete-session/template and configuration validation into the - appropriate supported callbacks. Retain identity/previous-session assertions - as explicit request checks with typed errors where necessary. Assert final - identities after resolution; configuration-derived defaults must not conceal - a supplied mismatch. -4. Replace `mergeCampaignSession`'s mutations with derivation of stable-input - references from the completed configuration and selected sources. Preserve - useful standalone campaign/session renderers and their authoring semantics; - they are materialization services, not alternative inspection authorities. - -Check: native session show, planning, execution, and standalone initialization -agree on effective defaults and identity. Imports retain correct resource bases; -no later operation changes the captured session or campaign result. - -## Stage 4: Derive party products and concrete artifact catalogs - -**Completed.** Exposed the retained typed party result and exact explicit-roster -selection. Concrete artifacts, dependency closure, family/member provenance, -players, and publication rules are derived separately; completed pipeline and -campaign values remain unchanged. Removed expansion state from PipelineConfig. -The full configuration package and maintained examples pass with Go 1.27.4, -including the dependency tests deferred from Stage 2 and independent-roster -regressions. Whole-application compilation awaits the family-origin consumers' -catalog wiring in Stage 5; no compatibility dispatcher or snapshot was added. - -Read: `internal/config/party*.go`, `artifact_families.go`, `artifact_catalog.go`, -the family declarations in `config.go`, and the roadmap's ownership boundary. - -1. Retain the actual party ConfigKit result and select either an explicit party - file or the campaign-default file. A CLI-relative explicit file uses the - invocation directory; missing or invalid explicit selections fail exactly. - Selection does not rewrite the campaign's configured party field. - Update stable-party references and `validateResolvedPartyInputs` so its - current campaign-only source assertion accepts the agreed explicit selection. -2. Turn family expansion into domain derivation from the completed pipeline and - selected canonical roster. Return concrete definitions, family/member origins, - resolved inputs/dependencies, and expanded publication rules separately. - Keep declarations in the authoritative pipeline configuration. -3. Derive the players projection from the same selected roster. Retain current - family ordering, member-variable semantics, collision and dependency checks, - and family/concrete artifact selection meanings. Validate these products when - deriving the catalog, rather than requiring them for app inspection. -4. Remove family-expansion flags and generated catalogs stored as mutations of - `PipelineConfig`. Reuse existing catalog/model types where appropriate rather - than adding a second family engine or a generic workflow abstraction. - -Check: deriving catalogs for two different parties from one loaded pipeline -does not affect either the native result or the other catalog. Explicit roster -selection reaches family members and players; collisions remain errors. - -## Stage 5: Wire derived products through planning and execution - -**Completed.** Analysis selection, planning, fingerprinting, reconciliation, -previous-session requirements, runtime registration, and artifact inspection use -the separate concrete catalog. Run options retain normalized family selection -and origins through the stage environment; Scriptorium requests borrow genuine -shared settings. An assembled bounded run verifies explicit-roster players, -family dependencies, adapter variables, durable identities, and unchanged native -configuration. Configuration, stage, and artifact suites, focused runner and -bounded-composition checks, and builds pass with Go 1.27.4. CLI tests awaiting -the later native command/inspection conversion remain transitional. - -Read: `internal/app/analyze_artifacts.go`, `runner_runtime.go`, -`bounded_prerequisites.go`, `internal/stage/stage.go`, `analyze*.go`, -`trim.go`, and the artifact catalog/resolution owners. - -1. Carry the derived catalog and roster products through the invocation context, - `RunOptions`/`stage.Env`, effective artifact registration, and selected-target - resolution. Treat loaded configuration as read-only. -2. Update analysis work planning, dependency/input resolution, previous-session - requirements, and relevant adapter composition to consume concrete domain - definitions rather than expecting expansion inside pipeline configuration. -3. Update all stage consumers, including built-in artifact handling and trim's - Scriptorium use, through their actual dependencies. Preserve full-bundle - extraction and analysis-specific target closure semantics. -4. Keep one shared derivation path for catalog listing, planning, execution, and - session artifact inspection. Remove old paths that expand configuration on - demand or implicitly select a different roster. - -Check: a character-family pipeline executes concrete members through the real -planner with fake external adapters; dependencies, family selections, and -selected-party players match the separately derived catalog. Bounded execution -still composes only the collaborators it needs. - -## Stage 6: Preserve preparation, publication, and semantic resume evidence - -**Completed.** Publication selection, lock reporting, and semantic fingerprints -consume derived publication actions and concrete artifacts. Preparation and -resume use the selected roster's players projection. Safe pipeline materialization -preserves credential references, family declarations, and null/empty distinctions -without a fabricated native result; comparison uses the actual retained results. -Configuration, stage, and artifact suites plus focused prepared-publication, -restore, semantic-resume, and bounded-family composition checks pass with Go -1.27.4. Later native CLI conversion tests remain transitional. - -Read: `internal/stage/prepare*.go`, `publish*.go`, `semantic_contracts*.go`, -the prepared-input and remote publication/restore contracts, and -`internal/config/effective_projection.go` and `effective_pipeline.go`. - -1. Pass derived publication actions to publish selection and lock validation - rather than inserting them into pipeline configuration. Preserve source IDs, - destinations, selected-artifact filtering, remote lock behavior, and the - immutable commit/current-pointer boundary. -2. Adapt prepared party/players, campaign/session YAML, pipeline snapshots, and - required domain records to the new data flow. A secret-safe serialization - DTO may remain for preparation or persistence where it preserves required - credential references and effective inputs. It must not become a fabricated - native inspection result. Persist additional domain evidence only where an - existing preparation/resume consumer needs it. -3. Update stage/artifact semantic projections to read the same concrete products - used by execution. Preserve result-affecting evidence; overlay labels, - source arrangement, provenance, and a whole-configuration digest remain - informational rather than blanket invalidation keys. -4. Remove `newPipelineInspection`, its `NoSource` freeze, and the separate - authoring-versus-final inspection machinery when their consumers have moved. - Reuse safe digest/materialization code where still needed, separating it from - native show/diff instead of deleting useful persistence indiscriminately. - -Check: prepared inputs and publication round-trip through existing restore -fixtures; locked/current outputs keep their behavior. Equivalent authored -configurations reuse valid work, while changed concrete prompts/variables, -dependencies, roster-derived inputs, or publish destinations affect the relevant -semantic checks. Inspection never depends on persisted domain snapshots. - -## Stage 7: Share lazy identity and source acquisition - -**Completed.** Invocation-owned typed selections now acquire pipeline, campaign, -party, session, and derived execution context lazily, retaining native results. -Exact files and identity assertions are complementary; local errors never trigger -remote fallback. Remote session acquisition avoids roster/catalog work, retains -storage and cleanup causes, distinguishes committed snapshots, and releases its -temporary download through the shared owner. Bootstrap runs once and retains -failures. Direct acquisition, identity, discovery, cleanup, pipeline-retention, -and prepared-publication round-trip checks and builds pass with Go 1.27.4. -Native inspection and command declarations consume this owner in later stages; -old CLI invocation tests remain transitional. - -Read: `internal/app/config_loader.go`, `campaign_config_path.go`, -`session_config_path.go`, `remote_session.go`, `command_dependencies.go`, -and the roadmap's selector and session-acquisition sections. - -1. Replace positional strings passed among loaders with invocation-owned typed - selections. Distinguish `--campaign` identity from `--campaign-file`, and - inspection `--session` identity from `--session-file`. A supplied identity - and file are complementary: load the exact file and assert the effective ID. - Remove the old campaign-ID/file mutual exclusion. Session domain commands - retain their positional ID; file-only inspection may obtain its ID from data. -2. Acquire supporting configuration only as needed. App needs no campaign/party - expansion; campaign needs no session; an explicit party file needs no session. - Resolve campaign defaults when a session load requires them. Keep the same - selections and acquired results for an operation rather than reloading the - pipeline during session resolution. -3. Share exact-file, local-discovery, and remote-discovery acquisition between - inspection and execution. Only local absence enables remote fallback, with a - known session identity. Prefer remote authored session-root configuration; - use the committed prepared snapshot only on authored-object absence. - Preserve checksum/size and pointer-selected snapshot checks already required. -4. Keep read-only acquisition free of workspace creation, restore, manifest - updates, execution locks, and stage adapters. Remote reads are permitted. - Preserve underlying storage/access/parse errors and source labels for local, - remote-authored, and committed-snapshot inputs. Release temporary resources - on success and failure and retain cleanup errors. -5. Bootstrap secrets once per reached invocation after parsed-input checks and - before typed loading, using the existing explicit/default directory contract. - Invocation caches may share already acquired context, but must not conflate - different files, overlays, campaigns, or comparison sides. - -Check: exact-file errors and ID mismatches never fall back; only absence does. -Remote inspection leaves no runtime session state. A selected party file is -used consistently, and app-only acquisition avoids unrelated inputs/services. - -## Stage 8: Expose genuine configuration scopes through ckconfig - -**Completed.** Installed native app (default), campaign, session, and party -scopes using retained typed results, native diagnostics, and lazy acquisition. -Pipeline file-local checks now run within ConfigKit validation. Removed the -handwritten config dispatcher, show/validation/source renderers, and synthetic -source reconciliation; comparison and domain-catalog conversion retain their -scheduled owners. Native scope/path/explanation, filtered validation, aggregate -failure, lazy discovery, secret-safe bundle inspection, and bounded execution -checks pass. Configuration, stage, and artifact suites and builds pass with Go -1.27.4; remaining command and comparison tests await their scheduled conversion. - -Read: upstream `Suite`, `Selector`, `FromConfigKit`, `AdaptDiagnostics`, and the -native configuration CLI contract; Narratio's old `config_commands.go`. - -1. Register `app`, `campaign`, `session`, and `party` through - `ckconfig.FromConfigKit`, making `app` the default. Supply actual results and - native load errors from Stages 2–7. Expose all applicable standard operations, - including native path show/discovery, sources, history/explanation, and - detailed validation findings/filters. -2. Bind custom selectors only to scopes needing them. Native `--config` always - selects the pipeline root and `--overlay` its overlay; neither changes - meaning with scope. `--party-file` selects roster input, not an app override. - Use `Suite.CheckInput` for scope request checks and shared lazy state for - reached acquisitions. Keep factories free of side effects. -3. Retain upstream per-operation aggregate-scope behavior. Missing required - context fails visibly rather than dropping a scope. Filtering displayed - findings does not turn invalid configuration into success. -4. Remove Narratio-owned show/source/explanation renderers and synthetic - path/provenance reconciliation after replacement. `config artifacts` moves - to the domain catalog command in Stage 11; the generated config group needs - no custom child-command extension. - -Check: native commands inspect each scope with real imported configurations; -show values, path explanations, and diagnostics come from that scope's real -result. Scope discovery and help cause no bootstrap, loading, or storage calls. - -## Stage 9: Add independent comparisons and app inspection overrides - -**Completed.** Native comparisons expose independent campaign/session/party -selectors and load each side separately with normal defaults. App-only native -overrides flow directly into ConfigKit; neither comparison sides nor later -invocations share overridden results. Removed the custom comparison parser, -renderer, same-campaign restrictions, and overlay-pair helper. Tests cover -independent root defaults, campaign/session files, explicit and default rosters, -one-time bootstrap, detailed differences/status 3, hidden credentials, and -unsupported-scope overrides. Configuration, stage, and artifact suites, focused -native/composition checks, builds, and diff checks pass with Go 1.27.4. - -Read: upstream `ComparisonSide`, `Selector.AgainstName`, native override/diff -contracts, and Narratio's old `config_diff.go`. - -1. Bind `--against-campaign`, `--against-campaign-file`, `--against-session`, - `--against-session-file`, and `--against-party-file` for their applicable - scopes alongside native `--against-config` and `--against-overlay`. -2. Resolve each side from its own fresh typed input and normal defaults. An - omitted right-side selector does not inherit its left counterpart. Party - defaults follow that side's campaign. Share only invocation-wide services - such as credential bootstrap and correctly keyed acquisitions. -3. Pass native `Selection.Overrides` to the app ConfigKit load and advertise - `Scope.Features.Overrides` only for app. Let ckconfig parse `--set` and - `--against-set`; ConfigKit applies them during the load. Keep native ordering, - merge, path, explicitness, secret, and conflict semantics. -4. Use native comparisons and report shapes, including detailed output and - `diff --check` status 3. Remove overlay-only/same-campaign restrictions and - the final-domain snapshot diff. Other scopes and aggregate inspection reject - overrides according to ckconfig's advertised capabilities. - -Check: independent files/campaigns/rosters on each side, right-side defaults, -safe hidden values, and native app override behavior. Execution and planning -declarations expose no field overrides; comparison results do not mutate inputs. - -## Stage 10: Replace run and regeneration command wiring - -**Completed.** Native run and artifacts-regenerate declarations use typed -selectors and the shared bounded planner/execution operation. Single-stage -selection rejects explicit range flags before bootstrap; regeneration fixes -forced extract-through-analyze without argv construction. Native reports expose -identities, selected/executed/skipped work, artifact targets, and completion -after acquisition cleanup. Removed the synthetic regeneration implementation; -obsolete routes and duplicate session identity flags are absent from the native -tree. Focused native execution, regeneration, selection, inspection, and bounded -family composition checks, builds, and diff checks pass with Go 1.27.4. Detailed -partial-run accounting remains assigned to Stage 15. - -Read: `internal/app/run.go`, `bounded_run.go`, `regenerate_artifacts.go`, -`planner.go`, analysis-selection helpers, and the roadmap command tree. - -1. Declare `run SESSION` and `artifacts regenerate SESSION` as typed native - commands. Bind common file/identity selectors, overlay/secrets selection, - force, and repeatable analysis artifact values. Remove the compatibility - `--session-id` path. Retain previous-session assertion support where applicable. -2. Extend the shared bounded-plan request with `--stage`: translate it to equal - inclusive endpoints and reject combining it with explicitly supplied - `--from` or `--through`. Preserve full-range defaults, endpoint validation, - and independent force. Check configuration-independent combinations before - bootstrap; check configured targets after catalog derivation. -3. Regeneration calls the same typed execution operation with forced extract - through analyze. Its public options select configuration and analysis targets; - fixed range/force are not independent user options on that command. Never - construct and reparse a synthetic argv string. -4. Return one public run report containing identities, selected stages, - executed/reused outcomes, artifacts, and completion. Stage 15 completes - failure reporting. Use native outcomes and presentation; remove the - `run-stage`, `analyze`, `publish`, and `regenerate-artifacts` command routes. - -Check: single-stage, full/ranged execution and force map to the same planner. -Regeneration always includes the complete extraction bundle and never publish; -`--artifacts` narrows analysis targets/prerequisites, not Notarius lanes. Artifact -selection retains its existing analyze/publish consumer and family rules. - -## Stage 11: Expose planning and the configured artifact catalog - -**Completed.** Native session-plan declarations share run selection/checking and -one read-only decision builder, including analysis prerequisites/reuse and -publication mappings from the same pure publication owner used by execution. -Native artifacts-list reports built-in, configured, and extraction sources, -producers, dependencies, and selected-party family provenance without requiring -a session. Acquisition retains one derived catalog; removed config-artifacts and -its custom inspection dependency. Native planning/catalog, existing planning and -manifest-cloning, explicit-roster immutability, and bounded composition checks -pass. Configuration, stage, and artifact suites, builds, and diff checks pass -with Go 1.27.4. Remaining unrelated command conversions stay scheduled below. - -Read: `internal/app/plan.go`, `analyze_artifacts.go`, old config artifact listing, -and the shared domain products from Stages 4–6. - -1. Declare `session plan SESSION` with the same stage/range, force, artifact, - and context selection semantics as run. Use the shared pure decision and - analysis work planners; keep planning read-only with cloned manifest state. -2. Include derived publication work as well as stage execution/reuse reasons, - selected analysis targets, prerequisite work, and reuse. Describe decisions - from the planner without predicting unknown output identities or claiming - execution occurred. -3. Declare `artifacts list` as a domain catalog requiring the context needed for - family derivation, but no session solely to list configured definitions. - Include configured/built-in extraction and artifact sources, producers, - dependencies, and family/member identities from existing catalog ownership. -4. Present public plan/catalog DTOs through native reporting. Remove the old - `config artifacts` implementation and its custom inspection dependency. - -Check: representative plans and execution make the same reuse/selection -decisions. Listing family members leaves the app result unchanged; neither -operation initializes a runtime workspace or stage adapters. - -## Stage 12: Convert read-only session inspection commands - -**Completed.** Native session validate/status/artifacts declarations share typed -selectors and read-only acquisition. Structured reports retain findings, local -stage/artifact evidence, audio/previous-input readiness, configuration provenance, -remote current state, locks, and exact published-output declarations including -carried origins. Partial failures retain reports and original causes; local-only -artifact inspection avoids storage and payload reads. Existing entry points -consume the same collectors. Native JSON/YAML, readiness/configuration failure, -remote partial-failure, carried-output identity, and manifest-immutability checks -pass alongside configuration, stage, artifact, focused composition, and build -checks with Go 1.27.4. Superseded parser/text assertions await final cleanup. - -Read: `operator_session_validate.go`, `operator_status.go`, -`operator_artifacts_list.go`, `operator_findings.go`, and `operator_inspection.go`. - -1. Declare `session validate SESSION`, `session status SESSION`, and - `session artifacts SESSION` with shared selectors and their relevant existing - options, including optional remote artifact inspection. -2. Separate inspection operations from handwritten parsing/text output. Return - structured findings, local stage state, artifact availability, and available - remote publication/lock information. Preserve pointer-selected remote state, - declared destinations, carried-output identities, and current publish outcomes. -3. Map genuine invalid configuration to the native input classification and - operational/readiness failures to operational failure. Preserve useful - findings or partial inspection data on failure. Material limitations belong - in the report even when informational messages are hidden. -4. Acquire only each operation's required inputs; preserve current payload-free - artifact inspection and runtime-state-free preflight behavior. - -Check: local and remote fixtures yield structured reports with expected domain -state in automatic/YAML/JSON modes. Read-only commands leave manifests and work -directories unchanged and expose no credential values. - -## Stage 13: Convert initialization and remote lock commands - -**Completed.** Native session-init and explicit locks list/add/remove commands -return structured confirmations and lock lists. Initialization shares authoring -and persistence with the existing entry point, retains template/import resource -bases and invocation-relative audio, and keeps generated YAML independent of -report formatting. Target/operand checks precede acquisition; bare locks shows -lazy help. Shared lock mutations retain conditional replacement, original -conflict causes, and idempotent removal, including failure reports. Native local/ -remote round-trips, imported-template/context, lock-concurrency, and input-check -fixtures pass alongside configuration, stage, artifact, focused composition, -build, and diff checks with Go 1.27.4. - -Read: `operator_session_init.go`, `operator_locks.go`, their focused tests, and -the initialization/lock contracts in the current CLI and operations guides. - -1. Declare `session init SESSION`; retain exactly-one output/remote target, - date/title, previous-session, mutually exclusive audio choices, force, and - applicable configuration/overlay/secrets selectors. Input combinations are - checked before loading. It creates new session input rather than acquiring an - existing session-file selection. -2. Keep native authoring-context loading, template imports, source-relative - resources, invocation-relative audio CLI paths, and standalone generated YAML. - Native report `--format` changes only the confirmation, not that YAML file. -3. Declare explicit `session locks list SESSION`, `add SESSION SOURCE`, and - `remove SESSION SOURCE`, preserving reason, add-force, conditional generation - updates, and idempotent removal. A bare locks group displays help. -4. Return concise initialization/mutation reports and a structured lock list, - retaining meaningful state when an operation cannot finish. Replace raw - success printing with CommandKit reporting. - -Check: local and remote initialization round-trips and lock concurrency -semantics continue to pass. Missing target/operand combinations fail before -bootstrap. File formatting is independent of report formatting. - -## Stage 14: Convert restore and local cleanup commands - -**Completed.** Native restore and session-clean declarations return separate -public action reports after resource release, including conflicts and completed -work on failure. Restore retains pointer authority, transition locking, recovery -markers, confined replacements, and manifest-last installation; its planner no -longer creates directories or changes their modes during dry runs. Cleanup -retains the exact workspace/spool/cache scopes and cache opt-in, checks --all -selection conflicts before bootstrap, and loads only pipeline context for that -mode. Removed top-level cleanup implementation. Focused restore/recovery/cache, -cleanup confinement/partial-failure, native selection, composition, package, -build, and diff checks pass with Go 1.27.4. Legacy workflow/parser assertions -continue their scheduled conversion before final verification. - -Read: `restore.go`, `restore_plan.go`, `restore_execute.go`, -`restore_report.go`, `clean.go`, and their focused safety tests. - -1. Declare `session restore SESSION`, retaining dry-run, force, include-audio, - and applicable context selectors. Present a public action/conflict/completion - report separately from any existing persisted restore execution report. -2. Preserve pointer-bound discovery, confined paths, the shared session lock, - incomplete-restore recovery marker, eligible regular-file replacement rules, - and manifest-last installation. Reporting does not change restore authority. -3. Move cleanup to `session clean [SESSION | --all]`, retaining dry-run and - clear-cache. Check target exclusivity and incompatible session/campaign - selections for `--all` before bootstrap. Load only the context required by - the chosen cleanup mode. -4. Return structured planned/completed cleanup actions and any failure context. - Preserve exact deletion scope, cache opt-in, confinement, and operator choice. - Retire the top-level clean route. No new directory layout or retention policy - is part of this conversion. - -Check: dry runs do not mutate; actual restore/cleanup behavior remains protected -by existing safety fixtures. Failures after completed actions report those -actions accurately instead of losing all result data. - -## Stage 15: Preserve partial run results and terminal failures - -**Completed.** Runner summaries retain the full selected plan, attempted/failed, -completed, reused/self-skipped, and unattempted outcomes, plus durable output and -speaker/artifact checkpoint evidence. One finalizer owns terminal persistence -and summary construction; failed completion/self-skip saves remain failures, -earlier successes survive, and original, persistence, and release causes stay -inspectable. Native reports follow resource release, honor quiet visibility, -and cannot undo completed work when their writer fails. Runner, checkpoint, -retry, composition, native-report, focused race, configuration/stage/artifact, -build, and diff checks pass with Go 1.27.4. Updated an existing publication test -fixture to supply its separately derived publication rules; remaining legacy -CLI conversions stay assigned to final cleanup. - -Read: `runner.go`, `runner_stage_execution.go`, `runner_runtime.go`, -`runner_finalization.go`, `run_control.go`, and manifest checkpoint contracts. - -1. Refactor the runner to return a summary alongside an error after meaningful - work. Initialize selected-stage reporting from the bounded plan; distinguish - attempted/failed, completed, reused/self-skipped, and not-attempted work. - Do not report a failed or unpersisted stage as completed. -2. Account for the current failed attempt before returning from stage execution; - today an error returns an empty outcome and the caller loses that attempt. - Include useful available artifacts through existing catalog/checkpoint - evidence without returning raw manifests as public reports or adding a - separate progress ledger. -3. Give terminal failure persistence and summary construction one clear owner - across stage execution, composition, and finalization errors. Preserve durable - earlier successes and current invalidation rules; join terminal-save and lock - release failures with the originating error. Avoid duplicate terminal saves - and duplicated failure output. -4. Finalize domain state and release resources before handlers deliver the final - report. Present partial results with the failed native outcome; early input - failures need no empty result. Report-write errors never undo completed work. - Subject all reports to native visibility, including quiet mode. - -Check: fail a later stage after earlier successes and checkpoints; the returned -summary/report and manifest agree and a retry reuses valid work. Inject terminal -save and lock-release failures and verify underlying causes remain inspectable. - -## Stage 16: Propagate process cancellation and finalize resumable state - -**Completed.** Interrupt/termination signals now cancel the native invocation. -Acquisition checks synchronous native loads, planning and scheduling check the -operation context, and running adapters/storage retain it. Transcription settles -workers and preserves returned shutdown/checkpoint causes alongside cancellation. -Terminal local manifest persistence and evidence readback detach cancellation -without detaching active work; acquired locks still release and completed speaker -and artifact checkpoints remain resumable. Restore preserves its incomplete marker -on cancellation, and initialization/cleanup stop before further local actions. -Deterministic process-signal, mid-transcription/analysis, failed-terminal-save, -retry, acquisition, runner/restore, adapter cancellation, focused race, package, -build, and diff checks pass with Go 1.27.4. - -Read: process entry, runner lifecycle, HTTP/subprocess adapter cancellation, -restore finalization, and CommandKit's invocation-context/outcome contracts. - -1. Create a signal-aware process context for interrupt and termination and pass - it through native execution to configuration acquisition, storage, planning, - stages, and existing HTTP/subprocess operations. Check around synchronous - native loads where their public API has no context parameter. Keep the - application entry injectable for deterministic tests. -2. Check cancellation before scheduling another stage or unit of work and use - existing adapter shutdown paths. Wait for started workers to settle before - closing reporting handles or returning their final result. -3. Attempt terminal local manifest persistence even when the operation context - is canceled; a narrowly scoped `context.WithoutCancel` for that final local - save can use the existing store. Keep active I/O and adapters on the canceled - operation context. Always release acquired local resources and retain - persistence/shutdown errors alongside the original cancellation. -4. Preserve successful speaker/artifact checkpoints, report a meaningful partial - result, and return operational status 1. Cancellation never schedules the - cleanup stage or deletes resumable work. Abrupt process death has no added - finalization guarantee. - -Check: controlled mid-operation cancellation reaches an adapter, starts no -later work, retains successful checkpoints, attempts a terminal save, and -releases the lock. A subsequent invocation can resume. Include a failed-save -case and representative process signal wiring without timing-based sleeps. - -## Stage 17: Centralize native console diagnostics and stage progress - -**Completed.** Replaced the runner's direct stderr logger with a small -Narratio-owned reporting interface adapted to the native invocation reporter. -Stage starts, durable completions, self-skips, and reuse emit Progress; selected -configuration, destinations, and reuse decisions emit Info, with native Warning -and Debug channels. Console delivery failures stay with native finalization and -do not interrupt checkpointing. Native visibility/boolean/repetition, report, -single-failure, downstream-capture isolation, runner/composition, focused race, -configuration/stage/artifact, build, and diff checks pass with Go 1.27.4. -Removed obsolete logger-construction tests in favor of real native invocation -coverage; captured downstream content is never replayed. - -Read: CommandKit reporting/visibility contracts, `internal/logging`, -`runner_runtime.go`, `runner_stage_execution.go`, and application print/log sites. - -1. Route console messages through the invocation's CommandKit reporter. Keep - any stage-facing reporting seam small and Narratio-owned; the application - adapts it to CommandKit. Avoid a general event bus or backend-specific types - in stage/domain contracts. -2. Emit Progress for stage start, completion, self-skip, and reuse. Emit Info for - selected configuration, sources/destinations, and decision context; retain - warnings/debug with their native severities. Replace the runner's default - direct `os.Stderr` logger so it cannot bypass reporting selection. -3. Keep failures in terminal native outcomes and material conditions in report - data. Preserve existing captured subprocess/file diagnostics and safe log - locations. Do not replay downstream streams automatically or print secrets, - transcript bodies, or prompt bodies as incidental messages. -4. Rely on native quiet, verbose, debug, progress, and format behavior, including - explicit boolean settings and upstream repetition/placement. Console debug - selection does not turn on downstream debug capture or alter retention. - -Check: ordinary mode, verbose-only, progress-only, their combination, quiet, -and debug produce the expected channel categories without changing the report -data or execution selection. Check semantic messages/categories, not complete -incidental text snapshots. Terminal failures appear once, including in quiet. - -## Stage 18: Add useful sub-stage progress - -**Completed.** Audio acquisition reports actual transfer starts, validated spool -completion, and cache reuse. Transcription and analysis report individual durable -checkpoint completions and validated reuse during partial execution, while paths, -source identity, and reuse decisions remain Info. All messages use the existing -invocation-safe seam and finish with the workers; no ledger or durable schema was -added. Native progress-only and combined-context fixtures verify concurrent -speakers, selective retry, artifacts, audio cache reuse, failed downloads, and -checkpoint-save failures without false completions or multiple final reports. -Focused race, native reporting/checkpoint/resume, configuration/stage/audio/ -artifact, build, and diff checks pass with Go 1.27.4. - -Read: existing audio acquisition/materialization, `internal/stage/transcribe.go`, -`analyze*.go`, checkpoint reconciliation, and the reporting seam from Stage 17. - -1. Emit Progress at actual audio download starts/completions, individual speaker - transcription completions, configured artifact completions, and reuse of - speaker/artifact work. Connect to the existing operation owners rather than - estimating progress from logs or introducing another durable ledger. -2. Keep source identity, selected paths, and decision explanations as Info where - they are context rather than work progress. Fine-grained progress requires - `--progress` alone; there is no extra verbose-progress mode. -3. Handle concurrent worker submissions through the invocation-safe seam and - ensure reporting is finished before invocation finalization. Concurrent - completion order need not become a deterministic public ordering contract. - Preserve checkpoint validation and actual completion timing semantics. - -Check: fake downloader/transcriber/artifact adapters demonstrate progress-only -download, speaker, artifact, and reuse events. Combining verbose and progress -adds contextual diagnostics while stdout remains one final report. No new -checkpoint schema, per-invocation history, or progress-output file is needed. - -## Stage 19: Finish completion and remove superseded CLI machinery - -**Completed.** Installed native shell completion with canonical stage/scope -choices and file/directory hints; discovery remains lazy with unusable inputs -and guarded credential/storage factories. Removed handwritten dispatch/parsing, -singleton and positional reconciliation, legacy command/loading wrappers, manual -renderers, and the obsolete flattened inspection-value projection. Maintained -workflow, restore, publication, credential, and resume tests now use native -invocations and structured reports, with separate derived products and explicit -bounded prerequisites in fixtures. Safe bootstrap diagnostics use native Info. -Application/configuration race suites, complete application/configuration/stage/ -artifact/audio/process tests, build, focused vet, and diff checks pass with Go -1.27.4. Remaining old route spellings are rejection tests or deferred canonical -documentation/migration work owned by Stage 20. - -Read: native shell completion contracts, the finished command declarations, -and the old session/flag/help/rendering helpers and their callers. - -1. Install native completion for `narratio`, with static stage/scope choices and - applicable file/directory hints. Supply hints at flag/argument declarations; - dynamic campaign/artifact completion remains deferred. -2. Delete unused handwritten dispatch, `flag.FlagSet` command parsing, - singleton/repetition enforcement, positional-session reconciliation, manual - usage/help, legacy command wrappers, custom inspection/diff renderers, and - obsolete output helpers. Keep genuinely reused domain validation and safe - persistence helpers. Trace callers before deleting shared code. -3. Remove old command aliases/routes and duplicate session identity flags. - Preserve native group help. Update or consolidate tests so real native - invocation coverage replaces private-parser or exact-output expectations. -4. Confirm help, version, completion generation/query, and `config scopes` - remain lazy even with invalid files, unavailable secrets, and failing external - dependency factories. No code path should bootstrap before these operations. - -Check: every command path in the roadmap is installed and removed paths are -absent. Native completion emits static/file hints without acquisition. Search -for obsolete routes/parsers and classify any remaining matches as current -contracts, deliberate migration guidance, or immutable historical records. - -## Stage 20: Update canonical documentation and migration workflows - -**Completed.** Rewrote the CLI reference for the native command tree, selectors, -inspection/comparison/override behavior, public reports, visibility, completion, -cancellation, and exit classifications. Updated configuration/domain ownership, -operator and troubleshooting workflows, maintained examples, README, and current -publication-policy references while preserving accepted ADRs and release history. -Audited actual runtime/test imports against existing pinned dependency notices; -no additional incorporated components were missing. The offline example -walkthrough, documentation links, maintained configuration examples, native -version/release-builder tests, and diff checks pass with Go 1.27.4. - -Read: documentation and OSS policies, roadmap migration requirements, current -CLI/config/operations/internal owners, maintained examples, and release scripts. - -1. Rewrite the CLI reference for the finished tree, selectors, native flag - placement/repetition, stage selection, inspection capabilities/overrides, - report fields, visibility, completion, and outcome statuses 0/1/2/3. - Document structured output from report DTOs without a schema-version system. -2. Update configuration and internal documentation for genuine native results, - lifecycle defaults, separate catalog/plan/publication derivation, and semantic - resume evidence. Revise obsolete current-policy references such as standalone - `publish` to `run --stage publish` without changing publication policy. - Leave the accepted ADR decision intact. -3. Update README workflows, operator development/regeneration examples, - troubleshooting, relevant integration references, maintained examples, and - affected scripts/tests. Highlight `run SESSION --stage analyze --force` for - analysis-only rebuilding and `artifacts regenerate SESSION` for forced - extraction plus analysis. Explain console versus downstream debug behavior. -4. Add a concise old-to-new command migration table in the CLI reference and - link to it from operations. Cover identity/file selectors, party selection, - native inspection versus domain views, independent comparison defaults, - inspection-only overrides, structured output, and cancellation. Versioned - release-note selection remains release preparation, not a sprint blocker. -5. Update dependency attribution for core, ckconfig, and relevant incorporated - transitive code using selected-version source commits and existing local - license texts where applicable. Audit actual runtime/test imports rather - than assuming upstream maintainer tools are distributed. Preserve Narratio's - existing grant and policy attribution. - -Check: documentation links and maintained examples pass; command examples match -real declarations. Release version embedding/checks work with native version -output. Historical released notes remain unchanged. No release, push, or CI -polling is part of this implementation sprint. - -## Stage 21: Verify the complete cutover and close implementation - -**Completed.** The final audit covers the native tree and four scopes, -independent comparisons and invocation-local overrides, immutable configuration -handoff and selected-roster derivation, reports/visibility/progress, partial -failure and cancellation, semantic resume, publication, restore, and cleanup. -Strengthened the assembled offline family workflow with imported configuration, -an overlay, native inspection/catalog/planning, safe synthetic credentials, -a later artifact failure, durable checkpoint reuse, and unchanged loaded values. -Native inspection now explains typed identity mismatches and labels campaign -sources correctly; focused regressions preserve the original causes. Removed -stale test diagnostic names. Full tests, race tests, vet, build, module-tidiness, -documentation/examples, and diff checks pass with official Go 1.27.4. The existing -nonpublishing builder produced Linux amd64/arm64 and macOS arm64 assets and -verified the embedded test version; macOS evidence remains compilation-only. -Deferred features and separate pending roadmaps remain unchanged. No release, -tag, push, or CI polling was performed. - -Read: roadmap verification/completion, this plan, the canonical updated -documentation, and [Development validation](../development.md#validation). - -1. Run an assembled offline workflow with real CommandKit/ConfigKit and fake - downstream/storage boundaries: imported pipeline and overlay, campaign and - party selection, native app/session/party inspection, independent comparison, - domain catalog/plan, execution with families, a later failure, resume, and - publication/restore where covered by existing fixtures. Preserve existing - consequential tests rather than rebuilding upstream parser/configuration - test suites inside Narratio. -2. Verify invocation-local overrides, absence of post-handoff mutation, and - roster consistency across inspection/domain consumers. Exercise safe - diagnostics, serialized configuration, public reports, failure/quiet modes, - report-writer errors, and cancellation/finalization. No resolved secret may - leak through these paths. -3. Run the established full checks with the selected Go 1.27.4 toolchain: - `go test ./...`, `go test -race ./...`, `go vet ./...`, and `go build ./...`. - These include documentation and maintained-example checks. Run - `go mod tidy -diff` for the changed dependency graph and `git diff --check`; - verify the existing supported-target asset builds with the repository's - nonpublishing builder. Do not introduce new permanent release gates. -4. Review the final diff against every roadmap target and deferred item. - Confirm all commands, four native scopes, supported inspection modes, - independent comparisons, reports, progress categories, cancellation, and - preserved safety/resume behavior have authoritative test or code evidence. - Repair real defects instead of weakening the agreed final state to fit tests. -5. Mark completed stages here with brief results, commit remaining sprint work, - and report verification and any genuine limitations. Leave pending roadmaps - intact. Tagging/pushing and the final release number require the separate - release task. - -Completion means the entire application compiles, the relevant behavior and -repository checks pass, and obsolete CLI/inspection machinery has been retired. -An intermediate green subset is not completion of this migration. - -## Open Questions - -None. The agreed roadmap settles the implementation's product decisions. -Release-number selection and execution/planning overrides remain deferred as -described there and do not block this sprint. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 73c4cb5..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,491 +0,0 @@ -# CommandKit migration review - -## Scope - -Reviewed all 21 stages of [implementation.md](implementation.md) against the -[approved roadmap](commandkit.md), the implementation from `86692d5` through -`0117f3f35d73b3025a0222d25b6924ef455592e1`, current source and documentation, -behavioral tests, and focused diagnostic reproductions. Completion annotations -were treated as claims to verify rather than proof of completion. - -The native command boundary, actual ConfigKit inspection results, separate domain -catalogs, independent selections, reporting, and cancellation architecture are -substantially implemented. The initial pass identified findings 1–3. An -integration-focused follow-up at `8be65fd` adds finding 4; the application source -is unchanged from the initial review. An algorithm-focused follow-up at -`65d24fb` adds findings 5–8: two P2 findings and two P3 optimization -opportunities. Remediation status is recorded under each finding. The review -passes made no application changes. - -## 1. P2: Preserve campaign inheritance when materializing session documents - -Status: Completed. The shared renderer uses one native explicitness snapshot to -omit inherited stable inputs and retain authored overrides and optional clearing. -Regression coverage checks changed receiving-campaign defaults, imported explicit -overrides, inherited optional inputs, preparation/publication fallback, and local -and remote initialization. Configuration, application, and stage suites pass. - -Stages: 3, 6, and 13; preservation checks in Stage 21. - -Locations: `internal/config/load.go:254–255`, -`internal/config/document_render.go:52–62`, and -`internal/app/prepared_session_roundtrip_test.go:88`. - -`SessionDocument` now points to the completed session, which already contains -campaign defaults. `RenderSessionDocument` writes those inherited stable input -paths as explicit session fields. Reloading generated session YAML or the -published prepared-session fallback therefore replaces inheritance with fixed -paths from the producing campaign. The optional spell-catalog default is also -written whenever nonempty. - -Reproduction: load a session whose authored inputs contain only audio; render its -prepared session; change the selected campaign's `speakers_file`; reload the -prepared bytes with that campaign. The received session still selects the old -speaker file. Its native metadata now calls this a session override. The -round-trip test was changed to expect `session_config` ownership for the formerly -inherited speaker path, so it accepts the regression rather than protecting the -prior behavior. - -The [configuration contract](../config.md#pipeline-imports-and-overlays) says -omitted session fields remain eligible to inherit the receiving campaign's -values. Preserving materialization/authoring semantics was part of the plan. -Freezing inherited resource paths is an additional semantic commitment, not a -requirement of completing defaults before the ConfigKit handoff. It can select -obsolete resources after campaign updates and retain paths unavailable on a -receiving host. - -Required remediation: - -- Keep the actual completed ConfigKit result authoritative for inspection and - execution. Use native explicitness metadata when serializing session-owned - fields so inherited campaign input defaults remain omitted in materialized - session YAML. Retain explicit session overrides and explicit optional-input - clearing. Preserve import flattening and source-relative resolution. -- Apply the distinction to preparation, local/remote initialization, and - committed-session fallback through their shared renderer. Do not introduce a - replacement configuration result or reparse authoring files for inspection. -- Restore coverage that materializes an omitted default, changes the receiving - campaign, and observes the new default. Include an explicit override and an - explicit empty optional spell catalog to establish the difference. Update the - existing round-trip expectation rather than preserving the newly frozen - behavior as a contract. - -## 2. Simplify native validation and error presentation - -Status: Completed. Identity assertions now use ConfigKit validation, and domain -commands present loading diagnostics through `ckconfig.AdaptDiagnostics`. Removed -the custom identity error and its classification/presentation branches, along -with configuration-layer diagnostic rewriting. Native causes and joined -operational failures remain intact. Regression coverage exercises campaign and -session mismatches (including right-side comparisons), malformed and missing -sources, and native warning visibility alongside joined failures. Full tests, -race tests, vet, and build pass with Go 1.27.4. No upstream changes were needed. - -Use ConfigKit's validation lifecycle for campaign/session identity assertions, -including comparison selections. Let ckconfig present and classify their native -diagnostics. Remove the separate identity error type and its presentation and -classification branches; no upstream extension is required. - -Retain native ConfigKit load errors and source evidence. Use ckconfig's existing -diagnostic adapter at Narratio's command boundary instead of rewriting source IDs -and embedding rendered diagnostics in configuration errors. Preserve useful -operation context and all joined operational causes. - -Keep minimal outcome adaptation for Narratio-owned input errors, partial reports, -and execution/persistence/resource-release failures. Preserve native visibility, -classification, and secret-safe diagnostics. Verify identity mismatches, malformed -and missing sources, comparison sides, and mixed operational failures through -native invocations. - -## 3. P2: Correct the documented empty spell-catalog behavior - -Status: Completed. The session field reference distinguishes omission (inherit) -from an explicit empty string (clear). Documentation checks pass; existing -native-default and materialization tests cover both behaviors. - -Stage: 20. - -Location: `docs/config.md:706`, the `inputs.spell_catalog_file` session field row. - -The field table says an empty or omitted value inherits the campaign value. -The implemented native-default behavior distinguishes them: omission inherits; -an explicit empty string clears the optional input. This behavior is exercised -by `TestSessionDefaultsCompleteBeforeNativeHandoff` and is consistent with the -agreed explicit-presence rules. The table contradicts both implementation and -the reference's general defaults section. - -Required remediation: update the field row and any equivalent current guidance -to state the implemented distinction. Retain optional-input clearing behavior; -do not change it to satisfy the obsolete wording. This is a documentation-only -correction, separate from preserving inherited versus explicit fields during -materialization in finding 1. - -## 4. P2: Keep party-selection provenance out of prepare resume identity - -Status: Completed. Prepare's versioned semantic projection excludes stable-input -ownership labels; provenance remains available in source records. Native planning -and execution regressions retain identical prepared checksums and downstream -success for default/explicit selection of the same roster, and invalidate work -for changed roster contents. Focused resume and semantic tests pass under race -detection. - -Stages: 4–7, 10–11, and 21. - -Locations: `internal/config/party_resolution.go:70–81`, -`internal/stage/semantic_contracts_initial.go:151–171`, -`internal/app/semantic_resume.go:44–45`, and -`internal/app/run_control.go:78–83`. - -The new explicit party selector labels its source `party_file`; campaign-default -selection labels the same roster `campaign_config`. Existing prepare semantic -evidence hashes that source label in `prepareStableInputSelection.Source`. -Selecting the very same file explicitly changes the semantic fingerprint even -when the canonical roster, players projection, derived catalog, and all prepared -file contents are unchanged. - -An end-to-end native-invocation diagnostic reproduced this sequence: - -1. Run `prepare` successfully using the campaign-default roster and record a - succeeded downstream `transcribe` stage. -2. Plan the same session with `--party-file` pointing to that same roster. The - planner reports `result-affecting configuration changed; rerun the stage`. -3. Execute that bounded prepare request without force. Prepare executes again, - and the previously succeeded transcription stage becomes stale. Every - prepared input path/checksum pair remains identical. - -The normal invalidation relation also applies to other succeeded downstream -stages. A subsequent full run can therefore schedule unnecessary downstream -processing solely because the operator made the source selection explicit. -The selector and existing semantic-resume implementation work independently, -but their integration turns provenance into result identity, contrary to the -roadmap's result-affecting evidence boundary. This is separate from finding 1: -the reproduction requires no serialization/reload or campaign-default change. - -Required remediation: - -- Keep the selected source label in appropriate provenance and inspection - records. Exclude or semantically normalize the provenance-only distinction in - prepare's reuse fingerprint. Review equivalent stable-input source labels - under the same result-affecting rule without weakening content validation. -- Continue detecting actual roster/players/input changes through existing - canonical payload checksums and stage/artifact evidence. Preserve the fixed - downstream invalidation relation when inputs really change. -- Add an integration regression covering native planning and execution with - default versus explicit selection of the same roster, unchanged prepared - bytes, and retained downstream success. Include changed roster contents as a - counterexample that must invalidate affected work. - -## 5. P2: Capture pipeline provenance once per catalog derivation - -Status: Completed. One derivation-local resolver shares a native metadata snapshot -and source-path index across family and authored-artifact lookups. Snapshot work -is `O(M)` per derivation, plus existing native queries. The configuration suite -and focused catalog, inheritance, overlay, and provenance tests pass; existing -immutability and independent-roster coverage remains intact. - -Stages: 2 and 4; catalog consumers in Stages 5, 7, and 11. - -Locations: `internal/config/artifact_catalog.go:49–65,127–143`, -`internal/config/artifact_families.go:78`, and -`internal/config/pipeline_configkit.go:291–307`. - -Every authored concrete-artifact provenance lookup calls -`cfg.ConfigKitResult().Metadata()`. Family derivation calls the same accessor -once per family. In the selected ConfigKit v1.2.1, this accessor calls -`Inspection.Metadata`, which clones the complete effective-path and field-info -maps. It is a detached snapshot operation, rather than a constant-time handle -lookup. The immutable history model is shared; the effective metadata maps and -their nested records are copied. - -Let `D` be the number of authored artifact and family declarations, and `M` the -size of the metadata copied by one snapshot. Snapshot acquisition alone costs -`O(D * M)` work and cumulative allocation per catalog derivation, in addition to -the actual history queries. With bounded-size declarations and `M = Θ(D)`, this -is quadratic work. Family lookups also rebuild the complete source-path index -per family. The prior implementation reused its retained metadata snapshot; -the repeated full-snapshot acquisition is introduced by this migration. - -Refactoring opportunity: - -- Acquire one native metadata snapshot and one source-ID/path index inside a - catalog derivation. Share them across authored-artifact and family provenance - queries through a small local resolver or explicit helper parameters. -- This reduces snapshot work to `O(M)` plus the existing per-declaration native - queries and output processing. Build the source index once as well. The - existing `partyCharacterSourceLookup` already demonstrates an invocation-local - snapshot and lookup cache. -- Retain native history, inheritance, and source semantics, with no global - cache or replacement inspection result. Verify equivalent provenance for - concrete definitions, families, imports, and overlays; preserve the completed - ConfigKit values. - -This opportunity concerns how Narratio calls the public API. It does not require -changing ConfigKit's detached-snapshot contract or reviewing its selected -configuration algorithms. - -## 6. P3: Avoid cloning the complete family catalog for individual origins - -Status: Completed. Resume reports, checkpoint records, and analysis diagnostics -use a constant-time member lookup returning only scalar family/character identity. -Origin lookup work is `O(L)` with no whole-catalog copies. Detached full records -remain available through the existing catalog-copy API. Focused analysis, family, -planning, and catalog tests pass. - -Stages: 5 and 11; artifact evidence in Stage 15. - -Locations: `internal/stage/analyze_resume.go:88–103`, -`internal/stage/analyze.go:179–185`, `internal/config/config.go:71–76`, and -`internal/config/artifact_families.go:395–418`. - -`exportAnalyzeResumeItems` calls `config.ArtifactFamilies(catalog)` inside its -per-item loop, then reads one member's family and character ID. That accessor -deep-copies every family and member, including dependency slices and input maps. -`analyzeArtifactOrigin` performs the same whole-catalog copy for individual -artifact checkpoint and diagnostic records. - -Let `L` be the number of origin lookups and `K` the total family/member catalog -payload. These lookups cost `O(L * K)` work and cumulative allocation. When a -plan reports a catalog-sized set of bounded-size members, `L` and `K` both grow -with member count, producing quadratic copy work for a linear-sized report. -This inefficiency predates the migration and remains in the rewired domain -consumers; it is an optimization opportunity, rather than a new regression. - -Refactoring opportunity: - -- Capture one detached family catalog outside a batch and reuse its member - index across all three resume-summary lists. Copy work becomes `O(K + L)`. -- For execution's individual lookups, read the already-derived member index - through a narrow accessor that returns the needed identity fields by value. - If a caller needs mutable dependency/input records, copy that member only. -- Preserve family identities, deterministic report ordering, and detached - records where callers need them. A new defensive-copy framework or global - cache is unnecessary. - -## 7. P3: Build publication-report lookup tables once - -Status: Completed. Publication reports build configured/extraction lookup tables -and a first-declaration-per-source commit index once. Local lookup work is now -`O(A log A + X log X + K + R)` plus copied entry payloads. Removed the unused -destination-display helper. Status, remote artifact inspection, exact carried-key, -partial-failure, and catalog-evidence tests pass. - -Stage: 12. - -Locations: `internal/app/session_state_command.go:256–270`, -`internal/app/artifact_inspection.go:79–131`, and -`internal/artifacts/catalog.go:255–289`. - -For every publication rule, `inspectPublishedOutputs` calls -`helperPublishedOutputDest`, which rebuilds the configured-output map and -extraction-output set. Building each map first enumerates, sorts, and copies -the entire corresponding runtime catalog. The same loop calls -`committedPublishedOutput`, scanning all commit declarations to find one source. - -Let `R` be publication-rule count, `A` configured-artifact count, `X` -extraction-artifact count, and `K` commit-declaration count. With bounded-size -catalog records, the repeated local work is -`O(R * (A log A + X log X + K))`, plus copying any variable-sized entry payloads. -When `R`, `A`, and `K` grow together, this includes quadratic-logarithmic sorting -work and quadratic declaration scans. Required remote existence checks are -separate from this CPU/allocation cost. These helpers and repeated lookups -predate the migration and remain in the new structured-report collector. - -Refactoring opportunity: - -- Build configured/extraction lookup tables once per report and pass them to - destination resolution, or share the existing publication-action planner - where its selection and failure behavior fit inspection. -- Index published commit declarations once by source, preserving the existing - declaration-selection behavior. Resolve each report row through that index. -- Reusing the existing catalog enumeration APIs reduces this local lookup work - to `O(A log A + X log X + K + R)`, plus entry-payload copying. Keep configured - report ordering, partial error reporting, pointer-selected authority, exact - declared object keys, and the existing remote checks. - -There is no need to add remote concurrency, caching across invocations, or a new -publication protocol to obtain this improvement. - -## 8. P2: Use canonical publication sources in session reports - -Status: Completed. Inspection canonicalizes each source once for the report and -all evidence lookups, preserving the authored configuration. A regression using -real loading, validation, and publication planning failed for an accepted spaced -source before the fix and now passes alongside its unspaced control. Carried-key, -lock/outcome, partial-failure, and read-only inspection tests pass. - -Stage: 12; assembled correctness verification in Stage 21. - -Location: `internal/app/session_state_command.go:256–270`. - -Publication validation and execution accept a source with surrounding whitespace -and resolve it to its trimmed artifact identity. The new report collector uses -the authored `rule.Source` unchanged for lock, outcome, and committed-declaration -lookups, although its destination helper trims the same source. - -A focused diagnostic loads and validates `source: -" narratio.transcript.final_trimmed "` through the actual configuration loaders. -The shared publication planner produces `narratio.transcript.final_trimmed`. -Given a matching committed artifact and lock, inspection reports `missing`, -`unlocked`, an empty outcome, and no declared key. The otherwise identical -unspaced case reports `published`, `locked`, and `carried_locked`, with the -declared object identity. The previous collector normalized the source before -these lookups, making this a migration regression. - -Required remediation: - -- Use the same canonical source identity as publication for every report - lookup and the report's source field. Derive this identity locally or consume - the shared publication-action representation; keep completed ConfigKit values - unchanged. -- Preserve accepted configuration, rather than adding a whitespace prohibition. - Retain exact committed destination and origin-run reporting, including carried - outputs. -- Add a collector/native-report regression with an accepted spaced source, - committed output, lock, and outcome, alongside an unspaced control. - -Canonicalizing once per row is linear in the source's string length and adds no -catalog-wide traversal. This correctness fix is independent of the repeated -sorting/scanning optimization in finding 7. - -## Stage-by-stage coverage - -| Stage | Evidence reviewed and result | -| --- | --- | -| 1 | Released module selection, native application/process boundary, lazy construction, and outcome adapter; finding 2 covers classification gaps. | -| 2 | Retained pipeline/campaign results, lifecycle normalization/validation, native metadata, and absence of replacement inspection results. | -| 3 | Campaign defaults, explicit presence, session assertions, and resource bases; findings 1 and 2 cover materialization and native assertion outcomes. | -| 4 | Selected-party results/players, pure family expansion, collision/dependency checks, and separate publication rules; finding 4 covers resume integration. | -| 5 | Concrete-catalog consumers in planning, analysis, semantic fingerprints, adapter composition, and runtime registration. | -| 6 | Preparation, publication, restore round trips, and semantic evidence; findings 1 and 4 cover serialization and provenance-sensitive reuse. | -| 7 | Exact-file/local/remote acquisition, source labels, one-time bootstrap, retained errors, and resource cleanup; finding 2 covers typed input classification. | -| 8 | Four native scopes, path/history/findings capabilities, aggregate behavior, and lazy acquisition. | -| 9 | Independent comparison inputs/defaults, app-only overrides, native difference reports and status 3; finding 2 covers assertion failures. | -| 10 | Native bounded run/regeneration wiring, force and artifact selection, and shared execution; finding 2 covers invalid domain configuration. | -| 11 | Shared read-only planning, publication actions, and configured artifact catalog reporting; finding 4 affects planned reuse. | -| 12 | Structured validation/status/artifact inspection, partial findings, local/remote evidence, and read-only behavior. | -| 13 | Initialization targets, authoring contexts, generated YAML, explicit lock commands, and conditional lock mutations; finding 1 covers inherited defaults. | -| 14 | Restore and cleanup declarations, dry-run behavior, exact action reports, confinement, conflicts, and partial failure handling. | -| 15 | Failed-attempt accounting, partial run results, checkpoint evidence, terminal persistence, cause retention, and lock release. | -| 16 | Interrupt/termination wiring, context propagation, worker settlement, detached terminal local persistence, and resumability. | -| 17 | Narratio-owned reporting seam, native visibility/channels, stage progress, and isolation from downstream debug capture. | -| 18 | Audio transfers/cache reuse, speaker/artifact durable completions, checkpoint reuse, and progress-only integration tests. | -| 19 | Native completion/static/path hints, lazy discovery tests, and retirement of handwritten routing/parsing/rendering. | -| 20 | Canonical references, workflows, migration table, attribution, supported platforms, and preserved release history; findings 1 and 3 cover remaining documentation divergence. | -| 21 | Full test/race suites, vet/build, assembled workflows, focused review reproductions, and remaining verification recorded below. | - -The initial pass identified no other actionable requirement strengthening. -The removed CLI aliases, native flag conventions, inspection-only overrides, domain/native-view -separation, and deferred workspace/downstream changes follow the agreed scope. - -## Review validation - -Using official Go 1.27.4 against the reviewed application source: - -- `go test ./...` passed. -- `go test -race ./...` passed. -- `go vet ./...` and `go build ./...` passed. -- Focused Go-overlay diagnostics reproduced both cases in finding 2 and the - campaign-default freezing in finding 1. Diagnostic files remained under - `/tmp`; application source and tests were unchanged. -- `go mod tidy -diff` passed. -- The nonpublishing asset builder passed with `v0.0.0-test`, producing Linux - amd64/arm64 and macOS arm64 binaries under `/tmp` and verifying the host - binary's embedded version. macOS evidence is compilation-only. -- Documentation-link checks and `git diff --check` passed with this review - document present. - -Only this remediation document changes the repository. No release, tag, push, -CI polling, or live downstream service invocation was performed. - -## Integration follow-up coverage and validation - -The follow-up traversed the stages in order, emphasizing existing consumers: - -- Stages 1–3: process/CLI inputs into existing loaders, validation callers, - authoring-context initialization, and materialized input consumers. -- Stage 4: selected-party provenance and canonical roster/players derivation. -- Stage 5: concrete catalog/dependency consumers in planning, execution, - adapter composition, previous-session requirements, and runtime registration. -- Stage 6: semantic fingerprints, preparation comparisons, publication actions, - and restored input records. Finding 4 is an additional resume-integration gap. -- Stage 7: explicit/default source selection into existing identity checks, - local/remote acquisition, storage, and temporary-resource cleanup. -- Stages 8–9: native scope acquisition, independent comparison state, and - invocation-local overrides against the retained load results. -- Stages 10–11: bounded run/regeneration and plan decisions against existing - resume validation, analysis target closure, and dependent invalidation. -- Stages 12–14: inspection, initialization, locks, restore, and cleanup against - local manifests, pointer-selected remote declarations, conditional lock - updates, confinement, and existing action-report persistence. -- Stages 15–16: partial results and cancellation against speaker/artifact - checkpoints, terminal persistence, adapter contexts, and local lock release. -- Stages 17–19: progress/diagnostic delivery across existing concurrent workers - and captured subprocess logs; native lazy discovery and retired routing. -- Stages 20–21: current operator contracts and maintained workflow fixtures - against the assembled boundaries above. - -Application source remained unchanged. The initial full tests, race tests, vet, -build, module, and supported-target checks therefore remain applicable. This -pass additionally reran `go test` for `internal/app`, `internal/config`, -`internal/stage`, `internal/artifacts`, `internal/audio`, and `internal/manifest`; -all passed on Go 1.27.4. A new native-invocation Go-overlay diagnostic reproduced -finding 4 under `-race` and verified unchanged prepared input checksums. Its -files remained under `/tmp`. Documentation checks and whitespace validation -were repeated for this document update. - -## Algorithmic follow-up coverage and validation - -The algorithmic pass reviewed the completed stages in order, using the same -application source and the exact pinned dependency implementations. Findings 5–8 -add three scaling opportunities and one correctness regression. It preserves the -algorithms and consequential behavior expressly selected in the plan. - -| Stage | Algorithmic review | -| --- | --- | -| 1 | Fixed command declarations and native outcome adaptation; parser/reporting algorithms remain upstream-owned. | -| 2 | Native-result and source-metadata access; repeated catalog snapshots are covered by finding 5. | -| 3 | Presence-based defaults and fixed stable-input fields; no additional scaling finding. | -| 4 | Family expansion is output-sensitive in families × characters; maps support collision checks. Finding 5 covers avoidable metadata copies beyond that output work. | -| 5 | Concrete dependency/input consumers and family-origin access; finding 6 covers whole-catalog copies per lookup. | -| 6 | Safe materialization and semantic projections; retained checkpoint and publication algorithms were not reopened. | -| 7 | Invocation-local lazy acquisition reuses acquired products and keeps comparison sides separate. | -| 8 | Native scope dispatch and actual-result inspection; no replacement traversal algorithm is introduced. | -| 9 | Independent side acquisition and upstream comparison/override semantics; no additional scaling finding. | -| 10 | Artifact-name token processing uses a set and sorting; family selection expands the selected members, and stage selection operates on the fixed inventory. | -| 11 | Planning and catalog reporting use derived products; findings 5 and 6 apply before/during report construction. | -| 12 | Artifact/publication report collection; findings 7 and 8 cover repeated lookup construction and inconsistent source normalization. | -| 13 | Initialization and lock commands retain shared operations; no additional scaling finding. | -| 14 | Restore/cleanup action reports retain their existing planners and safety rules; no additional scaling finding in the conversion. | -| 15 | Durable summary collection traverses recorded outputs/checkpoints with per-stage sets and sorted keys; finding 6 also affects individual analysis records. | -| 16 | Cancellation propagation and worker settlement retain selected execution/checkpoint behavior. | -| 17 | Native diagnostic adaptation adds no growing history or extra durable progress collection. | -| 18 | Per-unit progress attaches to existing audio, speaker, and artifact completion/reuse events. | -| 19 | Static completion and legacy deletion add no campaign/artifact enumeration algorithm. | -| 20 | Documentation-only stage; no runtime algorithm to reassess. | -| 21 | Cross-checked assembled paths, selected dependency source, and focused correctness reproduction. | - -Complexity statements are derived from loops, call paths, and copied payloads; -they are not benchmark results or predictions of measured operator latency. -The pass did not reconsider upstream configuration algorithms, fixed pipeline -ordering/invalidation, immediate durable checkpoints, or the selected publication -and restore protocols. - -The focused Go-overlay diagnostic used real configuration loading/validation and -the shared publication planner with in-memory remote state. Its canonical-source -control passed; its spaced-source case failed the expected-state assertions as -described in finding 8. Diagnostic files remained under `/tmp`, with no repository -test or application changes. Earlier full application verification remains -applicable to the unchanged source. Documentation checks and `git diff --check` -were repeated for this update. No live services or performance measurements were -used. - -## Remediation verification - -The completed Narratio units and domain-validation portion of finding 2 pass -`go test ./...`, `go test -race ./...`, `go vet ./...`, and `go build ./...` -with official Go 1.27.4. Documentation checks and `git diff --check` also pass. -Native regression coverage verifies equivalent-party resume, real roster changes, -source normalization, and typed domain-input outcomes. No upstream files, -dependencies, releases, tags, or remote repositories were changed. Finding 2's -local simplifications are also complete and verified with the same checks on -Go 1.27.4, as recorded above.