diff --git a/docs/roadmap/commandkit-adoption.md b/docs/roadmap/commandkit-adoption.md index 40f931a..45867c1 100644 --- a/docs/roadmap/commandkit-adoption.md +++ b/docs/roadmap/commandkit-adoption.md @@ -3,11 +3,13 @@ ## Status And Intent Initial roadmap, prepared 2026-11-19. The product direction is agreed; -implementation has not begun. This document owns the target and implementation -sequence, incorporating the decisions in the -[migration notepad](commandkit-notepad.md). The notepad remains the discussion +implementation has not begun. This document owns the target and product intent, +incorporating the decisions in the [migration notepad](commandkit-notepad.md). +The notepad remains the discussion record. The [Narratio proposal](narratio-feature-request.md) supplies consumer -acceptance scenarios for requirements discovery. +acceptance scenarios for requirements discovery. The +[implementation plan](implementation.md) owns the ordered stages and concrete +adapter choices. [ADR 0006](../adr/0006-adopt-commandkit-native-operator-interfaces.md) records the native-interface adoption and ownership boundaries, complementing ADRs 0004 @@ -53,13 +55,19 @@ and ckprompt v0.10.0, documented at revision reporting, and outcomes. - [CLI reference][upstream-cli]: native command grammar and observable behavior. -Confirm a released dependency combination during implementation. The reviewed -ckprompt release declares PromptKit v0.23.2 and ConfigKit v1.2.1; retain -Scriptorium's adopted v0.23.3 fixes and verify the combined module graph. -Do not infer compatibility merely from successful dependency resolution. -Use supported public APIs and released dependencies under the existing +The selected released combination retains Scriptorium's adopted PromptKit +v0.23.3 fixes and ConfigKit v1.2.1. The reviewed ckprompt release's PromptKit +v0.23.2 minimum does not require a downgrade. Supported public APIs and released +dependencies remain subject to the existing [architecture policy](../policy/architecture.md). +One approved upstream addition is needed: a released selected-format accessor. +CommandKit v0.7.0 provides a writer-level renderer but no public access to the +invocation's selected format. The accessor enables formatted application-owned +output files, including in quiet mode, without duplicating reporting-flag +parsing. The implementation plan tracks adoption of its released API; the +agreed file-output contract remains the target. + ## Target End State ### Responsibilities And Scope @@ -160,9 +168,10 @@ inspection must work without constructing a PromptKit engine. `--config` selects Scriptorium application configuration on every applicable command, including framework inspection. That configuration selects framework sources. Adopt native `--overlay` selection and upstream selection presence -semantics. Map selections through the application's ConfigKit loader rather -than retaining the old flag parser as a compatibility layer. Preserve useful -application source and server controls through appropriate selectors. +semantics. For ConfigKit v1.2.1 the supplied value selects an overlay by name, +not an arbitrary file path. Map selections through the application's ConfigKit +loader rather than retaining the old flag parser as a compatibility layer. +Preserve useful application source and server controls through appropriate selectors. Enable temporary configuration overrides where the loader supports them, passing native override records into loading. The reviewed PromptKit scope @@ -230,28 +239,24 @@ retained generation summary into an appropriate native message category. Map typed upstream failures and configuration diagnostics into native outcomes, without parsing error text. Standard statuses remain upstream-owned. Use an -application domain outcome for completed validation failure; status `4` is the -proposed mapping, avoiding usage status `2` and diff-check status `3`. +application domain outcome for completed validation failure, distinct from +usage and diff-check outcomes. The implementation plan specifies the mapping. Delivery failures must produce a nonzero outcome and explanation, preserving native presentation-failure behavior where applicable, including an already nonzero domain outcome. Requested file errors must not disappear merely because they occur outside the invocation reporter. -Use ordinary file operations. During implementation, settle and document -overwrite behavior, missing parent directories, destination collisions, and -delivery ordering. The recommended starting point is the existing simple file -behavior: replace files without creating parents, reject output/report -destinations known to name the same file before execution, attempt the main -output followed by the report, and attempt the report even if main delivery -fails. This adds no rollback, transactional delivery, or stronger filesystem -protection. Already delivered output may remain after failure. +Use ordinary file operations: replace files without creating parents, reject +output/report destinations known to name the same file before execution, +attempt the main output followed by the report, and attempt the report even if +main delivery fails. This adds no rollback, transactional delivery, or stronger +filesystem protection. Already delivered output may remain after failure. Wire signal cancellation through the invocation context. Keep the application whole-operation budget separate from the native per-attempt timeout binding. -Retaining `--run-timeout` with Go duration values is a suitable application -choice; map it to PromptKit's `OperationTimeout` and preserve upstream zero, -negative-value, and timeout behavior. Finalize that mapping with command help -and outcome tests rather than adding a second timeout policy. +Use `--run-timeout` with Go duration values mapped to PromptKit's +`OperationTimeout`, preserving upstream zero, negative-value, and timeout +behavior without a second timeout policy. ### Preview And Consumer Inspection @@ -274,122 +279,43 @@ readiness; partial discovery can succeed with status zero. Narratio owns its pipeline, binding graph, and policy for incomplete findings. Preserve native metadata without inventing prompt versions, variable defaults, or input schemas. -## Implementation Sequence +## Acceptance And Scope Boundaries -### 1. Establish Dependencies And Command Composition +The target is met when Scriptorium exposes the native command surface and +application behavior above, superseded CLI machinery is retired, and maintained +documentation and examples describe the shipped interface. In particular: -- Confirm released module versions and public integration APIs; update modules - and applicable [attribution](../oss/attribution.md) when dependencies land. -- Build the CommandKit application with identity/version, standard families, - application commands, aliases, and completion declarations. Keep help and - declaration construction independent of runtime configuration loading. -- Inventory existing CLI flags against reusable bindings and application - selectors. Resolve spelling collisions in favor of native meanings, using - canonical spellings where shorter aliases would be ambiguous. -- Specify the concrete outcome and deadline mappings described above. Confirm - public namespace binding support for the application command; raise any - genuine missing shared adapter capability upstream. +- Configuration callbacks, runtime consumers, and native inspection agree on + ConfigKit's completed values and evidence. Application-only inspection needs + no PromptKit engine, and comparison sides retain independent selections. +- Applicable selection and request context remain consistent across execution, + preparation, discovery, and inspection of multiple independently maintained + collections. Their upstream lifecycle differences remain visible. +- Generation defaults to the artifact alone; structured output and separate + reports expose the native result. Requested files remain independent of + console quiet mode, with useful failure outcomes and nontransactional delivery. +- Narratio's linked acceptance scenarios are supported, including useful partial + discovery and discovery before generated artifacts exist. +- Native help, aliases, completion, and explicit smoke testing are available. + Including smoke testing adds no provider access prerequisite to startup or + release validation. +- Server startup reaches the existing HTTP adapter with its transport, + restricted-reading, execution-report, and shared-capacity behavior intact. +- Canonical contracts, examples, dependency attribution, and migration guidance + reflect the cutover. The CLI migration includes positional prompt selection, + command replacements, literal versus file inputs, repeated mappings, overlay + selection, native endpoint/timeout controls, content-bearing reports, report + availability, quiet behavior, and changed exit statuses. -### 2. Adapt Loading And Install Native Inspection - -- Adapt `internal/config` loading, explicit overrides, diagnostics, and - application selection to ckconfig while preserving the - [configuration authority boundary](#configuration-authority-and-native-inspection). - Keep application resolution callbacks inside ConfigKit's lifecycle and - return its completed result to the native adapter. Inspect the native overlay - API mapping and update examples to its selected syntax; do not assume renaming - the old flag alone completes adoption. -- Install the native application and PromptKit configuration adapters with - native comparisons and independent comparison inputs. Avoid accidentally - sharing one side's selected values or cached engine with the other. -- Compose ckprompt inventories, named inspection, requirements discovery, - validation, preparation, and smoke testing through public adapters. -- Share request construction where applicable, starting from upstream seed - values and preserving explicitly supplied values. Keep application-only - inspection and early discovery free of execution-only prerequisites. - -### 3. Cut Over Execution And Server Startup - -- Bind `run PROMPT_ID` through reusable request flags and application selectors; - invoke PromptKit using the mapped request and cancellation context. -- Implement native result reporting, artifact text output, explicit output and - report destinations, and selected delivery/outcome handling. -- Register `serve` with CommandKit and route its startup diagnostics through - native reporting. Retain HTTP limits, restricted artifact reading, shared - engine/backend admission state, DTOs, and optional HTTP execution reporting. -- Complete the full command cutover without exposing legacy compatibility - aliases or alternate old report formats. - -### 4. Remove Superseded Code And Update Consumer Guidance - -- Remove obsolete CLI parsing, dispatch, custom config/source projections, - definition and prepared presentation, and CLI observer-report wiring. - Delete packages only after checking remaining consumers; HTTP still uses - `internal/report` at the baseline. -- Replace tests tied to deliberately removed behavior with protection for the - selected adapter contracts. Retain relevant HTTP and dependency-boundary - coverage; avoid copying upstream parser or formatter test matrices. -- Update canonical CLI, configuration, subprocess, operations, and internal - documentation alongside the behavior they own. Update architecture and test - ownership descriptions to reflect the new library boundaries. Preserve - accepted ADR history under the documentation policy. -- Update README workflows and maintained examples, including the existing - render script and collection workflows, with preferred native aliases. -- Prepare release migration guidance covering positional prompt selection, - command replacements, literal versus file inputs, repeated mappings, - overlay selection, endpoint/timeout names, output/report formats and - availability, quiet behavior, and exit-status changes. Explain native - content-bearing reports and changed inspection presentation. +The [implementation plan](implementation.md) owns task sequencing, concrete +adapter mappings, verification steps, and sprint status. There are no unresolved +product decisions; the approved upstream selected-format accessor remains an +external release prerequisite. The [release policy](../policy/release.md#versioning-and-compatibility) governs -the eventual breaking release version. Preparing this roadmap does not select -or publish a release. - -## Verification And Completion - -Verify the application integration with real library collaborators and -deterministic substitutes at provider boundaries. Prioritize these observable -scenarios rather than exhaustive upstream behavior matrices: - -- Positional generation and preferred aliases preserve native input mapping, - explicit zero/empty values, and independent namespace selection. -- Multiple imported collections work across execution, preview, discovery, - and inspection with consistent applicable selections; app-only inspection - works without an engine. Comparison sides remain independent. -- Application callback changes appear consistently in ConfigKit's completed - values and inspection evidence. Runtime conversion preserves those values, - and standard inspection exposes the acquired result through native adapters, - including native explanation and path behavior. Reuse existing boundary - tests where they already provide this coverage. -- Default output is the artifact; structured output and separate reports use - the native result representation. Format and destination are independent; - quiet suppresses console results while requested files are still written. -- Completed invalid content, operational failures without results, cancellation, - and output/report failures produce the selected statuses and useful native - diagnostics. Representative partial delivery does not report success. -- Preview and discovery respect their distinct upstream lifecycles. Narratio's - linked scenarios exercise complete, incomplete, and failed discovery. -- Help, aliases, completion, and smoke-test wiring are reachable through the - assembled CLI. Smoke tests use controlled providers in automated checks; - live provider tests remain explicitly opt-in. -- Server startup still reaches the existing HTTP adapter, whose established - transport, reporting, restricted-reading, and shared-capacity behavior retains - appropriate coverage. - -Run the repository's [baseline validation](../development.md#baseline-validation) -for implementation changes: `go test ./...`, `go test -race ./...`, -`go vet ./...`, and `go build ./cmd/scriptorium`, plus formatting, diff, -affected example, and documentation-link checks. These checks validate the -implementation when it lands; this documentation-only roadmap does not claim -the target interface has been built or tested. - -Migration is complete when the native command surface and application behavior -above are integrated and verified, superseded CLI machinery has been retired, -and maintained documentation and examples describe the shipped interface. -Use the roadmap for implementation status until those changes land. No -significant product decision remains open; dependency confirmation and the -ordinary adapter details identified above are implementation work, not reasons -to reopen the agreed cutover. +the eventual breaking release version. This migration does not itself select +or publish a release, introduce a compatibility layer, or settle the separate +pre-v1.0 HTTP design review. [upstream-guide]: https://forge.maximumdirect.net/base-layer/commandkit/src/commit/d330d7637f2e7507eca931a899397222c82bc892/docs/consumers/guide.md [upstream-cli]: https://forge.maximumdirect.net/base-layer/commandkit/src/commit/d330d7637f2e7507eca931a899397222c82bc892/docs/cli.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..b0bbfe0 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,555 @@ +# CommandKit Adoption Implementation Plan + +## Purpose And Execution + +Implement the target in the [CommandKit adoption roadmap](commandkit-adoption.md). +This plan owns the implementation steps and concrete adapter choices; the roadmap +owns product intent and target behavior. The [migration notepad](commandkit-notepad.md) +records the discussion. [ADR 0006](../adr/0006-adopt-commandkit-native-operator-interfaces.md), +[ADR 0005](../adr/0005-configkit-boundary.md), and +[ADR 0004](../adr/0004-definition-boundary.md) remain authoritative. + +Audience: a Luna coding agent implementing one numbered stage per prompt, in +order, as one end-to-end sprint. Intermediate stages may leave compilation or +tests temporarily broken. Do not build compatibility dispatch, parallel CLI +surfaces, feature toggles, or temporary abstractions to make intermediate stages +shippable. The final assembled application is the completion boundary. Add the +stage's relevant tests with its implementation; run focused checks when runnable +and record deferred checks for Stage 9. Follow the agent policy for focused +commits, accurately identifying incomplete validation. + +Start with [development guidance](../development.md). Read the applicable +[agent](../policy/agents.md), [architecture](../policy/architecture.md), +[testing](../policy/testing.md), [documentation](../policy/documentation.md), and +[OSS](../policy/oss.md) policies. Use the [release policy](../policy/release.md) +for release-related documentation and build metadata. Preserve unrelated work. +This plan does not authorize release publication, upstream repository changes, +or rewriting accepted ADR decisions. + +Planning baseline: `bcb1a76e345ef7f3720dcbce7c446beb0d9db23b`. +The current application uses Go 1.27.0, ConfigKit v1.2.1, and PromptKit v0.23.3, +with validation/build toolchain Go 1.27.4. Relevant starting components are +`cmd/scriptorium`, `internal/adapter/cli`, `internal/config`, `internal/format`, +`internal/report`, and the unchanged `internal/adapter/http`. + +## Dependency Prerequisite And Selected Integration + +The reviewed public APIs are documented at CommandKit revision +`d330d7637f2e7507eca931a899397222c82bc892` in its +[consumer guide][upstream-guide] and [CLI contract][upstream-cli]. The released +baseline is: + +| Module | Reviewed release | Release source commit | +| --- | --- | --- | +| `go.maximumdirect.net/commandkit` | v0.7.0 | `b57bfe176cff05a892e6cb07ebb5f350239e55d4` | +| `go.maximumdirect.net/ckconfig` | v0.9.0 | `9962368f0c1ea1f6ad010638e94621091105d745` | +| `go.maximumdirect.net/ckprompt` | v0.10.0 | `c9a56e1b15f2169881a91fca4e1f4a1c475e057a` | + +There is one approved external prerequisite. CommandKit v0.7.0 has the public +`RenderReport(writer, report, format)` API but does not expose the invocation's +selected format. Its invocation reporter suppresses console writes in quiet +mode. Consequently that release alone cannot implement the agreed formatted +`--output` behavior, including quiet-mode file delivery, through the intended +APIs. The user selected a **released upstream selected-format accessor** as the +solution. Adopt that capability before implementing Stage 6. Its final exported +name and release number are determined upstream, not invented here. + +Keep ckconfig v0.9.0 and ckprompt v0.10.0 if they compose with the released core +addition; adopt corresponding released integration updates only if needed for +that capability. Preserve ConfigKit v1.2.1 and PromptKit v0.23.3 unless that +necessary released combination requires a separately reviewed change. In +particular, ckprompt's PromptKit v0.23.2 minimum is not a reason to downgrade. +Do not opportunistically expand this sprint into another framework upgrade. + +Stage 1 confirms the actual release and API. If unavailable, record the external +dependency and leave the affected work incomplete pending its release. Do not +silently reduce the output contract, parse reporting flags again, use reflection +or private APIs, or commit a local module replacement. The upstream addition is +not an unresolved product choice and is not work to implement in this repository. + +## Settled Application Choices + +### Command And Selection Mapping + +Install the native `config` and `promptkit` families, plus application commands +`run PROMPT_ID` and `serve`. Use `ckprompt.Declarations` to install native +convenience aliases. Install equivalent command-scoped aliases for applicable +reused bindings on `run`; do not add legacy command aliases or a duplicate prompt +flag. Native help, version, completion, format, verbosity, and outcomes replace +their old counterparts without a compatibility parser. + +| Input | Application mapping | +| --- | --- | +| `--config` | Application main-source selection on every configuration-consuming command, including the `promptkit` scope and family. Preserve explicit presence, including an empty value. | +| `--overlay` | Pass the supplied value to ConfigKit's `Options.Overlay`. At v1.2.1 this is a name, not a filesystem path: native empty/default selection and `overlays/NAME.yml` lookup apply. Do not add a second overlay interpreter or a fallback root. | +| Repeated `--promptkit-source PATH` | Replace the complete `promptkit_sources` list during ConfigKit loading with the selected files/directories in occurrence order, as a native override. Retain the useful existing file/directory classification, without comma splitting. | +| `--schema-dir` | Presence-aware override of application `schema_dir`. | +| `--addr`, `--artifact-root`, `--max-request-bytes`, `--max-artifact-bytes`, `--max-response-bytes` | Presence-aware application `server` overrides; available on `serve` and application-scope configuration inspection. | +| `config ... --set POINTER=JSON` | Ordered native override records for `app`; append after named application-selector overrides so native assignments are authoritative when both address the same value. Enable the feature only on `app`. | +| `config diff --against-*` | Independent right-side selections and corresponding application-selector overrides, initialized independently rather than inherited from the left. | +| `--run-timeout DURATION` on `run` | `time.ParseDuration`, then unchanged into `RunRequest.OperationTimeout`. Positive values add the native operation budget; zero and negative values add none under PromptKit v0.23.3. No second application timer or positivity restriction. | +| Native `--llm-timeout-seconds` / `--timeout-seconds` | Integer per-attempt timeout through the ckprompt execution binding; no old duration conversion. | + +Framework selectors (`promptkit-source`, `schema-dir`) apply to both config +scopes and to the native PromptKit family. Server selectors apply to the `app` +scope only. Provide each config selector with its matching `AgainstName`, such +as `against-promptkit-source`, `against-schema-dir`, or `against-addr`. +`run` accepts the framework selectors, request controls, and delivery controls; +it does not need server-only flags. `serve` accepts framework and server +selectors, not generation request controls. The configured `transient_retries` +value remains an engine default; the native execution flag overrides a request. + +ConfigKit remains responsible for loading and all precedence, resolution, and +inspection semantics. The override ordering above selects library inputs; it +does not authorize post-load merging. Keep cwd-relative application path values, +the existing discovery locations, and source-relative PromptKit resources. + +### Outcomes And Delivery + +| Condition | Outcome | +| --- | --- | +| Completed run without `ValidationFailed` | Native success, status 0, subject to delivery. | +| Completed run with `ValidationFailed` | `commandkit.Domain("validation_failed", 4)` with a useful explanation; deliver the returned artifact and native result. | +| Syntax, declaration-input, or destination-combination errors | Native usage, status 2. | +| Configuration load errors | Use `ckconfig.ClassifyConfigurationError` and native diagnostic evidence: usage 2 for classified invalid configuration, otherwise operational 1. | +| Lookup, preparation, generation, capacity, operation-timeout, cancellation, or other operational errors | Operational 1 with safe explanation and preserved typed cause. Request failures are not completed content-validation failures. | +| Native `config diff --check` differences | Native status 3, unchanged. | +| Result or diagnostic presentation failure | Preserve CommandKit's finalization rules: success/usage become operational 1; a nonzero domain status remains nonzero and retains its classification. | +| Explicit file delivery failure | Operational 1 for an otherwise successful run. Preserve an already nonzero domain status 4 while adding the delivery explanation/cause. Attempt remaining requested delivery. | + +Use `errors.Is`/`errors.As`, not error-text matching. Apply cancellation and +typed operation-timeout handling before configuration classification. Do not +classify every `ErrInvalidConfig` wrapper as invalid user input without native +evidence. Retain safe explanations for provider failures; causes are inspectable +Go errors, not permission to print credential-bearing provider payloads. + +Use one `commandkit.Report` with the actual `*promptkit.RunResult` as `Value`. +Its reusable text template emits `Artifact.Body` as bytes represented as a string, +with no added newline, header, escaping, or summary. Native JSON/YAML serialization +remains upstream-owned, including byte representation and native metadata. + +Without `--output`, submit the report through the invocation reporter. With +`--output PATH`, render through `commandkit.RenderReport` to that file using +the upstream selected-format accessor; do not also submit a console result. +`--report PATH` additionally renders the same native result as JSON, regardless +of main format or destination. Explicit files are application effects and are +written even in quiet mode. No separate report is created when the flag is absent. + +Use ordinary create/truncate file operations (mode 0644 for newly created files, +subject to umask), preserve normal OS behavior for existing files, and handle +render/write/close errors. Do not create parent directories. Explicit empty +destination values are usage errors; `-` is an ordinary filename. Before +generation, reject output/report paths known to identify the same file using +cleaned absolute path equality and `os.SameFile` for existing destinations. +This is a best-effort collision check, not a filesystem-race guarantee. + +Once a result is available, attempt main delivery first, then the separate +report, even if the first attempt fails. Do not precreate or truncate files +before a result exists. No native result means no replacement report or output +file; an existing destination can remain from a previous invocation. Delivery +is nontransactional: written bytes remain after failure. No rollback, atomic +replacement, symlink sandbox, or additional filesystem hardening is requested. + +## Stage 1: Select Released Dependencies And Establish Command Composition + +**Primary files:** `go.mod`, `go.sum`, `internal/adapter/cli` composition, +`docs/oss/attribution.md`, and applicable `docs/oss/licenses/` texts. + +1. Verify the released selected-format accessor prerequisite above and select + the minimal compatible released module combination. Record the actual + versions and complete source commits in this plan's implementation status + and attribution entries as appropriate; use released modules, not sibling + checkouts, workspaces, vendoring, or replacements. +2. Add CommandKit, ckconfig, and ckprompt dependencies. Keep the existing Go + minimum and validation toolchain unless the selected addition supplies a + concrete contrary requirement; raise such an expansion rather than guessing. + Verify selected PromptKit/ConfigKit versions and tidy the module metadata. +3. Create the CLI composition entry point in `internal/adapter/cli`. Accept a + context, application identity, argv, and output streams; let `App.Execute` + own parsing and finalization and return its outcome/status to the entrypoint. + Keep package/file choices straightforward rather than introducing a new + framework or interface for each factory. +4. Establish declarations for `run`, `serve`, the native configuration suite, + native PromptKit suite, and `commandkit.Completion("scriptorium")`. Fill the + application loaders and handlers in later stages. Native root/group/help + behavior replaces the dispatch switch. Declaration construction, help, + version, scope discovery, and static completion must not load application + or framework configuration or invoke providers. +5. Apply the OSS House Format to CommandKit, ckconfig, and ckprompt using their + actual release commits and module-subdirectory notice paths. Inspect linked + dependencies, including Cobra, pflag, and any applicable mousetrap use, + rather than treating every module-graph tool as a runtime dependency. + Reuse existing verbatim license texts and add missing applicable ones. + Preserve current project licensing and notice navigation. + +**Evidence:** selected module metadata has no replacements or unintended +PromptKit downgrade; declaration construction is valid when wired; help/version +can be exercised without a readable application configuration. Check notice +paths against the selected commits. Final composed behavior is checked in Stage 9. + +## Stage 2: Adapt Authoritative Application Loading And Shared Selectors + +**Primary files:** `internal/config/{load,document,settings}.go`, their tests, +and CLI application-loading/source-selector helpers. + +1. Retain one `configkit.Load` boundary returning its actual + `*configkit.Result[Document]`. Adapt selection into `LoadOptions` or its + replacement: explicit config presence chooses `File`, omission chooses the + existing ordered `SearchConfig`, imports remain enabled, and overlay and + override records go directly into ConfigKit. Preserve wrapped error identity. + Respect the invocation context at the application boundary without claiming + ConfigKit's synchronous work can be forcibly interrupted. +2. Replace `flag.FlagSet.Visit` and parser-owned override collection with typed + CommandKit bindings retaining `Supplied.Set`. Convert only supplied + application selectors into correctly labeled ConfigKit overrides, then append + native `Selection.Overrides` as specified above. Keep source-list order and + replacement. Use ordinary OS classification for explicitly supplied source + paths; do not load framework definitions during application inspection. + Preserve the existing nonnegative `int64` domain for HTTP byte limits, + including zero as unlimited; do not narrow it to a platform `int` or add a cap. +3. Keep schema, defaults, normalization, and domain validation inside the + existing ConfigKit definition callbacks. Record normalization through native + `configkit.Normalize`/resolution evidence. `Settings` remains a conversion + of completed values, not another resolver or merger. +4. Remove `Document.Defaults`, `DefaultsDocument`, `defaults.render_format`, + `AppSettings.DefaultRenderFormat`, associated normalization/validation, and + the resulting `internal/config` dependency on `internal/format`. Strict + ConfigKit decoding handles the retired field; add no compatibility mapping. +5. Retain shared engine construction from resolved settings: one ordered + underlay per app selector, import-enabled file inputs, directory expansion + through `promptkit.DirectoryInputs`, no implicit main, existing schema base + and engine retry setting, and the default upstream bundle. Do not change + collection traversal, peer composition, or whole-record replacement. +6. Use fresh application input per comparison side and invocation-owned state + under the native factory lifecycle; comparison may share that state without + sharing either side's selected values. Start without an engine/config cache: + separately reached loads are ordinary + loads, not an immutable-snapshot contract. Server engine lifetime is handled + separately in Stage 7. Small invocation state for acquisition diagnostics is + acceptable and must not alter effective configuration. + +**Evidence:** real ConfigKit-backed tests cover omitted versus explicit-empty +config/overlay selection, a supplied zero or empty app override, native `--set` +precedence, ordered source replacement, normalized values and their evidence, +and removal of `render_format`. Reuse existing import/search/source tests; +do not reproduce ConfigKit's exhaustive composition matrix. + +## Stage 3: Install Native Configuration Scopes And Independent Comparison + +**Primary files:** CLI configuration-suite composition and loaders; replace +`internal/adapter/cli/config.go` and `framework_config.go` responsibilities. + +1. Construct `ckconfig.Command(ckconfig.Suite[...])` with `app` from + `ckconfig.FromConfigKit` and `promptkit` from `ckprompt.ConfigScope`. + Enable native validate/show/sources/explain/diff and supported path/metadata + modes. Keep native scope discovery and default scope selection. +2. Mark only the application scope as supporting temporary overrides, passing + those records through Stage 2. Retain the PromptKit scope's native disabled + override feature, including native capability checks for all-scope requests. +3. Install selectors with their scopes and independent `AgainstName` bindings + from the mapping above. Use fresh `NewInput` results, not shared pointers. + Each comparison loader must consume that side's `Selection` and input; + do not cache the first engine into the second side. Delegate comparisons + to the returned native adapters instead of assembling a synthetic document. +4. Preserve native view metadata, findings, sources, path discovery, explanation, + redaction, diff reports, and failure classification. No local source DTO, + JSON Pointer interpretation, custom effective document, explanation-tree + reconciliation, or replacement formatter is needed. +5. Route application-load diagnostics for framework acquisitions through + `ckconfig.AdaptDiagnostics` and `commandkit.Acquired` diagnostics/explanations, + retaining the adapter's existing framework evidence. Small callback + decoration may attach application acquisition messages; do not insert app + findings into the PromptKit configuration view or merge their evidence trees. + Retain diagnostics from each reached comparison side and label their context. + Application-scope acquisition itself stays with `FromConfigKit`. + +**Evidence:** app inspection works even with unreadable framework sources; +framework inspection uses sources selected by `--config`/`--overlay`; native +show and explain agree with the resolved callback values. Exercise native +root/index/escaped-key paths and redaction through representative integration +cases, independent left/right configs/overlays/selectors, and diff-check status 3. +Verify app overrides are available and framework overrides are not emulated. + +## Stage 4: Install Native PromptKit Resources, Requirements, And Preview + +**Primary files:** CLI PromptKit-suite composition, engine-loading adapters, +and behavior tests replacing custom definition/prepared inspection tests. + +1. Compose `ckprompt.FromEngine`, `WithRequestValidation`, + `WithRequestPreparation`, and `WithSmokeTesting`, using the common application + selection and engine loading from Stage 2. Enable all twelve callbacks: + five inventories, three named resource inspections, requirements discovery, + concrete validation, preparation, and smoke testing. +2. Enable `ckprompt.WithNamespaces` with empty application defaults and install + `ckprompt.Declarations`. Let the adapters apply their native engine views; + the common engine loader returns an unscoped engine. Retain inventory filters + and default listing behavior rather than forcing all namespaces. +3. Request builders should be nil where the seed already is the desired + request. If application mapping needs a builder, start with the supplied + seed and preserve its fields, explicit values, and namespace context; do not + reread bindings or merge a different request after the builder returns. + The configured retry default belongs in the engine, not a second request pass. +4. Carry application acquisition diagnostics through the native callback return + channels as in Stage 3, preserving framework messages and typed failures. + Let CommandKit control presentation and verbosity; do not write directly to + stderr from loaders or suppress native reports based on local interpretations. +5. Expose native prompt messages, profile/backend inspection, and prepared-run + reports as returned, including their content-bearing nature. No old summary + projection or local schema-version envelope. Preview remains native + `promptkit show prepared PROMPT_ID`, with native text/YAML/JSON behavior and + stdout capture for file saving. +6. Keep discovery distinct from preparation: requirements discovery does not + need generated input artifacts, model credentials, schemas, provider calls, + or a generation output path. Loading configuration and captured message + resources is still expected. Concrete validation/preview keep their upstream + preparation-time prerequisites. Do not reject partial discovery or convert + unsuccessful/unsupported smoke findings into extra application failures. + +**Evidence:** implement the nine linked +[Narratio scenarios](narratio-feature-request.md#consumer-acceptance-scenarios) +with a small set of real-library fixtures covering exact required names, +undeclared references, optional/conditional inputs, variable presence including +empty values, complete versus partial reports, profile/prompt identity, and +selection failures. Check native inventories/namespace filters, one named report +per resource kind, and preparation/validation without generation. Smoke-test +wiring uses a controlled local provider or injected public LLM client, never real +credentials or paid calls. Reuse fixture coverage rather than duplicating the +upstream analysis or rendering suite. + +## Stage 5: Implement Positional Generation, Request Mapping, And Outcomes + +**Primary files:** the new application `run` declaration, request/outcome +helpers in `internal/adapter/cli`, and run integration tests. + +1. Declare exactly one required positional `PROMPT_ID` with CommandKit, using + shared app config/overlay/framework-selector bindings. Reuse + `ckprompt.NewResourceFlags` for profile and session selection, + `NewPromptInputFlags` for variables/text/files, and `NewExecutionFlags` for + all native execution controls. Do not install the reusable prompt flag: + the operand supplies that ID. The native run request has no direct backend-ID + override, so do not invent a backend flag or synthetic profile for `run`. +2. Map the bound values directly into a native `RunRequest` (profile, session, + inputs, vars, execution, retry count). Preserve pointers and explicit values + rather than implementing local inheritance/clearing rules. Keep native + assignment semantics: first `=` separates a mapping, commas are literal, + and the last text/file occurrence for an input wins. +3. Install the native short convenience spellings on `run` with + `commandkit.FlagAliasFor`, matching the reused binding meanings. Cover + profile/session, var, input-text/input-file, all execution controls, and + `--input`/`--llm-input` as literal-text aliases. Do not retain `--out`, + `--execution-report`, `--llm-base-url`, old `--timeout`, `--prompt-id`, or + `--profile-id` merely for compatibility. +4. For `run`, declare the three standard namespace flags and convenience aliases + through ordinary typed CommandKit string flags. The reviewed reusable + resource bindings do not export namespace bindings; use public + `Engine.WithNamespaces(promptkit.Namespaces{...})`, which is the supported + application-engine-view integration. Leave validation of namespace values + and ID selection to PromptKit. No local qualification or rewriting of stored + references, and no configurable default namespace. +5. Pass the invocation context to `Engine.Run` and map `--run-timeout` unchanged + to `OperationTimeout`. Keep generation single-dispatch at the adapter level; + retries, repairs, backend admission, credentials, and preparation belong + upstream. Do not call `ValidateRequest` first as a new execution prerequisite. +6. Implement the outcome table above. A completed invalid result is still + delivered and returns domain status 4. A resultless operational error has + no synthesized result or alternative report shape. Treat an unexpected nil + result without an error as an operational integration failure, not success. + Preserve typed causes and safe terminal explanations. +7. Build the native result report/text template and submit stdout results + through the invocation reporter. Keep any retained generation summary as a + native Info message, normally hidden at ordinary verbosity. Stage 6 adds + explicit destinations without changing the result shape. + +**Evidence:** real-engine/controlled-provider tests exercise the assembled +positional command, preferred aliases, literal/file precedence, comma-containing +values, numeric zero and reasoning empty values, service tier/extra parameters, +qualified IDs and independent namespaces, raw byte output, native structured +serialization, completed invalid content/status 4, and resultless operational +failure/status 1. Include a small typed-error mapping table rather than parsing +error prose. Use native config classification for representative load failures. + +## Stage 6: Add Explicit Output And Native JSON Report Delivery + +**Primary files:** CLI generation-delivery helpers and focused delivery tests. +Requires the released upstream selected-format accessor confirmed in Stage 1. + +1. Add `--output` and `--report` as presence-aware application file flags with + path completion. Validate empty values and known destination collisions + before executing the model. Do not create files during validation. +2. Implement the settled delivery rules using the same report as Stage 5. + Main stdout uses `Invocation.Reporter().Report`; main files use + `RenderReport` and the upstream selected format; separate reports use + `RenderReport(..., FormatJSON)`. The default selected format must continue + to use the artifact-only text template. Extensions never select formats. +3. Keep explicit file effects independent of console quiet selection. Do not + temporarily change reporting mode, rerun `App.Execute`, capture stdout and + parse it back, or replace native structured serialization to obtain files. + Keep file rendering outside reporter render callbacks to avoid reentrancy. +4. Attempt main delivery and then the requested report even when the first + fails, retaining every relevant delivery error. Include useful failure + explanations and preserve the selected nonzero domain outcome. Console + presentation failures remain recorded by CommandKit finalization; explicit + file errors need their own outcome handling because they bypass its reporter. +5. Remove the CLI observer/report collector wiring and old report-file format. + Leave `internal/report` and HTTP report attachment intact. Only a returned + native run result is eligible for the new CLI report file. + +**Evidence:** cover default/text raw bytes, JSON and YAML main files, JSON +separate report equivalence to inline native JSON, no duplicate stdout, +format/extension independence, and quiet with/without requested files. Test +empty paths, same cleaned path, existing same-file aliases, replacement without +parent creation, first-delivery failure followed by report attempt, report +failure after successful main delivery, and an unchanged existing report on a +resultless operational failure. Use temporary files and a small failing-writer +seam for stdout/render failures. Confirm status 4 survives a delivery failure +while its explanation identifies the failure. No exhaustive filesystem matrix. + +## Stage 7: Wire Process Metadata, Cancellation, And Server Startup + +**Primary files:** `cmd/scriptorium/main.go`, CLI `serve` declaration, +`scripts/release.sh`, `.woodpecker/release.yml`, and lifecycle tests. + +1. Build the CLI identity as `scriptorium`, with caller-supplied version and + revision. Add entrypoint string variables defaulting to `dev` and empty + revision for ordinary builds, suitable for `-X main.version=...` and + `-X main.revision=...`. Supply the release version/source commit in the + existing release script and hosted build flags. Keep Go 1.27.4, build targets, + publication guards, and release-completion semantics unchanged. +2. At the process entrypoint, create a signal-aware context for interrupt and + SIGTERM, pass it into execution, clean it up before `os.Exit`, and return + CommandKit's finalized status without translating it to the old CLI constants. + Keep this separately testable without signaling the test runner itself. +3. Implement `serve` as an application command consuming Stage 2's resolved + server settings. Construct the existing restricted artifact reader and inject + it into one PromptKit engine shared by that server's requests. Install the + existing HTTP handler and limits; retain `ReadHeaderTimeout` and engine retry + defaults. Do not construct engines per request or remove backend capacity + coordination. +4. Send startup information through native Info reporting and failures through + native explanations/causes. Do not add raw stdout server banners. Integrate + context cancellation with `http.Server.Close` so the listener and active + connections are closed; treat the resulting `ErrServerClosed` as expected + cancellation with a useful nonzero operational outcome. Retain operational + failure for unrelated listen/serve errors. This adds no graceful-drain deadline + or guarantee that context-ignoring work has finished. +5. Leave HTTP routes, DTOs, status mappings, optional execution reports, + restricted-reader policy, and concurrency contracts unchanged. Continue using + the HTTP observer collector; native CLI result reporting does not replace it. + +**Evidence:** test injected context cancellation reaching a real engine operation +without a duplicate timer; distinguish native operation timeout from earlier +parent cancellation. Include zero/negative operation budgets and independent +per-attempt timeout mapping. Exercise actual `serve` startup with a loopback +listener, representative HTTP request, and controlled shutdown using explicit +readiness coordination. Reuse the HTTP reader/DTO/report/shared-capacity suite +rather than duplicating it. Build a binary with test identity flags and check native +version output. Test-local fallback deadlines prevent regressions from hanging. + +## Stage 8: Retire Superseded Machinery And Update Canonical Documentation + +**Primary files:** old CLI parser/dispatch/inspection files and tests, +`internal/format`, documentation and maintained examples listed below. + +1. Remove obsolete `flag` parsing, comma mapping, old dispatch/help/exit + constants, legacy `render`/`inspect` commands, custom source DTOs and + inspection writers, custom prepared/definition presentation, and unused + CLI report helpers. Delete `internal/format` if it has no remaining consumers; + preserve `internal/report` for HTTP. Keep the useful source-to-underlay wiring + in its new CLI home, not a resurrected framework facade. +2. Replace obsolete tests with selected native integration protection from + Stages 2–7. Retain architectural import boundaries and HTTP regression tests. + Do not make intentionally retired snapshots or aliases pass by retaining + production compatibility code. Avoid testing CommandKit's complete parser + or formatting matrix again in Scriptorium. +3. Update canonical owners with implemented behavior: + + | Owner | Required changes | + | --- | --- | + | `docs/cli.md` | New command tree, applicable flags/aliases, native selection/reporting, run destinations, file semantics, statuses, completion, explicit smoke tests. Link upstream for standard family details. | + | `docs/config.md` | Removed render default; native overlay-name selection; application overrides and loader authority; app/framework scopes and native comparison. Preserve collection/source/default semantics. | + | `docs/integrations/subprocess.md` | Artifact-first execution, native reports and result availability, content-bearing metadata reports, stream capture, partial delivery, cancellation, and caller responsibility. Link exact CLI syntax rather than duplicate it. | + | `docs/operations.md` | Updated discovery/preview/execution workflow, explicit smoke-test effects, potentially sensitive reports, file retention, server process lifecycle, unchanged HTTP limits. | + | `docs/internal/overview.md`, `adapters.md`, `sources.md` | Actual component inventory and data flow, native acquisition boundaries, CLI result versus retained HTTP observation report, selector/engine lifecycle. | + | `docs/development.md`, `docs/policy/architecture.md`, `docs/policy/testing.md` | Accurate component/dependency/reading and test ownership descriptions; retain adopted constraints, policy provenance, and accepted ADR decisions. | + | `README.md`, `docs/consumers/api.md`, `docs/consumers/upgrading-promptkit.md` | Current quickstart/navigation and any affected examples; distinguish framework-format migration from the new CLI migration. | + +4. Add a focused CLI migration guide under `docs/consumers/`, linked from the + CLI/consumer overview. Cover positional prompt selection; `render` and + `inspect` replacements; file versus literal inputs; repeated mappings without + comma splitting; `--overlay` names; endpoint and timeout spelling/units; + output/report destinations, native formats, and result availability; quiet + behavior; status changes; and content-bearing native inspection/reports. + This prepares release migration information without selecting a version, + editing published notes, tagging, pushing, or adding release gates. Eventual + release notes must summarize the breaking change and link the guide under + the existing release/OSS policies. +5. Update maintained application configs to remove the retired render default. + Update `examples/render-markdown-summary.sh`, `render-v0.9-features.sh`, + README/config examples, and collection workflows to native prepared preview, + inspection, positional IDs, and preferred aliases. Existing example script + filenames may remain; they are not CLI compatibility commands. Retain both + collection-entry spellings and the multi-definition discovery example. +6. Keep historical release notes unchanged. Update roadmap/Narratio status only + after their acceptance behavior is implemented and verified. Do not silently + retire deferred HTTP redesign or dynamic-resource completion proposals. + +**Evidence:** no production command routes to retired parsing/presentation; +remaining imports justify every retained helper/package. Maintained YAML loads +under the new schema, scripts run without provider access where intended, and +current documentation links resolve. Migration examples match the built CLI. + +## Stage 9: Verify The Complete Native Cutover + +1. Finish any deferred stage checks against the complete application. Use real + ConfigKit, PromptKit, and CommandKit with deterministic provider substitutes + at the external boundary. Tests remain offline and independent of real keys. +2. Run a representative end-to-end collection workflow using + `examples/config.discovery.yml`: discover resources in support/review, + show requirements before input artifacts exist, validate/preview once inputs + exist, and run through a controlled provider with consistent selections. + Include native app and framework configuration comparisons with independent + sides. Reuse the stage tests to cover Narratio's scenarios, not a parallel + test harness. Confirm both optional entry-file globs still work and a new + checkout is seen on a later invocation, not a hot-reload promise. +3. Verify reachable help/version, preferred/canonical aliases, completion + generation for the upstream-supported shells, and controlled smoke testing. + Help/version/scope discovery/static completion work without configuration or + provider availability. Confirm retired commands/flags are absent or rejected + except where a spelling now intentionally has its native meaning (`--input`). +4. Review the output/outcome matrix across stdout, requested files, quiet, + completed validation failure, operational failures, presentation errors, + cancellation, and partial delivery. Verify reports are native values with + no Scriptorium wrapper, and HTTP's distinct contract remains unchanged. +5. Run the repository baseline on the selected toolchain: + + ```sh + go test ./... + go test -race ./... + go vet ./... + go build ./cmd/scriptorium + ``` + + Check `gofmt`, `git diff --check`, affected examples and Markdown links, + `go mod tidy` cleanliness, and the selected module graph. Check linked + runtime packages separately with `go list -deps ./cmd/scriptorium` when + validating attribution. Build the existing Linux amd64/arm64 targets with + the unchanged release build options plus the selected identity metadata; + this is migration verification, not a new publication gate. +6. Audit the final diff against the roadmap and ADRs. In particular, confirm + no post-handoff effective-config rewriting, synthetic inspection evidence, + legacy CLI compatibility layer, new HTTP restriction, arbitrary secret + redaction, extra resource-readiness prerequisite, automatic provider probe, + workflow state, snapshot guarantee, or transactional file-delivery claim + was introduced. Upstream limits/semantics remain upstream-owned. +7. Record completion and the actual dependency/API selections after successful + verification. Commit the finished implementation units under the agent + policy and report checks, commits, and any remaining work accurately. Do + not mark the migration complete while the approved upstream accessor or + required acceptance behavior is missing. + +There are no unresolved product questions. The selected-format accessor is an +approved external release prerequisite, with its exact API/version to be +confirmed in Stage 1; it does not authorize a local workaround. + +[upstream-guide]: https://forge.maximumdirect.net/base-layer/commandkit/src/commit/d330d7637f2e7507eca931a899397222c82bc892/docs/consumers/guide.md +[upstream-cli]: https://forge.maximumdirect.net/base-layer/commandkit/src/commit/d330d7637f2e7507eca931a899397222c82bc892/docs/cli.md