diff --git a/docs/roadmap/commandkit-migration-discussion.md b/docs/roadmap/commandkit-migration-discussion.md index 387d709c..e0a79ccb 100644 --- a/docs/roadmap/commandkit-migration-discussion.md +++ b/docs/roadmap/commandkit-migration-discussion.md @@ -21,8 +21,9 @@ this discussion. Item 1 selects the dependency versions and standard capabilities, including PromptKit v0.23.3 in place of the master roadmap's initial v0.23.2 target. -Item 2 selects the application command structure and input conventions. Loader -wiring and detailed reporting choices remain open below. +Item 2 selects the application command structure and input conventions. Item 3 +selects native composition, shared loaders, and application preflight. Detailed +reporting, debug capture, and verification choices remain open below. The PromptKit architecture and checkpoint decisions remain in force. This discussion develops the CommandKit portion of that migration and its interactions @@ -80,8 +81,9 @@ spending migration effort recreating existing CLI conventions. | Receipt content, debug artifact lifecycle, and application outcome meaning | Notarius, expressed through CommandKit's reporting and outcome conventions. | Adopt the standard capability suite selected in item 1, including upstream -commands and recommendations beyond the existing Notarius surface. The remaining -discussion determines application-specific inputs, wiring, and result content. +commands and recommendations beyond the existing Notarius surface. Items 2–3 +select application inputs and composition; the remaining discussion determines +result presentation, debug capture, and verification. Pipeline and domain code should remain independent of CommandKit. Native PromptKit observations supply execution facts; Notarius attaches application @@ -101,9 +103,9 @@ Two application needs already identified for deliberate treatment are: 1. Incorporate items 1–2's selected dependency baseline, standard capabilities, and native application command design, bringing U10 forward. -2. Introduce CommandKit composition alongside native source assembly. Connect - shared loaders and standard inspection while retaining application - orchestration responsibilities. +2. Introduce item 3's CommandKit composition alongside native source assembly. + Connect shared loaders, standard inspection, and the focused application + preflight package. 3. Develop native execution and accounting against the intended CLI boundary, with pipeline and domain tests independent of CommandKit. 4. Complete command and reporting migration before final end-to-end verification @@ -115,9 +117,10 @@ work units and their placement in the master roadmap follow the discussion below ## 1. Adoption Scope And Dependency Baseline **Discussion / decision:** Versions and standard capability scope agreed. Adopt -the set below, including the move to PromptKit v0.23.3. Application-specific -command design and reporting details remain with the subsequent topics. These -are selected migration targets; the application dependencies have not changed. +the set below, including the move to PromptKit v0.23.3. Items 2–3 select +application commands and composition; subsequent topics address reporting and +debug capture. These are selected migration targets; the application dependencies +have not changed. ### Selected Versions @@ -264,7 +267,7 @@ Use filesystem and choice completion hints where applicable. Keep worker counts and semantic continuation budgets in configuration rather than introducing additional dedicated CLI controls now. Their field names, -defaults, and domain meaning remain master-roadmap decisions. Item 3 will settle +defaults, and domain meaning remain master-roadmap decisions. Item 3 selects shared loading and override wiring across application and standard commands. ### Native LLM Controls @@ -312,8 +315,8 @@ the existing resolver rather than writing a second resolution path for `show`. Generic effective configuration and source explanations remain available through `config show` and `config explain`. ConfigKit's safe inspection facilities do not automatically redact an arbitrary application report; select appropriate report -content rather than dumping the raw typed configuration. Detailed report content -and loader wiring remain with items 3–4. +content rather than dumping the raw typed configuration. Item 3 selects loader +wiring; detailed report content remains with item 4. ### Removals And Remaining Work @@ -337,22 +340,162 @@ compatibility constraint. ## 3. Application Composition, Loaders, And Preflight -**Discussion / decision:** Pending. +**Discussion / decision:** Agreed. Fully adopt native ConfigKit, PromptKit, and +CommandKit workflows through a small composition root, shared loaders, and +invocation-local state. Follow substantially the same composition/loading pattern +as Audita while retaining Notarius's pipeline, artifact, validation, recovery, +and publication responsibilities. Centralize application prompt/profile +dependency checks in a focused `internal/preflight` package. Implementation is +pending. -- How should command declarations, application handlers, and invocation state be - composed without introducing another general CLI or LLM wrapper? -- How should native command selections and overrides reach shared loaders while - retaining the selected ConfigKit source composition and main-source-relative - companion behavior across execution and inspection? -- How should `pipeline show`, `pipeline validate`, and `pipeline run` share - application resolution alongside standard configuration validation, and what - state should be acquired only when a command needs it? +### Native Application Composition -Retain the selected offline preflight boundary: it checks application composition -and known native resources without synthesizing a stage request or requiring -provider credentials. Help/version should not require runtime configuration or -provider work. Model-executing roles share one native engine within an execution; -the CLI wiring must preserve that ownership. +One application constructor installs the pipeline declarations and native `run` +alias, a ckconfig suite with `app` and `promptkit` scopes, the selected ckprompt +capabilities, and native help/version/completion/reporting. Supply ordinary +callbacks and typed inputs; do not introduce another CLI framework or LLM wrapper. + +Constructing the application registers declarations without loading configuration, +building an engine, or initializing run storage. Help, version, and completion +remain independent of runtime configuration and provider work. Application +handlers acquire only what their operation requires. + +### Shared Configuration And Engine Loaders + +Use two shared loading responsibilities across application and standard commands: + +1. **Application configuration:** Select explicit `--config`, then the existing + environment/discovery policy; apply native `--overlay` and supported + application `--set` overrides. ConfigKit owns composition, imports, + inheritance, presence, provenance, and inspection. Supply application + defaults, normalization, and structural validation through its extension + points. Retain the native ConfigKit result and selected main-source context; + derive runtime values where needed without copying inspection machinery into + another result abstraction. +2. **PromptKit engine:** Build a native `*promptkit.Engine` from that loaded + application's source context and the agreed native source assembly. Runtime, + preflight, and inspection use the same builder and definitions. + +Preserve supplied-value information throughout loading. An explicitly empty +configuration path receives deliberate validation rather than silently falling +through to discovery. Across application and PromptKit commands, `--config` +selects the Notarius application configuration; the PromptKit scope inspects the +engine assembled for that application selection. + +Retain the master roadmap's [source assembly](promptkit.md#configuration-sources-and-prompt-assets): +PromptKit's bundled underlay, embedded Notarius resources, then optional operator +companions beside the main application source actually selected by ConfigKit. +Use native imports and filesystem contexts. There is no second source search or +working-directory fallback; strict composition, missing-companion handling, and +load-error behavior remain as selected in the master roadmap. Do not reinterpret +the application document as a standalone PromptKit configuration or add another +inspection-only assembly path. + +Application `--set` overrides flow through ConfigKit's load. Native request +execution overrides remain request controls; they do not mutate captured engine +configuration. Preserve the selected PromptKit configuration scope's lack of +arbitrary `--set` support. + +### Invocation State And Engine Ownership + +Use CommandKit's supported invocation state to retain configuration and an engine +when callbacks need them repeatedly. Application handlers use the same loader +functions with small invocation-owned state. One pipeline execution shares one +native engine across producers, validators, and continuations, preserving native +backend admission limits. + +Do not assume one invocation always means one configuration selection. Native +`config diff` acquires left and right selections with potentially different +configuration, overlays, and overrides while sharing invocation state. Load the +sides independently or distinguish selections in any local reuse. A single +unqualified cached result would be incorrect. No persistent loader cache or +cryptographic configuration identity is needed. + +### Shared Pipeline Resolution And Command Work + +Reuse the existing application resolver with loaded configuration, selected +pipeline/lanes/reference changes, applicable overrides, and the module catalog. +It owns module/options validation, typed artifact compatibility, validator-chain +selection, ordered dependencies, and reference-binding compatibility. + +| Command | Required application work | +| --- | --- | +| `pipeline list` | Load application configuration and enumerate pipelines. | +| `pipeline show` | Resolve the selected pipeline and report its composition. | +| `pipeline validate` | Resolve the pipeline and check its known native resource dependencies offline. | +| `pipeline run` / `run` | Use the same resolution and preflight checks, then prepare and execute the pipeline. | +| `config … --scope app` | Supply the native ConfigKit result directly to the integration. | +| PromptKit inspection commands | Acquire the shared engine assembly and delegate to native inspection. | + +Run handlers call shared application functions directly; they do not invoke the +validation command or parse its output. `pipeline show` can describe configured +prompt/profile selections without constructing an engine just to expand them +into execution settings or performing every execution-readiness check. + +### Prompt Dependencies And Offline Preflight + +Notarius knows which operations need prompts; PromptKit knows whether those +definitions resolve in the assembled engine. Check every statically known prompt +dependency of the selected pipeline, including LLM-backed validators and any +separately selected correction prompts. A deterministic stage may require none, +an LLM-backed stage may require several, and multiple stages may share one. +Configuration can select the required set. For data-dependent selections, check +what is knowable before execution and resolve the final choice at runtime. + +Share prompt identifiers and selection logic with execution through module or +operation declarations. Do not maintain a separate hardcoded preflight inventory. +These small dependency declarations identify required resources; they do not +restore asset manifests, prompt filesystem registration, message assembly, or +schema registries. Native sources remain the owners of prompt definitions. + +Use the same engine and intended namespaces as execution to inspect known prompts +and applicable profile selections, including a known prompt's default profile +when effective. Partial profiles alone are not failures. Report missing +dependencies with pipeline, step/lane, stage or validator, and resource context +where applicable. + +Offline preflight does not read the transcript, materialize reference contents, +synthesize stage requests, require credential values, or contact providers. +Engine assembly may read configuration and prompt resources. Availability does +not certify prompt domain correctness or guarantee that future requests pass +schema, credential, or provider checks. Concrete `Prepare`/`ValidateRequest` +remains the boundary for completed request validation, and smoke commands remain +explicit provider operations. + +### Dedicated Preflight Package + +Add a small application-level `internal/preflight` package accepting the resolved +pipeline, relevant module declarations, and the native engine. It checks the +pipeline's known prompt/profile dependencies and returns contextual findings or +errors for the command boundary to present. + +Keep configuration validation in ConfigKit/application definitions, structural +resolution in the pipeline resolver, and concrete preparation in PromptKit. +Preflight does not discover configuration, register production modules, construct +engines, prepare synthetic requests, initialize storage, or own CLI reporting. +The composition root supplies its collaborators. Module dependency declarations +remain in existing module contracts; modules do not import the preflight package. +Exact function signatures and finding representation remain implementation choices. + +### Request Construction, Storage, And Removals + +Use native request builders directly for standard PromptKit commands where +sufficient. Add an application callback only for a demonstrated application need, +preserving supplied operator values. Pipeline modules construct their own domain +requests; share concrete request-building functions when useful without +simulating pipeline execution to populate an inspection command. + +Output publication, checkpoints, debug capture, and execution records belong to +the run path. Loaders do not initialize those resources; debug allocation timing +remains with item 5. Remove command-specific loading paths, copied configuration +and resource inspection, eager runtime setup, and duplicated preflight +orchestration. No broader application/package reorganization is required by this +decision. + +Evidence: [native suite and loader ownership](../../../commandkit/docs/consumers/guide.md#configkit-scopes), +[PromptKit integration callbacks](../../../commandkit/docs/consumers/guide.md#request-builders-and-loading), +[Audita composition](../../../audita/internal/cli/app_wiring.go), and the +[existing configuration ownership decision](../adr/0020-delegate-configuration-mechanics-to-configkit.md). ## 4. Reports, Machine Output, And Outcomes