Files
scriptorium/docs/roadmap/commandkit-notepad.md

16 KiB

CommandKit migration notepad

Status: discussion in progress. This is the working record for agreeing on Scriptorium's CommandKit migration before preparing a feature roadmap. It captures direction, open decisions, and discussion notes; implementation scope and sequencing will follow once the relevant decisions are settled.

Product context

Scriptorium is a neutral executable interface over PromptKit. Its primary consumers are applications that need to run stored prompts and consume their output without importing PromptKit or building an internal prompting workflow. The CLI and its subprocess contract are therefore central to this migration.

An installation may load many independently maintained collections. The existing collection model remains the starting point: ConfigKit imports discover collection.yml or collection.yaml entry points, each collection declares its own namespace, and relative imports assemble its profiles and prompts. See the configuration contract and discovery example.

HTTP remains secondary to the CLI discussion. A broader HTTP design review is planned before v1.0; adopting CommandKit does not itself settle that design.

Agreed direction

Full native cutover

Decision recorded 2026-11-19: become a CommandKit-native application, including ckconfig and ckprompt, and adopt upstream public conventions and recommendations as fully as possible.

Backward compatibility with existing CLI conventions is an explicit non-goal. Legacy command names, flag spellings, parsing rules, report shapes, and exit semantics do not constrain the new interface. Do not add compatibility wrappers, legacy aliases, or duplicate inspection implementations solely to preserve the old CLI. Upstream convenience aliases can be considered on their own merits.

Provide migration guidance in release documentation so consumers can update their invocations and output handling. Guidance replaces a compatibility layer; it does not require a transition period with both interfaces. The release policy governs release version selection and documentation of breaking changes.

Ownership and integration

Use CommandKit for command declarations, parsing, help, reporting, and outcomes; use ckconfig and ckprompt for their standardized operator interfaces. Scriptorium should retire the corresponding custom inspection machinery as those interfaces take over.

PromptKit continues to own preparation and execution semantics. ConfigKit continues to own configuration resolution. Scriptorium supplies application configuration and source selection, generation and server commands, process lifecycle, and application-specific artifact handling. CommandKit adoption does not introduce workflow or conversation state.

Prefer shared upstream capabilities over new Scriptorium-specific equivalents. If a required shared capability is missing, discuss an upstream request before designing a local substitute. Application-specific behavior still belongs in Scriptorium.

Initial capability set

Decision recorded 2026-11-19: include configuration comparison, provider smoke tests, and shell completion in the initial migration, alongside application and PromptKit configuration scopes, resource inventories and inspection, requirements discovery, concrete request validation, and prepared-request preview.

Use the existing upstream capabilities. Configuration comparison should cover both application and PromptKit scopes, with independent selections for each side. Provider smoke tests run when explicitly invoked; including them does not make them startup checks or release prerequisites. Document provider usage and the distinction between successful diagnostic execution and positive findings. Shell completion should use upstream command, flag, static-value, and path completion, with appropriate hints for application-owned inputs. Dynamic completion of installed resource names is a separate possible enhancement.

Application-specific wiring remains to be designed, but inclusion of these capabilities is settled. The prepared-request preview decision is recorded below.

Generation and subprocess requirements

Decision recorded 2026-11-19: optimize the default generation response for the majority of consumers, who want only the raw resulting artifact. Metadata is optional and available through the native result reporting choices below.

Prefer native CommandKit reporting and PromptKit result types before introducing a Scriptorium-specific response format. A failure must produce a nonzero process status and information explaining the failure. This does not require a new structured error envelope: native diagnostics are an acceptable way to deliver that information.

Scriptorium remains a neutral interface to PromptKit. Pass through the errors, validation outcomes, and result information that PromptKit supplies rather than adding independent judgments about generated content or redefining upstream failure semantics. Adapting those outcomes to CommandKit and process exit codes is application work; deciding what constitutes a PromptKit failure is not.

Native result reporting and separate report files

Decision recorded 2026-11-19: use a CommandKit Report containing PromptKit's native RunResult, with a text template that emits only generated content. Default/text output serves artifact consumers; standard JSON/YAML formats expose the native result and its metadata. This composes existing upstream APIs; ckprompt does not supply a general generation command or prescribe this template.

Include an optional --report PATH flag requiring an explicit file destination. Omitting it creates no separate report file. When requested, save the native result as JSON through CommandKit's renderer, using the same representation as inline JSON reporting. The report file's format is JSON independently of the main result's --format; requesting a report does not change the main result's representation or destination. Failure to write a requested report is a delivery failure and must produce a nonzero outcome and diagnostic.

Do not include a configured reporting directory or automatic report collection in the initial migration. The caller selects the report path and manages its lifecycle. A configured directory may be considered later if automatic collection becomes useful; an explicit report path would take precedence.

Native result reports are available when PromptKit returns a result. Operational failures produce a nonzero exit and native diagnostics. This follows PromptKit's capabilities and is the intended boundary, not a gap for Scriptorium to address. Do not synthesize replacement results or add an alternative failure-report format to compensate for the absence of a native result.

Output destinations and verbosity

Decision recorded 2026-11-19: use --output PATH for the main result's file destination. Without it, the main result goes to stdout. With it, the selected representation goes to the file without duplicate stdout output. Use the spelling --output, with no legacy --out compatibility alias.

--format selects the representation independently of the destination: default/text emits the raw artifact, while JSON/YAML emits the native result with metadata. The filename does not select the format. --report PATH remains an additional native JSON report, so callers can save the raw artifact and its report to separate files.

Adopt CommandKit's upstream verbosity and --quiet semantics fully, including mode selection, message visibility, rendering, and presentation failures. Do not add Scriptorium-specific exceptions to normal command-output suppression. Quiet mode does not cancel generation, and failure explanations remain visible through the native diagnostic channel.

Explicitly requested --output and --report writes are application-owned file operations and still occur in quiet mode. Render those files through CommandKit's writer API; the invocation reporter governs console output. This follows the upstream distinction between reporting visibility and application effects. With quiet mode and no requested files, generation still runs while ordinary result output is suppressed. The upstream reporting contract owns the shared verbosity rules.

Prepared-request preview

Decision recorded 2026-11-19: replace render entirely with the native promptkit show prepared PROMPT_ID command, enabled through ckprompt's request preparation integration. Remove render without a compatibility alias.

Use the native prepared-run report, standard request flags, diagnostics, verbosity, and outcomes. Default and explicit text output use the upstream YAML presentation; JSON remains available through --format json. Remove defaults.render_format and any custom prepared-output formatting code left unused by the migration.

Share application configuration selection, collection loading, and applicable request construction with generation so previews describe the corresponding execution request. PromptKit continues to own preparation semantics.

Use the native result stream for preview output; consumers can capture or redirect stdout to save it. The generation command's --output decision does not itself extend the native inspection commands. File-output convenience does not justify retaining a separate preview command. Update maintained examples and release migration guidance to use promptkit show prepared.

Upstream baseline

The initial review covered CommandKit v0.7.0, ckconfig v0.9.0, and ckprompt v0.10.0. These are a review baseline, not a final dependency choice for implementation. The upstream consumer guide describes their composition and supported extension points.

Component Relevant capabilities
CommandKit Typed flags and operands with explicit-value presence, generated help and version, shared reporting and outcomes, optional shell completion.
ckconfig Configuration scopes; validation, effective values, sources, explanations, and comparisons; temporary overrides where supported by the loader.
ckprompt Prompt, profile, backend, namespace, and collection inventories; resource inspection; requirements discovery; optional concrete request validation, prepared-request previews, and provider smoke tests. Reusable request flags also serve application-owned commands.

Scriptorium currently uses PromptKit v0.23.3 and ConfigKit v1.2.1. The reviewed ckprompt release declares PromptKit v0.23.2 and ConfigKit v1.2.1. Validate the chosen dependency combination while retaining Scriptorium's adopted PromptKit fixes; the older declared minimum is not a reason to downgrade.

The upstream CLI reference owns the detailed conventions. The current Scriptorium CLI and subprocess contract describe the starting point, not requirements to reproduce during migration.

Open decisions

1 Generation integration details

The consumer requirements, native result reporting, output destinations, and verbosity decisions above are settled. The remaining work is to specify their integration in the feature roadmap.

Map PromptKit's typed errors and reported outcomes into CommandKit conventions, preserving its distinction between a completed content check reporting ValidationFailed and an operational error. Select any necessary domain exit code without reusing the current validation-failure exit 2, which conflicts with CommandKit's usage convention. Also specify artifact/report delivery failures using CommandKit's presentation-failure behavior where applicable. These are adapter mappings, not a new application validation policy.

Signal cancellation and the whole-operation deadline remain application responsibilities. Decide their mapping to the CommandKit invocation context and PromptKit operation timeout, separately from the standard per-attempt timeout flag.

2 Configuration selection and namespace experience

Use consistent application configuration and PromptKit source selection across generation, inspection, requirements, and previews. Application-only configuration inspection should not require constructing a PromptKit engine.

Decide how the standard app and promptkit configuration scopes are exposed, how application selectors map into their loaders, and which temporary overrides are supported. Configuration scope, resource namespace, and profile selection are distinct concepts and should remain clear to operators.

For installations with many collections, decide whether to enable the optional prompt, profile, and backend namespace selectors and how to present the native namespace-filtered and all-namespace inventory views. Begin with upstream behavior; identify any unmet collection-navigation need before adding local conventions. Collection discovery itself does not need redesign for CommandKit.

3 Native reports and executable consumer needs

Adopt native inspection reports as the starting point. Standard prompt inspection includes captured prompt messages, unlike the current Scriptorium summary. Document that change and confirm how native inspection and preparation reports meet operator needs without rebuilding custom projections.

Reconcile the proposed Narratio requirements request with native promptkit requirements show. Its central need remains valid: discover input requirements before expensive upstream work and before input files exist. Preserve the distinction between successful discovery and complete discovery; a successful partial report does not establish all requirements.

The proposal's legacy syntax and prompt-version assumptions need revision. Its illustrative versioned JSON envelope and structured-error request also need a decision: native reports do not add that Scriptorium envelope, and --format json does not automatically turn diagnostics into a JSON error protocol. Determine which consumer requirements are essential and whether any genuine gap belongs upstream. Avoid treating the existing proposal as a mandate for a new compatibility facade.

Keep discovery distinct from concrete request validation: validation prepares a real request and may read input artifacts and check relevant credentials; discovery must work before those artifacts exist. Neither guarantees a future generation will succeed.

4 Application commands and standard request flags

Decide the final generation command operands and application-specific flags around the reusable ckprompt request flag groups. Use their standard names, value-presence behavior, and parsing semantics wherever applicable.

Migration guidance will need to explain changes such as separate literal-text and file-input flags, endpoint and timeout naming, and repeated mapping values. In particular, upstream's optional --input alias denotes literal text, while Scriptorium's existing flag denotes a file. Choose any upstream convenience aliases deliberately and document the new meanings; preserving old meanings is not a design requirement.

Continuing the discussion

Record discussion notes under the relevant open decision. When a choice is settled, move its conclusion and rationale into the agreed direction with its decision date, and remove superseded alternatives. Add new questions when a concrete product need emerges.

Once the command surface, consumer outputs and outcomes, loader and namespace behavior, and any upstream gaps are settled, fold the decisions into a feature roadmap. That roadmap will define implementation sequence, removal of replaced CLI code, appropriate validation, and migration documentation. This notepad does not prescribe implementation phases or treat the open proposals as commitments.