From d107b1235aa1a363443ca150e03ed8530486450a Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Tue, 17 Nov 2026 00:11:00 +0000 Subject: [PATCH] Finalize the PromptKit adoption plan --- docs/roadmap/implementation.md | 242 ++++++++++++++++++ .../promptkit-companion-yaml-exit-status.md | 85 ++++++ docs/roadmap/promptkit-v0.23.1.md | 101 +++----- 3 files changed, 368 insertions(+), 60 deletions(-) create mode 100644 docs/roadmap/implementation.md create mode 100644 docs/roadmap/promptkit-companion-yaml-exit-status.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..6d162fd --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,242 @@ +# PromptKit v0.23.1 implementation plan + +## Purpose and execution + +Implement the target state in the [adoption roadmap](promptkit-v0.23.1.md). +The [companion-YAML report](promptkit-companion-yaml-exit-status.md) provides the +original reproduction and expected outcomes. This plan is written for a Luna +coding agent working through one end-to-end sprint. + +Follow Stage 1 through Stage 5 in order. Continue between stages using the +decisions below. Intermediate stages may be incomplete while the sprint moves +toward the final target. Record each completed stage and its verification here, +then commit each completed nonempty unit using a short plain English message, +following the [agent policy](../policy/agents.md#git-commit-policy). + +Read the [developer guide](../development.md) and its applicable policy links. +Use the installed codebase knowledge graph for code discovery. Use Go 1.27.0 +on `PATH` with `GOTOOLCHAIN=local`. The final deliverable is a locally committed, +release-ready ckprompt v0.9.0 candidate. Publication and hosted-page delivery +follow the existing [release procedure](../release.md) when authorized. + +## Baseline and selected approach + +The planning baseline is `8513949`. ckprompt v0.8.0 selects PromptKit v0.21.0, +ckconfig v0.8.0, core v0.6.0, and ConfigKit v1.2.0, with Go 1.27.0 as its +consumer minimum. PromptKit v0.23.1 corresponds to upstream source commit +`74066167942fcf6f70c112359995739ec9789fa6`. + +Select PromptKit v0.23.1 and retain the other baseline dependency selections. +Use the existing `ckconfig.ClassifyConfigurationError`, ckprompt diagnostic +adaptation, and core outcome presentation. The upstream constructor fix enables +those mechanisms to classify malformed YAML correctly. Any production-code +adjustment should address a failure demonstrated by the integration checks. + +Existing application callbacks continue to own engine construction and native +request construction. Upstream owns conversation execution, operation timeouts, +and prepared-run notifications. Additional CLI proposals retain their status in +[future work](future.md). The [testing policy](../policy/testing.md#upstream-integration-testing-ownership) +governs assertions about these integrations. + +## Stage 1 — Select PromptKit v0.23.1 and update attribution + +Status — pending. + +1. Inspect `integrations/ckprompt/go.mod`, `go.sum`, and the module-local + `docs/oss/attribution.md`. Prepare the published dependency from the ckprompt + directory with the following command. + + ```sh + GOWORK=off GOTOOLCHAIN=local go get go.maximumdirect.net/promptkit@v0.23.1 + ``` + +2. Review the resulting module diff and resolved versions with + `GOWORK=off GOTOOLCHAIN=local go list -m all`. Keep the selected ckconfig, + core, ConfigKit, and Go minimum from the baseline. Resolve incidental module + changes using ordinary Go module tooling and the actual dependency graph. +3. Compare PromptKit's license and copyright notices at the selected source. + Update only the affected attribution entry and its source/license links to + v0.23.1 and commit `74066167942fcf6f70c112359995739ec9789fa6`, applying the + [OSS policy](../policy/oss.md). Existing reviewed license texts match. +4. Verify the published module selection with + `GOWORK=off GOTOOLCHAIN=local go list -m go.maximumdirect.net/promptkit`. + Keep any local `go.work` and `go.work.sum` as ignored development files. + +Completion evidence consists of the selected published version, reviewed +module/checksum diff, and updated attribution. Focused behavioral tests follow +in Stage 2. + +## Stage 2 — Cover real constructor failures through both integrations + +Status — pending. + +1. Read the existing test wiring in ckprompt's `command_test.go`, + `configuration_outcomes_test.go`, and + `configuration_composition_outcomes_test.go`. In particular, + `TestPromptNativeConfigurationFailures` and + `TestComposedConfigurationInputOutcomes` currently inject ConfigKit errors + directly. Reuse `newPromptTestApp` and the existing suite composition. +2. Add focused regression tests in + `integrations/ckprompt/constructor_outcomes_test.go`, using package + `ckprompt`. Build a small test-owned loader with `t.TempDir()` files, + including malformed `prompts.yml` containing `profiles: [`. Resolve the + companion from a valid application-config fixture, then construct the + engine using `promptkit.NewEngine` with a `configkit.File` input for that + companion. Use the existing fixture's `WithBundledOpenRouter(false)` option + to keep construction self-contained. Capture the actual constructor error + for identity assertions. +3. Let the test loader's `ckconfig.Selection` distinguish valid and malformed + configurations so it can serve both comparison sides. Use the existing + `adapterEngine` fixture for a successful side. Register a `FromEngine` + suite and a `ckconfig.Command` containing `ConfigScope(load)` through the + usual public command declarations. +4. Add `TestPromptConstructorConfigurationOutcomes` with the following + invocation cases. Pass the fixture's selected main path through `--config` + and use `--scope=promptkit` on configuration commands. + + | Invocation | Additional assertion | + | --- | --- | + | `promptkit list prompts` | Catalog acquisition preserves the constructor error. | + | `config show` | Configuration acquisition classifies the same error. | + | `config sources --format=json` | Available source evidence is still reported. | + | `config validate --details --format=json` | Available findings include the returned YAML diagnostic. | + | `config explain --list` | Discovery follows the configuration acquisition outcome. | + | `config diff` with malformed left input | Explanation identifies the left side. | + | `config diff` with valid left and malformed right inputs | Explanation identifies the right side. | + +5. For each case, assert usage classification and status 2, propagation of + the captured error through `errors.Is`, access to the same typed + `*configkit.LoadError` through `errors.As`, and recognition of + `promptkit.ErrInvalidConfig`. Check the returned `yaml.invalid` diagnostic + and companion source context through stable structured fields or small + semantic output fragments. Assert the available source/validation report + for the two reporting cases. Keep comparisons about adaptation and outcome + selection at the CommandKit boundary. +6. Run the focused test from the ckprompt module directory. + + ```sh + GOWORK=off GOTOOLCHAIN=local go test -count=1 -run '^TestPromptConstructorConfigurationOutcomes$' . + ``` + +Completion evidence consists of real constructor errors producing the expected +CommandKit outcomes, identity, diagnostics, reports, and comparison-side context. + +## Stage 3 — Preserve failure precedence and native report compatibility + +Status — pending. + +1. Add `TestPromptConstructorFailurePrecedence` alongside Stage 2's tests. + Exercise a representative catalog command and `config show --scope=promptkit` + for each distinct failure category below. + + - Use `NewEngine` with `WithSchemaFile` pointing to an absent file to obtain + an actual operational constructor error. Assert operational classification, + status 1, and preservation of both `ErrInvalidConfig` and `os.ErrNotExist`. + - Join the real malformed-YAML constructor error with a test-owned + operational sentinel using `errors.Join` in the loader. Assert status 1 + and preservation of both causes. + - Join the real malformed-YAML constructor error with `context.Canceled`. + Assert status 1 and preservation of the cancellation and constructor + causes. + - Execute with an already-canceled context and a loader that could otherwise + construct an engine. Assert status 1, `context.Canceled`, empty stdout, + and acquisition skipped, reusing the existing cancellation test pattern. + +2. Review `WithRequestPreparation` in `preparation.go` and + `WithRequestValidation` in `adapters.go` against the selected upstream + `RunRequest` type. Their existing whole-request assignment forwards native + builder results. Account for the new fields using that path. +3. Extend `TestPreparationCommandForwardsOrderedInputsAndNativeReport` in + `preparation_test.go` with a nonempty `ParentRunID` on its supplied native + `PreparedRun`. Keep its existing logical comparison with upstream JSON + serialization to cover additive report data through CommandKit's renderer. +4. Use the upstream public contracts when interpreting new options. + `Prepare` and `ValidateRequest` use the supplied context and leave + `OnPrepared` invocation to execution operations. Operation-timeout behavior + belongs to the upstream operations that apply it. Review keyed struct + literals and native-type use for compatibility during compilation. +5. Run the constructor tests and the affected preparation test, then ckprompt's + full default suite with published dependency selections. + + ```sh + GOWORK=off GOTOOLCHAIN=local go test -count=1 -run '^(TestPromptConstructor.*|TestPreparationCommandForwardsOrderedInputsAndNativeReport)$' . + GOWORK=off GOTOOLCHAIN=local go test ./... + ``` + +Completion evidence consists of operational/cancellation precedence, retained +causes, native prepared-report forwarding, and the passing independent suite. + +## Stage 4 — Update current guidance and prepare ckprompt v0.9.0 + +Status — pending. + +1. Update PromptKit references in `docs/consumers/guide.md` and `docs/cli.md` + to the selected source revision. Verify destinations and fragments against + v0.23.1. Review affected ckprompt GoDoc and correct version-specific claims + where needed. +2. Keep the currently published v0.8.0 installation instructions and historical + release facts accurate. Add a clearly identified prepared ckprompt v0.9.0 + candidate paragraph beside the consumer guide's version table, selecting core + v0.6.0, ckconfig v0.8.0, ConfigKit v1.2.0, and PromptKit v0.23.1. Current + checkout guidance can describe its selected v0.23.1 capabilities. Published + installation guidance can advance when publication occurs. +3. Document the corrected constructor-error behavior by referencing the existing + configuration-failure contract. Explain the application-owned engine/request + callbacks briefly and link upstream for conversation and execution features. + The existing maintained consumer supplies the composition example for this + adoption. Keep examples focused on CommandKit-specific integration behavior. +4. Create `integrations/ckprompt/docs/releases/v0.9.0.md` with title + `CommandKit ckprompt v0.9.0`. Describe the PromptKit upgrade, malformed + companion YAML changing from status 1 to status 2, preserved diagnostics + and typed causes, operational/cancellation status 1, dependency pairing, and + Go 1.27.0 minimum. Explain relevant additive native-type/report compatibility + and link to upstream upgrade guidance. Link current contracts for details. +5. End the release note with `COPYRIGHT | LICENSE | ATTRIBUTION` links using + full tag `integrations/ckprompt/v0.9.0` and module-prefixed paths, following + the [release-note procedure](../release.md#modules-versions-notes-and-shared-files). + Treat this as preparation for subsequent publication. + +Completion evidence consists of accurate current-versus-published guidance, +updated pinned references, and a version-matched release note. + +## Stage 5 — Verify the assembled candidate and complete the records + +Status — pending. + +1. Ensure the ignored local workspace includes root, ckconfig, and ckprompt, + following the developer guide. Run the following from the repository root + to validate aligned local sources and the maintained composed consumer. + + ```sh + GOTOOLCHAIN=local go test ./... + GOTOOLCHAIN=local go test ./integrations/ckconfig/... + GOTOOLCHAIN=local go test ./integrations/ckprompt/... + ``` + + The ckprompt workspace suite includes `examples/composed`; use that package's + result as the maintained-consumer evidence. + Reuse Stage 3's independent-suite result for the same Go inputs, or rerun + that suite if later Go/module changes affect it. +2. Review the aggregate diff against the roadmap's target state. Check Go + formatting for changed Go files, Markdown links and fragments, release-note + paths, attribution revision, and whitespace. Validate intended new files + along with tracked files. Resolve failures within their owning integration + paths and rerun affected checks. +3. Record stage completion and concrete verification commands/results. Mark the + companion-YAML report resolved with the selected upstream version and + integration regression evidence. Commit the verified implementation and + completed records under the local commit policy. +4. Complete roadmap retirement as release preparation. Preserve any unfinished + items in `future.md`. Record an immutable Git-history link in `future.md` + to the commit containing the completed records, then remove the completed + adoption roadmap, companion-YAML report, and this implementation plan. + Repair affected references and commit the cleanup. This sequence preserves + their resolution evidence in history while keeping active roadmaps current. +5. Confirm a clean worktree and summarize the dependency selection, tests, + documentation, commits, and prepared ckprompt version. Identify publication + as the subsequent separately authorized release action using the existing + procedure. + +The sprint is complete when the locally committed candidate meets the target +state, the scoped verification passes, and completed planning records have been +retired with their evidence retained in Git history. diff --git a/docs/roadmap/promptkit-companion-yaml-exit-status.md b/docs/roadmap/promptkit-companion-yaml-exit-status.md new file mode 100644 index 0000000..cbffbdc --- /dev/null +++ b/docs/roadmap/promptkit-companion-yaml-exit-status.md @@ -0,0 +1,85 @@ +# PromptKit companion YAML still exits with status 1 in v0.8.0 + +The original Audita report covered invalid configuration passing through both +`ckprompt` and `ckconfig` v0.7.0. Retesting with both integrations at v0.8.0 +confirmed that malformed main configuration now exits with status `2`. A +malformed `prompts.yml` companion still exits with status `1` through both +integrations, even though stderr includes the ConfigKit `yaml.invalid` diagnostic. +[Original Audita reproduction](https://forge.maximumdirect.net/eric/audita/src/commit/a355f05a273d6ac9d577eab8d30caa60cb74eee8/docs/roadmap/implementation.md#stage-1--establish-the-delegated-inspection-dependency) + +## Reproduction + +From the Audita repository root, build from a scratch copy of the reported +revision. The two integration pins are the only selected module versions that +change. The build uses CommandKit v0.6.0, ConfigKit v1.2.0, and PromptKit v0.23.0. + +```sh +scratch=$(mktemp -d) +git archive a355f05a273d6ac9d577eab8d30caa60cb74eee8 | tar -x -C "$scratch" +( + cd "$scratch" || exit + export GOWORK=off + go get go.maximumdirect.net/ckprompt@v0.8.0 go.maximumdirect.net/ckconfig@v0.8.0 && + go build -buildvcs=false -o "$scratch/audita" ./cmd/audita +) +mkdir "$scratch/repro" +cd "$scratch/repro" +printf '%s\n' 'version: 3' > config.yml +printf '%s\n' 'profiles: [' > prompts.yml +printf '%s\n' '[]' > transcript.json +printf '%s\n' 'version: [' > bad.yml +``` + +Run the following commands using `"$scratch/audita"` from the fixture +directory. They fail during configuration loading, before any provider calls. + +| Command arguments | v0.7.0 | v0.8.0 | Expected | Diagnostic | +| --- | ---: | ---: | ---: | --- | +| `promptkit list prompts --config config.yml` | 1 | 1 | 2 | `yaml.invalid` in `prompts.yml` | +| `config show --scope promptkit --config config.yml` | 1 | 1 | 2 | `yaml.invalid` in `prompts.yml` | +| `process transcript.json --config config.yml` | 2 | 2 | 2 | `yaml.invalid` in `prompts.yml` | +| `config show --scope app --config bad.yml` | 1 | 2 | 2 | `yaml.invalid` in `bad.yml` | + +`config sources`, `config validate`, `config explain --list`, and +`config diff --against-config config.yml` match the `config show` result in +each scope. The source and validation commands still deliver their available +reports on stdout. With `--config bad.yml`, both `promptkit list prompts` +and `config show --scope promptkit` also return `2` in v0.8.0 because the +main configuration load fails before construction of the PromptKit engine. + +## Why the companion case returns status 1 + +PromptKit's `NewEngine` wraps engine construction errors with +`fmt.Errorf("%w: %w", ErrInvalidConfig, err)`. In the reproduction the two +branches are the `promptkit.ErrInvalidConfig` category sentinel and a wrapped +`*configkit.LoadError` containing the `yaml.invalid` diagnostic. +[PromptKit v0.23.0 engine source](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fcbff80ab227397a03935ab3bf8114481c0736a7/engine.go) + +Both integrations can recover the typed load error through `errors.As`, so +the source and diagnostic reach stderr. `ckconfig.ClassifyConfigurationError` +classifies the typed load branch as usage and the standalone sentinel branch +as operational. The full error therefore becomes operational because the +classifier requires every branch to identify invalid input. Both +`ckprompt.definitionFailure` and `ckconfig.configFailure` use that shared +classification. +[ckconfig v0.8.0 classifier](../../integrations/ckconfig/errors.go) · +[ckconfig outcome handling](../../integrations/ckconfig/operations.go) · +[ckprompt outcome handling](../../integrations/ckprompt/command.go) + +## Expected behavior + +- Return status `2` when the typed diagnostic identifies malformed companion + YAML. Keep the diagnostic, source information, available reports, and typed + cause. Preserve recognition of `promptkit.ErrInvalidConfig` through + `errors.Is`. +- Return status `1` for cancellation and genuine operational failures, + including mixed errors with an independent operational cause. Audita's + already-canceled context test passed against v0.8.0 for catalog listing, + configuration show, and processing. It checks status `1`, + `context.Canceled` in the cause chain, and empty stdout. + +One possible upstream fix is for PromptKit to expose `ErrInvalidConfig` via +`Is` on a wrapper that unwraps to the underlying cause. This would let the +typed load branch determine the outcome for malformed YAML while preserving +the rule for independent operational branches. +[CommandKit configuration failure contract](../cli.md#configuration-failure-classification) diff --git a/docs/roadmap/promptkit-v0.23.1.md b/docs/roadmap/promptkit-v0.23.1.md index 264c9ba..e5ed4d0 100644 --- a/docs/roadmap/promptkit-v0.23.1.md +++ b/docs/roadmap/promptkit-v0.23.1.md @@ -2,9 +2,10 @@ ## Status and objective -Proposed. Adopt `go.maximumdirect.net/promptkit v0.23.1` in ckprompt and -prepare ckprompt v0.9.0. The work consists of dependency alignment, -CommandKit-level regression coverage, documentation, and release preparation. +Finalized for implementation. Adopt `go.maximumdirect.net/promptkit v0.23.1` +in ckprompt and prepare ckprompt v0.9.0. The +[implementation plan](implementation.md) owns sequencing, concrete steps, and +verification records. The targeted result is that malformed PromptKit configuration encountered through an application-owned engine loader receives CommandKit's existing @@ -41,7 +42,7 @@ independent sentinel branch and selects operational failure, producing status 1 for catalog and configuration-inspection commands. The intended input-error outcome is status 2 with the YAML diagnostic and source context preserved. -The user-supplied `promptkit-companion-yaml-exit-status.md` records the original +The [companion-YAML report](promptkit-companion-yaml-exit-status.md) records the original reproduction. Its observed application versions include PromptKit v0.23.0. The constructor wrapper responsible for that behavior also exists in the currently selected v0.21.0. @@ -70,40 +71,34 @@ operator needs recorded in [future work](future.md). Preparation and validation documentation should describe the upstream operations actually invoked, including their handling of contexts and request options. -## Work needed +## Targeted end state -### 1. Update the selected dependency +### Dependency and ownership -- Update ckprompt's `go.mod` and `go.sum` to PromptKit v0.23.1 using the - published module and its checksums. -- Review the resolved module graph against the baseline above. Upstream's - module requirements support the existing ConfigKit and Go selections. -- Check the selected source's license and notices. Update ckprompt's - [attribution](../../integrations/ckprompt/docs/oss/attribution.md) to v0.23.1 - and its exact source revision. The reviewed upstream license and copyright - texts match the v0.21.0 texts. +ckprompt selects the published PromptKit v0.23.1 module and verified checksums, +with the dependency versions and consumer minimum shown above. Its +[attribution](../../integrations/ckprompt/docs/oss/attribution.md) identifies the +selected upstream revision. The reviewed upstream license and copyright texts +match the v0.21.0 texts. -### 2. Add regression coverage through real engine construction +The existing shared classifier determines outcomes from the upstream error +evidence. CommandKit continues to own adaptation, presentation, and invocation +outcomes. Applications continue to supply engines and native requests through +the existing callbacks. -Use local fixtures and `promptkit.NewEngine` through application-owned loader -callbacks so the tests include PromptKit's actual public error wrapper. Build -on the existing integration tests and configuration-outcome helpers. +### Configuration outcomes -- Exercise malformed companion YAML through a representative `FromEngine` - catalog command and the `ConfigScope` show, sources, validate, explain - discovery, and diff paths. Cover comparison-side acquisition where it has a - distinct failure path. -- Verify the existing usage outcome and status 2, the `yaml.invalid` - diagnostic and its source context, and the reports supported by sources and - validate. Retain access to `*configkit.LoadError` and recognition of - `promptkit.ErrInvalidConfig` on the propagated error. -- Cover an operational constructor failure and genuine mixed causes so the - sentinel's presence does not turn operational failures into usage failures. - Include cancellation using the existing cancellation contract and test - helpers, retaining status 1 and the cancellation cause. -- Confirm that the existing classifier and diagnostic adapters handle these - upstream errors. Scope any implementation adjustment to a demonstrated - integration failure found during this work. +Real `promptkit.NewEngine` failures passing through a `FromEngine` catalog +command and `ConfigScope` show, sources, validate, explain discovery, and diff +receive the existing configuration-failure treatment. This includes either +comparison side. Malformed YAML selects status 2 with `yaml.invalid`, source +context, available source and validation reports, typed ConfigKit identity, and +recognition of `promptkit.ErrInvalidConfig` preserved. + +Operational constructor failures, independent operational branches joined with +an input failure, and cancellation select status 1 under the existing +[classification contract](../cli.md#configuration-failure-classification). +Maintained integration tests cover these CommandKit adaptation boundaries. The review already exercised real constructor failures against v0.23.1 using temporary module selection. Catalog and all five configuration operations @@ -111,33 +106,19 @@ returned status 2 for malformed YAML. Typed identity and mixed-cause classification were preserved. Maintained regression coverage is the remaining work needed to keep that integration behavior checked in the repository. -### 3. Validate compatibility and update documentation +### Compatibility and release readiness -- Run ckprompt's complete default suite independently against its published - dependency selections with `GOWORK=off`. -- Exercise the aligned local workspace and maintained composed consumer for - the changed integration paths, following the [developer guide](../development.md#documentation-validation). - The review's temporary v0.23.1 selection passed the independent ckprompt and - composed-consumer suites. -- Review native request and report use for the additive upstream fields, - including prepared-run parent identifiers where applicable. Keep tests - focused on CommandKit's forwarding and presentation responsibilities. -- Update the [consumer guide](../consumers/guide.md), relevant - [CLI reference](../cli.md) upstream links, and affected GoDoc to match the - selected dependency and existing integration behavior. Use brief integration - guidance and upstream references for application-owned capabilities. +Native request forwarding and prepared-report presentation accommodate the +additive upstream fields, including parent identifiers where supplied. Current +consumer guidance, CLI references, GoDoc, and upstream source links reflect the +selected library and the operations CommandKit invokes. -### 4. Prepare the integration release and completion records +The release-ready result includes a ckprompt v0.9.0 note describing the dependency +upgrade and corrected constructor-failure classification. Compatibility guidance +distinguishes the prepared candidate from published releases. Independent-module +tests and aligned-workspace composition substantiate the integration. -- Prepare `integrations/ckprompt/docs/releases/v0.9.0.md`, explaining the - PromptKit upgrade and corrected constructor-failure classification, with the - release navigation required by the existing policies. -- Update current compatibility documentation for the prepared ckprompt version - at the appropriate release-preparation point. Distinguish prepared and - published versions in completion records. -- Record completed work and verification. Retire this roadmap and the resolved - companion-YAML report when their work is complete, preserving any unfinished - items in [future work](future.md) and repairing affected references. -- Follow the existing [release procedure](../release.md) when publication is - authorized. ckprompt can release independently under the - [module alignment policy](../policy/release.md#commandkit-downstream-dependency-alignment). +The sprint ends with a locally committed candidate. Subsequent publication uses +the existing [release procedure](../release.md) under separate authorization. +ckprompt can release independently under the +[module alignment policy](../policy/release.md#commandkit-downstream-dependency-alignment).