Files
commandkit/docs/roadmap/implementation.md

374 lines
22 KiB
Markdown
Raw Normal View History

# Invalid configuration outcome implementation
Status — complete. Release coordination remains in the
[future work handoff](future.md#configuration-outcome-release-handoff).
## Sprint scope and baseline
Implement the accepted [feature roadmap](invalid-configuration-outcomes.md).
Its target and accepted decisions govern the work. This plan supplies the
specific API, diagnostic mapping, integration edits, and validation.
The planning baseline is commit `a53bf9f916ffb44d816f3929fb5297a68ad118c0`.
Production sources still match the reviewed v0.7.0 integration baseline. Read
[development guidance](../development.md), the [agent policy](../policy/agents.md),
and the applicable architecture, testing, documentation, and release policies.
Execute Stage 1 through Stage 5 in order as one sprint. Intermediate stages may
leave dependent work or test expectation updates for a later stage. Mark each
stage complete with its actual verification evidence and commit completed units
under the repository's local commit policy. Continue through the remaining
stages using the decisions in this plan.
Use Go 1.27.0 and the ignored workspace containing core, ckconfig, and ckprompt.
The new shared API will first exist in local ckconfig. ckprompt currently selects
published ckconfig v0.7.0, which lacks it. The aligned workspace supplies local
ckconfig throughout this sprint. Published dependency alignment is release
coordination work described in Stage 5. Record that pending state explicitly
when reporting validation.
## Stage 1 — Implement the shared classifier
2026-11-16 18:01:26 +00:00
Status — complete. Added the shared classifier and focused tests for the
selected diagnostic codes, fallback evidence, wrappers, joins, cancellation,
and real ConfigKit loading errors. Verified with Go 1.27.0 using
`GOTOOLCHAIN=local GOPROXY=off go test ./integrations/ckconfig -run 'TestConfigurationDiagnosticClassification|TestClassifyConfigurationError' -count=1`
in the aligned workspace (with a temporary writable Go build cache).
Add `integrations/ckconfig/errors.go` with the following exported function and
GoDoc describing its CLI classification purpose, accepted evidence, fallback,
nil behavior, and precedence.
```go
func ClassifyConfigurationError(err error) commandkit.OutcomeKind
```
The function returns an outcome kind and reads error evidence. Handlers own
outcome construction, explanations, causes, and invocation context.
### Exact behavior
1. A nil error returns `commandkit.OutcomeSuccess`.
2. A nonnil error matching `context.Canceled` or `context.DeadlineExceeded`
through `errors.Is` returns `commandkit.OutcomeOperational`.
3. Recognize a nonnil `*configkit.LoadError`, including ordinary wrapping, and
inspect its complete `Diagnostics()` result. Ignore warning diagnostics.
Require at least one error-severity diagnostic and classify as usage only
when every error diagnostic has a usage code in the table below. An unknown
severity, an operational or unknown error code, or insufficient error
evidence selects operational.
4. Traverse ordinary `Unwrap() error` wrappers and `Unwrap() []error` branches.
A join selects usage only when every nonnil failing branch selects usage.
An operational branch wins in either ordering. A nonnil error exposing an
empty child list is an unclassified failure and selects operational.
5. Other nonnil errors select `commandkit.OutcomeOperational`. This includes an
empty or typed-nil `*configkit.LoadError`, opaque application errors, and
native PromptKit errors carrying their existing domain classifications.
Use the public ConfigKit constants in an explicit switch. The following table is
the complete usage-code list for the selected ConfigKit v1.2.0 baseline. All
error codes outside it select operational.
| Evidence | ConfigKit constants selecting usage |
| --- | --- |
| YAML content and fixed parser input bounds | `CodeYAMLInvalid`, `CodeYAMLEmpty`, `CodeYAMLMultipleDocuments`, `CodeYAMLRootType`, `CodeYAMLDuplicateKey`, `CodeYAMLTag`, `CodeYAMLAnchor`, `CodeYAMLAlias`, `CodeYAMLMergeKey`, `CodeYAMLScalar`, `CodeYAMLDepthLimit`, `CodeYAMLNodeLimit` |
| Typed decoding | `CodeDecodeType`, `CodeDecodeNull`, `CodeDecodeRange`, `CodeDecodeDuration`, `CodeDecodeText`, `CodeDecodeUnknownField`, `CodeDecodeArrayLength` |
| Excessive individual source input | `CodeSourceLimitExceeded` |
| Unsupported import directives and invalid graph relationships | `CodeImportDisabled`, `CodeImportCycle`, `CodeImportRepeated`, `CodeImportVersion` |
| Sequence declarations and peer composition | `CodeSequenceInvalid`, `CodeSequenceConflict`, `CodeComposeInvalid`, `CodeComposeConflict` |
| Protected version contributions | `CodeUnderlayVersion`, `CodeOverlayVersion` |
| Invalid overlay relationships and directives | `CodeOverlayRepeated`, `CodeOverlayDirective` |
| Version and explicit semantic rejection | `CodeVersionType`, `CodeVersionError`, `CodeResolveError`, `CodeValidateError` |
| Inheritance declarations and references | `CodeInheritanceInvalid`, `CodeInheritanceMissing`, `CodeInheritanceCycle`, `CodeInheritanceVersion` |
| Invalid override paths and conflicting assignments | `CodeOverrideInvalidPath`, `CodeOverrideDuplicate`, `CodeOverrideOverlap`, `CodeOverrideDirective`, `CodeOverrideVersion` |
| Excessive override input | `CodeOverrideLimitExceeded` |
| Invalid or absent typed secret references | `CodeSecretRequired`, `CodeSecretInvalidName` |
The mapping follows the public code meanings and ConfigKit v1.2.0's
`docs/config.md` sections on input limits, YAML documents, composition,
inheritance, overrides, secret references, and safe failures. Review the pinned
module source and documentation in the Go module cache when implementing it.
The fallback intentionally includes these reviewed cases.
- Acquisition failures, including missing or unreadable sources and imports,
permission failures, non-regular sources, and unavailable overlays.
- `CodeImportInvalid` and `CodeOverlayInvalid`, which cover both invalid content
and acquisition/context failures. Their codes alone provide ambiguous evidence.
- `CodeImportOutsideRoot` and `CodeOverlayOutsideRoot`, where containment can
reflect source resolution and filesystem conditions.
- `CodeImportLimitExceeded` and `CodeOverlayLimitExceeded`, whose acquisition
budgets can encompass source graph and filesystem traversal work.
- Schema and options errors, source-description and reused-reader errors,
`CodeHistoryLimitExceeded`, and callback failures or invalid callback actions.
- `CodeOverrideInvalidLabel` and `CodeOverrideInvalidValue`, whose meanings also
cover malformed application-created records and unsupported Go values.
- `CodeSecretUnset` and `CodeSecretEmpty`, which describe credential acquisition
from the runtime environment. Secrets-directory bootstrap failures also follow
the operational fallback.
The fixed source/parser bounds and override input budgets in the usage list
identify excessive supplied input. The acquisition and retained-evidence budgets
listed above retain operational classification under the accepted fallback.
These choices classify returned evidence and leave upstream limits and
validation rules with ConfigKit.
Implement straightforward error-tree traversal. Recognize the current typed
loading error before following its children. Inspect join branches separately
so an `errors.As` search over a whole join cannot hide later failures. Perform
the cancellation `errors.Is` checks once at the public entry point, then walk
the tree and inspect each encountered loading error's diagnostics once. A small
private diagnostics classifier and recursive walker are appropriate here.
Add focused tests in `integrations/ckconfig/errors_test.go`.
- Exercise the mapping through the private diagnostics classifier using ordinary
diagnostic values, including warnings, mixed evidence, unknown codes, and
missing error evidence. Include each selected usage constant and representative
operational families.
- Exercise the exported function with real public loading-error fixtures,
wrappers, all-input joins, mixed joins in both orders, nested joins,
cancellation, deadlines, nil, and empty loading errors. Assert classification
and preservation of the original error's supported identity.
Generate representative fixtures through public ConfigKit loading APIs. Assertions
own CommandKit's mapping and traversal. Upstream domain validation remains owned
by ConfigKit under the [testing policy](../policy/testing.md#upstream-integration-testing-ownership).
Run the focused classifier tests. Stage 5 owns the final independent module suite.
## Stage 2 — Apply classification and cancellation precedence in ckconfig
Status — complete. The configuration failure boundary now incorporates
invocation cancellation and selects the shared classification. Tests cover all
five operations, both comparison sides, filtered validation, report delivery,
operational failures, and cancellation with retained causes. Updated the native
override and simple-example expectations for their input diagnostic codes.
Verified with Go 1.27.0: `GOTOOLCHAIN=local GOPROXY=off go test
./integrations/ckconfig/...` in the aligned workspace, using a temporary Go
build cache; both the integration and simple example passed.
Update `configFailure` in `integrations/ckconfig/operations.go` to receive
`context.Context` as its first argument. Update its callers in `operations.go`,
`show.go`, `sources.go`, `explain.go`, `validate.go`, and `diff.go` to pass the
invocation context. Add context to `configAcquisitionFailure` and pass it through
from `runConfigShow`.
At this failure boundary, sample `ctx.Err()` when context is available. If it is
nonnull, use it as the cause when the operation cause is nil, or join it with the
operation cause when `errors.Is` does not already identify it. Preserve the
original cause directly when additional cancellation evidence is absent.
Call `ClassifyConfigurationError` with that combined cause. Construct
`commandkit.Usage` for usage classification and `commandkit.Failure` otherwise.
Keep the latter default for intentionally reported failures with a nil cause.
Retain the existing explanation selection, details, and cause attachment.
Apply the result to all acquisition errors reaching the five configuration
operations. Classification uses the returned error's complete evidence. Detailed
validation filters and projected `View.Findings` are report inputs. Successful
callbacks continue through their existing result flow.
Follow the existing operation-specific reporting lifecycle. Sources can emit
their inventory after failed loading, validation can emit its reached-scope
report, and comparisons retain side-specific diagnostics. The existing
`config diff --check` domain outcome and core presentation rules apply.
Extend the existing command and native-adapter tests to cover these outcomes.
1. A wrapped malformed-YAML loading error returned through `FromConfigKit`
produces usage status `2` for show, sources, validate, explain discovery, and
diff. Assert the actionable diagnostic, typed cause, and appropriate report
evidence for each operation.
2. Diff handles a failure on either loading side and preserves the acquired
diagnostic context. A detailed-validation filter hiding an error has the same
classification as the unfiltered failure.
3. Ordinary acquisition failures and a representative operational loading error
produce status `1`.
4. Already-canceled invocations produce status `1`, retain cancellation identity,
and leave stdout empty before acquisition. A callback that cancels context
and returns a loading error retains both causes and selects status `1`.
5. An output failure while delivering a source or validation report after an
input failure follows core's operational presentation-failure rule while
retaining the configuration cause.
Use existing coverage in `detailed_validation_test.go`, `diff_execution_test.go`,
the source/report tests, and `contracts_adapter_test.go` where suitable. Update
the invalid-fixture expectations in `examples/simple/main_test.go`, which
currently assert operational outcomes for native loading errors. Check related
expectations against their actual fixture codes before changing them.
Run the changed command tests and the affected simple-example tests. Stage 5
owns the final independent ckconfig suite.
## Stage 3 — Apply the shared function to PromptKit inspection and requests
2026-11-16 18:05:24 +00:00
Status — complete. PromptKit failure handling now samples invocation
cancellation and applies the shared classifier, including the smoke handler
call sites needed for compilation. Tests cover every catalog, named inspection,
requirements, validation, preparation, native loader and builder failures,
and cancellation with preserved causes. With Go 1.27.0, the aligned workspace
passed `GOTOOLCHAIN=local GOPROXY=off go test ./integrations/ckprompt -skip
'^TestPublicConfigDiffUsesPromptKitPairAndPreparedInputs$'`; that composition
test and the composed example retain expectations for Stage 4. A temporary Go
build cache was used.
Update `definitionFailure` in `integrations/ckprompt/command.go` to receive the
invocation context and combine observed cancellation with the operation cause
using the same handler-level approach as Stage 2. Call
`ckconfig.ClassifyConfigurationError` and choose Usage or Failure while retaining
the existing explanation and cause. A nil cause for a missing provider result
still yields an operational failure.
Update all callers, including the catalog fallback in `newCatalogListCommand`,
the handlers in `command.go`, `validation_command.go`, `preparation.go`, and the
smoke handlers that Stage 4 will verify.
Cover every installed catalog and named prompt/profile/backend inspection,
requirements discovery, concrete validation, and prepared-request preview.
Configuration loading errors may come from engine loaders or request builders.
The returned ConfigKit evidence selects the classification in each path.
Native lookup, preparation, validation, and provider errors with insufficient
configuration evidence take the operational fallback. Existing preflight and
application input-check contracts continue to govern their own failures.
Retain the original upstream error and diagnostic adaptation in `adapters.go`.
Use the newly context-aware failure boundary to observe cancellation when a
callback returns an error before the handler's later success checkpoint.
Extend tests around `command_test.go`, `requirements_command_test.go`,
`validation_command_test.go`, and `preparation_test.go`.
- Supply a shared real ConfigKit loading-error fixture through each relevant
suite callback and representative native loader/builder paths. Assert usage
status `2`, diagnostics, and `errors.As`/`errors.Is` preservation.
- Cover an ordinary PromptKit lookup error, an ordinary loader or builder
failure, and missing-result handling as operational outcomes.
- Cover pre-canceled input and cancellation occurring with a returned loading
error, checking status `1` and retained cancellation/loading identity.
Use table-driven cases for handlers with equivalent failure behavior. Reuse
existing tests for forwarding and lifecycle semantics. Verify the affected
packages using the aligned workspace.
## Stage 4 — Verify smoke aggregation and configuration composition
Status — complete. Verified individual and aggregate smoke outcomes, mixed
failures and cancellation with retained reports and causes, and all five
operations through the composed PromptKit configuration scope, including the
right comparison side and pre-canceled invocations. Updated the existing
comparison and composed-example expectations for input diagnostics. With Go
1.27.0, `GOTOOLCHAIN=local GOPROXY=off go test ./integrations/ckprompt/...`
passed in the aligned workspace using a temporary Go build cache.
In `integrations/ckprompt/smoke_command.go`, complete the context argument updates
and apply the shared classification to individual failures and the final joined
aggregate cause. Follow the existing sequential four-kind execution, cancellation
stop behavior, report delivery, explanation combination, and acquired diagnostic
flow. The classifier supplies the new aggregate outcome.
Extend `smoke_outcome_test.go`, `smoke_aggregate_test.go`, and related command
tests to verify a single input failure, an all-input aggregate, a mixed
input/operational aggregate, and cancellation joined with an input failure.
Assert the available reports and all supplied error identities. Existing
report-with-error behavior remains part of those assertions.
Exercise `ConfigScope` through the public ckconfig family in ckprompt tests.
Use a loader returning a wrapped malformed-YAML loading error and cover show,
sources, validate, explain discovery, and diff, including the right comparison
side. This establishes that composed PromptKit configuration commands use the
shared ckconfig policy. Check the accompanying source and validation report
evidence and the already-canceled behavior.
Update affected assertions in `examples/composed/main_test.go` against the
fixture's actual evidence. Add a focused fixture only where it establishes a
CommandKit-specific composition or outcome boundary. Reuse the existing example
and adapter tests for ordinary upstream capability forwarding.
Run the focused smoke/composition tests. Stage 5 owns the final aligned-workspace
ckprompt suite. Review native lookup and operational error tests for accidental
classification broadening.
## Stage 5 — Update documentation, validate, and record release coordination
Status — complete. Updated the CLI and consumer contracts, development
guidance, ckconfig v0.8.0 release note, and dependency-first release handoff.
With Go 1.27.0 and a temporary writable Go build cache, the independent ckconfig
suite passed with `GOWORK=off GOTOOLCHAIN=local GOPROXY=off go test
-mod=readonly ./...`. In the aligned workspace, the three prescribed
`GOTOOLCHAIN=local GOPROXY=off go test -mod=readonly` commands passed for
`./...`, `./integrations/ckconfig/...`, and `./integrations/ckprompt/...`.
Reviewed the diff, local Markdown links and fragments, and whitespace. The
independent ckprompt build remains pending publication and dependency alignment.
Update the implemented CLI contract in `docs/cli.md`, including the outcome
table and affected load-failure wording in configuration comparison, inspection,
requirements discovery, concrete validation, catalogs, preparation, and smoke
testing. Document input classification, operational fallback, mixed-error
precedence, and the existing output-failure rules. Add the new exported API's
GoDoc and concise consumer guidance linked to it.
Prepare `integrations/ckconfig/docs/releases/v0.8.0.md` explaining the status
contract change and shared classification API, using the module's existing
release-note conventions and immutable licensing footer. Record the planned
ckprompt v0.8.0 note and published-dependency update as release follow-up under
`docs/roadmap/future.md`. The ckprompt candidate is prepared after ckconfig
publication under the [release procedure](../release.md#scope-and-release-order).
### Final sprint verification
Format changed Go files. Run the commands below from the indicated module
directories, with Go 1.27.0 on PATH. The default suites are deterministic and
offline once selected dependencies are present in the module cache.
From `integrations/ckconfig`, verify its published dependencies independently.
```sh
GOWORK=off GOTOOLCHAIN=local GOPROXY=off go test -mod=readonly ./...
```
From the repository root, verify all three local modules together.
```sh
GOTOOLCHAIN=local GOPROXY=off go test -mod=readonly ./...
GOTOOLCHAIN=local GOPROXY=off go test -mod=readonly ./integrations/ckconfig/...
GOTOOLCHAIN=local GOPROXY=off go test -mod=readonly ./integrations/ckprompt/...
```
The integration commands include their maintained example packages. Review
the final diff, Markdown links/fragments, and `git diff --check`. Confirm that
the shared code mapping, error trees, cancellation, reporting, both scope
adapters, and command families have the planned regression evidence. Report
actual checks and any remaining failures.
ckprompt's independent published-dependency build is pending the ckconfig release
because its current v0.7.0 selection lacks the new function. Keep that limitation
visible in the development/consumer guidance and release handoff. Retain the
existing published requirements through the implementation sprint. The ignored
aligned workspace is the local build setup for the new source combination.
### Release handoff
Record these follow-up actions together in `future.md` so they survive later
retirement of the implementation documents.
1. Publish ckconfig v0.8.0 from its independently validated candidate using the
documented release process when publication is authorized.
2. Prepare ckprompt with actual published ckconfig v0.8.0 using `go get` and
`go mod download` with `GOWORK=off GOTOOLCHAIN=local`. Retain real checksums,
then finish its v0.8.0 release note, consumer compatibility guidance, and
release preparation. The other dependency selections follow their existing
release preparation rules.
3. Validate ckprompt independently against its updated published dependencies
through its release candidate suite and publish ckprompt v0.8.0 under the
same authorization and procedure. Track hosted release delivery under the
runbook's separate delivery process.
These actions follow the existing dependency-first release procedure. The sprint
can complete with verified local behavior and explicit pending release alignment.
The release handoff identifies the remaining independent-build evidence.
Mark each completed stage with its implementation and verification evidence.
Commit the completed sprint work under the repository policy and report its
commits, checks, and pending release coordination.