commandkit/docs/roadmap/implementation.md

# Implement ConfigKit adoption and configuration comparison

Status: implementation plan; no implementation stages are complete.

## Scope and authority

Follow the stages in order, starting with Stage 1. Each stage is sized for one
Luna implementation assignment. The [feature roadmap](configkit.md) owns the
selected end state, operator behavior, target API, and scope limits. This document
owns the steps to reach that state. Read [development guidance](../development.md)
and the applicable policies before implementation. Accepted ADRs and the
[architecture policy](../policy/architecture.md) control if a conflict is found.

The baseline inspected for this plan is `f8bb789`. ConfigKit comparison decisions
are accepted in [ADR 0008](../adr/0008-semantic-configuration-diff.md) and
[ADR 0012](../adr/0012-typed-configuration-comparison-pairs.md). ADR 0008 delegates
comparison semantics to the adopted upstream version; it does not freeze those
semantics as separate CommandKit guarantees. Report details belong in the feature
roadmap until implemented, then in the CLI reference.

This is one sprint against an unreleased prototype. Preserve deliberately selected
contracts, but give existing implementation shapes no compatibility preference.
Intermediate commits need not build. Do not introduce wrappers or extra machinery
to make intermediate states usable. The completed sprint must compile and provide
the complete selected behavior. Core needs no new comparison model or dependency;
reuse its existing typed bindings, captured declarations, and outcomes.

During implementation, add focused coverage with the stage that owns its behavior,
format changed Go files, and record completion evidence here after each stage.
Commit completed nonempty units under the agent policy. Evidence must describe
actual checks and any limitations, not anticipated success. These development
checks do not create new release gates. This planning pass runs no Go tests and
changes no implementation, manifests, examples, or ADRs.

## Stage 1: Adopt ConfigKit and establish local validation

**Outcome:** both integrations select ConfigKit v0.7.1, with a usable development
graph and an honest distinction between local and published dependency evidence.

1. Read both integration manifests and the pinned ConfigKit release/comparison
   contracts linked from the feature roadmap. Update the two direct ConfigKit
   requirements from v0.5.0 to v0.7.1. Keep PromptKit v0.12.0 and consumer Go
   minimums at 1.26.0. Use the selected Go 1.26.8 development/release toolchain.
2. Preserve an existing ignored `go.work`; ensure it includes the root, ckconfig,
   and ckprompt modules. Do not commit workspace files or add filesystem
   replacements to distributed manifests. Use released upstream dependencies,
   not sibling source replacements.
3. Resolve actual upstream dependencies and sums. ConfigKit's tool requirement
   selects ReleaseKit v0.4.1 in integration graphs; retain whatever legitimate
   indirect requirements normal resolution needs. Do not change core's declared
   ReleaseKit v0.4.0 tool, launcher, or profiles as an incidental upgrade.
4. For unpublished CommandKit requirements, use the disposable module-file
   procedure in Stage 9. Generate real external dependency checksums; never
   invent sums for unpublished CommandKit versions. Keep normal manifests at
   their intended versioned requirements. Reconcile module-local sums from real
   dependency downloads, excluding disposable local wiring.
5. Run existing ckconfig and ckprompt suites against the upgraded dependency.
   Fix actual upstream API incompatibilities within this scope, if any. Do not
   compensate for a test that independently freezes a superseded upstream detail.

**Completion evidence:** reviewed manifest/sum changes; upgraded integration
graph selections; existing inspection and PromptKit suite results. Record any
unpublished dependency limitation explicitly.

## Stage 2: Add the comparison capability and captured provider contract

**Primary files:** `integrations/ckconfig/contracts.go`, `family.go`.

**Outcome:** scopes can independently declare comparison, without requiring
ordinary acquisition for a comparison-only scope.

1. Add `Diff Operation = "diff"`. Extend normalization, generated-operation
   inventory, and selector-union assembly to recognize it. Append it after the
   four existing operations so their relative order stays unchanged.
2. Add `ComparisonSide[I]`, `ComparisonRequest[I]`, and `Scope[I, S].Compare`
   exactly as selected under the roadmap's
   [provider contract](configkit.md#comparison-provider-and-configkit-adapters).
   Keep these exports integration-owned; keep `Request`, `View`, and ordinary `Acquire` focused on
   existing operations. Add GoDoc for sides, diagnostics, errors, and lifetimes.
3. Change capture-time wiring checks: declared `Diff` requires `Compare`;
   declared validate/show/sources/explain operations require `Acquire`. A scope
   declaring both kinds needs both. Metadata-only scopes need neither.
   Preserve retained adapter issues even when capability lists are narrowed.
4. Capture the comparison callback reference with the other scope metadata.
   Keep existing collection copies, original binding identities, root-only
   declaration placement, batch validation, and namespace ownership. Factories
   and providers must remain uninvoked during capture and installation.
5. Extend scope discovery and conditional family generation. Generate `diff`
   only when a scope supports it; selectors are accepted by that command only
   when applicable to a comparison-capable scope. Rely on existing core binding
   validation for collisions with generated flags.

**Completion evidence:** focused installation/discovery checks for comparison-only,
mixed, unsupported, and metadata-only scopes; missing callback and retained nil
loader issues; captured metadata surviving source edits. Execution is wired in
Stages 3 and 6; do not add temporary handlers for intermediate completeness.

## Stage 3: Prepare two presence-aware selections and typed inputs

**Primary file:** `integrations/ckconfig/input.go`.

**Outcome:** the comparison plan contains one selected scope and two independently
prepared inputs, ready for one shared acquisition state.

1. Add right-side selection storage and expose `--against-config` and
   `--against-overlay` only on `diff`. Keep `--config` and `--overlay` as the left
   selection. Default the comparison format to YAML; accept explicit YAML and
   reject unsupported formats through the existing usage-error path.
2. Resolve right fields independently: if the right `Supplied.Set` is false,
   copy the corresponding complete left `Supplied` value; otherwise keep the
   right value, including an explicit empty string. Do not interpret an empty
   overlay, normalize provider defaults, or require unequal selections.
3. Reuse existing scope/capability preflight. Comparison defaults to `app`,
   selects exactly one scope, and rejects `all`, unknown scopes, and scopes
   without comparison before suite input/state factories or provider calls.
4. Capture the operation's original selector handles through `CaptureFlags` once.
   Prepare left input and then right input with `NewInput` when reached. Apply
   only the selected scope's bindings independently through `FlagValues.ApplyTo`.
   Call `CheckInput` with the scope name, that side's selection, and its input.
   Store both checked sides in the operation plan; leave shared-state creation
   to execution after both checks succeed.
5. Preserve nil-factory zero values, caller-owned objects/aliases, copied
   core-owned repeated collections, and early failure. Do not clone arbitrary
   inputs or add pointer allocation. Preserve existing ordinary-operation paths.

**Completion evidence:** table-driven selection/presence coverage for omission,
explicit empty overrides, single-field inheritance, and equal selections;
applicable required selectors, explicit false/zero, parser conversion once, and
repeated-value independence. Verify preflight/check failures stop state/provider
work. Help converts supplied values without invoking suite factories/providers.
Do not assert calls that the accepted early-failure contract allows to be skipped.

## Stage 4: Implement typed native comparison and diagnostic retention

**Primary file:** `integrations/ckconfig/configkit_adapter.go`.

**Outcome:** `FromConfigKit` and consequently `AppCommand` support comparison using
their existing typed loader, with native records and diagnostics preserved.

1. Include `Diff` in `FromConfigKit`'s supported operation list and wire its
   comparison callback. Keep the generic `T` inside that adapter; no new view
   facet, result assertion, reflection, or private upstream import is needed.
2. Load left and then right with their respective selection/input and the same
   supplied state. Check context at ordinary boundaries, stop after a failed
   reached load, and do not skip either reached load just because selections
   compare equal. A successful callback returning nil is operational failure.
3. Adapt diagnostics from each reached successful result or `LoadError` using
   existing conversion. Accumulate left diagnostics before loading right, and
   retain both loads' diagnostics before invoking native comparison. Preserve
   side identity in comparison diagnostic headings so source IDs from two loads
   are not confused. Leave ordinary diagnostic headings unchanged.
4. Call `configkit.Diff(left, right)` and return its native records in
   `commandkit.Acquired[[]configkit.Difference]`. On error, return accumulated
   diagnostics with a cause preserving supported `errors.Is`/`errors.As`
   identity. Add useful scope/side context without making exact error wording
   or callback internals a new contract.
5. Reuse small local acquisition/diagnostic helpers only where they simplify
   ordinary and paired loading. Keep cache policy and arbitrary callback output
   application-owned. Update adapter/convenience GoDoc from four operations.

**Completion evidence:** direct adapter tests with real ConfigKit loads for
public changes, base-versus-overlay loading, equal effective values,
defaults/resolution, and paired-safe concealed changes. Test right failure
retaining left warnings, wrapped load-error
identity, nil results, and cancellation between loads. Exercise a native
comparison error with a non-nil uninitialized result and retain available
diagnostics. Do not build a global fake comparator just to force an unreachable
branch; handler-level provider errors can carry diagnostics from both sides.

## Stage 5: Render the selected three-category YAML report

**Primary new file:** `integrations/ckconfig/diff_report.go`.

**Outcome:** native safe changed regions become the selected YAML document,
without another equality engine or an untyped value round trip.

1. Build a private report representation using `go.yaml.in/yaml/v4` nodes.
   Initialize ordered `additions`, `changes`, `deletions` entries, with empty
   mapping, empty mapping, and empty sequence nodes respectively. Use the
   existing two-space encoder convention. No report type needs to be exported.
2. Classify each record solely by native left/right `Present`. Append deletion
   paths using `Path.String()`. For additions and changes, decode only the
   paired-safe right snapshot YAML to a node and place it in the appropriate
   category at the record's semantic path. Preserve scalar tags/value types,
   explicit null, empty collections, and whole sequence/structural regions.
3. ConfigKit's public path API exposes canonical JSON Pointer spelling, not
   public segment access. Handle the empty spelling as the root before splitting;
   `/` identifies one empty mapping key. For non-root pointers, remove the leading
   separator, split on `/`, and decode `~1` and `~0` once. Preserve
   empty, numeric, dot, percent, slash-containing, and tilde-containing keys.
   Do not use filesystem cleaning, URL decoding, or numeric-index guessing.
   Native sequence changes are atomic, so report ancestors for descendant
   records are mappings. Construct mapping keys as YAML strings.
4. Create category ancestors as needed, inserting nodes in the native traversal
   order. A root record assigns its safe node directly to the category; it need
   not be a mapping when the safe snapshot is null or wholly masked. Keep the
   accepted whole-root ambiguity; add no special root envelope or hidden-value
   analysis. Mapping-to-empty changes use the native deletion records.
5. Do not inspect left YAML to derive changes, compare rendered snapshots,
   expand atomic regions, infer sensitivity from marker text, append full
   configurations, or expose `Redacted`/`HiddenChanged` report fields. Concealed
   changes remain included because the native record exists, even when safe
   snapshots look equal. Do not claim every masked member changed.
6. Encode the completed document into a buffer before report delivery. Parsing
   or encoding failures return an error; no successful report bytes have been
   written. Apply ordinary error handling rather than adding a general provider
   schema-validation framework or new restrictions on legitimate native records.

**Completion evidence:** semantic YAML checks for all three categories and the
empty document; nested additions/changes, deletion versus null, empty nested
containers, escaped/empty/numeric keys, and large integers or scalar-looking
strings. Cover atomic sequences, structural replacements, native masks, literal
marker-looking public values, and concealed-only changes. Protect ordered root
keys and types, without freezing incidental emitter spelling or mapping styles.

## Stage 6: Execute comparison through existing outcomes and delivery

**Primary files:** `integrations/ckconfig/operations.go`; new `diff.go`.

**Outcome:** an installed `config diff` works end to end with the selected
preflight, acquisition, reporting, cancellation, and outcome behavior.

1. Add the operation name/summary and dispatch case. Use the existing execution
   path to create shared state once after checked pair preparation. Pass the
   invocation context, paired request, and state to the selected `Compare`.
   Do not call ordinary `Acquire` for comparison.
2. Copy/retain provider diagnostics through the outcome. On provider/native
   comparison failure, return operational failure with supported cause identity
   and no successful report. Check context after acquisition and before delivery;
   do not promise interruption inside synchronous upstream calls.
3. Render with Stage 5, then use existing checked report delivery. Parsing or
   encoding errors are operational failures before report writing. Output errors
   or short writes preserve the established delivery error and incomplete-report
   behavior. No new exit status, assertion mode, or process termination is added.
4. Successful comparisons return success for public differences, concealed
   differences, and no records. Nil/empty returned record slices both represent
   an empty successful comparison. Diagnostic stream handling remains core-owned.
5. Update focused public execution tests to assemble real declarations and run
   through core/Cobra. Reuse existing test helpers rather than constructing
   another dispatcher or private core bridge.

**Completion evidence:** default/explicit YAML, text rejection, file/overlay
selection inheritance, comparison-only scope, discovery/help availability, and
unsupported scopes with zero provider work. Cover provider diagnostics with an
error, wrapped identity, report preparation failure, cancellation, output/short
writes, and repeated invocations with fresh inputs/state. Existing four operations
and their output defaults must continue to work.

## Stage 7: Demonstrate correct consumer composition and shared caches

**Primary owners:** ckconfig's simple example and ckprompt's composed example.

**Outcome:** maintained consumers demonstrate meaningful comparison while
PromptKit inspection remains independent and usable.

1. Extend the simple example's application loader to honor a supplied config
   file through ConfigKit's public source API, retaining its existing default
   configuration when omitted. Add two small local YAML fixtures with different
   greetings and a documented `config diff` invocation. Keep `AppCommand` wiring
   as the minimal consumer path; do not add a bespoke comparison handler.
2. Repair the composed example's single-result `runCache` before demonstrating
   paired loads. Key cached results by the complete selection, including presence,
   and the supplied `Campaign` input used below. These fields are comparable;
   a small application-owned key and map suffice. This is example guidance, not
   a CommandKit cache guarantee or service.
3. Make the composed application loader honor a supplied ConfigKit source;
   retain its synthetic default when omitted. Add two local fixtures with valid
   existing campaign definitions and different default campaigns. Demonstrate
   application-scope comparison through the shared state, and verify the two
   source selections produce an actual changed value rather than cache equality.
   To demonstrate meaningful typed input, apply explicitly supplied `Campaign`
   through ConfigKit's public `Options.Overrides` to the existing
   `default_campaign` field, using `configkit.Keys("default_campaign")` and the
   safe flag label `--campaign`. Apply only when supplied; do not mutate
   `Result.Config` after loading. `Limit` remains an input check rather than a
   loading dependency. Explain these example-specific choices and cache identity;
   they prescribe no downstream application behavior.
4. Keep `ckprompt.ConfigScope` without `Diff`. Verify comparison selects the
   application scope while engine validate/show/sources/explain, named prompt and
   profile inspection, and `session restore` still work under PromptKit v0.12.0.
   Do not add PromptKit comparison by reaching into its ConfigKit internals.
5. Add small consumer tests for fixture comparisons and cache reuse/separation,
   including separation for different supplied campaign values and presence.
   Use local sources and synthetic values; no model requests or credentials.

**Completion evidence:** meaningful simple and composed diff reports, cache
separation for different selections, unchanged representative PromptKit/domain
commands, and a usage failure for unsupported `--scope promptkit` comparison.

## Stage 8: Reconcile documentation and distributed dependency material

**Outcome:** current documentation and module notices describe implemented
behavior, while future/deferred work stays clearly separate.

1. Update ckconfig GoDoc, its README, the CLI reference, consumer guide,
   internal overview, relevant example READMEs, and development commands.
   Document exact new exports in GoDoc and exact CLI/report semantics in the CLI
   owner; other documents summarize and link. Include inheritance/presence,
   lifecycle, independent capabilities, native semantics, reduced safe reports,
   ordinary null/deletion distinction, accepted root ambiguity, and outcome rules.
2. Move the implemented DIFF-01 portion out of deferred work. Retain unsupported
   PromptKit comparison, catalogs, independent per-side domain selectors, and
   unrelated deferred ideas. Update this plan's completion evidence and the
   feature roadmap's implementation status; do not keep proposals disguised as
   current operator/API documentation.
3. Reconcile module-local ConfigKit notices to v0.7.1 and its pinned source commit
   from the roadmap. Inspect resolved integration tool graphs for ReleaseKit
   v0.4.1 and relevant transitives; update actual versions, roles, pinned links,
   and any required license/notice material under the OSS policy. Root notices
   describe core's own graph, not the combined workspace's selected versions.
4. Finalize normal manifests/sums using real released upstream dependencies.
   Preserve intended CommandKit requirements without local replacement clauses
   or fabricated unavailable checksums. Any remaining prepublication limitations
   belong in the validation evidence and existing release procedure.
5. Update the still-unpublished integration v0.1.0 release notes to describe
   dependency adoption and comparison availability where appropriate. Recheck
   tag status first: never revise a published release record. If publication has
   occurred since this baseline, prepare a fresh version under the release
   policy rather than rewriting history. Do not add a core release solely for
   integration behavior, bump PromptKit, publish tags, or run the release launcher.
6. Check local links/fragments, module notice paths, whitespace, and stale claims.
   Keep accepted ADR decision text unchanged; implementation/report details do
   not require moving them back into ADRs.

**Completion evidence:** reviewed current owners and notices, correct native
dependency attribution, no stale four-operation convenience claims or text-diff
promises, and normal distributed manifests without local wiring.

## Stage 9: Validate the complete sprint and finish

**Outcome:** evidence proves the complete feature end state across all three
module boundaries, without implying publication or new guarantees.

1. With upstream dependencies prepared, run the deterministic default suites
   in the development workspace:

   ```sh
   go test ./...
   (cd integrations/ckconfig && go test ./...)
   (cd integrations/ckprompt && go test ./...)
   ```

   These commands traverse their own modules only. Do not run paid/live calls,
   release publication exercises, or add coverage/performance/platform gates.
2. Independently check core with `GOWORK=off GOTOOLCHAIN=local go test -mod=readonly
   ./...`. Inspect its workspace-off `go list -m all` and `go list -deps ./...`
   output for the established absence of optional integration/owning-library
   dependencies and for the unchanged declared maintainer tool selection.
3. Check each integration outside workspace mode. If its normal CommandKit
   prerequisites are downloadable, use its normal manifest. Otherwise use a
   disposable alternate manifest, keeping its actual ConfigKit/PromptKit versions:

   - Create a temporary directory; copy that integration's `go.mod` to a
     `.mod` file there and copy any existing sum file to the matching `.sum`.
   - From the integration directory, use `go mod edit -modfile=<temporary.mod>`
     to replace the root CommandKit module with the absolute repository root.
     For ckprompt, also replace the ckconfig module with the absolute local
     ckconfig directory. Do not replace ConfigKit, PromptKit, or ReleaseKit.
   - With `GOWORK=off GOTOOLCHAIN=local`, tidy the alternate file, then run
     `go test -modfile=<temporary.mod> -mod=readonly ./...`, `go list
     -modfile=<temporary.mod> -m all`, and `go list -modfile=<temporary.mod> -deps
     ./...`. Inspect selected versions, package imports, and boundary violations.
   - Discard alternate files after review. They establish local-source module
     isolation and upgraded external compatibility, not availability of future
     CommandKit tags. Never copy their replacements into committed manifests.

4. Run the maintained simple/composed comparison examples and representative
   existing inspection/domain commands. Inspect YAML content, diagnostic streams,
   and outcomes. Confirm no comparison is advertised by the PromptKit adapter.
5. Audit every feature-roadmap work area against changed code, tests, current
   docs, examples, manifests, and notices. Include both sides' input/check timing,
   callback-free discovery, equal-selection loading, warning retention, native
   safe values, all report categories, null/deletions, output errors, and ordinary
   inspection regressions. Reuse earlier successful evidence for unchanged areas.
6. Record actual commands/results and unresolved external limitations here.
   Review the complete diff including new files; check documentation whitespace
   and links. Commit intended implementation work under the agent policy and
   report remaining unrelated work. Follow existing release preparation later;
   this sprint does not publish or wait for module proxy availability.

**Completion evidence:** all three default suites, independent module graph
reviews (with local-source limits identified when applicable), maintained consumer
results, feature-to-evidence audit, and reviewed final documentation/distribution
changes. No significant product/API decisions remain open in this plan.