From 6830b96821d660da7321aacc3e567533faaddd78 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 6 Nov 2026 03:03:31 +0000 Subject: [PATCH] Expand operator roadmap with ConfigKit CLI features --- docs/roadmap/future.md | 39 +++--- docs/roadmap/promptkit-v0.19.0.md | 206 +++++++++++++++++++++++++++++- 2 files changed, 222 insertions(+), 23 deletions(-) diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index f7c8318..b0a03da 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -36,7 +36,7 @@ The published ckprompt v0.2.0 retains its earlier dependencies; see the ### PromptKit v0.19.0 adoption The [PromptKit v0.19.0 roadmap](promptkit-v0.19.0.md) owns the proposed adoption, -operator CLI additions, shared configuration inspection work, open decisions, +operator CLI additions for PromptKit and ConfigKit, open decisions, and completion checklist. Scope is agreed in principle and remains subject to roadmap review. Implementation has not started. Use released upstream contracts only; do not inspect or monitor unreleased work in anticipation of adoption. @@ -45,32 +45,35 @@ only; do not inspect or monitor unreleased work in anticipation of adoption. ### Comparison extensions -Independent per-side domain selectors and an optional assertion mode remain -unselected extensions to the delivered ConfigKit comparison. Consider them only -for a concrete consumer need and a deliberate API/behavior decision under the -relevant ADRs. +Independent per-side domain selectors and opt-in comparison assertions are now +included in the [working roadmap](promptkit-v0.19.0.md#additional-configkit-operator-workflows). +Their binding and outcome contracts remain open under roadmap OD-9 and OD-10; +retaining those proposals does not change current comparison behavior. ### Configuration inspection extensions The [PromptKit v0.19.0 roadmap](promptkit-v0.19.0.md#shared-configuration-capabilities) now includes the shared operator tasks previously retained here: metadata/path discovery and explicitness, failed-load source inventory, detailed comparison, -and structured diagnostics. Their earlier ConfigKit adoption identifiers were -OD-3 through OD-6; the new roadmap has its own decision numbering. Implementation +and structured diagnostics. It also includes focused effective display, +source-ID selection, metadata/diagnostic filters, independent comparison +selectors, and assertion mode. The original four shared proposals had ConfigKit +adoption identifiers OD-3 through OD-6; the new roadmap has its own decision numbering. Implementation belongs in `ckconfig`, with PromptKit exposing native evidence through its configuration scope. CLI syntax and report contracts remain subject to review. -Additional unselected work: - -- **Source-ID filtering.** Consider selecting a record already available in the - source inventory. Native `Source(id)` lookup is a convenience; this proposal - need not introduce another command or require a lookup method from every - custom source provider. +Transient acquisition overrides are an explicit open scope decision in +[roadmap OD-11](promptkit-v0.19.0.md#od-11-transient-configuration-overrides), +with a recommendation for opt-in CLI support and alternatives using +application-specific flags or a structured override document. Persistent +configuration editing remains deferred. Discovery, import defaults, collection identities, and field-level resolution -recording remain available through application loaders. Standard inspection of -resource contexts or secrets-directory bootstrap has no immediate proposal; -those capabilities primarily serve application loading and startup. +recording remain available through application loaders. Discovery tracing, +resource-context inspection, and secrets-directory bootstrap commands remain +deferred. Acquired source records do not provide a complete trace of rejected +discovery candidates; bootstrap and resource contexts primarily serve +application loading and startup. ### PromptKit inspection and selection extensions @@ -140,8 +143,8 @@ operator or consumer need and an appropriate ownership/behavior decision: - JSON reports and output-file management or reloadable export. - Shell completion extensions, including alias discovery described above and dynamic discovery of configured or remote values. -- Configuration mutation and generic value assignment. Field metadata and - provenance inspection are scoped under +- Persistent configuration mutation. Transient CLI override scope is owned by + roadmap OD-11; field metadata and provenance inspection are scoped under [configuration inspection extensions](#configuration-inspection-extensions). - Credential diagnostics beyond PromptKit's native smoke checks and additional live provider probes. diff --git a/docs/roadmap/promptkit-v0.19.0.md b/docs/roadmap/promptkit-v0.19.0.md index ff5b68c..b2943e2 100644 --- a/docs/roadmap/promptkit-v0.19.0.md +++ b/docs/roadmap/promptkit-v0.19.0.md @@ -1,4 +1,4 @@ -# PromptKit v0.19.0 adoption and operator CLI +# PromptKit v0.19.0 adoption and ConfigKit operator CLI Status: proposed roadmap for review. Scope is agreed in principle; implementation has not started. Command spellings, report designs, and API changes below are @@ -8,7 +8,7 @@ recommendations until the open decisions are resolved. Adopt released PromptKit v0.19.0 and expose useful operator tasks through standard CommandKit commands, flags, and reports. This roadmap covers CLI -exposure, including shared configuration inspection in `ckconfig`. It does not +exposure, including ConfigKit v1.1.0 operator features in `ckconfig`. It does not turn `ckprompt` into a convenience library for application-internal PromptKit operations. Applications continue to own engine construction, domain inputs, credentials, execution, and their executable lifecycle. @@ -45,6 +45,16 @@ and [smoke API](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/ Work against released versions only; do not inspect or monitor unreleased upstream work in anticipation of another adoption. +ConfigKit v1.1.0 is already adopted at released commit +`191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9`. Its +[inspection API](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/inspection.go), +[metadata API](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/metadata.go), +and [configuration reference](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/docs/config.md) +support the additional CLI work below without an upstream dependency upgrade. +Its matched history for registered records moving within lists flows through +native inspection; it does not introduce identity-based CLI paths or +element-level effective diffs. + Current `ckprompt` supplies prompt/profile inspection, requirements discovery, concrete request validation, prompt/profile/backend/namespace/collection catalogs, and four smoke operations plus `test all`. Smoke commands already @@ -62,6 +72,12 @@ are [ckconfig contracts](../../integrations/ckconfig/contracts.go), [ConfigKit adapters](../../integrations/ckconfig/configkit_adapter.go), and [PromptKit adapters](../../integrations/ckprompt/adapters.go). +Effective display currently writes the whole configuration. Sources are reported +as a complete inventory. Comparison accepts independent config/overlay choices, +but applies the same application selectors to both typed inputs and succeeds +when a comparison reports differences. The new tasks extend those boundaries; +they do not replace already-delivered explanation or smoke functionality. + ## Targeted end state The original adoption tasks remain included: @@ -84,6 +100,17 @@ the decisions at the end of this document. | Find configuration paths and inspect their explicitness | `config paths --scope promptkit`, with historical selection; metadata alongside path explanation | `ckconfig`, wired through `ckprompt.ConfigScope`: observed paths, effective presence, explicitness/null/origin evidence, and available history. | | Understand both sides of configuration changes | `config diff --scope promptkit --details` | `ckconfig`: native paths, left/right snapshots, presence/redaction, and hidden-change evidence. | | Investigate a failed configuration load | `config sources --scope promptkit` on failure; a structured diagnostics operation | `ckconfig`, with both native adapters: acquired source inventory and native codes, severities, paths, and locations while retaining the load failure. | +| Inspect one effective value or subtree | `config show --scope app --path /server` | `ckconfig`: the selected native effective snapshot, retaining redaction and absence/null distinctions. | +| Follow a source reference | `config sources --source-id ID` | `ckconfig`: the matching acquired source record, including evidence available after failure. | +| Compare different application selections | For example, `config diff --campaign spring --against-campaign autumn` | `ckconfig`: independently prepared typed inputs passed to the existing native comparison provider. | +| Detect configuration differences in scripts | `config diff --check` | `ckconfig`: a deliberate difference outcome based on native records, alongside the selected comparison report. | +| Narrow an inspection to relevant evidence | Subtree/explicitness filters on `config paths`; severity/code filters on `config diagnostics` | `ckconfig`: filtered native evidence with source references retained. | + +Transient configuration overrides are an open scope decision under OD-11, with +a recommendation to include them as an optional loader capability. Illustrative +`config validate --set /workers=0` and `config show --set /server/port=8080` +would apply explicit overrides during acquisition without editing source files. +Their inclusion and exact CLI/API contract remain for review. ### Compatibility and native semantics @@ -180,6 +207,50 @@ cancellation with partial evidence explicitly without promising interruption of an arbitrary writer or rollback of already-delivered bytes. OD-6 selects the structured diagnostics command and its interaction with ordinary diagnostics. +### Additional ConfigKit operator workflows + +**Focused effective display.** Use the effective snapshot from native +`Explain(path)` for path-selected display. The ordinary whole-document path +continues to use `WriteEffective`. A historical-only path can be explainable +while lacking an effective value; distinguish that case from a present YAML +null. Select the missing-value behavior, provider availability, and aggregate +scope applicability under OD-8. Do not read mutable `Result.Config` or source +files to reconstruct inspection values. + +**Source selection and report filters.** Select source IDs from the acquired +inventory, preserving existing record fields and upstream ordering. Native +`Source(id)` is a convenience, not an additional provider requirement. IDs are +local to a load and do not guarantee the same source in a later acquisition. +Apply subtree filtering to parsed configuration paths, not raw string prefixes +that would confuse `/server` with `/server2`. Specify explicitness filtering for +historical-only paths without treating unavailable effective metadata as false. +Diagnostic filters affect the presented records, not the underlying acquisition +outcome; retain the source references needed to interpret selected records. + +**Independent comparison selections.** Extend the per-side typed-input design +in [ADR 0012](../adr/0012-typed-configuration-comparison-pairs.md). Capture +applicable selector bindings for each side, prepare and check both inputs before +comparison, and preserve the existing shared invocation state. Resolve omitted +right selectors, explicit empty values, repeated selectors, and flag-name +collisions under OD-9. Domain interpretation and loader caching remain with the +application; the comparison provider still receives two selections. + +**Comparison assertions.** Base a difference outcome on native record presence, +including hidden changes whose redacted snapshots look equal. Equal effective +values can have different histories or explicitness. Comparison also does not +detect changed credential bytes behind an unchanged secret reference. Define +the opt-in outcome and report-delivery behavior under OD-10; retain the current +ordinary diff outcome. A load or comparison failure is not evidence of equality. + +**Transient override candidate.** If selected in OD-11, parse explicitly supplied +CLI values into native `Override` records and pass them through a declared loader +capability. ConfigKit owns override path validity, supported values, precedence, +decoding, resolution, provenance, and validation. Its configuration overrides +recursively overlay mappings and replace sequences; this differs from the +PromptKit execution override's complete provider-map replacement. Do not reuse +one grammar implementation in a way that erases that distinction. Source-load +errors remain errors even if an override would replace the affected value. + ## Work sequence 1. **Resolve the significant CLI choices.** Review the open decisions, record @@ -196,7 +267,11 @@ structured diagnostics command and its interaction with ordinary diagnostics. 4. **Implement shared configuration inspection.** Add metadata discovery and explicitness, detailed diff presentation, source evidence on failure, and structured diagnostics in `ckconfig`; wire native PromptKit facets through - `ConfigScope`. Keep each change reviewable with its boundary coverage. + `ConfigScope`. Add focused effective display, source selection, and bounded + metadata/diagnostic filters using those evidence capabilities. Extend typed + comparison selection and add the selected assertion outcome. If OD-11 is + accepted, implement explicit override acquisition and comparison-side wiring + as a separate reviewable unit. Keep boundary coverage with each change. 5. **Update operator and consumer documentation.** The CLI reference owns syntax, report schemas, availability, and outcomes; GoDoc owns callback and value contracts. Extend maintained simple/composed examples to demonstrate @@ -235,6 +310,21 @@ Default tests remain offline and independent of real credentials. available history on absent lookup, and projects native field/change records. - [ ] Detailed diff retains both snapshots and presence/redaction/hidden-change evidence, including empty comparisons and sequence-level changes. +- [ ] Focused display preserves native snapshot content and distinguishes + missing/historical-only paths from explicit null. Whole-document display + retains its current contract. +- [ ] Source-ID selection works on successful and failed acquisitions. Path, + explicitness, severity, and code filters preserve the selected evidence and + source references without masking the acquisition outcome. +- [ ] Independent comparison selectors preserve the chosen inheritance and + repeatable-value rules, side-specific checks, and shared-state lifecycle. +- [ ] Assertion mode distinguishes equality, native differences (including + hidden changes), usage errors, and operational failures under the selected + outcome contract. Report-delivery failures remain observable. +- [ ] OD-11 has an explicit scope decision. If included, only supplied CLI + overrides reach opted-in loaders, with typed values, safe flag labels, + selected comparison-side behavior, native errors, and returned provenance + preserved. No tests independently re-establish ConfigKit override semantics. - [ ] Source and diagnostic evidence can be reported with an acquisition error; the original failure, unavailable-versus-empty evidence, cancellation, and report-delivery failures remain distinguishable. @@ -273,9 +363,12 @@ Focused backend inspection remains in [future work](future.md#promptkit-inspecti until a released upstream API supports it. Do not reconstruct backend selection from configuration to deliver it in this roadmap. General execution, prepared handle lifecycle, sessions, and observer convenience APIs remain application -responsibilities. Configuration mutation, new live probes, dynamic completion, +responsibilities. Persistent configuration editing, new live probes, dynamic completion, JSON report formats, and unrelated deferred proposals are not added by this -roadmap. A later ConfigKit review may propose further CLI tasks separately. +roadmap. Transient acquisition overrides remain an explicit choice in OD-11. +Discovery tracing and secrets-directory bootstrap commands remain deferred: +the source inventory is not a trace of rejected search candidates, and bootstrap +performs application startup effects. ## Open Decisions @@ -396,3 +489,106 @@ available sooner, but creates an extra consumer upgrade and does not complete this roadmap's targeted end state. A disposable local module replacement can help test intermediate changes, but cannot substitute for final validation of the declared released dependency. + +### OD-8: Focused configuration reports and filtering + +**Recommendation:** extend existing commands: `config show --path POINTER` and +`config sources --source-id ID`; add subtree/explicitness filters to +`config paths` and severity/code filters to `config diagnostics`. Keep full reports as +the default. Use the existing explanation capability for selected effective +YAML, and recommend an operational failure for an absent effective path or +unknown source ID, consistent with unknown-path explanation. Recommend starting +path-selected display with one scope; whole-document `--scope all` remains +available. Specify presence-aware explicitness filtering so historical-only +entries are not labeled implicitly defined. These choices reuse existing +operator vocabulary and native evidence with little additional acquisition work. + +**Alternative:** introduce a `config get --path POINTER` command with a snapshot +envelope containing `present`, `redacted`, and `yaml`, and return absence as +report data. This is convenient for scripts and preserves unknown/null/absence +distinctions without relying on exit status, but adds another command/report +contract. Aggregate path-selected reports are also viable if scope-labeled +per-scope absence and delivery semantics are designed. Filters can be deferred +in favor of downstream YAML tools, reducing CLI surface but making routine +operator inspection more cumbersome. + +### OD-9: Independent comparison selector bindings + +**Recommendation:** let applications declare which domain selectors support +right-side comparison values, with explicit metadata for their `--against-*` +spellings. Omission inherits the left value and presence; an explicit right +value replaces it. For repeatable selectors, recommend replacing the inherited +collection with the supplied right collection. This follows config/overlay +selection conventions and supports comparisons such as two campaigns without +requiring different files. Review the selected refinement of ADR 0012 and flag +collision handling before implementation; exact helper signatures remain open. + +**Alternative:** require symmetric explicit left/right selectors for opted-in +bindings. This makes each side independent and easier to reason about when +defaults are complex, but adds repetition and departs from the existing +right-inherits-left convention. Either design keeps application domain parsing +in its declared bindings and avoids inventing a generic selector language. + +### OD-10: Comparison assertion outcomes + +**Recommendation:** add opt-in `--check`, retaining normal report delivery and +ordinary diff behavior when omitted. Recommend status `0` for equality and a +domain difference status `3`; retain current usage `2` and operational `1` +classification. This lets scripts distinguish differences from failed +comparison. The status is a proposal, not an existing contract. Apply existing +reporting outcome rules, including preservation of a nonzero domain outcome +when presentation also fails; callers can inspect the separate presentation +evidence. Document that interaction explicitly. + +**Alternative:** add a separate `config compare` assertion command, leaving +`diff` exclusively informational. This makes the intent visible in the command +path but duplicates selection/report options. Another viable choice is status +`1` for both differences and operational failures, which fits simple shell +checks but requires additional evidence to distinguish the two conditions. + +### OD-11: Transient configuration overrides + +**Recommendation:** include a declared, opt-in override capability for standard +configuration commands, with repeatable `--set POINTER=JSON` values passed to +application acquisition as native ConfigKit overrides. This enables validation, +inspection, and comparison of temporary settings through consistent CLI syntax. +Providers that do not accept overrides retain their existing operations. No +source file is edited, and no general application-internal convenience API is +required. The loader contract must state how it combines these records with +any existing application-owned overrides; the generic adapter cannot inject +them into an arbitrary loader after the load has completed. + +If selected, settle these details before implementation: + +- Split at the first `=`; JSON supplies typed null, boolean, string, numeric, + array, and object values. Specify numeric conversion to ConfigKit-supported + built-in forms rather than passing `json.Number` or silently promising + arbitrary numeric precision. An unambiguous alternate input form may be + needed for pointer keys containing `=`. +- Forward explicit records only, with safe display labels such as `--set`, + never raw value-bearing arguments as provenance labels. Preserve native + duplicate/overlap handling rather than silently collapsing assignments. +- Follow the released mapping-path constraints, including root/index and + schema-version restrictions, without extending them into a generic patch + language. Empty, false, zero, and null remain intentional supplied values. +- For comparison, recommend `--set` as the shared baseline and + `--against-set` as a right-side replacement list when supplied, following + OD-9's proposed repeatable-selector rule. Define an explicit way to select + an empty right list before finalizing this grammar; appending or replacing + individual paths would need a different selected rule. +- Reflect native CLI origins and subsequent resolution in existing reports. + Test CommandKit parsing, forwarding, and outcome/report adaptation, leaving + ConfigKit's override algorithm to its own tests. + +**Alternative A:** retain application-specific typed flags and loader translation +to `Options.Overrides`. This gives applications tailored vocabulary and avoids +generic JSON numeric/path syntax, but repeats CLI infrastructure and prevents +uniform temporary-setting workflows across applications. This is a viable +deferral if the generic acquisition contract is too disruptive for this sprint. + +**Alternative B:** accept a structured override document containing path/value +records instead of inline `--set` assignments. It handles complex values and +unusual pointer keys naturally and makes an empty comparison-side list explicit, +but adds file-input handling and is less convenient for one-value shell changes. +It still needs deliberate typed-value decoding and loader opt-in. Neither +alternative authorizes persistent configuration mutation.