From 90d9706f6de6c855ca253a776bf90992408635bb Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 7 Nov 2026 01:19:05 +0000 Subject: [PATCH] Plan independent defaults for both configuration comparison sides --- docs/roadmap/future.md | 12 ++- docs/roadmap/promptkit-v0.19.0.md | 136 ++++++++++++++++++++++-------- 2 files changed, 108 insertions(+), 40 deletions(-) diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index e6abcf0..e6347de 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -48,8 +48,11 @@ only; do not inspect or monitor unreleased work in anticipation of adoption. Independent per-side domain selectors and opt-in comparison assertions are now included in the [working roadmap](promptkit-v0.19.0.md#additional-configkit-operator-workflows). -Roadmap OD-9 selects explicit application-declared right-side bindings with -inheritance and collection replacement; empty-collection grammar remains open. +Roadmap OD-9 selects independent ordinary defaults for both sides and explicit +application-declared right-side bindings. Configuration selections and overrides +apply only to their own side; omission never inherits from the opposite side. +Whether to standardize empty collections overriding nonempty application defaults +remains open; clearing inherited comparison values is no longer needed. OD-10 selects an optional assertion outcome with a distinct difference status. These accepted plans do not change the currently implemented behavior. @@ -85,8 +88,9 @@ projections. Further presentation refinements should follow practical feedback. Transient acquisition overrides are included as an opt-in loader capability in [roadmap OD-11](promptkit-v0.19.0.md#od-11-transient-configuration-overrides). -Loader composition, numeric conversion, unusual path syntax, and comparison-side -clearing remain design tasks. Persistent configuration editing remains deferred. +Comparison overrides follow OD-9's independent-side model. Loader composition, +numeric conversion, and unusual path syntax remain design tasks. Persistent +configuration editing remains deferred. Discovery, import defaults, collection identities, and field-level resolution recording remain available through application loaders. Discovery tracing, diff --git a/docs/roadmap/promptkit-v0.19.0.md b/docs/roadmap/promptkit-v0.19.0.md index 01a991e..84db1ce 100644 --- a/docs/roadmap/promptkit-v0.19.0.md +++ b/docs/roadmap/promptkit-v0.19.0.md @@ -149,7 +149,9 @@ Transient configuration overrides are included under OD-11 as an opt-in loader capability. `config validate --set /workers=0` and `config show --set /server/port=8080` apply temporary overrides during acquisition without editing source files. Numeric conversion, unusual pointer keys, loader -composition, and comparison-side grammar remain to be finalized. +composition, and supplemental input grammar remain to be finalized. Comparison +overrides follow OD-9's independent-side model: `--set` affects only the left +side and `--against-set` only the right. ### Compatibility and native semantics @@ -301,12 +303,13 @@ Diagnostic filters affect the presented records, not the underlying acquisition outcome; retain the source references needed to interpret selected records. **Independent comparison selections.** Extend the per-side typed-input design -in [ADR 0012](../adr/0012-typed-configuration-comparison-pairs.md). Capture -applicable selector bindings for each side, prepare and check both inputs before -comparison, and preserve the existing shared invocation state. Resolve omitted -right selectors, explicit empty values, repeated selectors, and flag-name -collisions under OD-9. Domain interpretation and loader caching remain with the -application; the comparison provider still receives two selections. +in [ADR 0012](../adr/0012-typed-configuration-comparison-pairs.md), deliberately +replacing its left-to-right inheritance and shared application-selector rules. +Initialize each side from ordinary application defaults and apply only that +side's explicitly supplied configuration selections and overrides under OD-9. +Prepare and check both inputs before comparison and preserve the existing shared +invocation state. Domain parsing, default interpretation, and loader caching +remain with the application; the comparison provider still receives two selections. **Comparison assertions.** Base a difference outcome on native record presence, including hidden changes whose redacted snapshots look equal. Equal effective @@ -813,17 +816,25 @@ Default tests remain offline and independent of real credentials. - [ ] Source-ID selection works on successful and failed acquisitions. Path, explicitness, severity, and code filters preserve the selected evidence and source references without masking the acquisition outcome. -- [ ] Independent comparison selectors preserve the chosen inheritance and - repeatable-value rules, side-specific checks, and shared-state lifecycle. - Explicit empty right-side collection selection is designed and covered. +- [ ] Comparison sides start independently from ordinary application defaults. + Supplied config/overlay selections, domain selectors, and their presence apply + only to their own side, including repeatable values. Cover left-only, + right-only, both-side, neither-side, and partially supplied right selections; + no omitted right field inherits from the left. Preserve side-specific checks + and the shared-state lifecycle. Command-wide controls remain shared. +- [ ] A superseding ADR records OD-9's replacement of ADR 0012's inheritance and + shared-selector application rules. Update canonical CLI/API contracts and + examples with implementation, including cases that now require repeating a + common nondefault selection on both sides. - [ ] Assertion mode distinguishes equality, native differences (including hidden changes), usage errors, and operational failures with statuses `0`, `3`, `2`, and `1` respectively, without changing ordinary diff outcomes. Report-delivery failures remain observable. - [ ] Only supplied CLI overrides reach opted-in loaders under OD-11, with typed - values and safe flag labels, - selected comparison-side behavior, native errors, and returned provenance - preserved. No tests independently re-establish ConfigKit override semantics. + values and safe flag labels. `--set` applies only to the left comparison side, + and `--against-set` only to the right; omission supplies no CLI override records + for that side. Native errors and returned provenance are preserved. No tests + independently re-establish ConfigKit override semantics. - [ ] Default validation remains concise; `--details` selects the additional diagnostic report with a text template. Core automatically selects text when that template is supplied, while explicit formats still win. Changing format @@ -897,11 +908,13 @@ The core ownership, JSON representation conventions, text printing/templates, universal-format, automatic-selection, and joint format-delivery decisions are now agreed above. Remaining details are listed under [remaining core reporting implementation details](#remaining-core-reporting-implementation-details). -Provider JSON number decoding and namespace applicability are agreed in OD-2 -and OD-3. Outstanding integration choices are empty comparison collections and -binding design in OD-9, and override grammar/acquisition details in OD-11. Exact helper -names and report layouts otherwise remain ordinary implementation choices, not -unresolved scope. +Provider JSON number decoding, namespace applicability, and independent comparison +defaults are agreed in OD-2, OD-3, and OD-9. Outstanding operator choices are +whether to standardize explicit empty collections when application defaults are +nonempty (OD-9), and configuration-override numeric conversion, unusual-path +grammar, and loader composition (OD-11). Inheritance-clearing flags are no longer +needed. Exact helper signatures, typed binding composition, flag collision +handling, and report layouts remain implementation choices, not unresolved scope. ### OD-1: Preview command and report presentation @@ -1324,20 +1337,62 @@ or aggregate path-selected display is selected for this adoption. ### OD-9: Independent comparison selector bindings -**Agreed:** applications declare which domain selectors support right-side -comparison values, with explicit metadata for their `--against-*` spellings. -Do not generate comparison flags for every application option automatically. -Omission inherits the left value and presence; an explicit right value replaces -it. For repeatable selectors, the supplied right collection replaces the -inherited collection rather than appending to it. This supports comparisons -such as two campaigns without requiring different files or repeating every -unchanged selection. +**Agreed comparison model:** initialize the left and right sides independently +from the configuration the application ordinarily loads without CLI configuration +selections or overrides. Ordinary loading retains application discovery, files, +defaults, environment handling, and resolution; it does not mean an empty or +defaults-only configuration. -Domain parsing stays in the declared typed bindings; no generic selector -language is introduced. Review the refinement of ADR 0012 and flag collision -handling before implementation. Exact helper signatures and an explicit way to -select an empty right-side collection remain open. An empty string element is -not necessarily an empty collection; do not equate them implicitly. +Apply each side's explicitly supplied values only to that side: + +- `--config`, `--overlay`, application configuration selectors, and `--set` + affect the left side. +- Their declared `--against-*` counterparts affect the right side. +- Every omitted field retains that side's ordinary application default and + unsupplied state. Never copy the opposite side's value or supplied state, + even when other fields on the right are explicitly supplied. +- Repeatable selectors follow their declared typed binding behavior on their + own side. Left-side elements are never copied into the right collection. +- Command-wide controls such as `--scope`, `--format`, `--details`, and `--check` + govern the comparison as a whole and remain outside this separation. + +| Invocation after `app config diff` | Left selection | Right selection | +| --- | --- | --- | +| `--config candidate.yml` | `candidate.yml` | Ordinary configuration | +| `--config before.yml --against-config after.yml` | `before.yml` | `after.yml` | +| `--config base.yml --against-overlay experimental` | `base.yml` | Ordinary configuration with the selected overlay | +| `--config base.yml --against-config base.yml --against-overlay experimental` | `base.yml` | `base.yml` with the selected overlay | +| `--against-config candidate.yml` | Ordinary configuration | `candidate.yml` | +| No side-specific flags | Ordinary configuration | Ordinary configuration | + +The examples retain normal application interpretation of file and overlay +selectors. Explicitly selecting the same nondefault input on both sides requires +supplying it on both sides. Equal selections remain allowed and do not skip +acquisition or comparison. + +Applications declare which domain selectors support right-side values, with +explicit metadata for their `--against-*` spellings. Do not generate comparison +flags for every application option automatically. A configuration selector +without a right-side binding still affects only the left when supplied; it is +not implicitly shared. Domain parsing stays in declared typed bindings; no +generic selector language is introduced. Retain the two input preparations and +checks, one shared invocation state, and existing provider lifecycle. + +This deliberately replaces the previous inheritance model, including existing +config/overlay behavior and shared application selectors. When implementation +lands, record the change in a superseding ADR for the affected decisions in +ADR 0012, retaining its accepted decision text as history, and update current +CLI/API contracts and examples. Exact helper signatures and flag collision +handling remain implementation design tasks. + +**Remaining collection question:** omission already means the ordinary collection +for that side, so no flag is needed to clear inherited left-side values. The +earlier proposed `--against-clear-*` flags are not selected. Explicitly replacing +an application's nonempty default collection with an empty collection is a +separate, symmetric question for either side: decide whether it needs a standard +capability or should remain with application-declared bindings. No empty-collection +grammar is selected here, and an empty string element is not implicitly an empty +collection. ### OD-10: Comparison assertion outcomes @@ -1367,6 +1422,20 @@ required. The loader contract must state how it combines these records with any existing application-owned overrides; the generic adapter cannot inject them into an arbitrary loader after the load has completed. +**Agreed comparison behavior:** `--set` supplies override records only to the +left side; `--against-set` supplies records only to the right. Each side starts +from its ordinary configuration under OD-9. Omission supplies no CLI override +records for that side; it never inherits records from the other side. Normal +application-owned overrides are part of ordinary loading and are not erased by +omitting these flags. + +For example, `config diff --set /workers=4` compares the left configuration with +that override against the ordinary right configuration. Adding +`--against-set /workers=8` explicitly overrides the right configuration as well. +Repeat the same `--set` assignments as `--against-set` assignments when both +sides should use them. There is no shared CLI override baseline and no +`--against-clear-set` flag is needed to undo inheritance. + Prefer inline assignments for ordinary one-value experiments. This is the largest remaining loader integration design item; revisit scope explicitly if the required acquisition changes prove disproportionate rather than silently @@ -1383,11 +1452,6 @@ dropping the agreed feature. Settle these details before implementation: - Follow the released mapping-path constraints, including root/index and schema-version restrictions, without extending them into a generic patch language. Empty, false, zero, and null remain intentional supplied values. -- For comparison, recommend `--set` as the shared baseline and - `--against-set` as a right-side replacement list when supplied, following - OD-9's agreed repeatable-selector rule. Define an explicit way to select - an empty right list before finalizing this grammar; appending or replacing - individual paths would need a different selected rule. - Reflect native CLI origins and subsequent resolution in existing reports. Test CommandKit parsing, forwarding, and outcome/report adaptation, leaving ConfigKit's override algorithm to its own tests.