diff --git a/docs/roadmap/config-directives.md b/docs/roadmap/config-directives.md index 9737bea..7ac1d17 100644 --- a/docs/roadmap/config-directives.md +++ b/docs/roadmap/config-directives.md @@ -13,9 +13,11 @@ to use nested `config` fields for their own settings. An application that select ConfigKit adopts one control syntax; it should not need a configurable keyword or a second configuration-processing pass. -This roadmap defines the target and the work needed to reach it. It does not -change current behavior. The [configuration contract](../config.md) and public -Go declarations remain authoritative until implementation lands. +This roadmap defines the target and the work needed to reach it. The +[implementation plan](implementation.md) owns staged execution and progress. +Neither document changes current behavior. The +[configuration contract](../config.md) and public Go declarations remain +authoritative until implementation lands. ## Current Baseline diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..8a4155e --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,338 @@ +# Implement Document-Root Configuration Controls + +Status: ready for implementation by a Luna coding agent. + +Implementation baseline: `b27fe0598a9540ad05eefe2868a5a243eb28111e`. +The [feature roadmap](config-directives.md) owns the target and its rationale; +this document owns execution order and verification. + +## Working rules and settled decisions + +Follow the [development guide](../development.md), [agent policy](../policy/agents.md), +[architecture policy](../policy/architecture.md), [testing policy](../policy/testing.md), +and [documentation policy](../policy/documentation.md). This is one end-to-end +sprint. Intermediate stages may leave compilation or existing fixtures broken; +do not introduce compatibility code, skip tests, or weaken assertions to make +an intermediate stage green. Use focused checks as their fixtures become ready, +then run the complete development check at the end. + +The following decisions are settled: + +| Input or operation | Target behavior | +| --- | --- | +| Root `config` in any acquired source document | Parse as the document's control mapping before placement, preserving the existing context-specific fields, validation, and import opt-in. | +| Nested application `config` | Ordinary data, including inside sequence records and selected overlays. Children named `imports` or `sequences` do not activate controls. | +| A source mounted beneath an application destination | Interpret that source's own controls first; its mounting location does not change its document root. | +| Application-root `/config` produced by placement, defaults, CLI, or resolution | Ordinary data. Do not re-extract controls from assembled or typed values. | +| `configkit` and `_configkit` application keys | Ordinary data at any depth, with no control alias or special legacy-key error. Normal schema and other structural rules still apply. | +| `extends` | Preserve its existing inheritance semantics and restrictions, including CLI rejection. | +| Source versions | Protect actual source-local controls; retain existing header modes, conversion, main ownership, consumption, and disclosure rules. A matching application-path spelling alone is not control overlap. | + +Keep exported signatures, the module/package name, `configkit:"secret"`, +diagnostic identifiers, and the `configkit:` library error prefix. Preserve +composition, ordering, limits, ownership, and secret-handling behavior. Do not +add keyword configuration, control discovery after acquisition, public record +fields, runtime watchers, dependencies, or release gates. + +When migrating literals, distinguish root control syntax and control-evidence +paths from package imports, struct tags, error text, ordinary application keys, +and application paths. A repository-wide replacement of `configkit` or +`configkit:` is incorrect. Keep useful old-spelling inputs as positive ordinary +data regressions. A former rejection test needs a deliberate new expectation; +renaming its input while retaining the old rejection is not sufficient. + +## Stage 1 — Record the agreed architectural boundary + +**Scope:** one decision record, without changing current-behavior documentation. + +1. Create `docs/adr/0020-document-root-controls.md`, using the repository's ADR + format, status Accepted, and the date of authorship. Record the decisions in + the table above: per-source root recognition, ordinary nested and assembled + root data, and one canonical spelling. +2. Explain the source-local versus application-coordinate distinction and why + placement and later lifecycle operations do not create another parsing pass. + Keep the decision focused on ConfigKit. Discuss rejected choices only under + Alternatives Considered; avoid framing the decision around downstream apps. +3. Identify the reservation-scope portion of ADR 0011 as amended, while retaining + its placement decisions. Update only that earlier ADR's status section with + a link to the new decision and a statement that its other decisions remain + in force. Leave accepted decision bodies and historical release notes intact. + +**Complete when:** the ADR records the approved scope, including ordinary +application-root data, without claiming implementation is finished. Add its +development-guide navigation with the final documentation stage. + +## Stage 2 — Implement source-root control extraction and guards + +**Owners:** [directives.go](../../internal/engine/directives.go), its callers in +[acquisition.go](../../internal/engine/acquisition.go) and +[selected_overlay.go](../../internal/engine/selected_overlay.go), and focused +directive/declaration tests. + +1. Change the engine's control-key constant to `config`. Keep the existing + `parseRootDirectiveBlock` grammar and context distinctions. Root extraction + must clone/remove the selected block as before and return the remaining + application body without recursively rejecting names in that body. +2. Remove `nestedDirectiveDiagnostics` and `disabledNestedDirectiveDiagnostics` + once their callers no longer need them. Make both imports-disabled guard + paths inspect only the original document's root. Preserve the distinction + between ordinary/layered sequence declarations and the private imports-only + extraction context. Root imports with imports disabled still fail. +3. Make `overlayDirectiveDiagnostics` inspect only the original overlay root. + Preserve its current root acceptance rules and diagnostics, including the + allowed collection-only block and rejected imports. Nested application + `config` values must not trigger overlay-directive errors. +4. Check acquisition call sites so these guards receive acquired source trees, + not placed or composed application trees. Preserve extraction before + mounting and existing processing-result reuse; add no second extraction + pass. Leave source-version overlap changes to Stage 4. +5. In a focused `internal/engine/directive_scope_test.go`, cover a valid new + root block; nested `config` with control-looking children in a mapping and + record; ordinary old spellings; malformed/unknown root controls; disabled + imports; and overlay root restrictions. Reuse existing declaration fixtures + for their detailed grammar and migrate the literals needed by these tests. + +**Complete when:** these focused cases distinguish source roots from data, with +unchanged error families for still-invalid controls. Removing a recursive guard +must not remove validation of fields inside the actual control block. + +## Stage 3 — Permit application paths and CLI data + +**Owners:** [destination.go](../../internal/engine/destination.go), +[sequence_registration.go](../../internal/engine/sequence_registration.go), +[override.go](../../internal/engine/override.go), and their existing tests. + +1. Remove the `configkit` name ban from `normalizeDestination`; do not replace + it with a `config` ban or an application-root prefix check. Retain path + validity, mapping-key normalization, explicit-index rejection, and the + existing `extends` restriction. This applies to root and nested destinations + and to relative suffixes beginning with `config`. +2. Let exact and wildcard identity registrations use literal `config` and + `configkit` segments through the shared validation. Preserve wildcard + matching, identity normalization, conflicting-registration checks, and + final sequence validation. YAML declarations inherit the same path change. +3. Remove `pathContainsDirective` and its CLI path rejection. Preserve the + independent inheritance-path check. Narrow recursive override-value control + detection to its continuing `extends` responsibility; `config` and old + spellings are ordinary data anywhere in a CLI value. Simplify private names + and parameters where the removed responsibility makes them misleading. +4. Keep CLI type checking, root-path/index restrictions, overlap detection, + protected-version checks, labels, accounting, and assignment order intact. + Ordinary root data at `Keys("config")` is not the empty/root override path. +5. Revise destination, registration, and override tests. Move formerly invalid + `configkit` cases into positive data cases. Where a table still needs a + reserved-key rejection, use the continuing `extends` rule rather than + treating `config` as reserved application data. Cover root and nested + `config`, relative suffixes, exact/wildcard registrations, and CLI values + containing ordinary `imports`/`sequences` fields. Retain meaningful negative + cases for indexes, invalid paths, inheritance controls, and version writes. + +**Complete when:** literal application keys work through the shared path APIs +without a new per-caller exception or configurable reservation policy. Typed +defaults and resolution must not gain a new schema/key rejection. + +## Stage 4 — Verify source-version and acquisition integration + +**Owners:** [source_version.go](../../internal/engine/source_version.go), +`processAcquiredInputVersion` in acquisition, import processing, and existing +source-version/item-policy tests. + +1. Use the new control constant in actual source-root overlap detection. Keep + `sourceVersionHeaderControlOverlap` tied to an observed source-local header + and the original source tree. Preserve malformed-control validation before + a checker can consume that content, along with `extends` overlap handling. +2. Trace ordinary, layered, imported, sequence-item, and overlay processing. + Retain the distinction between local version lookup and effective main + ownership after mounting. Do not add a preflight ban on a version path merely + because it contains or begins with `config`. +3. Migrate the source-version test inputs and source-local expected paths. + Preserve absent/null behavior, callback order, consumption of accepted + non-main headers, conservative error/callback redaction, and existing codes. +4. Add focused cases for a header under an ordinary nested `config`; a real + header inside root `config.imports` being rejected before its checker; and + an enabled-mode main with local `version: 4` mounted at `[config]`. For the + mounted case, check the local header, final `/config/version`, and rejection + of an override targeting that protected application path. Include an + assembled-root case under the default mode using its existing effective-path + rule, rather than changing MainOnly semantics to match enabled modes. + +**Complete when:** application-root data and source-root controls are not +confused by version checking. Existing version behavior needs no new mode, +schema exception, or record format. Run focused SourceVersion tests once their +affected fixtures have been migrated. + +## Stage 5 — Migrate engine source and acquisition fixtures + +**Scope:** remaining engine source/discovery/import/placement/resource-context +tests and acquisition-related benchmarks, excluding structural/history fixtures +assigned to Stage 6. + +1. Migrate genuine root control blocks in import specs, import discovery and + evidence tests, filesystem/equivalence tests, discovery-load tests, + destination/import lifecycle tests, and standalone/grouped acquisition + tests. Cover escaped Go strings, raw multiline YAML, and mapping-source + keys; do not rely on a root-line text replacement alone. +2. Update expected declaration/empty-selection paths and coordinates to the + new source-local spelling. Preserve source labels and IDs, physical paths, + namespace identity, acquisition order, and effective destinations. +3. Convert nested-name rejection cases into ordinary-data cases where required. + Keep the file/containment/permission/repetition/limit tests testing their + original risks with valid new control syntax. +4. Include an acquired fragment with its own `config` controls mounted below a + nested destination or as a record. Include a payload mounted at root + `[config]` and a transitive relative destination starting with `config`. + Check successful assembly and source/resource attribution using existing + filesystem test patterns, without a mock traversal-count requirement. + +**Complete when:** these tests reach the behavior they intend to exercise with +the new grammar. Run the relevant Source/Import/Glob/Discovery/Destination/ +Resource tests; defer unrelated stale structural fixtures to Stage 6. + +## Stage 6 — Migrate engine structural and inspection fixtures + +**Scope:** remaining engine composition, collection, inheritance, lifecycle, +history, provenance, explanation, sensitivity, and rendering tests/benchmarks. + +1. Migrate genuine root controls and control-path expectations in the remaining + engine fixtures, including collection declarations, overlay policy, + inheritance-policy propagation, and item-local policy cases. Preserve + application keys and Go secret tags. Rename fixture comments when they + describe the changed syntax, not the library identity. +2. Retain the existing strict-peer, precedence, collection-operation, and + inheritance assertions. Upsert remains whole-record replacement; the + namespace change does not authorize a different combination algorithm. +3. Extend existing behavioral owners to cover ordinary `config` data surviving + strict composition, a higher-tier override, and sibling inheritance. Verify + explicit false/zero and representative permitted empty/null values at these + paths; reuse existing presence tests rather than constructing a Cartesian + product of every input kind and lifecycle operation. +4. Verify value inspection and source-local control evidence can coexist even + when path text shares a `/config/...` prefix. Use the existing source IDs, + locations, and record contexts; do not invent a path-kind flag or format. + Preserve historical redaction and snapshot ownership through existing test + coverage, adding a scope-specific case only where that coverage leaves a gap. + +**Complete when:** the engine suite passes with no obsolete global-name +rejections or control aliases. Run `go test ./internal/engine`; use the final +stage for the complete repository check. + +## Stage 7 — Update facade fixtures and public integration coverage + +**Scope:** root facade tests and executable package examples. + +1. Migrate root-level test and example inputs that intentionally use controls, + including source equivalence, import-default evidence, mounted collections, + version modes, diagnostics, and underlay-origin fixtures. Update expected + source-local paths, preserving package names, error prefixes, and secret tags. +2. Add public API regressions in `facade_config_directives_test.go`, using the + external `configkit_test` package and independently authored expectations: + - A normal load with a new root control and nested typed `config` data. + - A declared import mounted at `[config]`, with ordinary data children named + `imports` or `sequences`. It must not attempt another import pass. + - Defaults-only loading with a typed root `config` field, an explicit CLI + false/zero assignment below it, and a recorded resolution change. Check + entry-state versus final presence/origins and effective values through the + existing public APIs. + - Old spellings as normal data, including a control-looking old block whose + nonexistent import is not executed. Use a permissive schema for this + positive case; separately retain strict unknown-field failures. +3. Use the mounted case to verify import-declaration evidence, final data + lookup, explanation, and effective output describe their proper contexts. + Include a field tagged `configkit:"secret"` under application `config` and + verify ordinary hiding still applies. Do not compare full incidental error + wording or add duplicate engine tests for every parser shape. +4. Keep root facade conversions and exported signatures unchanged unless a + genuine existing conversion bug is exposed; this feature requires no new + bridge API or inspection fields. + +**Complete when:** `go test .` passes, including executable API examples and +public/import-boundary checks, with working external use of root and nested +application `config` data. + +## Stage 8 — Migrate maintained consumer fixtures and show the boundary + +**Scope:** `examples/consumers`, its YAML fixtures, and their executable tests. + +1. Migrate genuine root controls in consumer fixtures and inline generated + documents. Update expected declaration paths or displayed errors as needed; + retain domain behavior, application-owned profile concepts, and the existing + source/resource ownership assertions. +2. Add `Example_documentRootConfig` in + `examples/consumers/config_controls_test.go`. Use a small `fstest.MapFS` main + document with `config.imports`, importing a fragment containing ordinary + settings at destination `[config]`. Decode into an application type with a + `yaml:"config"` field and show the resulting value. The example demonstrates + that the source control is consumed while the fragment becomes root data. + Keep the example offline, credential-free, and small enough to copy. +3. Preserve existing complete consumer examples instead of adding a parallel + loader abstraction. Use existing imports, destinations, and recorded-helper + conventions without a second application-side merge. + +**Complete when:** `go test ./examples/consumers` passes, including the new +executable example. Existing scenarios still exercise their original features. + +## Stage 9 — Update current documentation and public comments + +**Scope:** canonical behavior, local architectural descriptions, consumer +migration guidance, and navigation after the implementation is in place. + +1. Update the configuration reference's imports, overlays, destinations, + collection declarations/registrations, CLI, version overlap, and earlier + directive spelling sections as one consistency pass. Replace the obsolete + nested-key prohibition with the per-source root boundary. Explain ordinary + application-root data and the existing non-reloadability of inspection YAML. +2. Show a source-root control beside a nested application `config`, and link + the maintained mounted-root example. Explain that every imported file has + its own root, even when mounted below another value or as a sequence item. +3. Add consumer migration guidance: rename genuine old control blocks; move, + rename, or separately mount source-root application fields formerly named + `config`; update stored `/configkit/...` declaration paths to `/config/...`. + Old blocks can decode as data in permissive schemas and no longer execute + controls. Preserve historical release notes and accepted ADR bodies. +4. Update relevant public Go comments and local architecture-policy wording + for source-local controls versus application data. Keep exact YAML rules + in the configuration reference. Explain only affected implementation flow + in internal docs; add the ADR and example to existing development/consumer + navigation. Review README examples and links for any obsolete control syntax. +5. Leave unrelated policy language, licensing, module identity, Go minimum, + dependencies, release commands, and release gates unchanged. Do not prepare + v1.0.0 notes or publish anything during this sprint. + +**Complete when:** current docs, declarations, and examples agree on both the +spelling and the boundary, with historical records clearly historical. Keep +implementation progress in this plan; the feature roadmap remains an end-state +document. + +## Stage 10 — Complete the integration and scope review + +1. Search current production code, tests, examples, and docs for old root + controls, recursive name bans, and stale declaration paths. Review each + remaining old spelling in context; retained package names, tags, error + prefixes, ordinary data tests, and historical records are intentional. +2. Review the complete result against the settled-decision table. In particular, + prove new source controls work across supported source forms; nested and + constructed root data are accepted; controls are interpreted before + placement; CLI/collection/version checks use the right coordinates; and + `extends`/protected-version restrictions remain effective. Use the owning + tests from prior stages instead of adding another duplicate integration suite. +3. Review for unintended scope changes: no source-root data fallback, old-key + compatibility alias, configurable keyword, post-acquisition control parsing, + application-root schema ban, new metadata format, or changes to composition, + defaults, ownership, secret handling, limits, or ordering. Fix discrepancies + rather than altering the approved target to match the implementation. +4. Run `./scripts/check.sh` with dependencies already available. It supplies + formatting/whitespace, all tests and examples, local links, vet, and build. + Reuse successful focused checks for unchanged work; broaden testing only for + concrete unresolved risks. No benchmark, CI wait, macOS gate, or live service + check is required for this migration. +5. Review the final diff and preserve unrelated work and ignored credentials. + Record actual completion/validation in this plan and update the feature + roadmap's overall status when the whole feature is implemented, without + adding stage checklists there. Commit validated, reviewable work under the + agent policy with plain-English messages; no + empty commits, push, release tag, hosted delivery, or consumer dependency + update is part of this implementation sprint. + +**Complete when:** the targeted behavior is implemented and documented, the +complete check passes, and the final review finds no remaining gap against the +feature roadmap. There are no unresolved design decisions in this plan.