From 3e50388b73babfc46b02550e207d6c6a90a0977d Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Tue, 10 Nov 2026 14:34:12 +0000 Subject: [PATCH] Make overlay version checks file-scoped --- docs/roadmap/overlays.md | 139 +++++++++++++++++++++++++++++---------- 1 file changed, 105 insertions(+), 34 deletions(-) diff --git a/docs/roadmap/overlays.md b/docs/roadmap/overlays.md index 28cb1dc..ec31699 100644 --- a/docs/roadmap/overlays.md +++ b/docs/roadmap/overlays.md @@ -1,7 +1,7 @@ # Single-file named overlays -Status: strict composition between the two forms and direct catalog structure -are settled. Remaining decisions are listed below. Not implemented. +Status: strict composition, direct catalog structure, and file-level version +checking are settled. Remaining decisions are listed below. Not implemented. Implementation baseline: `ce6005c`. ## Purpose @@ -47,8 +47,9 @@ cleanup already cover the individual-file form. ### File layout and selection -The catalog is a mapping from overlay names directly to overlay bodies, without -an enclosing `overlays` key: +The catalog maps overlay names directly to overlay bodies, without an enclosing +`overlays` key. It may also carry file-level version metadata under the application's +configured version path. The following example omits optional version metadata: ```yaml # overlays.yml — planned format @@ -81,8 +82,9 @@ service: If the individual file instead also defines `service.workers: 12`, ordinary strict-composition conflict rules apply. File order grants no precedence. The other catalog entries do not contribute to the selected configuration. -OD-1 settles this catalog structure. Source-version/control scope and the -validation boundary remain in OD-2 and OD-3. +OD-1 settles the direct catalog structure. Version checks are file-scoped as +described below; metadata-name collisions, collection-control placement, and +the validation boundary remain in OD-2 and OD-3. Use the same lookup directory for the catalog and the existing `overlays/` directory. Preserve ordinary versus layered lookup semantics, explicit context @@ -98,6 +100,45 @@ missing-overlay condition. A selected empty mapping is a present contribution, not a missing overlay. Acquisition or parsing failures are not a reason to silently prefer the other form. +### File-level version checks + +**Settled:** a version declaration checks whether a physical configuration file +declares an acceptable specification version. It does not independently version +each overlay body. Use the existing application-configured `Definition.Version` +path and source-version mode; do not hard-code the spelling `version`. + +For an application using the root `version` key, a catalog can contain: + +```yaml +version: 2 +production: + service: + workers: 8 +development: + service: + workers: 1 +``` + +Apply the configured source-version policy once to each acquired physical file, +before its contribution enters strict composition. The catalog check is independent +of which name is selected and also applies when the acquired catalog lacks that +name. No selected overlay still means neither form is read or checked. In enabled +modes, evaluate the catalog declaration at its physical document root, not inside +the selected body. Preserve the existing distinction between check-when-present +and required modes and the application's authority to accept the declared value. + +An acquired individual overlay file has its own declaration and existing version +processing. When both files contribute, each is checked independently; one file +cannot satisfy another's required declaration. Do not check or require separate +version declarations in the catalog's named bodies. Keep catalog version metadata +out of the applied overlay, and keep its evidence attached to the actual catalog +source and authored location. Body values remain subject to normal protection of +the application's version path when applying the overlay. + +This decision concerns stale-file detection. Collection controls describe overlay +operations and need not share the version declaration's scope. OD-2 separates the +remaining control-placement and catalog-metadata recognition decisions. + ### Strict composition, then one overlay application **Settled:** contributions for the selected name from both forms are peers. @@ -144,8 +185,8 @@ history or a new inspection API solely for this feature. Use deterministic acquisition order for source IDs and sequence assembly. The recommended order is catalog contribution, then individual-file contribution; this orders evidence and composed records, not precedence. Reuse existing -diagnostic classifications where they fit. Source-version/control scope is an -open decision in OD-2 and must be reflected consistently in inspection. +diagnostic classifications where they fit. Record file-level version evidence +once per acquired source; collection-control scope remains open in OD-2. ## Work from the baseline @@ -155,9 +196,11 @@ open decision in OD-2 and must be reflected consistently in inspection. 2. **Separate overlay acquisition from application.** Adapt the selected-overlay loader to acquire optional contributions and distinguish missing selection from acquisition failure. Retain existing context selection and access rules. -3. **Select catalog bodies without losing source evidence.** Parse the physical - catalog, select the requested body, and retain its actual locations. Apply - the chosen version/directive scope and account for the complete acquired file. +3. **Check files and select catalog bodies without losing source evidence.** Parse + and apply file-level source-version policy to each acquired file. Separate + catalog metadata from named definitions, select the requested body, and retain + actual locations. Apply the chosen collection-control scope and account for + the complete acquired file. 4. **Compose peers and apply once.** Reuse strict composition with the lower collection policy and selected declarations, then feed the composed result into the existing overlay operation. Preserve contributor origins, history, @@ -183,6 +226,10 @@ Cover the public loading boundary with: roots, host and native filesystems, and existing link-handling capabilities. - Invalid YAML, catalog shape, selected/unselected body validation, source-version modes, protected version paths, and forbidden directives under the settled rules. +- One file-level version evaluation per acquired catalog, independent of the + selected name, including a catalog without that name. Cover both source forms + under required and check-when-present modes, configured metadata paths and name + collisions, metadata exclusion from payloads, and actual declaration locations. - Actual source paths and locations, origins and contributor history, sensitive values, distinct resource bases, source IDs, and detached snapshots. - Missing versus unreadable sources, repetition, complete catalog accounting, @@ -207,39 +254,63 @@ Overlay names retain existing spelling and selection rules, including empty selection and exact lowercase `default` meaning no overlay. This settles the storage structure within the narrower file-based feature. -Source-version declarations and document controls remain subject to OD-2. +File-level version metadata coexists with the named entries. Its recognition and +name-collision rules, and collection-control placement, remain subject to OD-2. ## Open Decisions -### OD-2: Source-version declarations and document controls +### OD-2: File-level versions and overlay controls — partially settled -**Recommendation:** treat each selected body as the overlay's logical document -root for existing source-version and overlay-control processing. A version header -or `config.sequences` declaration appears inside that body, just as it appears -at the root of an individual overlay file. Process each participating contribution -before strict composition, including its own required version check when enabled. -Preserve the physical file's authored locations and report version evidence for -the selected logical document. Imports remain forbidden under existing overlay -rules; nested application data is not reinterpreted as root controls. +**Settled:** source-version checking operates once per acquired physical file, +using the existing application rule and mode. The catalog's declaration belongs +at its configured file-level path, not inside every named body. Collection-control +placement is a separate decision. The targeted end state above records the +version behavior; the remaining questions follow. -This permits moving a body between forms without rewriting its payload or -changing its policy meaning. It requires explicitly distinguishing the catalog's -physical document from the selected logical overlay in version processing and -inspection; it must not silently change source-version semantics for other inputs. +#### Collection-control placement -**Alternative:** declare the version and shared controls once at catalog level, -with entries inheriting them. This reduces repeated headers but needs a metadata -convention alongside the named entries, including rules for collisions with overlay -names and shared versus per-entry controls. It must respect OD-1's direct mapping -and also makes a standalone overlay body differ from its catalog form. +**Recommendation:** interpret `config.sequences` inside the selected body, using +application-relative destinations such as `/prompts`. Process each selected +contribution's controls before strict composition under existing same-tier rules. +This keeps an overlay's collection operations beside its payload and preserves +their meaning when moving that payload between forms. The file-level version +declaration stays separate. Imports remain forbidden under existing overlay rules; +nested application data does not activate controls. + +**Alternative:** place shared collection controls at catalog level and apply them +to every selected body. This reduces repetition where profiles use identical +operations, but needs metadata-name rules, makes operations less local to their +payloads, and differs from the structure of individual overlay files. + +#### Version metadata recognition and name collisions + +**Recommendation:** when the application configures a version path, give that +path a defined metadata role in the catalog and reserve its enclosing top-level +entry from overlay-name selection. For a root `version` path this reserves only +the catalog name `version`; for a nested path it makes the enclosing entry a +metadata container. This avoids guessing whether a value is a header or a profile. +Do not reserve a hard-coded version name when no version rule is configured. + +Specify how recognition works in main-only mode separately from whether a checker +runs, keeping metadata out of overlay payloads. Retain support for existing version +path forms; concrete examples must cover nested paths and retained empty ancestors +after header consumption. These rules are still open, not implicit additions to +the settled file-level check. + +**Alternative:** reserve only the exact version path and allow other fields in +its enclosing entry to form an overlay body. This permits more names and avoids +reserving a whole metadata container, but gives one mapping both metadata and +profile roles. Selection, header removal, and empty-body behavior become harder +to explain and need explicit rules. ### OD-3: Validation of unselected bodies **Recommendation:** require a valid YAML mapping catalog and look up the selected name. Preserve parser-level validation of the whole file, including duplicate mapping-key rules and limits. Require the selected body to satisfy normal overlay -mapping, directive, and version rules, but do not run those semantic checks or -application callbacks for unselected bodies. Final application validation assesses +mapping and directive rules, but do not run those semantic checks or application +callbacks for unselected bodies. File-level version policy still applies to the +acquired catalog independently of selection. Final application validation assesses the resulting effective configuration, not each profile in isolation. This keeps a load focused on the requested configuration while acknowledging @@ -247,9 +318,9 @@ that malformed YAML anywhere prevents reliable parsing. An empty mapping catalog contains no definitions; selected null/scalar/sequence bodies follow the existing mapping-root rules rather than becoming shorthand for absence. -**Alternative:** validate every body's shape, name, directives, and version when +**Alternative:** validate every body's shape, name, and directives when reading the catalog. This catches unused-profile defects earlier, but makes one -profile's load depend on unrelated definitions and expands checker invocation. +profile's load depend on unrelated definitions and expands semantic validation. A whole-catalog lint facility could serve that purpose separately if requested. ### OD-4: Filename variants