Prepare release notes for ConfigKit 1.1.0
This commit is contained in:
@@ -60,5 +60,6 @@ and lifecycle ordering in [ADR 0018](0018-cli-after-structural-resolution.md).
|
||||
Resolution auditing and inspection must share entry-to-final correspondence,
|
||||
including explicit unmatched records, while preserving historical coordinates
|
||||
and conservative sensitivity masking. Source assembly and public path types do
|
||||
not change. The [implementation plan](../roadmap/implementation.md) tracks the
|
||||
work; the [configuration contract](../config.md) owns implemented behavior.
|
||||
not change. The completed implementation plan is preserved in Git history at
|
||||
`7e53863:docs/roadmap/implementation.md`; the
|
||||
[configuration contract](../config.md) owns implemented behavior.
|
||||
|
||||
@@ -93,6 +93,19 @@ and current facade ownership belongs to the [component overview](internal/overvi
|
||||
|
||||
## Baseline validation
|
||||
|
||||
The completed field-level sequence resolution roadmap, implementation plan, and
|
||||
resolved audit findings are preserved in Git history at `7e53863` under
|
||||
`docs/roadmap/`. For example:
|
||||
|
||||
```sh
|
||||
git show 7e53863:docs/roadmap/implementation.md
|
||||
git show 7e53863:docs/roadmap/remediation.md
|
||||
```
|
||||
|
||||
The [configuration contract](config.md#recorded-application-resolution) owns
|
||||
current resolution behavior, and [ADR 0021](adr/0021-sequence-resolution-correspondence.md)
|
||||
records the correspondence decision.
|
||||
|
||||
The completed filesystem-source and underlay roadmap, original feature request,
|
||||
implementation plan, and resolved remediation record are preserved in Git history
|
||||
at `b604aa1` under `docs/roadmap/`. Current behavior belongs to the
|
||||
|
||||
79
docs/releases/v1.1.0.md
Normal file
79
docs/releases/v1.1.0.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# ConfigKit v1.1.0
|
||||
|
||||
ConfigKit now accepts field-level resolution records inside lists and follows
|
||||
registered records across position changes when reporting their provenance and
|
||||
history. This release also standardizes more release validation on ReleaseKit.
|
||||
|
||||
## Field-level resolution in lists
|
||||
|
||||
Applications can normalize or derive a field inside a list and record that
|
||||
field directly. For example, changing `/resources/0/path` can be covered by
|
||||
`RecordNormalization` at that path, without recording the entire `/resources`
|
||||
list when no other changes require coverage.
|
||||
|
||||
For ordinary lists of equal length, resolution auditing compares corresponding
|
||||
positions recursively. Positions are addresses; ConfigKit does not infer an
|
||||
element's identity from similar values. A length change still requires a record
|
||||
for the list or an ancestor.
|
||||
|
||||
For collections registered through `Definition.SequenceIdentities`, ConfigKit
|
||||
matches records between the resolution-entry and final snapshots when both
|
||||
endpoints have valid, unique identities. Field edits can be recorded individually.
|
||||
Changes to identity order or membership still require list or ancestor coverage;
|
||||
changing an identity is treated as removal and addition. When matching is
|
||||
unavailable, a changed registered collection is audited as a whole. Existing
|
||||
final identity validation remains in place, allowing resolution to repair entry
|
||||
values.
|
||||
|
||||
See [recorded application resolution](../config.md#recorded-application-resolution)
|
||||
for the complete rules.
|
||||
|
||||
## Matched history and inspection
|
||||
|
||||
For eligible registered collections, inspection at a surviving record's final
|
||||
position follows its own entry evidence, including earlier values, input
|
||||
explicitness, and source attribution. A record no longer inherits the evidence
|
||||
of a different record that previously occupied its final position. Original
|
||||
source coordinates remain intact, while reported operation paths follow the
|
||||
matched final record.
|
||||
|
||||
Collection inspection retains removed records' evidence. Removed fields of
|
||||
surviving moved records remain available through metadata history and
|
||||
explanation, including deep inherited fields. Sensitivity propagation protects
|
||||
matched and removed evidence and recorded dependencies; detached before/after
|
||||
snapshots remain conservatively masked.
|
||||
|
||||
Correspondence lookup, sensitivity propagation, and resolution-change ordering
|
||||
reuse indexes, a work queue, and prepared pointer strings to avoid repeated
|
||||
prefix construction, full-map passes, and formatting during sorting. No
|
||||
end-to-end performance improvement is claimed.
|
||||
|
||||
## Compatibility
|
||||
|
||||
This is a backward-compatible extension of resolution auditing and inspection.
|
||||
Existing callbacks that record a whole list remain supported. Records and helper
|
||||
targets for surviving values refer to final positions; helpers do not follow
|
||||
later moves automatically. `Resolution.EntryInfo` and `Resolution.IsExplicit`
|
||||
retain their fixed entry-snapshot meaning.
|
||||
|
||||
Public effective-configuration diffs continue to report list changes atomically.
|
||||
Source composition, collection merge operations, configuration syntax, and public
|
||||
path addressing are unchanged. No new public Go symbols are required.
|
||||
|
||||
The consumer minimum remains Go 1.27.0, and the YAML dependency is unchanged.
|
||||
Linux amd64 and arm64 remain release-validated; macOS remains best-effort.
|
||||
|
||||
## Release tooling
|
||||
|
||||
ReleaseKit advances from v0.6.0 to v0.8.0. The runbook uses the module-selected
|
||||
`go tool releasekit` command and documents saved publication attempts and
|
||||
reconciliation after failed or uncertain tag pushes.
|
||||
|
||||
ReleaseKit now checks committed Markdown links and rejects filesystem dependency
|
||||
replacements during candidate validation. The custom documentation-link Go test
|
||||
is removed, so automated link feedback occurs during candidate validation rather
|
||||
than ordinary Go tests. Source publication and hosted Gitea delivery remain
|
||||
separate outcomes; CI remains outside release gates. See the
|
||||
[release procedure](../release.md).
|
||||
|
||||
[COPYRIGHT](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.1.0/COPYRIGHT) | [LICENSE](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.1.0/LICENSE) | [ATTRIBUTION](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.1.0/docs/oss/attribution.md)
|
||||
@@ -1,475 +0,0 @@
|
||||
# Sequence resolution and inspection implementation plan
|
||||
|
||||
Status: complete.
|
||||
|
||||
## Scope and execution
|
||||
|
||||
Implement the agreed [feature roadmap](resolution-sequences.md) from baseline
|
||||
`9adcf14`. The target is field-level resolution auditing for ordinary lists and
|
||||
identity-aware resolution history for application-registered collections. This
|
||||
plan is for a Luna coding agent executing the numbered stages in order during
|
||||
one end-to-end sprint. Intermediate stages may leave integration incomplete or
|
||||
temporarily fail to compile; do not build compatibility adapters merely to make
|
||||
each intermediate state green. The completed sprint must satisfy the whole
|
||||
roadmap, including history, metadata, redaction, and removed-record inspection.
|
||||
|
||||
Read [agent policy](../policy/agents.md), [architecture
|
||||
policy](../policy/architecture.md), [testing policy](../policy/testing.md), and
|
||||
[documentation policy](../policy/documentation.md), using the
|
||||
[development guide](../development.md) for orientation and commands. Read the
|
||||
roadmap as the target contract; current [resolution
|
||||
documentation](../config.md#recorded-application-resolution) describes the
|
||||
baseline that this work deliberately extends. Preserve unrelated changes.
|
||||
|
||||
Use the existing toolchain required by `go.mod` and acquired dependencies.
|
||||
This sprint does not change module versions, release machinery, configuration
|
||||
syntax, public path types, collection composition, or identity selector scope.
|
||||
There are no benchmarks, timing thresholds, CI waits, or additional release gates
|
||||
required by this plan. Prefer focused tests during implementation and the final
|
||||
checks in Stage 13. Commit completed nonempty units under the agent policy.
|
||||
|
||||
## Implementation decisions
|
||||
|
||||
- Keep `compareConfigurationValues` atomic for sequences. Add a separate private
|
||||
resolution comparison operation; validation and public effective-value diff
|
||||
retain their existing comparison semantics.
|
||||
- Ordinary equal-length lists compare recursively by position. A length change
|
||||
requires coverage at the list or an ancestor. Registered lists use matching
|
||||
only when both endpoint sequences have valid, unique identities under the
|
||||
applicable application registration. A registered collection that cannot be
|
||||
matched uses whole-list auditing when changed; equality still requires no
|
||||
record. Do not emit matching-eligibility errors or move final validation earlier.
|
||||
- Match within the same concrete registered collection destination, using the
|
||||
already compiled application policy. Use payload identity rules, including
|
||||
ordinary late `extends` fields. YAML-only declarations and matching across
|
||||
different collection destinations are outside this correspondence.
|
||||
- A valid registered collection's changed identity order or membership requires
|
||||
whole-list coverage. Continue matching surviving records for inspection even
|
||||
when that structural delta already covers all field changes.
|
||||
- Records and helper verification address final positions for surviving values;
|
||||
entry queries retain entry coordinates. A final record's missing descendant
|
||||
may still refer to a removed value on its matched entry record. A wholly removed
|
||||
position retains the existing historical meaning. When an address is reused,
|
||||
the surviving record takes precedence; use the container to record removal of
|
||||
the former occupant. Helpers never follow moves automatically.
|
||||
- Use one private correspondence result throughout resolution and inspection.
|
||||
Store concrete entry/final record-prefix pairs and explicit unmatched cases,
|
||||
not a mutable identity registry or a full tree copy per field. Source
|
||||
coordinates, original input payloads, and callback entry metadata remain fixed.
|
||||
- A surviving record query selects its own entry lineage. A newly added identity
|
||||
has no prior occupant's lineage. Collection queries retain physical before/after
|
||||
order and removed evidence; absent historical positions remain positional.
|
||||
Reuse existing source, inheritance, CLI, and decoding history semantics within
|
||||
the selected entry lineage; do not rematch identities across earlier phases.
|
||||
- Record reasons, dependencies, equal-valued intent, resolution origins, and
|
||||
coalescing retain their existing semantics. Identity correspondence supplies
|
||||
earlier evidence; it does not replace the recorded application operation.
|
||||
- Reuse the shared history model and snapshot masking boundary. Preserve
|
||||
historical sensitivity and dependency closure, including conservative masking
|
||||
already required by the current contract. This feature promises no reduction
|
||||
in masking and adds no unredacted inspection mode.
|
||||
|
||||
Suggested new private file names below are implementation locations, not public
|
||||
interfaces. Keep responsibilities cohesive and use existing helpers where they
|
||||
already express the needed behavior. Do not introduce a general comparison
|
||||
framework, new public selectors, caches, or mutation logging for this feature.
|
||||
|
||||
## Stage 1 — Record the architectural extension and inspect the baseline
|
||||
|
||||
**Status:** Complete. The focused engine baseline and documentation-link checks passed.
|
||||
|
||||
**Scope:** decision documentation and a focused starting check.
|
||||
|
||||
1. Add `docs/adr/0021-sequence-resolution-correspondence.md` with Accepted status
|
||||
using the documentation policy's ADR format. Record the approved positional
|
||||
and registered-identity rules, the entry-to-final boundary, and the distinction
|
||||
between resolution auditing and public atomic diff presentation. Include
|
||||
removed/reused-position inspection and final-only registration validation.
|
||||
Keep discussion of alternatives in Alternatives Considered; focus on ConfigKit.
|
||||
2. Relate the decision to ADRs 0003, 0010, 0017, and 0018. This extends resolution
|
||||
inspection and preserves their source-assembly, diff, and lifecycle choices;
|
||||
do not rewrite accepted decision bodies or mark them superseded wholesale.
|
||||
3. Read `resolution.go`, `load_metadata.go`, `load_result.go`, `override_context.go`,
|
||||
`history_model.go`, `history_resolution.go`, `history_select.go`, `history.go`,
|
||||
`explain.go`, `secret.go`, and `snapshot.go` in `internal/engine`. Note that
|
||||
engine `Resolution.InputInfo` backs public `Resolution.EntryInfo`.
|
||||
4. Run the existing focused engine tests for resolution, history, sequence
|
||||
identity, and diff to establish the starting state. Do not change existing
|
||||
expectations simply because the new contract will later allow more precision.
|
||||
|
||||
**Completion:** one architectural decision records the scope without claiming
|
||||
implementation completion. No current-behavior reference is updated yet.
|
||||
|
||||
## Stage 2 — Build endpoint collection correspondence
|
||||
|
||||
**Status:** Complete. Focused correspondence tests cover identity matching, fallback, and dynamic selectors.
|
||||
|
||||
**Scope:** private, pure matching logic, preferably in
|
||||
`internal/engine/resolution_correspondence.go`.
|
||||
|
||||
1. Build a correspondence object from the canonical post-CLI entry tree, final
|
||||
projected tree, and compiled `sequencePolicy`. Reuse `identityFieldsAt` and
|
||||
`validateNormalizedSequenceIdentityFrom` with `sequencePayloadRecord`, or
|
||||
extract a small shared non-reporting helper if necessary. Reuse exact tuple
|
||||
equality; do not implement another numeric or compound-key comparison.
|
||||
2. Distinguish ordinary, eligible registered, and ineligible registered sequences.
|
||||
For eligible pairs, retain entry-to-final and final-to-entry item indexes and
|
||||
whether ordered identity membership changed. Distinguish an unmatched final
|
||||
item from a matched item at the same numeric index.
|
||||
3. Use tuple lookup tables and ordered sequence traversal. Resolve each record's
|
||||
identity once per endpoint. Reuse mapping indexes rather than scanning every
|
||||
record for every match. Keep identity tuples private and out of diagnostics.
|
||||
4. Provide shape-aware prefix translation for final record descendants to their
|
||||
corresponding entry paths, with an explicit no-entry result for additions.
|
||||
Unregistered paths remain positional. Handle missing descendants without
|
||||
borrowing the previous occupant's field; distinguish numeric keys and indexes.
|
||||
5. Honor existing selector traversal: mappings and dynamic mapping namespaces
|
||||
can lead to registered lists; selectors do not traverse list indexes. Nested
|
||||
ordinary lists still receive recursive positional comparison later.
|
||||
|
||||
**Verification:** focused pure behavior tests for reorder, insertion/removal,
|
||||
identity change, empty collections, malformed/duplicate endpoints, compound and
|
||||
typed scalar identities, and exact versus dynamic registrations. Reuse existing
|
||||
identity equality tests rather than repeating their entire scalar matrix.
|
||||
|
||||
**Completion:** correspondence is deterministic and detached from callback state;
|
||||
invalid entry identities remain repairable and no new load failures are added.
|
||||
|
||||
## Stage 3 — Compare ordinary lists recursively during resolution
|
||||
|
||||
**Status:** Complete. Focused resolution and public diff checks passed.
|
||||
|
||||
**Scope:** resolution-only delta production and its initial integration.
|
||||
|
||||
1. Add a private resolution comparator, preferably in `resolution_compare.go`.
|
||||
Reuse canonical scalar equality and existing mapping comparison behavior.
|
||||
Compare equal-length ordinary sequences element by element recursively.
|
||||
2. Emit a list-level replacement for ordinary length changes. Keep additions,
|
||||
removals, null, and kind replacements at the appropriate affected path.
|
||||
Until Stage 4 connects identity matching, registered sequences can remain
|
||||
conservative; do not change the shared atomic comparator.
|
||||
3. Preserve canonical diagnostic ordering and existing exact/ancestor coverage
|
||||
through `resolutionCoverageIndex`. Never let a descendant record cover a
|
||||
container delta. Avoid comparing a complete subtree before recursively
|
||||
comparing it again at every level.
|
||||
4. Use the new comparator in `runResolution`. Review `resolutionResult.deltas`
|
||||
consumers; metadata integration is completed in Stage 6.
|
||||
|
||||
**Verification:** field-only recording inside an ordinary list succeeds; a second
|
||||
unrecorded field fails at its own path; length changes still need a container
|
||||
record. Cover nested mappings/lists and explicit zero, false, and null through
|
||||
representative cases. Existing public diff/comparator tests must remain atomic.
|
||||
|
||||
## Stage 4 — Integrate registered comparison and paired record snapshots
|
||||
|
||||
**Status:** Complete. Focused resolution and identity tests passed, including repair and helper-target checks.
|
||||
|
||||
**Scope:** auditing, record validation, and before/after capture.
|
||||
|
||||
1. Pass `request.sequencePolicy` through `resolveLoadInput` to `runResolution`;
|
||||
reuse the preflight-compiled application policy. Build correspondence once
|
||||
after final projection, before coverage and snapshot capture.
|
||||
2. In eligible collections, recursively compare matched records at final paths.
|
||||
Emit a structural list delta for membership/order changes independently of
|
||||
field deltas. Preserve pairs even when no additional field delta is needed.
|
||||
Ineligible changed registered lists use an atomic delta; unchanged ones do not.
|
||||
3. Validate each record against the correctly paired entry and final target.
|
||||
A helper must still equal the final value at its literal declared path. For
|
||||
additions, the before side is absent; do not fall back to an old occupant.
|
||||
For removals, use the addressing decisions above and existing schema checks.
|
||||
4. Extend private resolution snapshots/results with the paired path information
|
||||
needed downstream. Capture a field's before value from its matching entry
|
||||
record; a collection/root snapshot still contains the complete physical tree.
|
||||
Preserve cloned snapshots, record coalescing, reason/dependency order,
|
||||
normalization dominance, and equal-valued intent.
|
||||
|
||||
**Verification:** registered field edits succeed; reorder plus only a field
|
||||
record fails; container coverage succeeds; identity changes require container
|
||||
coverage. Check invalid-entry repair and unchanged invalid collections without
|
||||
inventing an early identity error. Final invalid values still fail at the
|
||||
existing final identity validation boundary. Include a helper whose target was
|
||||
moved afterward and no longer matches its declared final address.
|
||||
|
||||
## Stage 5 — Retain correspondence in the shared history owner
|
||||
|
||||
**Status:** Complete. Focused history, correspondence, and retained-budget checks passed.
|
||||
|
||||
**Scope:** evidence transport, ownership, and retained-node accounting.
|
||||
|
||||
1. Extend `historyResolutionPayload` to own the compact correspondence and paired
|
||||
snapshot coordinates. Transfer them from the audited result through
|
||||
`resolveLoadInput` and `observeResolutionEvidence`; retain no callback or
|
||||
mutable configuration references.
|
||||
2. Keep original record observations and `resolutionReplayGroup` coalescing.
|
||||
Index the actual concrete before and after shapes, using paired paths for
|
||||
fields and the physical paths for container snapshots. Do not globally
|
||||
rewrite earlier observations or duplicate all source evidence under each
|
||||
moved record's final prefix.
|
||||
3. Keep one scratch replay tree. Paired before paths describe evidence, not
|
||||
instructions to move scratch nodes one assignment at a time. Apply a
|
||||
structural container's final subtree once, as its existing replay group does.
|
||||
4. Include any newly retained document roots in `historyModel.documentRoots`
|
||||
and the existing admission check before secret lookup and validation. Prefer
|
||||
borrowing already immutable checkpoint/snapshot roots. Release temporary
|
||||
identity tables unless needed by frozen queries; retain positional pairs,
|
||||
not identity text, where sufficient.
|
||||
|
||||
**Verification:** extend the focused history-resolution tests to check behavior
|
||||
with paired sides and overlapping records. Reuse history-budget tests to prove
|
||||
that new retained roots cannot escape accounting and shared roots count once.
|
||||
Do not assert an incidental struct layout or a fixed number of internal calls.
|
||||
|
||||
## Stage 6 — Refresh effective metadata using the matched entry record
|
||||
|
||||
**Status:** Complete. Focused resolution, metadata, validation, and secret checks passed.
|
||||
|
||||
**Scope:** presence, origins, and post-resolution diagnostic locations.
|
||||
|
||||
1. In `applyResolutionMetadata` and `refreshResolutionFields`, seed final fields
|
||||
from their corresponding entry fields rather than the previous occupant of
|
||||
the final index. Use shape-aware entry lookup for exact descendants. Added
|
||||
identities and newly created fields have no invented input explicitness.
|
||||
2. Apply recorded resolution origins using the existing covering-record and
|
||||
finer-record semantics. A structural normalization remains a resolution
|
||||
operation, including when an unchanged record moves beneath it. Source
|
||||
lineage remains available through history; do not suppress the operation to
|
||||
make a field appear to come directly from its original file.
|
||||
3. Remove effective fields absent from the final tree and rebuild effective path
|
||||
shapes. Preserve positional behavior outside eligible registered collections.
|
||||
4. Ensure `documentWithMetadataLocations` and `metadataFieldLocation` can obtain
|
||||
the matched record's source evidence for later secret/validation diagnostics.
|
||||
Preserve source IDs and passive contexts. Do not mutate the fixed entry view,
|
||||
move source-document coordinates, or fabricate sources for additions.
|
||||
|
||||
**Verification:** moved records with different explicit/defaulted descendants
|
||||
retain their own presence; added records do not inherit old explicitness. Test
|
||||
deleted fields and zero/null states, resolution origins, and source attribution.
|
||||
During the callback, `EntryInfo` and `IsExplicit` must keep their old coordinate
|
||||
meanings. History-based location fallback becomes fully usable in Stage 7.
|
||||
|
||||
## Stage 7 — Select history through the correct entry coordinate
|
||||
|
||||
**Status:** Complete. Focused history and matched-location checks passed.
|
||||
|
||||
**Scope:** shared history query selection and metadata history projection.
|
||||
|
||||
1. Add a query-local coordinate description to the existing history selection
|
||||
machinery. For a final item or a descendant under an eligible collection,
|
||||
distinguish matched entry coordinates, no-entry additions, and unchanged
|
||||
positional queries. A removed descendant of a surviving record uses that
|
||||
record's correspondence even if the descendant is absent finally.
|
||||
2. For baseline, sources, inheritance, decoding, and CLI, select the matched
|
||||
entry path's existing evidence. Do not select the prior occupant of the final
|
||||
index. For an added identity, suppress that prior occupant's earlier evidence.
|
||||
Preserve the existing chronological and source-composition behavior at the
|
||||
selected entry path; this does not infer identity across historical tiers.
|
||||
3. For resolution, select paired observations and their covering operation.
|
||||
Keep original-record attribution and reason/dependency lists available even
|
||||
when explanation uses a coalesced replay event.
|
||||
4. Project `Change.Path` into the queried final coordinate where lineage is
|
||||
followed, retaining a meaningful affected path or container ancestor. Keep
|
||||
original source `Location` coordinates and recorded dependency paths intact.
|
||||
Derive `Previous` from the selected lineage, not unrelated positional records.
|
||||
5. Keep collection/ancestor queries on physical history coordinates so they retain
|
||||
the complete operation. Use existing indexes for selection; do not rescan
|
||||
every historical fact separately for every final item.
|
||||
|
||||
**Verification:** `Metadata.History` and historical location lookup follow the
|
||||
matched record through a swap with distinct source locations. Added records do
|
||||
not claim the previous occupant's definitions. Existing key/index shape and
|
||||
ancestor-replacement selection tests remain valid. Stages 8 and 9 complete value
|
||||
replay and removed-position handling using this same selection owner.
|
||||
|
||||
## Stage 8 — Replay earlier explanation values at the entry coordinate
|
||||
|
||||
**Status:** Complete. Focused source, inheritance, CLI, history, and explanation checks passed.
|
||||
|
||||
**Scope:** pre-resolution explanation replay, including inheritance and CLI.
|
||||
|
||||
1. Extend the query-local replay selection context in `explain.go` to use the
|
||||
coordinate description from Stage 7. Keep `Explanation.Path` and its effective
|
||||
snapshot at the requested final address; read earlier matched values at the
|
||||
entry address instead of the final numeric slot.
|
||||
2. Apply source/CLI payloads to the existing physical scratch tree unchanged.
|
||||
Use the entry coordinate for before/after snapshots, contribution selection,
|
||||
baseline, and decoding checkpoints. For additions with no entry lineage,
|
||||
report absence before resolution rather than another record's value.
|
||||
3. Thread the same coordinate through `appendInheritanceExplanation` and its
|
||||
helpers. Preserve inherited source representation, original locations,
|
||||
consumed controls, chronological order, and the existing typed-decoding
|
||||
transition. Do not create a second independently retained history model.
|
||||
4. Retain snapshot-side path information so Stage 11 can mask values using the
|
||||
coordinates from which they were actually read. Do not rebase sensitive
|
||||
values and then mask them solely at their new index.
|
||||
|
||||
**Verification:** a record moved from index 0 to 1 shows its own earlier field
|
||||
value, including selected overlay, inheritance, typed defaults, and CLI evidence
|
||||
where applicable. Use focused representative fixtures rather than multiplying
|
||||
every source form across every case. Collection queries keep original list order.
|
||||
|
||||
## Stage 9 — Explain net resolution and preserve removed evidence
|
||||
|
||||
**Status:** Complete. Focused history and explanation checks passed for moves, removals, additions, and reused positions.
|
||||
|
||||
**Scope:** resolution replay groups, structural changes, and reused addresses.
|
||||
|
||||
1. For matched-record queries, read a group's before side using the entry path
|
||||
and after side using the final path. Apply the physical covering subtree once
|
||||
to scratch; do not replay overlapping child assignments or guess callback
|
||||
order. Preserve combined reasons/dependencies and explicit equal writes.
|
||||
2. A pure move may have equal before/after record values but still includes the
|
||||
recorded collection operation. A field edit plus move shows the matched old
|
||||
value and correct final value, with the application operation retained.
|
||||
3. For newly added records, begin their individual lineage with absent before
|
||||
values. For identity changes, use removal/addition, not a rename transfer.
|
||||
4. Implement the roadmap's removal example: `[reports, invoices]` becomes
|
||||
`[invoices]`. `/resources/0` follows invoices; `/resources` retains reports'
|
||||
removal, earlier values, and available attribution; absent `/resources/1`
|
||||
remains explainable using the established historical-position rules.
|
||||
5. Ensure collection-level selection returns the necessary removed definition
|
||||
evidence, not just a generic parent normalization. Do not expose a removed
|
||||
occupant as part of a surviving record's lineage. Preserve historical-path
|
||||
enumeration of actual observed shapes without inventing projected positions
|
||||
for values that never existed there. Surviving record prefixes take precedence
|
||||
when a missing descendant is queried; wholly absent positions stay historical.
|
||||
|
||||
**Verification:** swap, deletion followed by compaction, insertion with shifted
|
||||
survivors, identity replacement, removed field on a moved record, and queries of
|
||||
reused versus absent positions. Assert metadata attribution and explanation
|
||||
values agree, parent history retains removed evidence, and unknown paths remain
|
||||
unknown. Use both whole-list-only and overlapping list/field records.
|
||||
|
||||
## Stage 10 — Classify matched and removed sensitivity evidence
|
||||
|
||||
**Status:** Complete. Focused sensitivity, secret, and dependency checks passed.
|
||||
|
||||
**Scope:** existing classification and dependency closure.
|
||||
|
||||
1. Extend history sensitivity candidates to include actual entry/final shapes
|
||||
and mapped surviving evidence. Reuse schema tags, projected sensitive nodes,
|
||||
predicates, and recorded dependencies; classify historical values even when
|
||||
removed. Keep identity tuples and resolved credential bytes out of evidence.
|
||||
2. Transfer sensitivity from matched entry descendants to final descendants
|
||||
using the endpoint correspondence. Use the entry evidence as the source for
|
||||
this projection, not an in-place loop that confuses indexes during swaps.
|
||||
Preserve historical-coordinate masks as required for collection and removed
|
||||
queries. Avoid modeling item matching as arbitrary transitive mutation edges.
|
||||
3. Preserve supplied dependency paths and their existing directional semantics.
|
||||
A dependency path does not follow later moves as a public identity selector.
|
||||
Include dependency-derived sensitivity from retained matched evidence and
|
||||
original resolution records, including children hidden by a replay group.
|
||||
4. Keep typed secret lookup and final identity validation at their current
|
||||
phases. Preserve source-version evidence classification at its original
|
||||
coordinates. Do not introduce callbacks, credential reads, or new failure
|
||||
modes while collecting correspondence sensitivity.
|
||||
|
||||
**Verification:** focused cases for sensitivity transferred during a swap,
|
||||
removed evidence retained at collection level, and dependency-derived sensitivity
|
||||
from records hidden by a replay group. Reuse existing dependency-closure tests.
|
||||
Stage 11 verifies the resulting disclosure through the safe rendering boundary.
|
||||
|
||||
## Stage 11 — Mask paired snapshots and preserve safe disclosure
|
||||
|
||||
**Status:** Complete. Focused moved, removed, dynamic-tag, dependency, secret, and diff checks passed.
|
||||
|
||||
**Scope:** snapshot rendering and inspection integration.
|
||||
|
||||
1. Mask a moved explanation's before and after snapshots using their respective
|
||||
coordinates and required historical/dependency closure. Reuse `snapshot.go`;
|
||||
support paired query-local mask information if needed rather than creating
|
||||
another renderer. Conservative historical masking remains acceptable under
|
||||
the existing contract; do not weaken masking to make examples look narrower.
|
||||
2. Preserve effective rendering, public diff masking from both results, source
|
||||
version evidence redaction, and effective secret-reference disclosure rules.
|
||||
A positional historical query and an identity-following surviving query must
|
||||
both mask all sensitive values they actually render.
|
||||
3. Check the same safe-snapshot operation for whole-collection before/after
|
||||
values, including removed records. Container queries retain physical order
|
||||
and their original source evidence; per-record queries use paired positions.
|
||||
|
||||
**Verification:** disposable canaries for sensitive moved fields, sensitive
|
||||
removed records at collection level, dynamic sensitive tags, predicates, and
|
||||
derived dependencies. Verify absence of canaries in safe inspection and failure
|
||||
output. Include a dependency hidden by parent/child coalescing and a swap, so
|
||||
masking cannot accidentally protect only the previous numeric position. Reuse
|
||||
existing tests for ordinary dependency closure and credential non-retention.
|
||||
|
||||
## Stage 12 — Verify the complete feature through the public facade
|
||||
|
||||
**Status:** Complete. Public resolution, identity, diff, disclosure, and boundary checks passed.
|
||||
|
||||
**Scope:** representative integration and ownership regressions.
|
||||
|
||||
1. Add a focused public fixture, for example
|
||||
`facade_resolution_sequences_test.go`, using real `Load`/`LoadLayered`, public
|
||||
`Definition.SequenceIdentities`, resolution records/helpers, metadata, and
|
||||
explanation. Reuse engine coverage for the detailed algorithm matrix.
|
||||
2. Cover the motivating ordinary-list field edit, a registered reorder plus
|
||||
normalization, and removal/position reuse with distinct source evidence.
|
||||
Include one dynamic registration and one ordinary/layered equivalence case.
|
||||
Check codes and semantic paths for uncovered structural changes, rather than
|
||||
snapshotting entire diagnostic messages.
|
||||
3. Verify fixed callback entry queries versus final identity-aware queries;
|
||||
correct final values, explicitness, origins, history, and safe explanation;
|
||||
and a public `Diff` that still reports the containing list atomically.
|
||||
4. Mutate returned configuration and query records after loading and show that
|
||||
frozen explanation and other loads are unaffected. Reuse existing callback
|
||||
lifetime and concurrency tests; add only correspondence-specific ownership
|
||||
coverage where those tests would miss an alias.
|
||||
5. Confirm whole-list-recording consumers still work, YAML-only identities do
|
||||
not gain final matching semantics, and final registration checks can still
|
||||
validate or reject defaults/CLI/resolution output at the established boundary.
|
||||
|
||||
**Completion:** the feature is observable through the existing public API;
|
||||
no exported type, selector, or callback lifecycle change is needed.
|
||||
|
||||
## Stage 13 — Document the implemented contract and validate the sprint
|
||||
|
||||
**Status:** Complete. Focused engine, facade, and consumer checks, `go test -race ./...`, and `./scripts/check.sh` passed with Go 1.27.0 on Linux.
|
||||
|
||||
**Scope:** canonical documentation and final integration checks.
|
||||
|
||||
1. Update `docs/config.md` resolution, layered metadata, historical explanation,
|
||||
identity-registration, and private-comparison sections. Explain ordinary
|
||||
positional comparison, eligible registered correspondence, atomic structural
|
||||
coverage, final versus entry addresses, additions/removals, and reused-position
|
||||
history. Keep public effective diff atomic. Include a short field-edit and
|
||||
remove/compact example and link the ADR for rationale.
|
||||
2. Update public comments in `lifecycle.go`, `metadata.go`, `inspection.go`, and
|
||||
`sequence_identity.go` where their current descriptions need qualification.
|
||||
Check the engine-facing counterparts as well. Preserve fixed `EntryInfo` and
|
||||
`IsExplicit` semantics and avoid promising identity-addressed public paths.
|
||||
3. Update only relevant architecture-policy provisions to recognize registered
|
||||
identity correspondence during resolution inspection. Preserve composition,
|
||||
final-only identity validation, public facade ownership, resource lifetimes,
|
||||
and secret boundaries. Update `docs/internal/overview.md` for actual new
|
||||
responsibilities and `docs/development.md` for the ADR and test owners.
|
||||
Keep detailed current behavior in its canonical owner.
|
||||
4. Update roadmap/plan completion status after reviewing the full target, without
|
||||
deleting unrelated pending roadmaps. Do not prepare release notes or publish
|
||||
a release as part of this sprint.
|
||||
5. Format changed Go files and review whitespace. Run the checks below with the
|
||||
installed compatible Go toolchain. These are development checks, not new
|
||||
release gates; do not wait for CI.
|
||||
|
||||
```sh
|
||||
go test ./internal/engine -run 'Resolution|History|Explain|Sequence|Collection|Secret|Sensitivity|Diff|Independent|Provenance'
|
||||
go test . -run 'Resolution|SequenceIdentity|FacadeInspection|Diff|PublicFacadeBoundary|DependencyImportBoundary|LocalDocumentationLinks'
|
||||
go test ./examples/consumers
|
||||
go test -race ./...
|
||||
./scripts/check.sh
|
||||
```
|
||||
|
||||
Use narrower applicable commands during earlier stages. On failure, resolve the
|
||||
actual defect or deliberately changed contract rather than weakening unrelated
|
||||
assertions. Do not add traversal-count, benchmark-threshold, or configuration
|
||||
snapshot tests. No new network or credential requirement belongs in the suite.
|
||||
|
||||
**Final acceptance review:** verify all roadmap examples, ordinary and registered
|
||||
list rules, fallback without early rejection, complete field coverage, unchanged
|
||||
helper/entry semantics, matched source/presence/history, added-record absence,
|
||||
removed/reused-position evidence, safe moved/removed values, detached ownership,
|
||||
existing history admission, and atomic public diff. A passing coverage audit
|
||||
alone is insufficient. Review the final diff and commit the completed nonempty
|
||||
work with a plain-English message, leaving the workspace clean and reporting
|
||||
validation and any material limitations.
|
||||
@@ -1,290 +0,0 @@
|
||||
# Sequence resolution implementation review
|
||||
|
||||
Reviewed on 2026-11-02 against the [feature roadmap](resolution-sequences.md),
|
||||
[implementation plan](implementation.md), and
|
||||
[ADR 0021](../adr/0021-sequence-resolution-correspondence.md).
|
||||
|
||||
Scope: commits `1cee127..b066090` plus the pre-existing working-tree changes,
|
||||
including final documentation, an additional sensitivity regression, and the
|
||||
sensitivity fixed-point loop in `internal/engine/secret.go`. The latter loop is
|
||||
not present in `b066090`; Finding 3 specifically concerns the reviewed working
|
||||
tree. Findings 1 and 2 also affect the committed implementation. Existing work
|
||||
was preserved; this review changes no production code or tests.
|
||||
|
||||
The architecture now separates resolution comparison from the atomic comparator
|
||||
used by public diff. A private entry-to-final correspondence for registered
|
||||
collections feeds auditing, snapshots, metadata refresh, and shared history
|
||||
selection/replay. All five findings below have now been addressed. Original
|
||||
audit observations are retained for context; completion notes describe the fixes.
|
||||
|
||||
A follow-up integration and algorithm audit at `27be6ba`, including the same
|
||||
pre-existing working-tree changes, added Findings 4 and 5. It preserved the
|
||||
decided comparison, identity, ordering, and history policies. Findings 1–3 were
|
||||
still open at that audit; Finding 3 covers the principal collection-size quadratic loop and
|
||||
is not duplicated below.
|
||||
|
||||
## Finding 1 — Metadata history rejects a valid matched historical field
|
||||
|
||||
**Priority:** P2. **Status:** Complete.
|
||||
|
||||
Completion: metadata history now delegates correspondence-aware eligibility to
|
||||
the shared selector. A regression verifies agreement with explanation for a
|
||||
removed field absent from the previous final-slot occupant, while effective
|
||||
lookup, unknown paths, and explicit key/index distinctions remain correct.
|
||||
The regression failed before the fix; focused tests and `./scripts/check.sh`
|
||||
passed after it.
|
||||
|
||||
**Code:** [engine metadata](../../internal/engine/metadata.go),
|
||||
`Metadata.History` at lines 50–55; compare
|
||||
[shared history selection](../../internal/engine/history_select.go),
|
||||
`selectHistory` at lines 43–49.
|
||||
|
||||
`selectHistory` accepts a query whose matched entry coordinate has an observed
|
||||
shape, even if its final coordinate never had that field. `Metadata.History`
|
||||
still requires the requested coordinate itself to have been observed before
|
||||
calling that selector. Consequently metadata history and explanation disagree
|
||||
on a removed descendant of a moved record.
|
||||
|
||||
Reproduction using `Items []map[string]any`, with an application registration
|
||||
of `id` at `/items`:
|
||||
|
||||
```yaml
|
||||
items: [{id: a, extra: own}, {id: b}]
|
||||
```
|
||||
|
||||
During resolution, delete `extra` from `a`, swap the records, and record a
|
||||
normalization at `/items`. Query `/items/1/extra`, which follows surviving
|
||||
record `a` back to entry position 0.
|
||||
|
||||
**Expected:** both inspection APIs retain `a`'s earlier field evidence and its
|
||||
removal. Effective lookup remains absent. The roadmap and plan explicitly cover
|
||||
removed descendants of surviving records through correspondence.
|
||||
|
||||
**Actual, verified through the public API:** `Explain` succeeds with two history
|
||||
events, including a resolution transition from `own` to absence, while
|
||||
`Metadata.History` returns an empty slice. The requested physical path did not
|
||||
exist on either endpoint, although its matched entry path did.
|
||||
|
||||
**Remediation:** make metadata history use the shared selector's correspondence-
|
||||
aware eligibility rather than its stronger preliminary shape check. Preserve
|
||||
unknown-path and explicit key/index-shape handling. Add a focused regression
|
||||
where the old occupant of the final position never had the removed field;
|
||||
the existing removed-field test gives both original records that field and does
|
||||
not expose this inconsistency. This requires no synthetic historical-path
|
||||
enumeration or new public addressing mechanism.
|
||||
|
||||
## Finding 2 — Deep descendant history reports the old record's operation path
|
||||
|
||||
**Priority:** P2. **Status:** Complete.
|
||||
|
||||
Completion: pre-resolution operation paths now use the matched record-prefix
|
||||
translation, preserving ancestor scope and suffixes. Resolution records already
|
||||
in final coordinates are left unchanged. A regression covers deep inherited
|
||||
fields, record/container queries, original source coordinates, and overlapping
|
||||
final-position records. Focused history checks and `./scripts/check.sh` passed.
|
||||
|
||||
**Code:** [shared history selection](../../internal/engine/history_select.go),
|
||||
`selectHistory`, especially the reported-path translation at lines 116–120.
|
||||
|
||||
Selection follows the correct entry record, but reported-path translation uses
|
||||
the queried leaf and a depth heuristic rather than the matched record prefix.
|
||||
An operation reported at an entry record or other ancestor more than one level
|
||||
above the queried leaf can retain its old index. That `Change.Path` is neither
|
||||
the final queried path nor its ancestor. This diverges from Stage 7's requirement
|
||||
to project affected paths into the final coordinate while preserving original
|
||||
source locations separately.
|
||||
|
||||
Verified reproduction with registered `id` identity:
|
||||
|
||||
```yaml
|
||||
items:
|
||||
- id: parent
|
||||
nested: {more: {value: old}}
|
||||
- id: child
|
||||
extends: {id: parent}
|
||||
```
|
||||
|
||||
Swap the records during resolution, record `/items`, then inspect
|
||||
`Metadata.History(/items/0/nested/more/value)`. The child is now at position 0.
|
||||
|
||||
**Expected:** the inherited operation's reported path identifies the affected
|
||||
region under final `/items/0`, or its actual common container ancestor; source
|
||||
`Location` coordinates continue to describe the original input.
|
||||
|
||||
**Actual:** history contains a main-layer inherited operation reported at
|
||||
`/items/1`, followed by the resolution operation at `/items`. The first reported
|
||||
path now identifies the other record. Source/value matching alone does not
|
||||
repair this public path inconsistency.
|
||||
|
||||
**Remediation:** translate reported paths using the actual entry/final item-prefix
|
||||
pair, preserving the operation's relative suffix and ancestor scope. Distinguish
|
||||
entry-coordinate source/inheritance paths from resolution records that already
|
||||
use final coordinates, so the latter are not translated twice. Verify a deep
|
||||
descendant of a moved inherited record, a containing operation, and an unchanged
|
||||
common collection ancestor. Avoid substituting the queried leaf for every
|
||||
operation path solely to satisfy an ancestor check.
|
||||
|
||||
## Finding 3 — Sensitivity propagation rescans the whole collection quadratically
|
||||
|
||||
**Priority:** P2. **Status:** Complete.
|
||||
|
||||
Completion: matching edges and real dependencies now share one indexed work
|
||||
queue. Candidate coordinates are registered once, newly reached historical
|
||||
coordinates are registered on demand, and each trigger fires at most once;
|
||||
classification no longer rebuilds closure in full-map passes. Conservative
|
||||
historical masking is retained. The existing earlier-matched-value regression
|
||||
is retained in this fix, and a cyclic rotation regression covers removed fields
|
||||
and dependency-derived values. Focused sensitivity tests and
|
||||
`./scripts/check.sh` passed.
|
||||
|
||||
**Code:** [secret classification](../../internal/engine/secret.go),
|
||||
`classifySecretPathsWithEvidence`, working-tree lines 83–122; associated
|
||||
[correspondence translations](../../internal/engine/resolution_correspondence.go).
|
||||
|
||||
The working-tree fix repeatedly projects every currently sensitive path in both
|
||||
directions through the permutation, merges those paths into the same positional
|
||||
mask, and rebuilds dependency closure until no mask changes. Each pass scans all
|
||||
candidate paths and reruns `closeSecretDependencies` from scratch.
|
||||
|
||||
For an `n`-record cyclic rotation with one initially sensitive field and no
|
||||
dependency edges, the first pass reaches neighboring positions. Subsequent
|
||||
passes treat those newly projected positions as fresh entry/final seeds and
|
||||
advance around the cycle. There are Theta(n) passes over Theta(n) paths, giving
|
||||
Theta(n²) work at fixed record width and depth. This does not require a dense
|
||||
dependency graph or unusually deep configuration. The plan called for endpoint
|
||||
projection from entry evidence and explicitly cautioned against treating item
|
||||
matching as arbitrary transitive mutation edges.
|
||||
|
||||
A public-API probe with entry identities `[a, b, c, d]`, final identities
|
||||
`[d, a, b, c]`, and only `/items/0/value` predicate-classified confirms that all
|
||||
four final values become masked. Some masking at reused historical coordinates
|
||||
is deliberately conservative; the issue is reusing projected positional masks
|
||||
as further correspondence seeds and rescanning the complete population. This
|
||||
finding does not introduce a promise to disclose more values or weaken the
|
||||
existing historical sensitivity contract.
|
||||
|
||||
**Remediation:** retain the protection provided by the additional
|
||||
`TestMovedDependencyProtectsEarlierMatchedValue` regression, including backward
|
||||
protection of earlier matched values after dependency classification. Represent
|
||||
entry/final sensitivity correspondence explicitly and process newly classified
|
||||
states with indexed dependency/matching edges or another bounded worklist,
|
||||
rather than rebuilding full closure on every projection pass. Keep original
|
||||
historical-coordinate masks and required conservative redaction. Check cyclic
|
||||
rotations and interactions with dependencies without performance thresholds or
|
||||
tests of incidental traversal counts. Reverting to a single incomplete
|
||||
projection pass would lose the dependency-history protection the working-tree
|
||||
change was added to provide.
|
||||
|
||||
## Finding 4 — Correspondence lookup repeatedly serializes growing path prefixes
|
||||
|
||||
**Priority:** P2. **Status:** Complete.
|
||||
|
||||
Completion: eligible collection prefixes now use the existing prefix index.
|
||||
Translation formats the requested pointer once, retains concrete path-kind
|
||||
checks, and copies the resulting path independently. No-eligible cases return
|
||||
positionally without formatting prefixes; no-registration loads skip matching
|
||||
construction entirely. Cloning rebuilds an independently owned index. Existing
|
||||
matching/history/ownership coverage and a numeric-key/escaped-key regression
|
||||
passed, along with `./scripts/check.sh`.
|
||||
|
||||
**Code:** [correspondence translations](../../internal/engine/resolution_correspondence.go),
|
||||
`entryPath` at lines 96–125 and `finalPath` at lines 129–156; consumers include
|
||||
[metadata refresh](../../internal/engine/load_metadata.go), resolution record
|
||||
validation/snapshots, shared history queries, diagnostic locations, and sensitivity.
|
||||
|
||||
Both translation methods search for a containing collection by constructing and
|
||||
serializing successively longer prefixes of the requested path. For an unmatched
|
||||
path with `d` constant-width segments, prefix serialization performs work
|
||||
proportional to `0 + 1 + ... + (d-1)`, or Theta(d²), per lookup. Each nonempty
|
||||
serialization also allocates a pointer string. The final owned path copy only
|
||||
needs Theta(d) work. This prefix search is not an algorithm selected by the
|
||||
roadmap or plan.
|
||||
|
||||
This also affects ordinary configurations with no registered collections:
|
||||
`entryPath` still formats every prefix even when `collections` is empty.
|
||||
`applyResolutionMetadata` calls it while seeding every final field and again
|
||||
while refreshing explicitness. Thus a resolution callback introduces this cost
|
||||
across ordinary metadata, not only when records actually move. On a chain of
|
||||
depth `d`, the lookups at all levels contribute Theta(d³) prefix-serialization
|
||||
work, compared with Theta(d²) for linear lookup at each level. Existing depth
|
||||
limits bound the magnitude; this is an avoidable extra depth factor within those
|
||||
limits, not a claim of unbounded supported input depth.
|
||||
|
||||
**Remediation:** return an owned positional path immediately when there is no
|
||||
eligible correspondence. For registered scopes, use a segment-prefix index, or
|
||||
serialize the requested pointer once and query the existing `pathPrefixIndex`
|
||||
for matching ancestors. Retain concrete path-kind verification so numeric keys
|
||||
and sequence indexes remain distinct, and preserve additions, removals, and
|
||||
wholly absent historical-position behavior. Because registration selectors do
|
||||
not pass through sequence indexes, the relevant matching record prefix does
|
||||
not require an exhaustive scan of every collection. This can make translation
|
||||
linear in encoded path length while retaining the current compact pair tables.
|
||||
Also avoid the preparatory correspondence tree walk when the compiled
|
||||
application policy contains no registrations. Verify behavioral equivalence
|
||||
through existing translation, shape, history, and ownership tests rather than
|
||||
adding timing thresholds or exact allocation assertions.
|
||||
|
||||
## Finding 5 — Delta sorting formats pointers inside every comparison
|
||||
|
||||
**Priority:** P3. **Status:** Complete.
|
||||
|
||||
Completion: both comparison operations share the existing decorated delta sort,
|
||||
which constructs one pointer per delta and preserves canonical pointer ordering
|
||||
and the kind tie-break. Existing comparison and diff coverage passed, along with
|
||||
`./scripts/check.sh` and the final `go test -race ./...` run. No performance
|
||||
thresholds, persistent cache, or new public contract were introduced.
|
||||
|
||||
**Code:** [resolution comparison](../../internal/engine/resolution_compare.go),
|
||||
`compareResolutionValues` at lines 64–69; compare the existing decorated sort in
|
||||
[shared structural comparison](../../internal/engine/document_compare.go),
|
||||
`compareConfigurationValues`.
|
||||
|
||||
The new resolution comparator invokes `Path.String()` on both operands during
|
||||
every sorting comparison. With `k` deltas, comparison sorting can invoke this
|
||||
Theta(k log k) times. For pointer length `L`, that contributes Theta(k L log k)
|
||||
serialization work and Theta(k log k) non-root pointer allocations in the worst
|
||||
case. This is newly avoidable overhead; the existing shared comparator already
|
||||
formats one pointer per delta before sorting.
|
||||
|
||||
**Remediation:** follow the existing decorate/sort/unwrap pattern: compute each
|
||||
delta's pointer once, sort by the retained pointer and existing kind tie-break,
|
||||
then return the original semantic paths. Pointer construction becomes Theta(k L)
|
||||
with Theta(k) pointer allocations. Sorting still performs O(k log k) comparisons
|
||||
with normal string-comparison costs; this does not claim a better worst-case
|
||||
bound for string sorting itself. Keep canonical ordering as expressly selected
|
||||
in the plan. Reuse ordering tests; no alternate sort policy, persistent cache,
|
||||
benchmark gate, or API change is needed.
|
||||
|
||||
## Validation performed
|
||||
|
||||
- The current working tree passed `./scripts/check.sh` with Go 1.27.0 on Linux.
|
||||
- `go test -race ./...` passed with the same toolchain.
|
||||
- Separate disposable programs outside the repository reproduced Findings 1
|
||||
and 2 through the public facade and confirmed the cyclic masking behavior
|
||||
described in Finding 3. Its complexity follows from the implemented loop;
|
||||
no timing benchmarks were collected.
|
||||
- The review found no additional public API restrictions or earlier identity
|
||||
validation requirements. Shared atomic diff comparison, callback entry views,
|
||||
registered selector scope, and YAML-only identity lifetime remain preserved.
|
||||
- The follow-up traced identity-table construction, resolution comparison,
|
||||
coverage and snapshot handoffs, metadata refresh, history selection/replay,
|
||||
and sensitivity propagation. Endpoint matching uses tuple lookup tables rather
|
||||
than pairwise record searches. Existing whole-tree explanation replay and
|
||||
canonical ordering were retained as planned choices, not reconsidered.
|
||||
- Findings 4 and 5 follow directly from the loops, `Path.String` implementation,
|
||||
and existing prefix-index/decorated-sort alternatives. No new tests or
|
||||
benchmarks were needed; prior successful checks apply to the unchanged
|
||||
production and test files. The amended document's links and whitespace were
|
||||
checked separately.
|
||||
|
||||
## Remediation validation
|
||||
|
||||
All five issues were addressed sequentially in separate nonempty commits, each
|
||||
with its completion note and relevant verification. Final validation passed
|
||||
`./scripts/check.sh` and `go test -race ./...` with Go 1.27.0 on Linux. The
|
||||
development check includes formatting, whitespace, tests, local documentation
|
||||
links, vet, and build checks. No benchmark thresholds or release gates were added.
|
||||
|
||||
The directly related pending sensitivity regression was retained with Finding 3.
|
||||
Unrelated pre-existing documentation and public-comment changes were preserved
|
||||
outside the remediation commits.
|
||||
@@ -1,252 +0,0 @@
|
||||
# Field-level resolution and identity-aware sequence history
|
||||
|
||||
Status: implemented. This roadmap records field-level
|
||||
resolution auditing and identity-aware inspection of registered collections.
|
||||
The [implementation plan](implementation.md) owns execution order and verification.
|
||||
|
||||
## Purpose
|
||||
|
||||
Allow applications to normalize or derive individual fields inside lists and
|
||||
record those fields directly. Preserve the requirement that every net change
|
||||
made during resolution has an exact or ancestor record, together with coherent
|
||||
final values, provenance, history, and sensitivity metadata.
|
||||
|
||||
For a collection with an applicable application registration, the declared key
|
||||
defines a record across positions. Use that correspondence in auditing and
|
||||
inspection. For ordinary lists, position is the only available correspondence;
|
||||
compare positions as addresses without inferring which application object an
|
||||
element represents.
|
||||
|
||||
## Prior baseline
|
||||
|
||||
At the planning baseline, the resolution contract audited the typed configuration
|
||||
before and after `Definition.Resolve`. Mappings were compared recursively, while
|
||||
the private structural comparator reported any list change at the whole-list
|
||||
path. A field-level record inside a changed list therefore failed to cover that
|
||||
list-level delta. Recording the whole list worked.
|
||||
|
||||
For example, changing `/resources/0/path` and recording only that path previously
|
||||
produced an unrecorded-change diagnostic at `/resources`. That was established
|
||||
behavior rather than a new regression associated with v1.0.0.
|
||||
|
||||
The comparator also serves other inspection and metadata operations. Making
|
||||
resolution auditing more precise must account for those callers rather than
|
||||
silently changing the public diff contract.
|
||||
|
||||
The prior resolution implementation captured before and after values at the
|
||||
same numeric path and refreshed metadata at that path. Following a registered
|
||||
record through a move requires the matching entry and final record evidence.
|
||||
|
||||
Application registrations already define identities for collections and constrain
|
||||
their final values. YAML-only declarations have a shorter lifetime. The
|
||||
[identity contract](../config.md#application-registrations-and-identity-details)
|
||||
and [ADR 0017](../adr/0017-collection-sequence-policy.md) own those distinctions.
|
||||
|
||||
## Target behavior
|
||||
|
||||
### Mappings and ordinary lists
|
||||
|
||||
Mapping comparison remains recursive. For lists without applicable application
|
||||
identity registrations:
|
||||
|
||||
- If the lists have equal lengths, compare corresponding positions recursively.
|
||||
A position is an address, not an assertion of element identity.
|
||||
- If the length changes, report a structural change at the list path, requiring
|
||||
a record for that list or an ancestor.
|
||||
- For equal-length lists, a reorder manifests as changes at the affected
|
||||
positions. Every detected change needs coverage; a whole-list record remains
|
||||
a convenient way to cover the operation.
|
||||
|
||||
Apply these rules recursively to nested values. Preserve existing treatment of
|
||||
additions, removals, null, scalar differences, and changes of value kind except
|
||||
where finer sequence comparison deliberately changes the reported path.
|
||||
Ordinary-list inspection remains positional; similar field values do not justify
|
||||
moving provenance or history between positions.
|
||||
|
||||
### Lists with registered identity
|
||||
|
||||
Use applicable `Definition.SequenceIdentities` registrations to match records
|
||||
between the entry and final snapshots. Reuse existing identity-field selection,
|
||||
compound-identity, scalar-equality, and duplicate-detection semantics. Do not infer
|
||||
an identity from a field merely named `name` or `id`.
|
||||
|
||||
| Observed change | Proposed auditing behavior |
|
||||
| --- | --- |
|
||||
| An ordinary field changes on a matched record | Compare recursively and allow a record covering that field. |
|
||||
| Identity order changes | Report a structural change at the list path; require a list or ancestor record. |
|
||||
| An identity is added or removed | Report a structural change at the list path; require a list or ancestor record. |
|
||||
| An identity changes | Treat it as removal and addition, without inferring a rename. |
|
||||
|
||||
Compare matched records by identity even when their numeric positions differ.
|
||||
Structural changes still require whole-list coverage. A field record by itself
|
||||
does not cover an accompanying reorder or membership change.
|
||||
|
||||
Identity matching serves both audit coverage and inspection. Although a
|
||||
whole-list record covers every edit in a moved collection, matching still
|
||||
identifies which record's earlier values and source evidence belong to each
|
||||
final record. Stopping at a whole-list delta is insufficient for that purpose.
|
||||
|
||||
Match only the resolution-entry and final snapshots. Identity changes are
|
||||
removal and addition; temporary identities, intermediate moves, and the order of
|
||||
callback mutations are not reconstructed. Matching does not introduce a new
|
||||
collection merge operation or change whole-record `upsert`.
|
||||
|
||||
Use the existing registration selectors and scope. They traverse mapping keys,
|
||||
not sequence indexes; recursive comparison of nested lists does not expand the
|
||||
registration API. YAML-only declarations do not acquire a new final-resolution
|
||||
guarantee or become identity selectors for this work.
|
||||
|
||||
### Matching eligibility
|
||||
|
||||
Use identity matching when both snapshots provide valid, unique identities under
|
||||
the applicable registration. Otherwise use conservative whole-list auditing for
|
||||
a changed collection, without inventing correspondence. An unchanged collection
|
||||
does not require a resolution record merely because matching is unavailable.
|
||||
|
||||
Preserve existing final identity validation. Resolution may repair invalid entry
|
||||
values; this feature does not reject them earlier or require valid identities
|
||||
throughout the callback. When matching is unavailable, the recorded container
|
||||
operation describes the change without claiming that individual records survived.
|
||||
|
||||
### Example
|
||||
|
||||
Assume the application registers `name` as the identity for `resources`:
|
||||
|
||||
```yaml
|
||||
resources:
|
||||
- name: reports
|
||||
path: ./reports
|
||||
- name: invoices
|
||||
path: ./invoices
|
||||
```
|
||||
|
||||
Changing the first path to `/srv/reports` can be covered by a normalization record
|
||||
at `/resources/0/path`. The unchanged `invoices` record requires no record.
|
||||
|
||||
If resolution also swaps the records, the final collection is:
|
||||
|
||||
```yaml
|
||||
resources:
|
||||
- name: invoices
|
||||
path: ./invoices
|
||||
- name: reports
|
||||
path: /srv/reports
|
||||
```
|
||||
|
||||
Recording only `/resources/1/path` does not cover the reorder. Recording
|
||||
`/resources` covers that structural operation and any field edits within the
|
||||
list. A field record can additionally explain the path normalization.
|
||||
|
||||
Inspection at the final `reports` position includes that record's earlier
|
||||
evidence from position 0, with its original source coordinates, and the recorded
|
||||
collection operation explaining the move. Its previous path is `./reports`,
|
||||
not the `./invoices` value that previously occupied position 1. The `invoices`
|
||||
record moved without a field change. An ordinary unregistered list instead
|
||||
describes changes at positions 0 and 1, without identifying records across them.
|
||||
|
||||
### Recording, inspection, and ownership
|
||||
|
||||
Resolution records and helper targets for surviving values describe final
|
||||
positions. Preserve the existing ability to record removed values.
|
||||
`Resolution.EntryInfo` and `Resolution.IsExplicit` retain their existing fixed
|
||||
entry-snapshot meaning. Numeric paths and helper targets are not automatically
|
||||
rebound to an identity when the application later moves an element. A helper
|
||||
assignment must still agree with the final value at its declared target.
|
||||
|
||||
Identity establishes correspondence internally; this feature adds no public
|
||||
identity-addressed path or mutation-tracking API. An application that restructures
|
||||
a list can record its container rather than maintain individual helper targets
|
||||
through moves.
|
||||
|
||||
Preserve helper-value consistency checks, reason and dependency recording,
|
||||
presence distinctions, detached query results, and independent-load ownership.
|
||||
Fine-grained accepted changes must appear consistently in the returned values,
|
||||
metadata, history, explanation, and sensitivity propagation. For eligible
|
||||
registered collections, use the same entry-to-final correspondence for before
|
||||
values, source attribution, input explicitness, and sensitivity. Do not attach
|
||||
the previous occupant's evidence to a different record merely because it now
|
||||
occupies that index. Identity matching alone does not make a newly added record
|
||||
explicit input or give it an invented source.
|
||||
|
||||
A history query at a final record's position includes the matched record's
|
||||
earlier evidence from its entry position. Preserve original source coordinates
|
||||
and include the recorded collection operation explaining the move. Recorded
|
||||
normalization and derivation retain their resolution origins; following prior
|
||||
evidence does not erase the application operation. Removed evidence remains
|
||||
available as described below.
|
||||
|
||||
Explicit structural records remain the explanation of application-owned list
|
||||
restructuring. Dependency paths retain their existing recording semantics;
|
||||
identity matching does not turn supplied paths into selectors that follow later
|
||||
mutations. Check sensitivity propagation through both the matched evidence and
|
||||
recorded dependencies, including moves and removed values. The proposal does not
|
||||
reconstruct the callback's intermediate mutations.
|
||||
|
||||
Public effective-configuration diff output remains atomic for lists. Resolution
|
||||
comparison may need a dedicated internal operation or comparison mode so that
|
||||
its new behavior does not alter unrelated callers.
|
||||
|
||||
### Removed records and reused positions
|
||||
|
||||
For eligible registered collections, inspection distinguishes a surviving
|
||||
record's lineage from evidence of records removed during resolution:
|
||||
|
||||
- Queries at surviving positions describe the surviving record's lineage,
|
||||
including its matched entry evidence and the recorded collection operation.
|
||||
A removed record that previously occupied that position does not become part
|
||||
of the survivor's individual history.
|
||||
- Collection-level history and explanation retain the removal and the removed
|
||||
record's evidence, including available source attribution and before values.
|
||||
- Historical positions that no longer exist remain inspectable under the
|
||||
existing historical-path rules. Their availability does not give a reused
|
||||
position a second, identity-addressed meaning.
|
||||
|
||||
For example, if `[reports, invoices]` becomes `[invoices]`, a query at
|
||||
`/resources/0` follows `invoices` back to its entry position 1. The removal of
|
||||
`reports` and its earlier evidence remain inspectable at `/resources`, rather
|
||||
than being mixed into the individual history of `invoices`. The now-absent
|
||||
position `/resources/1` remains available under the historical-path contract.
|
||||
Original source coordinates are preserved, and existing sensitivity rules apply
|
||||
to all retained evidence.
|
||||
|
||||
## Work needed from the baseline
|
||||
|
||||
- Trace resolution comparison, record coverage, helper verification, metadata
|
||||
updates, history replay, and sensitivity handling to identify where each needs
|
||||
finer sequence changes or explicit before/after correspondence.
|
||||
- Add positional recursive comparison for ordinary equal-length lists and
|
||||
identity-aware comparison for eligible registered lists, while retaining
|
||||
structural coverage for the cases above.
|
||||
- Reuse collection identity machinery and validate matching eligibility without
|
||||
strengthening the existing final-only registration guarantee.
|
||||
- Carry matched entry and final positions into resolution snapshots, metadata,
|
||||
history, and sensitivity handling through the existing shared history model.
|
||||
A coverage-only implementation does not complete this feature. Preserve
|
||||
source coordinates and distinguish a record's lineage from the prior occupant
|
||||
of its final position.
|
||||
- Verify field edits, nested lists, zero/false/null values, multiple simultaneous
|
||||
changes, length changes, reorders, identity changes, helper targets, and
|
||||
metadata behavior across the public loading boundary. Include cases in which
|
||||
a field record must not conceal another unrecorded change.
|
||||
- Exercise moved records with distinct source evidence, explicitness, and
|
||||
sensitivity; combined reorders and field edits; removed values; repaired entry
|
||||
identities; and dynamic registration selectors. Check agreement between final
|
||||
metadata and explanation while retaining entry-query behavior and atomic public
|
||||
diff output. Verify the removal-and-position-reuse example above: surviving
|
||||
record queries follow their own lineage, collection queries retain removed
|
||||
evidence, and absent historical positions remain inspectable.
|
||||
- Update the resolution contract and relevant API comments when implemented.
|
||||
Review ADR and policy follow-up for registered identity as the correspondence
|
||||
used during resolution inspection, as well as the distinction between
|
||||
resolution comparison and public diff presentation. Preserve accepted ADR
|
||||
decision bodies.
|
||||
|
||||
Use identity lookup tables for expected linear matching work in collection size,
|
||||
plus recursive value-comparison work. Preserve deterministic output and existing
|
||||
equality semantics; avoid pairwise searches for matching records. This is a
|
||||
design objective, not a new public performance guarantee.
|
||||
|
||||
No configuration syntax, dependency, release gate, or new mutation-history system
|
||||
is proposed. Existing callbacks that record a whole list should continue to work.
|
||||
The matching-eligibility, final-position-addressing, and removed-record inspection
|
||||
decisions are settled and integrated above.
|
||||
Reference in New Issue
Block a user