Files
narratio/docs/roadmap/implementation.md

320 lines
19 KiB
Markdown

# ConfigKit v0.5.0 implementation plan
## Scope and authority
Implement [the ConfigKit roadmap](config.md). It owns the target, compatibility
choices, and rationale. This plan covers Narratio's upgrade from v0.4.2 to
v0.5.0 and integration with native sibling inheritance. Audita adoption and
workspace reorganization remain subsequent, separately planned work.
The reviewed upstream revision is
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`, tagged `v0.5.0`.
Use the released module, not a local replacement. The pinned
[configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/config.md)
and [consumer guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/consumers/README.md)
own upstream semantics. Read their sibling-inheritance and metadata sections
when working on the relevant stages; source may also be inspected in the
selected Go module cache. The sibling checkout is optional reference material,
not a build or test dependency.
## Working instructions
The audience is a Luna coding agent. Complete the numbered stages in order.
Each stage is a bounded unit with its own verification and completion criteria.
Inspect current code before editing; the paths below identify owners rather
than requiring changes to every listed file.
- Start with `docs/development.md`, `docs/policy/agents.md`, and `config.md` in
this directory. Follow the development policy-reading guide. Architecture,
testing, documentation, and OSS policies apply to this work; release policy
applies if discussing compatibility or eventual release preparation.
- Use Go 1.26. Prefer the repository's graph tools for code discovery and
inspect focused tests at each package boundary.
- Keep ordinary `Load`, typed defaults, explicit imports, one selected overlay,
domain validation, and current path-resolution rules. Native inheritance is
available wherever ConfigKit and Narratio's schema permit it, with no new
collection allowlist, opt-in flag, or Narratio-owned inheritance engine.
- Distinguish original declarations from inherited contributors. Preserve
genuine duplicate-base rejection and truthful provenance. Parent entries
remain ordinary entries subject to existing domain rules.
- Keep invocation-owned secrets bootstrap and shared exact-name credential
forwarding unchanged. Do not introduce a second base load, file reparse, or
historical-secret lookup to recover declaration information.
- Tests use temporary files, controlled synthetic secrets, and real internal
collaborators. No downstream executable, sibling checkout, cloud service,
or paid provider is needed. Protect integration behavior rather than
duplicating ConfigKit's generic algorithm tests or freezing incidental output.
- Commit each completed nonempty unit with a short plain-English message after
suitable validation. Report blockers and incomplete work accurately. If a
review-only task needs no source fix, retain useful behavioral coverage and
report that result rather than making a cosmetic rewrite.
Do not adopt layered loading, new source types, source-relative rebasing,
abstract artifacts, automatic YAML conversion, new execution selection rules,
new output collision policy, or changes to subprocess credentials and workspace
paths in this feature. Existing domain validation and reuse rules still apply;
do not add prohibitions, hardening, or guarantees merely because they can be
tested. This plan does not authorize a release, push, tag, ADR acceptance, or
retirement of pending roadmaps.
## Stage 1: Select the released dependency and update attribution
Purpose: establish the actual dependency baseline before integration work.
Read `go.mod`, `go.sum`, `docs/oss/attribution.md`, the OSS policy, and the
selected release's notes, `COPYRIGHT`, and `LICENSE`.
1. Select `gitea.maximumdirect.net/eric/ConfigKit@v0.5.0` with the normal Go
module workflow. Keep Go 1.26 and avoid unrelated dependency upgrades.
2. Update ConfigKit's attribution entry to v0.5.0 and use the full reviewed
commit for its Source, COPYRIGHT, and LICENSE links. Preserve upstream
copyright wording, SPDX expression, and applicable local license links.
Its license and YAML dependency are unchanged; no new license copy is needed.
3. Inspect module changes for unintended churn or a local `replace`. Existing
configuration loaders should compile without adopting optional APIs.
Verify with `go test ./internal/config ./internal/app ./internal/doccheck`
and `go mod tidy -diff`. Confirm
`go list -m gitea.maximumdirect.net/eric/ConfigKit` selects v0.5.0 without a
replacement.
Complete when the released dependency and notices agree and current focused
tests pass. The cross-file inheritance defect is addressed in the next stage;
this stage does not claim that new inheritance behavior is fully integrated.
## Stage 2: Correct direct declaration accounting
Purpose: allow distinct artifact and family keys to inherit across imported
files while preserving existing duplicate-declaration rejection.
Read `internal/config/pipeline_configkit.go`, particularly
`pipelineMetadataFromConfigKit` and `rejectDuplicateBaseDefinitions`, and
`internal/config/pipeline_configkit_test.go`. Inspect the public ConfigKit
`Metadata.History`, `HistoricalPaths`, `Origin`, and `ChangeKind` contracts.
1. Add package-boundary regression fixtures with a valid parent in one imported
file and a child using `extends` in another. Cover ordinary artifacts and
artifact families. Give parent and child distinct valid output paths or
family patterns, and provide canonical-party context for expanded families.
Both keys remain ordinary domain entries; child IDs must stay their own.
2. Adjust declaration accounting using retained public metadata. In v0.5.0,
original base-input events have `LayerBase` and no origin operation;
projected transfers carry `ChangeInheritance`. Count original base
declarations, not these transfers or subsequent decoding/resolution events.
Consumed structural `extends` paths are not artifact declarations.
3. Keep the historical-path check, including original declarations erased by
a selected overlay. Do not switch to surviving effective keys only or count
all displaced origins as fresh declarations. Reuse a small package-local
helper only if it makes the next stage clearer.
4. Retain `TestPipelineConfigKitRejectsDuplicateBaseDeclarations`, including
disjoint fields supplied to the same key, container removal, and removal of
Scriptorium by an overlay. An overlay may still modify an existing entry.
Add only any missing regression case needed to exercise the new filter.
Verify with `go test ./internal/config`. The positive cross-file regression
should expose the baseline defect before the fix. Assert loaded values and
identities, or the duplicate category and relevant sources, without matching an
entire diagnostic or ConfigKit's private event ordering.
Complete when cross-file inheritance loads and expands correctly, and actual
duplicates still fail even when the overlay hides them. Do not use the direct
declaration filter to remove contributor evidence from inspection.
## Stage 3: Preserve contributor provenance and family ownership
Purpose: keep inspection useful when one entry contains values from several
files, without confusing its defining file with those contributors.
Read `internal/config/artifact_catalog.go`,
`internal/config/artifact_families.go`, `internal/config/effective_pipeline.go`,
and metadata-to-ownership projection in `pipeline_configkit.go`. Inspect the
focused catalog, family, and effective-source tests.
1. Exercise the inherited configurations from Stage 2 through
`EffectiveArtifactCatalog` and `EffectivePipelineSources`. Check inherited
values against their actual parent contributors and child overrides against
the child/overlay contributors. Keep historical definition evidence distinct
from effective field ownership.
2. Preserve `DefinitionSources` aggregation, including inherited contributors.
Do not relabel all inherited values as authored in the child's file.
3. Correct `artifactFamilySource` where its first inherited field currently
selects the parent's file. For an explicitly declared child family, identify
its original declaration using retained metadata: use its base defining file
when present, or its overlay defining file for an overlay-only family.
Overlay edits to a base family do not relocate its base definition.
4. Retain the existing fallback for entries without a direct declaration or
programmatically constructed configuration. Do not restrict valid enclosing
inheritance merely to force every entry into a single-file ownership model.
5. Keep logical `SourceID`, artifact keys, family/character/party provenance,
existing source roles, and inspection interfaces. Update projections only
where inherited evidence is otherwise lost or incorrectly attributed.
Verify with `go test ./internal/config`. Cover an explicitly declared child
family inheriting from another file and a selected-overlay contribution. Check
meaningful records and source identities, not a complete formatted snapshot.
Complete when contributor reporting remains truthful and generated members
identify their child family definition rather than an arbitrary inherited field.
## Stage 4: Verify inherited presence, domain resolution, and path bases
Purpose: prevent Narratio's application defaults or normalization from undoing
the library's inherited explicit input.
Read `pipelineConfigDefaults`, `resolvePipelineConfig`, and
`internal/config/load.go`, including `finalizeLoadedPipeline` and
`resolveNotariusPaths`. Inspect the current presence/defaulting tests and
the relevant domain validators.
1. Add representative inheritance cases for explicit false, zero, empty,
and null input versus genuinely absent input. Include a pointer boolean
and an entry-level absence-based default where Narratio consumes metadata.
For example, inherited `render_debug: false` can exercise artifact presence;
inherited empty Scriptorium timeout can exercise an existing invalid value
that must not silently become its default.
2. Preserve the existing domain outcome for each chosen field. Valid explicit
values retain their value; explicit values already invalid must still fail
ordinary validation rather than be refilled. An empty mapping merges under
ConfigKit's rules and must not be described as universal container removal.
3. Exercise overlay composition before inheritance: modify a parent's field
in the selected overlay and confirm the child's inherited result uses that
effective value. A child override still wins over its parent.
4. Verify a known root-relative field, such as enabled Notarius `config_path`
or `working_directory`, when inherited from an imported declaration.
Resolve it exactly as the existing field contract does. Do not use passive
SourceContext to rebase it to the parent's or child's source directory, or
newly normalize other path fields merely to make their behavior uniform.
5. Fix any demonstrated Narratio integration defect at its existing defaulting
or domain-resolution owner. Use ConfigKit presence and recorded operations;
do not recreate structural inheritance or weaken ordinary validation.
Verify with `go test ./internal/config`. Keep coverage representative; generic
missing-parent, cycle, resource-context, and expansion-limit matrices belong to
ConfigKit. Parents must remain valid Narratio entries, not special templates.
Complete when inherited explicit input and absent defaults have distinct,
correct domain outcomes and existing path bases are unchanged.
## Stage 5: Verify safe effective projections and inherited credentials
Purpose: ensure inheritance changes neither semantic comparison nor the
existing secret boundary.
Read `internal/config/effective_pipeline.go`, `effective_digest.go`,
`effective_projection.go`, `scriptorium_credentials_test.go`, and the effective
projection tests in `load_validate_test.go`. For diff composition, read
`internal/app/config_diff.go`. Reuse
`TestPipelineOverlayDoesNotRequireUnusedBaseCredentials` and
`TestCredentialValuesNeverEnterEffectiveProjectionOrDigest` coverage.
1. Load two configurations that produce the same effective domain values:
one uses inheritance, the other writes the expanded values explicitly.
Include ordinary artifacts and representative resolved family members.
Verify equal safe effective mappings and digests and no effective diff.
Do not compare private provenance as part of semantic equality.
2. Add a representative typed credential inheritance case using shared fields
of valid Narratio sibling mappings and controlled synthetic environment
variables. Verify resolved credentials retain their exact configured names;
serialization and inspection expose those identities without values.
3. Verify overlay removal/replacement of a credential before inheritance does
not resolve the superseded reference. Reuse the existing selected-overlay
regression rather than creating a second loader or bootstrap algorithm.
4. Check that changing only synthetic secret values leaves semantic projections
and digest unchanged. Fix only demonstrated projection defects; preserve
the existing safe typed rendering boundary and semantic reuse evidence.
Verify with `go test ./internal/config ./internal/app`. Existing shared adapter
credential-forwarding tests remain the owner of subprocess forwarding; do not
rewrite adapters or duplicate those tests simply because input can be inherited.
Complete when resolved values, not source layout or secret bytes, determine
effective comparison, and inherited credentials obey the existing safe boundary.
## Stage 6: Document inheritance and maintain a small example
Purpose: make the adopted behavior understandable and usable without expanding
Narratio's configuration surface.
Read `docs/config.md`, `docs/internal/configuration.md`, `examples/README.md`,
the production/testing bundle, and `TestExamplesLoadAndValidate` in
`internal/config/load_validate_test.go`. Follow the documentation policy.
1. Add native sibling inheritance guidance to the configuration reference.
Explain exact sibling lookup, overlay-before-inheritance resolution,
explicit presence, ordinary parent validation, distinct child identity,
and cross-file declaration/contributor distinctions. Link to the pinned
upstream contract for detailed syntax and structural rules.
2. Explain the reserved `extends` key throughout ConfigKit inputs, including
dynamic artifact IDs and prompt-variable mappings. Operators must rename
existing ordinary-data keys with that name; there is no opt-in or automatic
conversion. Campaign, session, and party inputs also use ConfigKit.
3. Clarify the existing statement that overlays cannot inherit: overlay files
cannot inherit or stack, but the selected overlay may supply native sibling
directives within its values. Keep imports and overlay selection unchanged.
4. Update configuration internals for inheritance before typed decoding,
original-declaration counting, contributor projection, and the safe
effective boundary. Describe implemented mechanisms, linking to the public
reference for their operator-facing meaning.
5. In `examples/production-testing/overlays/testing.yml`, add
`extends: session_recap` to the existing `testing_notes` artifact. Keep its
disabled status and distinct output override. It inherits the imported
parent's prompt/input and the parent's testing profile after overlay
application. The parent remains an ordinary artifact.
6. Update the example guide and extend existing example validation to check
those inherited fields. Preserve the bundle's current identities and
artifact count; no new example root or template convention is necessary.
Verify with `go test ./internal/config -run '^TestExamplesLoadAndValidate$'`
and `go test ./internal/doccheck`. Review examples for synthetic-only content
and validate local links. Keep unimplemented Audita/workspace behavior in its
roadmaps. Do not create a release note under an invented Narratio version;
reserved-key compatibility impact remains an explicit release-preparation item.
Complete when canonical documentation and a tested copyable example describe
the implemented adoption, without promoting new restrictions to policy.
## Stage 7: Verify the assembled inspection workflow and close the feature
Purpose: establish that real command composition consumes the integrated
configuration correctly and finish the upgrade as a reviewable baseline.
Read `internal/app/config_commands_test.go`, `config_artifacts_test.go`,
and existing read-only inspection workflow coverage. Reuse the preceding
package coverage rather than repeating its entire matrix at the CLI boundary.
1. Add or extend a representative command-level temporary imported bundle with
inheritance. Exercise validation, effective show, source reporting, artifact
catalog, and diff through existing command entry points. Use a campaign/party
fixture for family inspection, and paired overlays with equivalent inherited
and explicitly expanded values when testing command-level diff equality.
2. Check semantic output: resolved child values, expected contributor/defining
sources, logical artifact identities, and no secret values. Keep inspection
read-only with respect to session/runtime state, as existing tests establish.
Do not invoke downstream applications or add CLI modes or columns.
3. Review the combined diff against `config.md` and this plan. Confirm that
ConfigKit v0.5.0 is selected, attribution links use the reviewed revision,
no optional loading feature was adopted, and no Audita/workspace change or
compatibility shim slipped into scope.
4. Run `go test ./...`, `go test -race ./...`, `go vet ./...`,
`go build ./...`, `go mod tidy -diff`, and `git diff --check` against the
final source. These reuse existing repository checks, not new release gates.
The full suite includes example and documentation-link validation; focused
checks need not be repeated solely for this stage if unchanged.
5. Commit the completed unit and report the dependency revision, checks,
compatibility impact, and any unfinished work. Leave the pending Audita and
workspace plans and Proposed ADRs intact. Retire completed planning files
only during separately requested cleanup/release preparation, repairing links.
Complete when the feature roadmap's completion criteria are evidenced by the
released dependency, integrated behavior, maintained documentation/examples,
and passing final checks. The next implementation scope is Audita; its concrete
plan is prepared separately. Release version selection and publication are
outside this implementation pass.
## Open Questions
None required for this adoption. Existing decisions select native inheritance,
current loading/path/credential semantics, and ConfigKit before Audita before
workspace work. Eventual release preparation must account for the reserved-key
compatibility impact under the release policy; no version is selected here.