Remove completed ConfigKit planning documents
All checks were successful
ci/woodpecker/push/verify Pipeline was successful
All checks were successful
ci/woodpecker/push/verify Pipeline was successful
This commit is contained in:
@@ -1,243 +0,0 @@
|
||||
# Adopt ConfigKit v0.5.0
|
||||
|
||||
## Purpose
|
||||
|
||||
Upgrade Narratio from ConfigKit v0.4.2 to v0.5.0 and integrate its native
|
||||
sibling inheritance with Narratio's configuration validation and inspection.
|
||||
Keep the upgrade focused on the existing loading model and the adjustments
|
||||
needed to consume the new release correctly. Rely on ConfigKit's assembly and
|
||||
metadata features; validate Narratio's effective domain configuration without
|
||||
adding acceptance rules about how that configuration was authored.
|
||||
|
||||
This adoption precedes [Audita adoption](audita.md) and the
|
||||
[workspace reorganization](workspace.md). This document defines the target
|
||||
and required changes; [the implementation plan](implementation.md) defines
|
||||
the steps for reaching that target.
|
||||
|
||||
## Reviewed Upstream Release
|
||||
|
||||
ConfigKit v0.5.0 identifies commit
|
||||
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. The
|
||||
[release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/releases/v0.5.0.md),
|
||||
[configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/config.md),
|
||||
and [consumer guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/consumers/README.md)
|
||||
own the upstream behavior.
|
||||
|
||||
The release introduces native `extends` relationships between sibling mappings,
|
||||
layered loading with ordered underlays, caller-owned filesystem and mapping
|
||||
sources, and passive source-resource contexts. Inheritance preserves explicit
|
||||
presence, provenance, and secret classification.
|
||||
|
||||
The existing loading API remains available. ConfigKit's minimum Go line, YAML
|
||||
dependency, and LGPL-3.0-or-later license are unchanged. Narratio can retain its
|
||||
current Go toolchain and ordinary loading calls.
|
||||
|
||||
## Current Baseline
|
||||
|
||||
The [configuration reference](../config.md) owns Narratio's current public
|
||||
semantics; [configuration internals](../internal/configuration.md) describe
|
||||
loading and inspection.
|
||||
|
||||
- Pipeline loading uses ConfigKit v0.5.0 with explicit additive imports and one
|
||||
selected overlay, typed defaults, recorded resolution, and domain validation.
|
||||
- Campaign, session, and canonical-party inputs use separate single-source
|
||||
loads. Secret-directory bootstrap remains an explicit invocation operation.
|
||||
- Fields with root-relative pipeline path semantics retain that base when their
|
||||
declarations move into imports. For example, enabled Notarius configuration
|
||||
and working-directory paths resolve from the root pipeline directory.
|
||||
- Distinct fields of one artifact or artifact-family entry can compose across
|
||||
base files under ConfigKit's structural rules. Narratio validates the resulting
|
||||
effective entry, and the selected overlay may replace or remove those values.
|
||||
- Source inspection and artifact catalogs consume retained ConfigKit metadata.
|
||||
Effective show, diff, and digest projections describe the resolved domain
|
||||
configuration rather than source layout.
|
||||
|
||||
The released v0.5.0 module is now selected directly, and its attribution points
|
||||
to the reviewed commit. Cross-file artifact and family inheritance and disjoint
|
||||
entry composition now pass through Narratio's effective configuration checks.
|
||||
The remaining integration work verifies native provenance, explicit presence,
|
||||
safe projections, and the assembled inspection workflow described below.
|
||||
|
||||
## Targeted End State
|
||||
|
||||
### Native inheritance
|
||||
|
||||
Use ConfigKit's built-in sibling inheritance wherever its input contract and
|
||||
Narratio's domain schema permit it. ConfigKit resolves `extends` after source
|
||||
composition and overlay precedence, before typed decoding and application
|
||||
resolution. Narratio receives the resulting typed configuration.
|
||||
|
||||
Inheritance can reduce repetition between artifacts and artifact families,
|
||||
including definitions in separate imported files. Distinct child keys retain
|
||||
their identities. Each resulting entry undergoes ordinary Narratio validation,
|
||||
family expansion, dependency resolution, and execution selection.
|
||||
|
||||
Parent entries remain ordinary domain entries. They must satisfy applicable
|
||||
validation and participate in the effective configuration. This adoption does
|
||||
not introduce abstract artifacts or a separate template registry. Operators
|
||||
must override inherited fields where needed to keep the resulting configuration
|
||||
valid, such as output destinations.
|
||||
|
||||
Inherited false, zero, empty, and null input retains ConfigKit's explicit
|
||||
presence semantics. Narratio's absence-based defaults must not refill such
|
||||
values. Normal domain validation still determines whether a resulting value
|
||||
is acceptable.
|
||||
|
||||
### Effective configuration and native provenance
|
||||
|
||||
Remove Narratio's one-base-file-per-artifact/family restriction. If ConfigKit
|
||||
accepts composition of one entry from several files, Narratio accepts that
|
||||
entry when its effective domain values are valid. ConfigKit continues to reject
|
||||
structural conflicts under its own contract. Narratio does not reconstruct
|
||||
declaration history to add another conflict rule or reject domain values that
|
||||
were removed or replaced before the final configuration handoff.
|
||||
|
||||
Domain validation concerns the effective configuration, including application
|
||||
resolution and family expansion. Family publication requires an applicable
|
||||
effective publish configuration, not separate evidence that the section was
|
||||
explicitly authored rather than defaulted. Existing checks on destinations,
|
||||
dependencies, identities, and effective publish rules remain.
|
||||
|
||||
Retain `config sources` and useful provenance in `config artifacts`. Consume
|
||||
ConfigKit's public source, origin, presence, history, and explanation APIs as
|
||||
intended. A thin formatting/projection layer may add Narratio-specific family,
|
||||
character, and party context. It must not duplicate upstream source tracking,
|
||||
inheritance analysis, or derivation bookkeeping.
|
||||
|
||||
An entry can have several contributors. Report the evidence ConfigKit provides
|
||||
without inventing a unique defining-file guarantee or choosing an arbitrary
|
||||
contributor as the owner. Keep logical artifact IDs independent of source files.
|
||||
Small representation changes needed for truthful composite provenance belong
|
||||
to this adoption; retiring inspection commands does not.
|
||||
|
||||
Effective configuration, diffs, and digests continue to reflect resolved
|
||||
domain values. Equivalent inherited and explicitly authored configurations
|
||||
should have equivalent effective projections, regardless of their provenance.
|
||||
Stage reuse continues to use existing semantic evidence.
|
||||
|
||||
### Loading and ownership boundaries
|
||||
|
||||
Retain ordinary ConfigKit loading, centralized typed defaults, imports, selected
|
||||
overlays, domain resolution, credential bootstrap, and exact-name credential
|
||||
forwarding. Preserve current path bases and read-only inspection workflows.
|
||||
|
||||
ConfigKit owns parsing, source composition, structural inheritance, presence,
|
||||
provenance, and secret resolution. Narratio owns its schema, domain defaults,
|
||||
semantic normalization and validation, campaign/family resolution, and runtime
|
||||
use. Supported defaults, resolver, and validator callbacks remain appropriate;
|
||||
there is no requirement to move them after `Load` merely to establish this
|
||||
ownership boundary. ConfigKit presence APIs remain suitable for absence-based
|
||||
defaults, and native metadata remains suitable for operator inspection and
|
||||
existing party ordering.
|
||||
|
||||
Layered loading, embedded configuration sources, programmatic mapping sources,
|
||||
and source-relative resource rebasing are available upstream but are outside
|
||||
this focused adoption. Do not relax empty/null default semantics or change
|
||||
party ordering just to avoid using useful native APIs.
|
||||
|
||||
## Required Changes
|
||||
|
||||
### Dependency and attribution
|
||||
|
||||
Update `go.mod` and `go.sum` to ConfigKit v0.5.0. Update its entry in
|
||||
[attribution](../oss/attribution.md) to the selected version and the reviewed
|
||||
commit's source and notice links. Existing applicable license texts can remain.
|
||||
|
||||
### Configuration integration
|
||||
|
||||
Delete the duplicate-definition check and its obsolete declaration-accounting
|
||||
tests. Remove the family-publish explicit-declaration gate and metadata retained
|
||||
only for that gate. Use existing final-value domain checks instead.
|
||||
|
||||
Simplify source and artifact-definition projections around ConfigKit's public
|
||||
metadata. Remove duplicated tracking or inferred ownership where native evidence
|
||||
already provides the needed information. Retain application context and
|
||||
formatting that actually serve operators. Use the completed load's evidence
|
||||
rather than reparsing sources or performing a second base-only load that could
|
||||
resolve superseded credential references.
|
||||
|
||||
Keep effective rendering and digest construction grounded in the existing safe
|
||||
typed projection, with credential identities rather than resolved values.
|
||||
|
||||
Use ConfigKit's native operation directly. No application-owned inheritance
|
||||
engine, collection registration, alternative keyword, or compatibility loader
|
||||
is needed.
|
||||
|
||||
### Policy alignment
|
||||
|
||||
Revise the Configuration section of the
|
||||
[architecture policy](../policy/architecture.md) to record this responsibility
|
||||
boundary: native loading and metadata belong to ConfigKit; Narratio owns domain
|
||||
meaning and fail-fast effective validation. Explicitly permit supported
|
||||
callbacks and native provenance inspection. Remove wording inconsistent with
|
||||
the established explicit-presence/defaulting semantics rather than introducing
|
||||
a new empty-value policy. General agent, testing, documentation, and OSS
|
||||
policies need no broad revisions for this feature.
|
||||
|
||||
### Compatibility and operator guidance
|
||||
|
||||
Document that `extends` is now reserved structural input throughout ConfigKit
|
||||
loads, including dynamic mappings such as artifact IDs and prompt variables.
|
||||
Operators using that key as ordinary data must rename it. Inheritance is active
|
||||
without an opt-in switch; imports and overlay-file selection remain separately
|
||||
controlled as before.
|
||||
|
||||
Explain native sibling lookup, inherited explicit presence, and the requirement
|
||||
that parent and child entries satisfy Narratio's domain rules. Link to the
|
||||
pinned upstream contract for detailed inheritance semantics. Clarify that a
|
||||
selected overlay may contribute sibling inheritance directives; this does not
|
||||
add inheritance or stacking between overlay files.
|
||||
|
||||
Replace documentation of the one-base-file restriction with ConfigKit's
|
||||
composition contract and Narratio's effective-value validation boundary.
|
||||
Describe any inspection representation changes without promising a unique
|
||||
defining file for a composite entry.
|
||||
|
||||
Update the configuration reference, internal configuration guide, and a small
|
||||
maintained example when implementation lands. Preserve each field's current
|
||||
path-resolution rule when its value is inherited. Record the reserved-key
|
||||
compatibility impact in the eventual release note; automatic configuration
|
||||
conversion is outside this feature.
|
||||
|
||||
### Verification
|
||||
|
||||
Add focused Narratio coverage for cross-file artifact/family inheritance and
|
||||
valid disjoint contributions to one entry. Verify that effective invalid
|
||||
configuration still fails before execution and that structural conflicts still
|
||||
surface as ConfigKit errors. Old source-history rejection tests are retired or
|
||||
rewritten for the new contract, including overlay replacement/removal cases.
|
||||
Verify inherited explicit values against Narratio's defaulting rules, credential
|
||||
safety, native contributor reporting, and equivalent effective projections.
|
||||
|
||||
Concentrate tests on Narratio's integration behavior. ConfigKit's own tests own
|
||||
its generic inheritance algorithm and source lifecycle. Narratio's default
|
||||
tests remain offline, use temporary configuration bundles and controlled secret
|
||||
values, and do not depend on the sibling ConfigKit checkout. Run the existing
|
||||
suite against the selected released module before completing the upgrade.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
Narratio selects the released ConfigKit v0.5.0 module without a local checkout
|
||||
replacement. Cross-file artifact and family inheritance works through the
|
||||
ordinary loader; valid composition is not rejected by Narratio's declaration
|
||||
history checks. Effective domain errors fail before execution. Native provenance
|
||||
remains inspectable without unique-file assumptions; effective projections,
|
||||
explicit-presence defaulting, and credential handling retain their semantics.
|
||||
|
||||
Maintained guidance and examples explain native inheritance and the reserved
|
||||
`extends` key, and the architecture policy reflects the ConfigKit/Narratio
|
||||
boundary. Attribution identifies the selected upstream revision. Focused
|
||||
regression coverage and the existing suite pass without implementing Audita or
|
||||
workspace changes as part of this upgrade.
|
||||
|
||||
## Relationship To Subsequent Work
|
||||
|
||||
ConfigKit adoption provides a verified configuration baseline for the later
|
||||
Audita and workspace features. It makes no changes to subprocess destinations,
|
||||
session layout, or cleanup behavior.
|
||||
|
||||
Audita adoption then migrates downstream profiles and adds adapter support for
|
||||
explicit report/debug destinations. Audita owns its separate ConfigKit and
|
||||
PromptKit dependencies. Shared latest-attempt links, final workspace paths, and
|
||||
retention protection remain part of the workspace reorganization, so those
|
||||
facilities are implemented once at their shared owner.
|
||||
@@ -1,315 +0,0 @@
|
||||
# Adopt ConfigKit v1.2.1 and native configuration ergonomics
|
||||
|
||||
## Purpose
|
||||
|
||||
Adopt current ConfigKit as Narratio's configuration owner and make its native
|
||||
authoring and inspection features useful to operators. Align configuration
|
||||
semantics with other ConfigKit consumers. Existing Narratio assembly conventions
|
||||
are migration inputs rather than compatibility requirements for this work.
|
||||
|
||||
This roadmap defines the target, user intent, and required changes. The
|
||||
[implementation plan](implementation.md) defines the work needed to reach that
|
||||
target. The earlier [v0.5.0 roadmap](config.md) records the completed adoption
|
||||
that forms the current baseline.
|
||||
|
||||
The ownership boundary in [ADR 0003](../adr/0003-assign-configuration-ownership.md)
|
||||
continues to apply: ConfigKit resolves configuration before its final handoff;
|
||||
Narratio owns domain validation, application derivation, and runtime use.
|
||||
Supported callbacks and native metadata provide the integration boundary.
|
||||
|
||||
## Reviewed baseline and releases
|
||||
|
||||
Narratio currently selects `gitea.maximumdirect.net/eric/ConfigKit v0.5.0`,
|
||||
commit `dc09959cca4e105f49ec316c6d4ba585bd2340bd`, with Go 1.26.0 as its
|
||||
consumer minimum. Release builds currently select Go 1.26.7.
|
||||
|
||||
The adoption target is `go.maximumdirect.net/configkit v1.2.1`, commit
|
||||
`51ef3144158db1a09fabdbd1c7ae59908c84923b`. The published tag and the sibling
|
||||
checkout agree on that revision. ConfigKit requires Go 1.27.0 and uses
|
||||
`go.yaml.in/yaml/v4 v4.0.0`. Its code license remains LGPL-3.0-or-later.
|
||||
|
||||
The pinned [configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md)
|
||||
and [consumer guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md)
|
||||
own upstream semantics. The intervening releases supply the following relevant
|
||||
changes:
|
||||
|
||||
| Release | Adoption relevance |
|
||||
| --- | --- |
|
||||
| v0.6.0 | Loading and metadata optimizations; Go minimum moves to 1.26.0. |
|
||||
| v0.7.0 | Native safe effective-configuration `Diff`. |
|
||||
| v0.7.1 | Maintainer-tool update; runtime behavior unchanged. |
|
||||
| v0.8.0 | Input/import destinations, sequence-item placement, shared source-version checks, and initial identified-sequence composition. |
|
||||
| v0.9.0 | Directory and expanded import selection; optional-by-default imports; identified-list replace/append/upsert and inheritance; CLI assignments move after decoding and inheritance. Its final collection APIs replace the initial v0.8 registrations. |
|
||||
| v0.10.0 | Native ordered configuration discovery through `SearchConfig`. |
|
||||
| v0.10.1 | Loading consolidation and import bookkeeping improvements. |
|
||||
| v0.10.2 | Correctness fixes for decoding, callback/resource lifetime, discovery, inspection budgets, and collection operations. |
|
||||
| v1.0.0 | Vanity module path, source-root `config` controls, public API cleanup, source lookup, provenance and sensitivity fixes, and Go 1.27.0 minimum. |
|
||||
| v1.1.0 | Field-level list resolution records and identity-aware inspection across list movement. |
|
||||
| v1.2.0 | Main-relative companion inputs and overlay context for layered loading; overlay filesystem fixes. |
|
||||
| v1.2.1 | Stable go-yaml v4.0.0 dependency; maintainer release-tool update. |
|
||||
|
||||
The corresponding `docs/releases/<version>.md` files at the pinned revision
|
||||
provide the migration details. Select v1.2.1 directly; intermediate releases
|
||||
provide context rather than required deployment stops.
|
||||
|
||||
## Current Narratio integration
|
||||
|
||||
The [configuration reference](../config.md) and
|
||||
[configuration internals](../internal/configuration.md) describe current behavior.
|
||||
|
||||
- Commands select a pipeline path through Narratio's search loop, bootstrap
|
||||
credentials from an explicit directory or a sibling `secrets/` directory,
|
||||
and then call ConfigKit.
|
||||
- Pipeline loading enables explicit path/glob imports and one selected overlay.
|
||||
Campaign, session, and canonical-party documents use separate loads. Session
|
||||
loading also supports command-scoped downloaded input snapshots.
|
||||
- Typed defaults, recorded normalization, domain validation, and native
|
||||
provenance are integrated. Additional path/dependency resolution and
|
||||
campaign-owned family expansion occur after the pipeline load.
|
||||
- Artifact files currently repeat enclosing `scriptorium.artifacts` mappings.
|
||||
Logical IDs come from map keys. Families expand against canonical party data.
|
||||
- Show, diff, and digest use a safe finalized-domain projection. Narratio owns
|
||||
the current mapping-diff algorithm. Native load metadata supplies source and
|
||||
artifact-catalog evidence.
|
||||
|
||||
Some public import descriptions still state restrictions from older ConfigKit
|
||||
behavior. Replace these with current native semantics during adoption rather
|
||||
than recreating them as Narratio checks.
|
||||
|
||||
## Targeted end state
|
||||
|
||||
### Native assembly and operator choice
|
||||
|
||||
ConfigKit owns source discovery, acquisition, composition, inheritance, presence,
|
||||
typed secrets, provenance, and safe native inspection. Narratio supplies its
|
||||
schema, domain defaults and validation, supported application resolution, and
|
||||
operator-selected inputs.
|
||||
|
||||
Use ordinary `Load` for an ordinary configuration. Use public native facilities
|
||||
where they serve the operator workflow. Layered loading is appropriate when
|
||||
actual lower-priority inputs are needed; typed defaults remain appropriate for
|
||||
simple defaults. Main-relative companion inputs allow such underlays to follow
|
||||
a discovered main without Narratio searching for it again.
|
||||
|
||||
Expose the library's authored `config.imports` and `config.sequences` semantics
|
||||
where applicable. Imports use ConfigKit defaults, including optional absence,
|
||||
unless the operator explicitly requests other supported behavior. An invalid
|
||||
selected source still fails. Operators mark essential imports `required: true`.
|
||||
Directory imports can select YAML with an explicit `pattern`; unrestricted
|
||||
filenames remain the library default.
|
||||
|
||||
Native destinations, nested imports, discovery filters, source-local relative
|
||||
import paths, collection operations, and inheritance work under ConfigKit's
|
||||
rules. Narratio validates the final effective domain entries. It adds no
|
||||
fragment-count rule, mandatory fragment version, import-mode allowlist, or
|
||||
parallel composition engine.
|
||||
|
||||
### Ergonomic artifact files and list customization
|
||||
|
||||
Support an artifact definition without repeated enclosing mappings through an
|
||||
explicit import destination. For example:
|
||||
|
||||
```yaml
|
||||
config:
|
||||
imports:
|
||||
- path: artifacts/session-recap.yml
|
||||
destination: [scriptorium, artifacts, session_recap]
|
||||
required: true
|
||||
```
|
||||
|
||||
The imported file contains the artifact fields such as `prompt_id`, `inputs`,
|
||||
and `output_path`. The literal destination defines the reusable key
|
||||
`session_recap`, with logical source `narratio.artifact.session_recap`.
|
||||
The filename is an organizational choice. Keyed files mounted at
|
||||
`[scriptorium, artifacts]` support declared directory or glob imports when many
|
||||
definitions should be discovered together. Apply the same pattern to families.
|
||||
|
||||
Keep keyed artifact/family mappings and campaign-owned character data. They
|
||||
already fit native inheritance and stable identity. Full-root and disjoint
|
||||
definitions remain valid native inputs; a one-artifact-per-file organization is
|
||||
guidance rather than a validity requirement.
|
||||
|
||||
Allow authored native collection declarations for existing list fields where
|
||||
their final shapes satisfy Narratio's schema. Operators can choose replacement,
|
||||
append, or upsert and identified-record inheritance. Explain that upsert
|
||||
replaces whole matching records and that an empty append/upsert contribution
|
||||
does not clear a list. Register application-owned sequence identities only
|
||||
where Narratio has a real domain identity requirement; generic library
|
||||
capabilities do not establish new uniqueness requirements for every list.
|
||||
|
||||
### Discovery, credentials, and resource paths
|
||||
|
||||
Replace Narratio-owned local candidate probing with ConfigKit search descriptions
|
||||
and `SearchConfig`. Use the native `WorkingDirectory`, `UserConfigDirectory`,
|
||||
and `UnixSystemDirectories` presets in that order with application name
|
||||
`narratio`. Use their documented filenames. Explicit `--config` selection
|
||||
remains an exclusive `File` input. Report the source actually selected by the
|
||||
library. This provides convenient development-local configuration and standard
|
||||
ConfigKit naming. Existing `pipeline.yml` files remain explicitly selectable;
|
||||
migration guidance explains the new discovery names and precedence.
|
||||
|
||||
Bootstrap uses `LoadSecretsIntoEnvironment` once per invocation before typed
|
||||
credential resolution and concurrent work. Existing environment variables win,
|
||||
including explicitly empty variables. Use the invocation-owned conventional
|
||||
directory `/usr/local/etc/narratio/secrets`, with `--secrets-dir` selecting
|
||||
another directory. A missing conventional directory remains optional; an
|
||||
explicit selection follows the existing explicit-selection behavior. Retain
|
||||
safe partial-report handling and exact-name forwarding to subprocesses.
|
||||
|
||||
The independent directory lets bootstrap precede ConfigKit's first and only
|
||||
configuration selection. Service environments can provide credentials directly;
|
||||
development invocations can choose a project-local secrets directory explicitly.
|
||||
The implicit sibling `secrets/` convention retires with this cutover.
|
||||
|
||||
Resolve relative file-resource references from their effective defining source's
|
||||
ConfigKit resource context. Inherited values follow their effective origin;
|
||||
CLI-relative paths use the invocation directory. Application-derived paths
|
||||
record their dependencies. Use the main source as a documented fallback for
|
||||
origins without a distinct usable resource base. This makes imported definitions
|
||||
portable across bundles.
|
||||
|
||||
Review actual path-bearing fields explicitly. Artifact destinations, command
|
||||
names, session-relative products, and remote object keys retain their domain
|
||||
meanings. This policy introduces no new target-existence or root-containment
|
||||
checks. ConfigKit's consumer examples illustrate application policy; Narratio
|
||||
owns the selected interpretation.
|
||||
|
||||
### Imports in secondary documents
|
||||
|
||||
Enable native explicit imports for locally sourced campaign, party, and session
|
||||
documents as well as pipelines. This supports split rosters and shared metadata
|
||||
with consistent library semantics. Campaign selection, party schema validation,
|
||||
and session domain rules remain application-owned. Pipeline overlay selection
|
||||
remains associated with the pipeline.
|
||||
|
||||
Materialize effective documents where raw root-file copying would leave
|
||||
unresolved imports in downstream inputs or published session data. Remote
|
||||
session snapshots retain their explicit source context; resolving imports
|
||||
against an incidental download directory would give them an incorrect base.
|
||||
Remote companion-file retrieval remains separate work.
|
||||
|
||||
### Final-domain inspection through native APIs
|
||||
|
||||
Keep validate, show, sources, artifact catalog, and diff useful for the operator's
|
||||
fully resolved pipeline, including campaign-driven members and publish rules.
|
||||
Use native effective rendering and `Diff` where they own the operation, with
|
||||
Narratio formatting and generated-domain context at the presentation boundary.
|
||||
|
||||
A raw pipeline `Result` is a frozen load snapshot. Mutating its `Config` later
|
||||
does not update rendering or diff. Integrate this distinction explicitly:
|
||||
record domain resolution within a supported resolver when its inputs are
|
||||
available there; where later campaign resolution remains appropriate, create
|
||||
a completed native inspection result from the finalized safe domain projection
|
||||
using ConfigKit's public loading APIs. That result describes application-derived
|
||||
values; retained authoring results supply actual file provenance.
|
||||
|
||||
Use native comparison rather than maintaining a second general mapping-diff
|
||||
engine. Preserve final-domain coverage and credential identities. Narratio's
|
||||
digest and stage-owned reuse evidence remain application responsibilities.
|
||||
Source-layout changes alone do not change equivalent domain values. Source
|
||||
inspection can report composite origins, empty optional-import selections, and
|
||||
application derivation without inventing a unique defining-file contract.
|
||||
|
||||
### Scope and compatibility
|
||||
|
||||
This is a clean native cutover. Update source-root `configkit:` controls to
|
||||
`config:` and remove old-module/API references in maintained code and fixtures.
|
||||
Nested `config` data remains ordinary data under ConfigKit's source-root rules.
|
||||
The `configkit` package name and `configkit:"secret"` Go tag retain their
|
||||
upstream spelling. Mapping and identified-list `extends` follow current native
|
||||
rules.
|
||||
|
||||
This roadmap deliberately permits revised configuration semantics. Update
|
||||
tests and documentation for selected changes rather than preserving historical
|
||||
acceptance rules through compatibility loaders or keyword aliases.
|
||||
|
||||
Application identity, artifact dependency meaning, runtime output destinations,
|
||||
campaign-owned party authority, remote session retrieval, and stage reuse remain
|
||||
Narratio domain concerns. This scope does not redesign the runtime workspace,
|
||||
change subprocess adapters, adopt ReleaseKit for Narratio, or add external
|
||||
provider probes. Audita and workspace work retain their separate roadmaps.
|
||||
|
||||
## Required changes
|
||||
|
||||
### Dependency, toolchain, and notices
|
||||
|
||||
- Select the released `go.maximumdirect.net/configkit v1.2.1` module and migrate
|
||||
all imports and exchanged named types together. Remove the old requirement;
|
||||
use the released dependency rather than a filesystem replacement.
|
||||
- Raise `go.mod` to Go 1.27.0 and select an available official Go 1.27 patch for
|
||||
development/release validation and release builds. Align CI, runbook setup,
|
||||
and incorporated Go attribution with that selection.
|
||||
- Update ConfigKit source/license links to the reviewed commit and YAML v4
|
||||
attribution to the selected stable release. Check the actual resulting module
|
||||
graph. Maintainer tool dependencies are not automatically Narratio runtime
|
||||
components. Narratio's separate direct YAML use need not be rewritten merely
|
||||
because ConfigKit changed its internal parser dependency.
|
||||
|
||||
### Integration and simplification
|
||||
|
||||
- Apply the v1 API replacements actually used here, including
|
||||
`LoadSecretsIntoEnvironment`, `LayerMain`, `RecordDerivedDefault`, and
|
||||
`RecordNormalization`. Review other public names against the migration guide.
|
||||
- Use keyed expanded options and definition literals. Keep metadata projections
|
||||
compatible with current source evidence, native layer names, and safe errors.
|
||||
- Replace broad list-record workarounds with supported field-level recording
|
||||
when the application changes only fields. Record whole-list changes where
|
||||
order or membership changes warrant them.
|
||||
- Adopt native discovery, destination-based artifact authoring, authored
|
||||
collection operations, and native final-domain rendering/comparison. Remove
|
||||
replaced search, comparison, and redundant metadata machinery.
|
||||
- Apply the agreed source-relative resource bases and secondary-document imports.
|
||||
Share definitions and loading context between execution and inspection so an
|
||||
operator validates the configuration the pipeline will actually consume.
|
||||
- Review secondary-document input materialization and publication as well as
|
||||
parsing. A copied root file can retain imports whose
|
||||
companions are absent at its new location. Materialize the effective domain
|
||||
document where a subprocess or restored session consumes it, retaining native
|
||||
authoring evidence separately. Review placeholder checks against the effective
|
||||
session input and preserve required external party/session formats.
|
||||
|
||||
### Documentation and maintained examples
|
||||
|
||||
Update the configuration reference, CLI reference, operations/bootstrap guidance,
|
||||
configuration internals, contributor prerequisites, and release procedure for
|
||||
the implemented semantics. Keep canonical facts in their respective owners.
|
||||
Update project-specific architecture wording where selected path/bootstrap
|
||||
decisions change it; preserve ADR 0003's ownership boundary.
|
||||
|
||||
Migrate the production/testing bundle to `config:`. Demonstrate destination-based
|
||||
artifact and family files, a native directory import restricted to YAML, an
|
||||
essential required import, native inheritance, and a useful selected-overlay
|
||||
list customization. Keep illustrative optional choices labeled as choices.
|
||||
Show how the declared artifact key is referenced by dependent artifacts.
|
||||
|
||||
Provide concise manual upgrade instructions for existing Narratio configurations:
|
||||
control-key/API changes, optional import defaults, collection behavior, source
|
||||
discovery, and selected path/bootstrap changes. Separately installed applications
|
||||
own their configuration versions; Narratio's migration does not automatically
|
||||
rewrite their files.
|
||||
|
||||
The eventual Narratio release requires a major version because existing valid
|
||||
configuration control blocks change. The Go minimum increase alone is permitted
|
||||
in a minor release under the adopted release policy. This roadmap does not
|
||||
authorize tagging, publication, or retrospective release-note changes.
|
||||
|
||||
### Verification
|
||||
|
||||
Use temporary inputs and controlled synthetic credentials. Exercise native
|
||||
discovery and the chosen bootstrap convention together, explicit file selection,
|
||||
selected-source errors, optional and required imports, import destinations,
|
||||
inherited artifact identities, native list customization, and final domain
|
||||
validation. Add coverage at Narratio boundaries rather than duplicating
|
||||
ConfigKit's complete structural test matrix.
|
||||
|
||||
Verify inherited presence/defaulting, party order, final campaign/family
|
||||
expansion, resource paths, source attribution, and secret-safe inspection.
|
||||
Diff/show must include derived members and publish rules; compare equal effective
|
||||
configurations across different source arrangements. Credential rotation behind
|
||||
the same reference name must remain absent from semantic diffs and digests.
|
||||
|
||||
Update tests that encode intentionally retired syntax or policies. Run focused
|
||||
configuration/command/example/documentation checks and the established full
|
||||
test, race, vet, build, and module checks on the adopted Go line. Preserve the
|
||||
current supported build targets. No new live-service or CI-completion gate is
|
||||
introduced by this migration.
|
||||
@@ -1,431 +0,0 @@
|
||||
# ConfigKit v1.2.1 implementation plan
|
||||
|
||||
## Scope and authority
|
||||
|
||||
Implement [the approved ConfigKit roadmap](configkit.md). It owns the target,
|
||||
native-semantics intent, and four approved product decisions. This plan replaces
|
||||
the completed v0.5.0 plan. Its audience is a Luna coding agent implementing one
|
||||
end-to-end sprint in the numbered order below.
|
||||
|
||||
Select `go.maximumdirect.net/configkit v1.2.1`, upstream commit
|
||||
`51ef3144158db1a09fabdbd1c7ae59908c84923b`. The pinned
|
||||
[consumer guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md)
|
||||
and [configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md)
|
||||
own upstream semantics. Read the sections identified by each stage and public
|
||||
declarations in the released module as needed. The sibling checkout is optional
|
||||
reference material, not a build or test prerequisite.
|
||||
|
||||
## Working instructions
|
||||
|
||||
- Start with `docs/development.md` and `docs/policy/agents.md`. Architecture,
|
||||
testing, documentation, OSS, and release policies apply; reuse unchanged
|
||||
policy text already read. Prefer graph tools for code discovery.
|
||||
- Implement the full native configuration workflow. Discovery, destinations,
|
||||
collection controls, resource bases, secondary imports, materialization, and
|
||||
native inspection are part of this sprint.
|
||||
- Stages are work boundaries, not deployment releases. Intermediate stages may
|
||||
fail to compile or have dependent tests awaiting later work. Record that state
|
||||
accurately. Avoid temporary adapters or extra abstractions merely to make
|
||||
every intermediate commit green. Final validation covers the assembled system.
|
||||
- Commit completed nonempty units with short plain-English messages. Add focused
|
||||
behavioral tests with their owner; run meaningful available checks. The final
|
||||
stage runs repository-wide validation against the finished source.
|
||||
- Use ConfigKit's public loading, presence, provenance, recording, rendering,
|
||||
and comparison APIs. Narratio supplies domain meaning and runtime use.
|
||||
Retire obsolete source-authoring checks and tests rather than reproducing them.
|
||||
- Leave import defaults unspecified so ConfigKit supplies its defaults.
|
||||
Explicit false/zero/empty/null and collection semantics follow the native
|
||||
contract and actual Narratio domain rules.
|
||||
- Use temporary inputs, real ConfigKit, and controlled synthetic credentials.
|
||||
The default tests remain offline and independent of installed subprocesses.
|
||||
- Write documentation in concise, affirmative, factual language. The roadmap
|
||||
owns intent; this file owns sequencing and implementation choices.
|
||||
|
||||
Preserve stage ordering, artifact dependency meaning, the canonical party schema,
|
||||
exact-name credential forwarding, publish commit semantics, and the existing
|
||||
Linux amd64/arm64 plus best-effort macOS arm64 targets. This sprint does not
|
||||
implement pending Audita/workspace features, migrate old sessions, rewrite
|
||||
downstream configurations, add fragment version mandates or new path checks,
|
||||
adopt ReleaseKit for Narratio, or publish a release.
|
||||
|
||||
## Concrete integration choices
|
||||
|
||||
- Consumer Go minimum: **1.27.0**. Development/release validation and build
|
||||
toolchain: official **Go 1.27.4**, an available patch on that line. Selecting
|
||||
it is not a new publication gate.
|
||||
- Pipeline source when no explicit file is supplied:
|
||||
`SearchConfig("narratio", Searches(WorkingDirectory(), UserConfigDirectory(),
|
||||
UnixSystemDirectories()), SearchOptions{Required: true})`.
|
||||
An explicit `--config` uses `File` exclusively.
|
||||
- Bootstrap: optional `/usr/local/etc/narratio/secrets`, or explicit
|
||||
`--secrets-dir`, through `LoadSecretsIntoEnvironment` before typed loading.
|
||||
Existing environment values win. The implicit sibling directory retires.
|
||||
- Local campaign/session/party loading enables imports. Only the pipeline gets
|
||||
its selected overlay. Party version checking remains main-only; fragments
|
||||
need no new version header. Remote session bytes are contextless snapshots.
|
||||
- Existing campaign registry selection and local session search locations remain
|
||||
domain conventions. Use native optional `SearchConfig` with `SearchPaths`
|
||||
for local session candidates. Only actual absence permits remote fallback.
|
||||
- Final-domain inspection uses a safe DTO and a defaults-backed
|
||||
`Load(NoSource(), Definition.Defaults)` snapshot after domain resolution.
|
||||
Credential fields contain reference names, not typed secrets that would
|
||||
resolve again. Original results supply actual file provenance. Stages 8–9
|
||||
explain this boundary.
|
||||
|
||||
## Stage 1: Establish the dependency and toolchain baseline [x]
|
||||
|
||||
Read `go.mod`, `go.sum`, `.woodpecker/*.yml`, release scripts/runbook, and
|
||||
attribution. Read the upstream v1.0.0 migration table and v1.2.1 release note.
|
||||
|
||||
1. Select Go 1.27.0 in `go.mod` and ConfigKit v1.2.1 under its vanity path.
|
||||
Migrate all production/test imports together and remove the old requirement.
|
||||
Use the released module rather than a filesystem replacement.
|
||||
2. Replace APIs actually used here: `LoadSecretsDir` →
|
||||
`LoadSecretsIntoEnvironment`, `LayerBase` → `LayerMain`, and generic
|
||||
`Resolution.Record` calls with `RecordDerivedDefault` or
|
||||
`RecordNormalization`. Audit other removed public names and compared layer
|
||||
or diagnostic strings. Use keyed expanded options/definition literals.
|
||||
3. Rename genuine source-root `configkit:` controls to `config:` throughout
|
||||
fixtures and examples. Keep the package name, `configkit:"secret"` Go tag,
|
||||
and ordinary data keys distinct from source controls.
|
||||
4. Align Go CI images and local/release instructions with Go 1.27.4. Preserve
|
||||
platform targets and publication mechanics. Update incorporated Go notices.
|
||||
5. Refresh ConfigKit and stable go-yaml v4.0.0 attribution from selected source
|
||||
notices. Review actual module graph churn and tool-only dependencies. Keep
|
||||
Narratio's separate YAML v3 use unless another scoped change needs it.
|
||||
|
||||
Check module identity and attribution. Attempt focused config/app tests; record
|
||||
failures assigned to later stages. Complete when the dependency/API baseline is
|
||||
established, without preserving old behavioral expectations as compatibility.
|
||||
|
||||
## Stage 2: Integrate native pipeline discovery and independent bootstrap [x]
|
||||
|
||||
Read `internal/app/pipeline_config_path.go`, `secrets_bootstrap.go`,
|
||||
`command_dependencies.go`, `loadPipelineConfigWithDependencies` and callers,
|
||||
inspection loaders, and `internal/config/pipeline_configkit.go`.
|
||||
Read upstream discovery and secrets-bootstrap contracts.
|
||||
|
||||
1. Have the configuration owner accept an explicit file or the native required
|
||||
discovery source above. Retain explicit-file package entry points where useful.
|
||||
Commands load through one composition path instead of first probing with
|
||||
`os.Stat` to obtain a default pathname.
|
||||
2. Bootstrap once before typed loading or adapter creation from the independent
|
||||
directory. Preserve explicit-path error behavior, safe partial reports, and
|
||||
existing-empty environment precedence.
|
||||
3. Obtain the actual selected root and import/overlay evidence from ConfigKit.
|
||||
Carry that root into campaign selection, provenance, and command summaries.
|
||||
Remove dependencies/search helpers used only for old pipeline preselection.
|
||||
4. Apply this composition to run/range/alias commands, session init/plan/status/
|
||||
restore, inspection, and remote-session workflows. A selected invalid source
|
||||
fails rather than falling through to another pipeline candidate.
|
||||
5. For overlay diff, load the left through native selection and the right from
|
||||
that selected root with its own overlay. Share invocation bootstrap; neither
|
||||
side performs a preliminary base-only load.
|
||||
|
||||
Test precedence and exclusive explicit selection with controlled working/user
|
||||
directories and temporary native search descriptions. Test selected invalid
|
||||
input, no source, bootstrap override/default behavior, and diff bootstrap once.
|
||||
Keep tests isolated from the operator's system configuration and secrets.
|
||||
|
||||
## Stage 3: Resolve pipeline resources from native origin context [x]
|
||||
|
||||
Read `resolvePipelineConfig`, `resolveNotariusPaths`,
|
||||
`finalizeLoadedPipeline`, `effective_digest.go`, and the upstream source-aware
|
||||
path example. Resolve paths before normalization erases entry origins.
|
||||
|
||||
1. Introduce one small config-owned host-resource base helper using
|
||||
`Resolution.EntryInfo`, `Resolution.Source`, and
|
||||
`Resolution.SourceContext`. Use the value's effective origin rather than
|
||||
its child's inheritance directive. A distinct usable source context supplies
|
||||
its canonical base; missing or composite context falls back to the main source.
|
||||
Explicit CLI-origin file references use the captured invocation directory.
|
||||
If the main input is contextless too, use that invocation directory rather
|
||||
than interpreting a diagnostic label as a filesystem base. This covers remote
|
||||
byte snapshots without turning their temporary filenames into resource roots.
|
||||
2. Normalize nonempty relative references with recorded helpers. Absolute values
|
||||
stay absolute. Derive Notarius working directory from its already resolved
|
||||
config path and record that dependency. Preserve domain validation of relevant
|
||||
empty/disabled values.
|
||||
3. Apply this classification:
|
||||
|
||||
| Fields | Interpretation |
|
||||
| --- | --- |
|
||||
| Audita/Scriptorium/Notarius `config_path`; Notarius `working_directory` | File/directory resource based on effective origin. |
|
||||
| Relative `workspace.root`, `campaigns.root`, `spool.root`, `cache.root` | Host directory reference based on effective origin; absolute defaults stay unchanged. |
|
||||
| Campaign template/stable-input paths; session stable-input/audio paths | Same resource policy, completed in Stage 4. |
|
||||
| Artifact output paths, family output patterns, publish destinations, source IDs, prompt/profile IDs, schema identifiers, binary commands, Notarius reference bindings and chunk-cache modes | Existing domain interpretation, not file-resource rebasing. |
|
||||
|
||||
4. Remove duplicate pipeline-root rebasing and logical-Notarius path caches once
|
||||
callers use resolved domain values. Show/digest describe those domain values,
|
||||
rather than reconstructing an alternative authoring-path view.
|
||||
5. Keep resource access at existing consumer boundaries; base selection adds no
|
||||
target-existence check, root restriction, or history acceptance gate.
|
||||
|
||||
Test imported, inherited, overlay, absent-default, absolute, and ambiguous-origin
|
||||
cases. Equal resolved resources have equal effective semantics across layouts;
|
||||
relative text resolving to a different resource is a meaningful change.
|
||||
|
||||
## Stage 4: Load campaign/session imports with correct contexts [x]
|
||||
|
||||
Read `document_configkit.go`, `load.go`, `pathresolve.go`,
|
||||
`party_resolution.go`, app session discovery/remote-session composition, and
|
||||
session init. Read native import and resource-context contracts.
|
||||
|
||||
1. Use `File` with imports enabled for local campaign/session inputs. Share
|
||||
definitions with byte loading and retain successful results/source contexts.
|
||||
ConfigKit owns per-source and aggregate acquisition limits.
|
||||
2. Replace local session candidate probing with native optional discovery over
|
||||
the existing candidate paths. Permit absent input solely for remote fallback;
|
||||
distinguish absence through native source evidence. Structure callbacks to
|
||||
allow that discovery-only no-source result. Acquired invalid files still
|
||||
receive ordinary validation and fail without remote fallback.
|
||||
3. Downloaded session bytes receive no context inferred from their temporary
|
||||
filename. Retain object provenance and current explicit session/previous-ID
|
||||
checks. Use native contextless-import behavior; no remote import fetcher or
|
||||
custom structural rejection layer is added.
|
||||
4. Resolve campaign `session_template_file`, stable-input paths, and session
|
||||
overrides/audio references with Stage 3's helper. Resolve each
|
||||
`audio_files` element at its entry index before ordering. Keep campaign/
|
||||
session stable-input precedence and campaign ownership of party input.
|
||||
Provenance identifies the effective supplying configuration source.
|
||||
5. Check unresolved template tokens in effective session string values, including
|
||||
imported values. Session init still reads an authored template for intentional
|
||||
substitution; runtime loading stays a configuration operation.
|
||||
6. Retain detached campaign and session document values after their own loading
|
||||
and resource resolution. Capture the session document before campaign-inherited
|
||||
fields are copied into runtime session state. Stage 6 serializes these values.
|
||||
|
||||
Test local imports, resource bases, effective placeholder rejection, identity
|
||||
mismatches, imported session-init template paths, absence versus selected-source
|
||||
failure, and remote snapshots independent of temporary directories. Add no
|
||||
secondary overlay flags.
|
||||
|
||||
## Stage 5: Resolve imported party definitions and canonical roster output [x]
|
||||
|
||||
Read `party.go`, `party_resolution.go`, `artifact_families.go`, the party
|
||||
integration contract, and focused party/family tests.
|
||||
|
||||
1. Add a file-backed party entry point with imports enabled for campaign
|
||||
resolution. Keep `ParseParty(bytes)` contextless for snapshots and tests.
|
||||
Reuse the definition, main-only version rule, and domain validator.
|
||||
2. Preserve `narratio.party.v1`, existing character/name/class/alias validation,
|
||||
and campaign ownership. Native destinations allow split roster contributions;
|
||||
no fragment version mandate or abstract/template record type is added.
|
||||
3. Use native metadata for roster order and evidence, preserving authored order
|
||||
within a source. Extend the current ordering helper using source acquisition
|
||||
order, then line/column and ID as deterministic ties across sources. Prefer a
|
||||
child's own declaration/inheritance location over its parent's location.
|
||||
This is a presentation convention, not a new source acceptance rule.
|
||||
4. Render canonical party YAML from the effective roster: schema version, keyed
|
||||
characters, singular external `alias`, classes and optional levels. Preserve
|
||||
meaningful roster/list order. A small typed DTO/YAML-node writer is suitable
|
||||
for this Narratio-owned external format.
|
||||
5. Keep players derivation and family expansion based on that same roster.
|
||||
Replace output/checksum uses of `CanonicalParty.Raw` with canonical rendering;
|
||||
retain raw data only where an actual diagnostic caller needs it.
|
||||
|
||||
Test split/inherited rosters, aliases/classes, effective duplicate-name rejection,
|
||||
order, reloadable standalone canonical output, players, and family identities.
|
||||
Use native evidence rather than a second source parse to reconstruct order.
|
||||
|
||||
## Stage 6: Materialize effective inputs and align prepare resume checks [x]
|
||||
|
||||
Read `internal/stage/prepare.go`, `prepare_party.go`, `prepare_resume.go`,
|
||||
prepared-input descriptors, and prepare semantic evidence/tests.
|
||||
|
||||
1. Add configuration-owned deterministic campaign/session renderers from Stage
|
||||
4's detached document values. Serialize public schemas and resolved file
|
||||
references, excluding native directives and runtime/provenance state.
|
||||
2. Preserve ownership: campaign-inherited stable inputs must not become explicit
|
||||
session overrides. A published session that omitted `speakers_file` must
|
||||
still inherit the receiving campaign's value. Explicit session overrides
|
||||
remain explicit. Include actual session identity values where application
|
||||
resolution supplies them, without freezing campaign defaults into the session.
|
||||
3. Prepare writes these documents and Stage 5's party bytes through existing
|
||||
artifact-store helpers. Keep filenames, input kinds, provenance fields, and
|
||||
write/checksum behavior. Resume uses the same renderers instead of comparing
|
||||
prepared effective output with a raw imported root.
|
||||
4. Ensure imported effective changes reach existing stage-owned semantic/input
|
||||
evidence. Equivalent domain values remain equivalent. Full pipeline digests,
|
||||
source mtimes, or every authoring file are not new blanket invalidation keys.
|
||||
5. Preserve ordinary stable-file copying/audio handling. Already resolved paths
|
||||
must not be rebased again against the session root. Retire raw campaign/session
|
||||
file requirements used solely for copying the old authoring documents.
|
||||
|
||||
Test standalone prepared documents, imported changes, equivalent layouts, and
|
||||
remote-origin metadata. Canonical-byte format changes can cause a normal prepare
|
||||
rerun; no session migration or new manifest schema is needed for serialization.
|
||||
|
||||
## Stage 7: Close publication, restore, and remote-session round trips [x]
|
||||
|
||||
Read publish input selection, remote-session loading, restore contracts, and
|
||||
existing in-memory storage integration tests.
|
||||
|
||||
1. Trace the actual bytes published as remote `session.yml`. Publish the canonical
|
||||
prepared session rather than the original root or command-scoped download.
|
||||
Preserve immutable object/commit-pointer ordering and lock behavior.
|
||||
2. Confirm campaign/party prepared products flow through existing selections
|
||||
with their identities intact and without unresolved companion imports.
|
||||
3. Verify a published canonical session loads through the existing contextless
|
||||
path after authoring fragments and temporary files disappear. Restore uses
|
||||
recorded contents and existing safety/locking. No remote import graph or
|
||||
producer-host filesystem search is introduced.
|
||||
4. Verify omitted session stable fields inherit the receiving campaign's values.
|
||||
Keep remote object identity evidence separate from checksums of locally
|
||||
materialized canonical bytes.
|
||||
|
||||
Use fake/in-memory storage for prepare → publish selection → remote load and
|
||||
relevant restore coverage. Fix actual producers/consumers rather than adding a
|
||||
parallel publication path. Existing atomic commit tests remain authoritative.
|
||||
|
||||
## Stage 8: Build native final-domain inspection snapshots [x]
|
||||
|
||||
Read `effective_projection.go`, `effective_digest.go`,
|
||||
`effective_pipeline.go`, pipeline finalization/family expansion, and native
|
||||
completion, defaults, rendering, and typed-secret contracts.
|
||||
|
||||
1. Keep one safe schema-specific final-domain DTO containing concrete expanded
|
||||
artifacts and generated publish outputs. Omit unexpanded family declarations
|
||||
and bookkeeping. Audit all credentials, including S3 fields and Audita/
|
||||
Notarius/Scriptorium lists: use exact reference names, with no resolved
|
||||
`Secret` fields in the inspection DTO.
|
||||
2. Build a native DTO result with `Load(NoSource(), Options{Definition:
|
||||
Definition{Defaults: ...}})`, supplying a fresh safe DTO and no external
|
||||
credential lookup or application callback rerun. ConfigKit copies and freezes
|
||||
these already resolved values. Using defaults avoids interpreting final
|
||||
`config`/`extends` data as fresh directives or converting precise numbers
|
||||
through a generic mapping-source reconstruction.
|
||||
3. Build/refresh the snapshot at existing domain-finalization boundaries, after
|
||||
applicable path/dependency/family resolution. Keep original authoring results
|
||||
separately. Synthetic default origins describe this derived snapshot, not
|
||||
original file provenance. No general snapshot service is needed.
|
||||
4. Route effective show and `pipeline.resolved.yml` rendering through native
|
||||
`WriteEffective`. Use the same DTO's deterministic JSON for Narratio's
|
||||
semantic digest. Preserve exact integers, domain null/empty values, and
|
||||
credential identities. Update the digest discriminator if its format changes.
|
||||
5. Remove flattening/serialization machinery used only to duplicate native
|
||||
show/diff. Keep narrow path enumeration and family/party context genuinely
|
||||
needed by catalog/source presentation and stage-owned reuse evidence.
|
||||
|
||||
Test final family/publish expansion, inherited/expanded equivalence, large
|
||||
integer variables, null/empty values, safe rendering, credential-reference
|
||||
changes versus rotation, and no credential resolution during snapshot creation.
|
||||
|
||||
## Stage 9: Adopt native diff and truthful operator source inspection [x]
|
||||
|
||||
Read `internal/app/config_diff.go`, config command loaders/tests,
|
||||
`artifact_catalog.go`, and native Diff/source/history contracts.
|
||||
|
||||
1. Fully resolve each overlay against the same selected campaign/party before
|
||||
comparing Stage 8 snapshots with `configkit.Diff`. Remove
|
||||
`diffEffectivePipelineValues`, its general map comparison, and consistency
|
||||
checks predicated on the old flattened-leaf change model.
|
||||
2. Present canonical JSON Pointer paths in native order. Derive added/removed/
|
||||
changed from snapshot `Present`, not nil or redaction text. Use compact JSON
|
||||
for safe values and expose `HiddenChanged` if present. Equal values produce
|
||||
`no differences`. Document pointer notation and native changed-region/list
|
||||
granularity instead of emulating the old engine.
|
||||
3. Source/catalog evidence comes from authoring results through public source
|
||||
lookup and metadata history/origins. Add clearly labeled empty optional-import
|
||||
selection evidence to `config sources`, without guessed candidates or
|
||||
reasons for absence.
|
||||
4. Preserve artifact/family/character IDs and generated party context. Distinguish
|
||||
normalization origins from retained definition evidence. Use native `main`
|
||||
where an upstream layer is reported; root/import presentation roles may remain
|
||||
where they express actual application context.
|
||||
|
||||
Test changed family members and publish rules, equivalent layouts, composite
|
||||
contributors, optional empty imports, atomic lists, and pointer escaping in
|
||||
allowed dynamic mapping keys. Inspection remains secret-safe and read-only,
|
||||
without source reparsing or provenance-based acceptance gates.
|
||||
|
||||
## Stage 10: Exercise native authoring and maintain ergonomic examples [x]
|
||||
|
||||
Read the production/testing bundle, artifact/family tests, native destinations
|
||||
and collection rules, and maintained example tests.
|
||||
|
||||
1. Convert artifact/family files to destination-based definitions. IDs remain
|
||||
explicit destination keys or keyed payloads; filenames do not imply IDs.
|
||||
2. Demonstrate a declared directory import with `pattern: "*.yml"`, essential
|
||||
`required: true` input, an optional selection, inheritance, and a useful
|
||||
testing-overlay list customization through `config.sequences`.
|
||||
3. Cover append/upsert/replace on an existing list using an appropriate authored
|
||||
identity, such as publish output `source`. Whole-record upsert and empty-list
|
||||
behavior follow ConfigKit. No application identity registration is required
|
||||
merely to enable authored collection controls.
|
||||
4. Demonstrate a split roster and local secondary imports using the sample
|
||||
campaign, with companion resource paths following source-origin policy.
|
||||
Explain that full-root and disjoint definitions remain accepted too.
|
||||
5. Use ordinary `Load` for this authored-import workflow. Layered loading is
|
||||
useful for actual lower-priority sources, not required for defaults alone.
|
||||
If such a source is genuinely needed, use `LoadLayered` and `MainRelative`
|
||||
directly with focused coverage instead of another merge/search mechanism.
|
||||
|
||||
Load examples through production definitions. Check dependent logical IDs,
|
||||
concrete family members, testing differences, and standalone prepared inputs.
|
||||
Keep examples synthetic and credential-free.
|
||||
|
||||
## Stage 11: Align canonical documentation and project-specific policy [x]
|
||||
|
||||
Read documentation/OSS/release policies, affected references, and the approved
|
||||
roadmap. Finish current-behavior documentation for the assembled implementation.
|
||||
|
||||
1. Update config/CLI references, operations/bootstrap guidance, troubleshooting,
|
||||
configuration internals, prepare internals, and party integration docs for the
|
||||
new discovery, import, resource-base, materialization, and inspection behavior.
|
||||
2. Replace obsolete import restrictions with an accurate native summary and
|
||||
pinned upstream links. Explain optional imports, `required`, filters,
|
||||
destinations, source-local root controls, list modes, and reserved-key scope.
|
||||
Keep effective domain validation Narratio-owned.
|
||||
3. Align project-specific architecture configuration/startup statements and the
|
||||
current raw-party-copy description in its actual owner. Preserve accepted
|
||||
ADR 0003 and generic adopted policies. Routine integration details do not
|
||||
require new ADRs.
|
||||
4. Write manual upgrade guidance for control names, native discovery filenames/
|
||||
precedence, explicit custom filenames, secrets override, relative path changes,
|
||||
required imports where desired, and list operations. Explain effective prepared
|
||||
documents versus authored roots; downstream migrations remain separate.
|
||||
5. Finish Go prerequisites, CI/runbook toolchain alignment, and attribution from
|
||||
Stage 1. Record impact for eventual major-release preparation without inventing
|
||||
a release version or changing published notes. Leave pending Audita/workspace
|
||||
roadmaps and Proposed ADRs intact.
|
||||
|
||||
Run doccheck and maintained example validation. Each volatile contract has one
|
||||
canonical owner. Retire completed roadmap files only when separately requested.
|
||||
|
||||
## Stage 12: Verify the assembled migration and close the sprint [x]
|
||||
|
||||
1. Extend a command-level temporary bundle with native discovery, explicit
|
||||
secrets override, imported campaign/party/session resources, destination-based
|
||||
artifacts, family expansion, and a testing overlay. Reuse preceding focused
|
||||
coverage instead of repeating ConfigKit's structural matrix.
|
||||
2. Exercise validate/show/sources/artifacts/diff through real composition. Check
|
||||
selected root, resources, generated values, optional-import evidence, safe
|
||||
credentials, and absence of runtime workspace creation by inspection. Exercise
|
||||
plan or prepare with fake external seams to establish execution parity.
|
||||
3. Trace remaining callers of old search, rebasing, raw-copy, and diff helpers.
|
||||
Remove dead code/test seams. Review intentional test expectation changes
|
||||
against the roadmap and preserve existing consequential domain/safety tests.
|
||||
4. With Go 1.27.4 and dependencies prepared, run `go test ./...`,
|
||||
`go test -race ./...`, `go vet ./...`, `go build ./...`,
|
||||
`go mod tidy -diff`, and `git diff --check`. Use the existing release asset
|
||||
builder with a temporary output directory and its existing `v0.0.0-test` label
|
||||
to check the three targets; this invokes no publication or new release note.
|
||||
5. Inspect `go list -m` for the exact release, absence of the old module path
|
||||
and filesystem replacements, and expected notices. Review every roadmap
|
||||
capability and all four approved decisions; native structural behavior must
|
||||
not be narrowed by new Narratio acceptance gates.
|
||||
6. Commit the finished unit and report checks, compatibility changes, revision,
|
||||
and any genuine unfinished work. Leave a clean workspace. Actual release
|
||||
publication and the next feature require separate authorization.
|
||||
|
||||
Complete when the native workflow works end to end, prepared inputs survive
|
||||
loss of authoring companions, operator inspection covers final domain values,
|
||||
and final validation passes.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None required for this sprint. The four product choices are approved. This plan
|
||||
selects concrete native snapshot, source-context, materialization, and toolchain
|
||||
approaches without adding compatibility shims, structural restrictions, or
|
||||
stronger runtime guarantees.
|
||||
@@ -1,459 +0,0 @@
|
||||
# ConfigKit adoption review remediation
|
||||
|
||||
## Review scope
|
||||
|
||||
Reviewed the twelve completed stages in [implementation.md](implementation.md)
|
||||
against [the approved target](configkit.md), the implementation diff from
|
||||
`65d0e51` through `39a958675d68e940799f4cb8c8e6748830d1483a`, current code,
|
||||
documentation, focused diagnostic reproductions, and repository checks.
|
||||
|
||||
An integration-focused follow-up reviewed the same implementation at
|
||||
`bccf03a99f45c2b9b342412b85f70e356663df61`, tracing each stage into existing
|
||||
command, preparation, publication, restore, and inspection workflows. Items
|
||||
5–6 were additional findings from that pass; items 1–4 were outstanding then.
|
||||
|
||||
An algorithm-focused pass reviewed all twelve stages at
|
||||
`583c5bdf1bc8f0af8ec2e257bcb03ecccbf283b1`. Items 7–10 are refactoring
|
||||
opportunities for consideration. Complexity follows the code and the selected
|
||||
ConfigKit v1.2.1 implementation; no operational timings or benchmarks were
|
||||
collected. The pass preserves algorithms expressly chosen by the plan.
|
||||
|
||||
All ten items below are now complete. The original findings are retained with
|
||||
completion annotations; final verification is recorded at the end of this file.
|
||||
|
||||
## 1. P2: Distinguish effective resource origins from superseded history
|
||||
|
||||
Status: Complete. Native pre-resolution Previous evidence supplies current
|
||||
contributors; regression coverage includes overlay replacement and inheritance.
|
||||
|
||||
Stage: 9, with documentation coverage in Stage 11.
|
||||
|
||||
Location: `internal/config/effective_pipeline.go:274–280`,
|
||||
`effectivePipelineOwners`.
|
||||
|
||||
For a resolution-origin field, the projection adds every main/overlay origin
|
||||
from native history to the same current source-role records. This conflates
|
||||
effective ownership with historical definitions. For example, a root containing
|
||||
`audita: {config_path: original.yml}` and an overlay containing
|
||||
`audita: {config_path: replacement.yml}` produce `audita.config_path` records
|
||||
for the root, overlay, and resolution. The root's superseded value did not
|
||||
supply the effective resource; its record is indistinguishable from an effective
|
||||
contributor. The error is visible to operators using `config sources` to locate
|
||||
the definition that controls an integration path.
|
||||
|
||||
Required change: use native effective/pre-normalization evidence for effective
|
||||
input attribution. If historical definitions are also displayed, label them
|
||||
separately. Preserve historical contributor information in the artifact catalog
|
||||
where it is explicitly definition evidence. Keep the projection thin; another
|
||||
provenance reconstruction service is unnecessary.
|
||||
|
||||
Verification: cover a normalized resource with a superseded root/import value,
|
||||
an overlay replacement, and inheritance. The effective overlay and resolution
|
||||
remain visible, while superseded evidence is either absent from current-owner
|
||||
records or explicitly identified as history. Update the command reference for
|
||||
the resulting roles.
|
||||
|
||||
## 2. P2: Remove the restore test's incidental startup deadline
|
||||
|
||||
Status: Complete. Cancellation is synchronized after discovery with the session
|
||||
lock held. Ten focused race repetitions and the full repository race suite pass.
|
||||
|
||||
Stage: 12; the fragile timing assumption predates this migration.
|
||||
|
||||
Location: `internal/app/restore_execution_test.go:383–387`,
|
||||
`TestExecuteRestoreLockConflictFailsAndWritesNothing`.
|
||||
|
||||
The test starts a 100 ms context deadline before calling the complete restore
|
||||
command, then requires an error from session-lock acquisition. With the current
|
||||
configuration workflow under the full race suite, the deadline can expire
|
||||
during earlier work. The reproduced failure was:
|
||||
|
||||
```text
|
||||
restore: remote check current commit pointer "dnd/campaigns/sample-campaign/sessions/2026-05-03/current/commit-pointer.json": context deadline exceeded, want lock failure
|
||||
```
|
||||
|
||||
An isolated application-suite run passed, while a subsequent full race run
|
||||
failed at this assertion. A green isolated run therefore does not establish
|
||||
Stage 12's repeatable final validation. The test implicitly requires unrelated
|
||||
startup work to finish within 100 ms, which is not an application requirement.
|
||||
This is not evidence that production session locking is incorrect.
|
||||
|
||||
Required change: make the contention/cancellation test target the intended
|
||||
lock boundary without timing all preceding configuration and remote discovery.
|
||||
Use synchronization at an appropriate existing test boundary, or separate
|
||||
lock-service contention coverage from command-level no-write coverage. A larger
|
||||
arbitrary startup timeout alone leaves the underlying timing dependence.
|
||||
Preserve the assertions that restore cannot write protected files while another
|
||||
invocation owns the session lock.
|
||||
|
||||
Verification: repeat the focused test and rerun the complete race suite under
|
||||
normal package concurrency. Report actual outcomes rather than treating an
|
||||
isolated pass as full-suite success.
|
||||
|
||||
## 3. P3: Finish the planned field-level publish resolution cleanup
|
||||
|
||||
Status: Complete. Field-only defaults and destinations retain precise native
|
||||
history. Publish and authored append/upsert/replace regression checks pass.
|
||||
|
||||
Stages: 1 and 3; the roadmap's integration scope requires field-level recording
|
||||
for field-only changes.
|
||||
|
||||
Location: `internal/config/pipeline_configkit.go:202–215,232–235`,
|
||||
`resolvePipelineConfig`.
|
||||
|
||||
Publish requiredness and derived destinations already use field-level native
|
||||
helpers. The resolver additionally tracks `defaultedOutput`, records the entire
|
||||
`publish.outputs` list for those field edits, and retains a comment claiming
|
||||
that child records cannot cover the changes. ConfigKit v1.2.1 supports exactly
|
||||
this field-level coverage. The ancestor record adds redundant history: querying
|
||||
an untouched output source also returns a whole-list normalization operation
|
||||
with publish-default/destination reasons.
|
||||
|
||||
Required change: remove the obsolete list-coverage tracking, redundant ancestor
|
||||
records, and inaccurate comment for these field-only edits. Keep field helpers
|
||||
and any whole-list records actually needed for structural membership/order
|
||||
changes. This is completion of the selected simplification, not a new ban on
|
||||
container-level records.
|
||||
|
||||
Verification: a temporary diagnostic overlay removing only the obsolete
|
||||
workarounds successfully loaded defaulted/derived publish outputs and passed
|
||||
the append/upsert/replace collection tests. Add suitable behavioral coverage
|
||||
for effective values and precise field history, without asserting private
|
||||
helper invocation counts.
|
||||
|
||||
## 4. P3: Pin Go attribution links to the selected source commit
|
||||
|
||||
Status: Complete. Go license, patent, and source links identify the verified
|
||||
Go 1.27.4 source commit; documentation checks pass.
|
||||
|
||||
Stages: 1 and 11.
|
||||
|
||||
Location: `docs/oss/attribution.md:125–128`.
|
||||
|
||||
The updated Go entry uses `go1.27.4` in the LICENSE, PATENTS, and Source URLs.
|
||||
The adopted OSS policy requires commit-specific upstream source and notice
|
||||
links. The preceding entry used a source hash; this update replaced it with
|
||||
tag-based references.
|
||||
|
||||
Required change: use the verified Go 1.27.4 source commit
|
||||
`7b2881024c95a11ddb86663a616340aa3a88b67f` in those three URLs. The hash was
|
||||
resolved through the official Go repository's release tag. Keep the existing
|
||||
license texts, factual notice wording, and Go version identification.
|
||||
|
||||
Verification: confirm the links identify that source revision and run the
|
||||
documentation checks. This requires no licensing change or new release gate.
|
||||
|
||||
## 5. P2: Give session initialization the correct authoring context
|
||||
|
||||
Status: Complete. Rendered templates load with native local context and imports;
|
||||
local and remote products are standalone effective documents. Relative CLI audio,
|
||||
import removal, placeholder/identity, and initialization race checks pass.
|
||||
|
||||
Stage: 4, interacting with materialization and command validation in Stages
|
||||
6, 11, and 12.
|
||||
|
||||
Locations: `internal/app/operator_session_init.go:78–102` and
|
||||
`internal/config/load.go`, `LoadSessionBytesWithOptions`.
|
||||
|
||||
The existing initializer validates its rendered document through
|
||||
`LoadSessionBytesWithOptions`. That loader now deliberately describes a
|
||||
contextless remote snapshot with imports disabled. A generated local session
|
||||
is an authored file, so this caller no longer uses the same semantics as the
|
||||
ordinary file loader.
|
||||
|
||||
Two focused reproductions show the mismatch:
|
||||
|
||||
- A local template with a required `shared.yml` import containing constant
|
||||
metadata fails with `import.disabled`, even when the import exists beside
|
||||
the generated destination and the resulting local session is otherwise valid.
|
||||
- With `--audio-dir ./audio` and an output in another directory, initialization
|
||||
validates the invocation-directory audio path, then writes the original
|
||||
relative text. Loading that generated file selects the output-directory
|
||||
audio path. The persisted resource differs from the validated resource and
|
||||
the documented invocation-relative CLI interpretation.
|
||||
|
||||
Required change: distinguish authored initialization from contextless snapshot
|
||||
loading. Use ConfigKit's supported source/context facilities for local
|
||||
composition, and make persisted resource references reproduce the validated
|
||||
configuration. Materialize effective values where moving or uploading the
|
||||
document would leave authoring imports unresolved. Preserve the narrow template
|
||||
substitution contract and the existing contextless remote-loading boundary.
|
||||
|
||||
Verification: initialize a local imported template, load its output through the
|
||||
ordinary loader, and check the imported values. Cover a destination outside the
|
||||
invocation directory and relative CLI audio input. Check that remote generation
|
||||
emits a consumable standalone snapshot and that ordinary placeholder and identity
|
||||
validation still applies. These cases exercise initialization through the real
|
||||
configuration owner without adding remote companion retrieval.
|
||||
|
||||
## 6. P2: Reconcile remote session updates with committed-session selection
|
||||
|
||||
Status: Complete. Authored remote input takes precedence for later commands;
|
||||
committed snapshots remain immutable fallback input and restore authority.
|
||||
Publication/update regression coverage checks changed title and audio selection.
|
||||
|
||||
Stage: 7, interacting with the existing initializer and shared command loader
|
||||
from Stages 2 and 4.
|
||||
|
||||
Locations: `internal/app/config_loader.go:194–223` and
|
||||
`internal/app/operator_session_init.go:113–138`.
|
||||
|
||||
The new remote loader selects a committed `run:prepare/inputs/session.yml`
|
||||
whenever the current commit declares one. The existing `session init --remote`
|
||||
writer continues to update the session-root `session.yml`. After publication,
|
||||
the writer and reader therefore use different authoritative objects.
|
||||
|
||||
A prepare → publish → `session init --remote --force --title "After publication"`
|
||||
reproduction succeeds and writes the requested title to the root object. A
|
||||
subsequent ordinary command load still returns the old title from the committed
|
||||
prepared session. The same selection would ignore changed audio or previous-session
|
||||
settings. The documented overwrite workflow silently stops affecting later runs
|
||||
once the session has been published.
|
||||
|
||||
Required change: reconcile remote input authoring and selection so successful
|
||||
operator updates can be used by subsequent runs. Preserve immutable published
|
||||
snapshots and committed restore authority. Keep an effective remote editing or
|
||||
override workflow; silently ignoring updates or introducing a blanket ban on
|
||||
editing published sessions would leave the integration unresolved. Update the
|
||||
canonical operator guidance for the resulting selection behavior.
|
||||
|
||||
Verification: publish a prepared session, update it through the supported remote
|
||||
workflow, then load it through the ordinary run/plan configuration path and check
|
||||
the updated values. Include audio or previous-session selection as well as title.
|
||||
Verify that restore still uses its selected immutable commit and that published
|
||||
objects are unchanged by the authoring update.
|
||||
|
||||
## 7. P3: Reuse enumerated source records and one inspection source index
|
||||
|
||||
Status: Complete. Source enumeration uses complete native records directly;
|
||||
catalog projection shares one index and retained family sources. Nested empty
|
||||
imports and existing composite/family inspection checks pass.
|
||||
|
||||
Stage: 9.
|
||||
|
||||
Locations: `internal/config/effective_pipeline.go:57–75`,
|
||||
`internal/config/pipeline_configkit.go:354–371`, and
|
||||
`internal/config/artifact_catalog.go`, `declarationDefinitionSources`.
|
||||
|
||||
`emptyImportSelections` enumerates `Result.Sources()` and then calls
|
||||
`Result.Source(id)` for every enumerated record. `pipelineSourcePaths` does the
|
||||
same lookup for every retained source. In ConfigKit v1.2.1, completed-result
|
||||
source lookup scans the source slice. Enumerating S sources this way performs
|
||||
Θ(S²) source-ID comparisons, although the enumerated records already contain
|
||||
the required information, including empty-import evidence.
|
||||
|
||||
The artifact catalog additionally rebuilds this source map for each declaration.
|
||||
For A entries this introduces O(A S²) source-index work; with one imported file
|
||||
per artifact, that component can become cubic in A. This is separate from the
|
||||
native history queries and necessary output construction.
|
||||
|
||||
Potential refactoring: consume the complete enumerated records directly. Build
|
||||
the source-ID/path map once per inspection operation and share it with declaration
|
||||
projection. This reduces empty-import enumeration to O(S + E), where E is the
|
||||
returned selection evidence, and source-index work across the catalog to
|
||||
expected O(S + A). Expanded members can also reuse the family-source lists
|
||||
already retained in the family catalog. Keep native history and origin queries
|
||||
as the provenance authority.
|
||||
|
||||
Verification: temporary functional checks confirmed that enumerated records and
|
||||
individual lookups have equal contents, and that direct enumeration preserves
|
||||
source paths and nested empty-import selections. Preserve operator source/catalog
|
||||
output, including composite and inherited contributors, when refactoring.
|
||||
|
||||
## 8. P3: Reuse party metadata and per-character provenance within inspection
|
||||
|
||||
Status: Complete. Each source/catalog projection shares one native metadata
|
||||
snapshot, one source index, and a per-character cache. Imported-party, composite
|
||||
provenance, family, and command agreement tests pass under the race detector.
|
||||
|
||||
Stage: 9, consuming Stage 5's imported party result.
|
||||
|
||||
Locations: `internal/config/artifact_catalog.go:67–105` and
|
||||
`internal/config/effective_pipeline.go`, `appendGeneratedArtifactSources`.
|
||||
|
||||
`partyCharacterSources` calls `result.Metadata()` separately for Lookup and
|
||||
History. ConfigKit v1.2.1 returns detached metadata by copying the complete
|
||||
effective-path and field maps, including their origin records. This helper is
|
||||
called for each generated artifact in the catalog and for each generated
|
||||
artifact field in source inspection. The same character's provenance is
|
||||
therefore repeatedly recomputed across fields and families.
|
||||
|
||||
Let M be the copied party-metadata size and P the number of provenance requests.
|
||||
The avoidable metadata-copy work and cumulative allocation are Θ(P M). Even
|
||||
with one family and bounded-size character entries, M and P both grow with the
|
||||
character count C, producing quadratic work. More families and fields multiply
|
||||
the requests. This describes the provenance pass, not the complexity of the
|
||||
whole command or unchanged party validation.
|
||||
|
||||
Potential refactoring: obtain one native Metadata snapshot and source index per
|
||||
inspection operation. Resolve each requested character's source list once and
|
||||
reuse it for that operation. The full-metadata copy term becomes O(M), with
|
||||
native per-character query costs and required output emission remaining. Keep
|
||||
the cache local to the projection; a new persistent provenance service is
|
||||
unnecessary. Apply the same snapshot reuse to campaign provenance helpers where
|
||||
convenient, although their fixed field count is not a quadratic case.
|
||||
|
||||
Verification: a temporary imported/inherited-roster check confirmed equivalent
|
||||
source lists using shared native metadata and an enumerated source index.
|
||||
Retain contributor accuracy, ordering, and secret-free inspection output.
|
||||
|
||||
## 9. P3: Read the selected remote session without enumerating run history
|
||||
|
||||
Status: Complete. Remote loading opens the selected object once, verifies its
|
||||
returned identity, and streams into a command-owned temporary file. Remote and
|
||||
publication round trips pass under race; a rejecting-list fake checks enumeration
|
||||
is unnecessary, and cleanup remains idempotent.
|
||||
|
||||
Stage: 7.
|
||||
|
||||
Location: `internal/app/config_loader.go:194–228`, `findRemoteSessionConfig`.
|
||||
|
||||
After selecting the committed prepared-session key, this function lists the
|
||||
entire session prefix and scans it for that exact key. If H objects are retained
|
||||
across historical runs, this adds Θ(H) listing/scan work and Θ(H) object-record
|
||||
memory to every remote session load. The production S3 adapter paginates and
|
||||
accumulates the complete listing. The scan predates the migration and was
|
||||
retained in the new committed-object selection path.
|
||||
|
||||
Potential refactoring: retrieve the exact selected key through the existing
|
||||
`ObjectStore.Read` boundary, which returns metadata with the opened body. Use
|
||||
that metadata for the existing size/generation checks and stream the body into
|
||||
the command-owned temporary file while retaining checksum verification. The
|
||||
object lookup/transfer then takes a constant number of requests independent of
|
||||
H, plus the necessary O(B) work for B session bytes. Current-commit acquisition
|
||||
and validation costs remain. Apply this after reconciling the selection policy
|
||||
in item 6; no new backend interface is needed.
|
||||
|
||||
Verification: the production S3 Read implementation performs one exact-key
|
||||
GetObject and returns its size and generation. Preserve fake-store coverage for
|
||||
missing objects, selected identity, checksum/size/generation mismatches, and
|
||||
temporary-file cleanup. Confirm that unrelated historical keys are not enumerated.
|
||||
|
||||
## 10. P3: Stream prepared-input archival copies
|
||||
|
||||
Status: Complete. Verified descriptors stream through the existing atomic-reader
|
||||
copy helper, with unchanged-destination reuse. Archive bytes, checksums, modes,
|
||||
and tamper rejection are covered; full ordinary and race suites pass.
|
||||
|
||||
Stage: 7.
|
||||
|
||||
Location: `internal/stage/publish_sources.go:29–64`,
|
||||
`collectPublishPreparedInputs`.
|
||||
|
||||
After opening and verifying each prepared document, the new archival path reads
|
||||
the whole file with `io.ReadAll` and passes the byte slice to
|
||||
`writeBytesIfChanged`. For a document of B bytes this requires Θ(B) auxiliary
|
||||
transfer memory. Expanded imported party documents can be larger than any one
|
||||
authoring fragment. A full in-memory document is unnecessary for this file copy.
|
||||
|
||||
Potential refactoring: preserve the current confined source opening and checksum
|
||||
checks, then stream the opened descriptor into the run archive. The existing
|
||||
`fileops.WriteReaderAtomicWithChecksum` already supplies reader-based atomic
|
||||
replacement and hashing through `io.Copy`. Preserve unchanged-destination reuse
|
||||
using the known checksum. Transfer time remains Θ(B), while auxiliary copy
|
||||
memory becomes O(1) with a fixed-size buffer. This does not alter the plan's
|
||||
separate canonical YAML rendering algorithm or total in-memory configuration size.
|
||||
|
||||
Verification: preserve archived bytes/checksums, workspace file modes, source
|
||||
validation, failure handling, and immutable publication behavior. Existing
|
||||
reader-copy and prepared-session publication tests provide suitable behavioral
|
||||
coverage; allocation or wall-clock thresholds are unnecessary.
|
||||
|
||||
## Stage-by-stage disposition at the initial reviews
|
||||
|
||||
| Stage | Review result |
|
||||
| --- | --- |
|
||||
| 1: Dependency/toolchain | ConfigKit v1.2.1 and Go 1.27.0/1.27.4 selected; remaining notice pinning in item 4. |
|
||||
| 2: Discovery/bootstrap | Native discovery, exclusive explicit selection, independent bootstrap, and shared diff bootstrap are present. |
|
||||
| 3: Resource bases | Native origin-based resolution and fallbacks are present; field-record cleanup remains in item 3. |
|
||||
| 4: Campaign/session imports | File contexts, contextless remote snapshots, native optional discovery, and detached document values are present; initialization still uses the snapshot path in item 5. |
|
||||
| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present; the planned roster sort is retained. |
|
||||
| 6: Prepared inputs/resume | Canonical document materialization and matching resume evidence are present. |
|
||||
| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; item 6 is a correctness gap and items 9–10 are scalability opportunities. |
|
||||
| 8: Final-domain snapshots | Defaults-backed native snapshots, safe credential identities, rendering, and digest integration are present. |
|
||||
| 9: Diff/provenance | Native diff and pointer paths are present; item 1 concerns attribution correctness and items 7–8 concern avoidable provenance work. |
|
||||
| 10: Examples | Destination-based definitions, directory imports, list operations, split roster, and secondary imports are demonstrated. |
|
||||
| 11: Documentation/policy | Current references cover the migration; finish attribution, source-role, and initialization/selection corrections above. |
|
||||
| 12: Final verification | Existing integration checks pass; add coverage for items 5–6. Full race validation remains unreliable because of item 2. |
|
||||
|
||||
The approved discovery, bootstrap, relative-resource, and secondary-import
|
||||
changes are deliberate contract changes. Items 5–6 describe implementation
|
||||
inconsistencies in existing workflows. Item 2 concerns an incidental test-time
|
||||
performance assumption, not a requested startup guarantee.
|
||||
|
||||
## Initial review validation
|
||||
|
||||
Using Go 1.27.4:
|
||||
|
||||
- `go test ./...` passed.
|
||||
- `go test -race ./...` failed at item 2. The isolated application race suite
|
||||
passed, as did five subsequent repetitions of the focused restore test.
|
||||
- `go vet ./...`, `go build ./...`, and `go mod tidy -diff` passed.
|
||||
- Release-asset cross-builds passed for Linux amd64, Linux arm64, and macOS
|
||||
arm64 with the existing `v0.0.0-test` label and temporary output directory.
|
||||
- Focused diagnostic reproductions confirmed items 1 and 3. Temporary Go
|
||||
overlays kept those diagnostics and proposed-code experiments outside the
|
||||
repository.
|
||||
- Documentation checks and `git diff --check` passed.
|
||||
|
||||
Review execution made no application changes and invoked no live subprocess
|
||||
services or release publication.
|
||||
|
||||
## Integration follow-up validation
|
||||
|
||||
Using Go 1.27.4 at the follow-up revision:
|
||||
|
||||
- `go test ./...` passed with local test-server sockets permitted. The sandboxed
|
||||
attempt was blocked by `httptest` socket restrictions.
|
||||
- Focused migration checks across config, stage, and app packages passed both
|
||||
ordinarily and under `-race`. These exercised native discovery/inspection,
|
||||
imported inputs, canonical materialization, published-session round trips,
|
||||
initialization, and command/execution agreement.
|
||||
- Temporary overlay diagnostics reproduced both aspects of item 5 and the
|
||||
publication/update conflict in item 6, ordinarily and under `-race`.
|
||||
- The full race suite was not repeated in this pass; item 2 retains the initial
|
||||
review's observed failure. Application code is unchanged.
|
||||
|
||||
Only this remediation document changed. Diagnostic files remained outside the
|
||||
repository, and object storage used an in-memory fake.
|
||||
|
||||
## Algorithm-focused review coverage and validation
|
||||
|
||||
The stage-by-stage pass covered dependency/API substitutions (1), native
|
||||
discovery/bootstrap (2), resource-base queries (3), secondary loading and
|
||||
materialization data flow (4), roster ordering/rendering (5), prepare/resume
|
||||
serialization (6), archive/remote/restore paths (7), finalized snapshots (8),
|
||||
diff/provenance projection (9), examples (10), documentation (11), and assembled
|
||||
verification (12).
|
||||
|
||||
The chosen roster ordering, defaults-backed snapshot construction, native Diff,
|
||||
and required content validation remain accepted. Canonical rendering is linear
|
||||
in emitted data; the planned roster comparison sort is O(C log C) after deriving
|
||||
ordering keys. Unchanged algorithms such as pairwise party-name validation and
|
||||
prepare-input matching are not new findings from this migration-focused pass.
|
||||
|
||||
Cost analysis followed the released module's public API wrappers through native
|
||||
source lookup and metadata-copy implementations. Temporary functional Go-overlay
|
||||
checks validated source-enumeration and shared-metadata equivalence, ordinarily
|
||||
and under `-race`. Focused configuration, file-reader copy, S3 opened-object
|
||||
metadata, native inspection, and prepared-session publication checks passed on
|
||||
Go 1.27.4. No performance data, timing assertions, benchmark gates, or application
|
||||
changes were introduced.
|
||||
|
||||
## Completed remediation validation
|
||||
|
||||
Each issue was addressed in order and committed with its completion marker.
|
||||
Using Go 1.27.4 on the completed implementation:
|
||||
|
||||
- `go test ./...` passed.
|
||||
- `go test -race ./...` passed, including the corrected restore contention test.
|
||||
- `go vet ./...`, `go build ./...`, and `go mod tidy -diff` passed.
|
||||
- Focused regressions cover effective provenance, field-level publish history,
|
||||
local/imported initialization, standalone remote input, remote authoring after
|
||||
publication, enumeration-free loading, and verified streaming archives.
|
||||
- Documentation checks and `git diff --check` passed.
|
||||
|
||||
Remote session authoring updates affect future command inputs. Committed
|
||||
publication objects remain immutable and restore continues to validate its
|
||||
pointer-selected remote state. Inspection caches are scoped to one operation;
|
||||
ConfigKit remains the authority for configuration and provenance.
|
||||
Reference in New Issue
Block a user