316 lines
18 KiB
Markdown
316 lines
18 KiB
Markdown
# Adopt ConfigKit v1.2.1 and native configuration ergonomics
|
|
|
|
## Purpose
|
|
|
|
Adopt current ConfigKit as Narratio's configuration owner and make its native
|
|
authoring and inspection features useful to operators. Align configuration
|
|
semantics with other ConfigKit consumers. Existing Narratio assembly conventions
|
|
are migration inputs rather than compatibility requirements for this work.
|
|
|
|
This roadmap defines the target, user intent, and required changes. The
|
|
[implementation plan](implementation.md) defines the work needed to reach that
|
|
target. The earlier [v0.5.0 roadmap](config.md) records the completed adoption
|
|
that forms the current baseline.
|
|
|
|
The ownership boundary in [ADR 0003](../adr/0003-assign-configuration-ownership.md)
|
|
continues to apply: ConfigKit resolves configuration before its final handoff;
|
|
Narratio owns domain validation, application derivation, and runtime use.
|
|
Supported callbacks and native metadata provide the integration boundary.
|
|
|
|
## Reviewed baseline and releases
|
|
|
|
Narratio currently selects `gitea.maximumdirect.net/eric/ConfigKit v0.5.0`,
|
|
commit `dc09959cca4e105f49ec316c6d4ba585bd2340bd`, with Go 1.26.0 as its
|
|
consumer minimum. Release builds currently select Go 1.26.7.
|
|
|
|
The adoption target is `go.maximumdirect.net/configkit v1.2.1`, commit
|
|
`51ef3144158db1a09fabdbd1c7ae59908c84923b`. The published tag and the sibling
|
|
checkout agree on that revision. ConfigKit requires Go 1.27.0 and uses
|
|
`go.yaml.in/yaml/v4 v4.0.0`. Its code license remains LGPL-3.0-or-later.
|
|
|
|
The pinned [configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md)
|
|
and [consumer guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md)
|
|
own upstream semantics. The intervening releases supply the following relevant
|
|
changes:
|
|
|
|
| Release | Adoption relevance |
|
|
| --- | --- |
|
|
| v0.6.0 | Loading and metadata optimizations; Go minimum moves to 1.26.0. |
|
|
| v0.7.0 | Native safe effective-configuration `Diff`. |
|
|
| v0.7.1 | Maintainer-tool update; runtime behavior unchanged. |
|
|
| v0.8.0 | Input/import destinations, sequence-item placement, shared source-version checks, and initial identified-sequence composition. |
|
|
| v0.9.0 | Directory and expanded import selection; optional-by-default imports; identified-list replace/append/upsert and inheritance; CLI assignments move after decoding and inheritance. Its final collection APIs replace the initial v0.8 registrations. |
|
|
| v0.10.0 | Native ordered configuration discovery through `SearchConfig`. |
|
|
| v0.10.1 | Loading consolidation and import bookkeeping improvements. |
|
|
| v0.10.2 | Correctness fixes for decoding, callback/resource lifetime, discovery, inspection budgets, and collection operations. |
|
|
| v1.0.0 | Vanity module path, source-root `config` controls, public API cleanup, source lookup, provenance and sensitivity fixes, and Go 1.27.0 minimum. |
|
|
| v1.1.0 | Field-level list resolution records and identity-aware inspection across list movement. |
|
|
| v1.2.0 | Main-relative companion inputs and overlay context for layered loading; overlay filesystem fixes. |
|
|
| v1.2.1 | Stable go-yaml v4.0.0 dependency; maintainer release-tool update. |
|
|
|
|
The corresponding `docs/releases/<version>.md` files at the pinned revision
|
|
provide the migration details. Select v1.2.1 directly; intermediate releases
|
|
provide context rather than required deployment stops.
|
|
|
|
## Current Narratio integration
|
|
|
|
The [configuration reference](../config.md) and
|
|
[configuration internals](../internal/configuration.md) describe current behavior.
|
|
|
|
- Commands select a pipeline path through Narratio's search loop, bootstrap
|
|
credentials from an explicit directory or a sibling `secrets/` directory,
|
|
and then call ConfigKit.
|
|
- Pipeline loading enables explicit path/glob imports and one selected overlay.
|
|
Campaign, session, and canonical-party documents use separate loads. Session
|
|
loading also supports command-scoped downloaded input snapshots.
|
|
- Typed defaults, recorded normalization, domain validation, and native
|
|
provenance are integrated. Additional path/dependency resolution and
|
|
campaign-owned family expansion occur after the pipeline load.
|
|
- Artifact files currently repeat enclosing `scriptorium.artifacts` mappings.
|
|
Logical IDs come from map keys. Families expand against canonical party data.
|
|
- Show, diff, and digest use a safe finalized-domain projection. Narratio owns
|
|
the current mapping-diff algorithm. Native load metadata supplies source and
|
|
artifact-catalog evidence.
|
|
|
|
Some public import descriptions still state restrictions from older ConfigKit
|
|
behavior. Replace these with current native semantics during adoption rather
|
|
than recreating them as Narratio checks.
|
|
|
|
## Targeted end state
|
|
|
|
### Native assembly and operator choice
|
|
|
|
ConfigKit owns source discovery, acquisition, composition, inheritance, presence,
|
|
typed secrets, provenance, and safe native inspection. Narratio supplies its
|
|
schema, domain defaults and validation, supported application resolution, and
|
|
operator-selected inputs.
|
|
|
|
Use ordinary `Load` for an ordinary configuration. Use public native facilities
|
|
where they serve the operator workflow. Layered loading is appropriate when
|
|
actual lower-priority inputs are needed; typed defaults remain appropriate for
|
|
simple defaults. Main-relative companion inputs allow such underlays to follow
|
|
a discovered main without Narratio searching for it again.
|
|
|
|
Expose the library's authored `config.imports` and `config.sequences` semantics
|
|
where applicable. Imports use ConfigKit defaults, including optional absence,
|
|
unless the operator explicitly requests other supported behavior. An invalid
|
|
selected source still fails. Operators mark essential imports `required: true`.
|
|
Directory imports can select YAML with an explicit `pattern`; unrestricted
|
|
filenames remain the library default.
|
|
|
|
Native destinations, nested imports, discovery filters, source-local relative
|
|
import paths, collection operations, and inheritance work under ConfigKit's
|
|
rules. Narratio validates the final effective domain entries. It adds no
|
|
fragment-count rule, mandatory fragment version, import-mode allowlist, or
|
|
parallel composition engine.
|
|
|
|
### Ergonomic artifact files and list customization
|
|
|
|
Support an artifact definition without repeated enclosing mappings through an
|
|
explicit import destination. For example:
|
|
|
|
```yaml
|
|
config:
|
|
imports:
|
|
- path: artifacts/session-recap.yml
|
|
destination: [scriptorium, artifacts, session_recap]
|
|
required: true
|
|
```
|
|
|
|
The imported file contains the artifact fields such as `prompt_id`, `inputs`,
|
|
and `output_path`. The literal destination defines the reusable key
|
|
`session_recap`, with logical source `narratio.artifact.session_recap`.
|
|
The filename is an organizational choice. Keyed files mounted at
|
|
`[scriptorium, artifacts]` support declared directory or glob imports when many
|
|
definitions should be discovered together. Apply the same pattern to families.
|
|
|
|
Keep keyed artifact/family mappings and campaign-owned character data. They
|
|
already fit native inheritance and stable identity. Full-root and disjoint
|
|
definitions remain valid native inputs; a one-artifact-per-file organization is
|
|
guidance rather than a validity requirement.
|
|
|
|
Allow authored native collection declarations for existing list fields where
|
|
their final shapes satisfy Narratio's schema. Operators can choose replacement,
|
|
append, or upsert and identified-record inheritance. Explain that upsert
|
|
replaces whole matching records and that an empty append/upsert contribution
|
|
does not clear a list. Register application-owned sequence identities only
|
|
where Narratio has a real domain identity requirement; generic library
|
|
capabilities do not establish new uniqueness requirements for every list.
|
|
|
|
### Discovery, credentials, and resource paths
|
|
|
|
Replace Narratio-owned local candidate probing with ConfigKit search descriptions
|
|
and `SearchConfig`. Use the native `WorkingDirectory`, `UserConfigDirectory`,
|
|
and `UnixSystemDirectories` presets in that order with application name
|
|
`narratio`. Use their documented filenames. Explicit `--config` selection
|
|
remains an exclusive `File` input. Report the source actually selected by the
|
|
library. This provides convenient development-local configuration and standard
|
|
ConfigKit naming. Existing `pipeline.yml` files remain explicitly selectable;
|
|
migration guidance explains the new discovery names and precedence.
|
|
|
|
Bootstrap uses `LoadSecretsIntoEnvironment` once per invocation before typed
|
|
credential resolution and concurrent work. Existing environment variables win,
|
|
including explicitly empty variables. Use the invocation-owned conventional
|
|
directory `/usr/local/etc/narratio/secrets`, with `--secrets-dir` selecting
|
|
another directory. A missing conventional directory remains optional; an
|
|
explicit selection follows the existing explicit-selection behavior. Retain
|
|
safe partial-report handling and exact-name forwarding to subprocesses.
|
|
|
|
The independent directory lets bootstrap precede ConfigKit's first and only
|
|
configuration selection. Service environments can provide credentials directly;
|
|
development invocations can choose a project-local secrets directory explicitly.
|
|
The implicit sibling `secrets/` convention retires with this cutover.
|
|
|
|
Resolve relative file-resource references from their effective defining source's
|
|
ConfigKit resource context. Inherited values follow their effective origin;
|
|
CLI-relative paths use the invocation directory. Application-derived paths
|
|
record their dependencies. Use the main source as a documented fallback for
|
|
origins without a distinct usable resource base. This makes imported definitions
|
|
portable across bundles.
|
|
|
|
Review actual path-bearing fields explicitly. Artifact destinations, command
|
|
names, session-relative products, and remote object keys retain their domain
|
|
meanings. This policy introduces no new target-existence or root-containment
|
|
checks. ConfigKit's consumer examples illustrate application policy; Narratio
|
|
owns the selected interpretation.
|
|
|
|
### Imports in secondary documents
|
|
|
|
Enable native explicit imports for locally sourced campaign, party, and session
|
|
documents as well as pipelines. This supports split rosters and shared metadata
|
|
with consistent library semantics. Campaign selection, party schema validation,
|
|
and session domain rules remain application-owned. Pipeline overlay selection
|
|
remains associated with the pipeline.
|
|
|
|
Materialize effective documents where raw root-file copying would leave
|
|
unresolved imports in downstream inputs or published session data. Remote
|
|
session snapshots retain their explicit source context; resolving imports
|
|
against an incidental download directory would give them an incorrect base.
|
|
Remote companion-file retrieval remains separate work.
|
|
|
|
### Final-domain inspection through native APIs
|
|
|
|
Keep validate, show, sources, artifact catalog, and diff useful for the operator's
|
|
fully resolved pipeline, including campaign-driven members and publish rules.
|
|
Use native effective rendering and `Diff` where they own the operation, with
|
|
Narratio formatting and generated-domain context at the presentation boundary.
|
|
|
|
A raw pipeline `Result` is a frozen load snapshot. Mutating its `Config` later
|
|
does not update rendering or diff. Integrate this distinction explicitly:
|
|
record domain resolution within a supported resolver when its inputs are
|
|
available there; where later campaign resolution remains appropriate, create
|
|
a completed native inspection result from the finalized safe domain projection
|
|
using ConfigKit's public loading APIs. That result describes application-derived
|
|
values; retained authoring results supply actual file provenance.
|
|
|
|
Use native comparison rather than maintaining a second general mapping-diff
|
|
engine. Preserve final-domain coverage and credential identities. Narratio's
|
|
digest and stage-owned reuse evidence remain application responsibilities.
|
|
Source-layout changes alone do not change equivalent domain values. Source
|
|
inspection can report composite origins, empty optional-import selections, and
|
|
application derivation without inventing a unique defining-file contract.
|
|
|
|
### Scope and compatibility
|
|
|
|
This is a clean native cutover. Update source-root `configkit:` controls to
|
|
`config:` and remove old-module/API references in maintained code and fixtures.
|
|
Nested `config` data remains ordinary data under ConfigKit's source-root rules.
|
|
The `configkit` package name and `configkit:"secret"` Go tag retain their
|
|
upstream spelling. Mapping and identified-list `extends` follow current native
|
|
rules.
|
|
|
|
This roadmap deliberately permits revised configuration semantics. Update
|
|
tests and documentation for selected changes rather than preserving historical
|
|
acceptance rules through compatibility loaders or keyword aliases.
|
|
|
|
Application identity, artifact dependency meaning, runtime output destinations,
|
|
campaign-owned party authority, remote session retrieval, and stage reuse remain
|
|
Narratio domain concerns. This scope does not redesign the runtime workspace,
|
|
change subprocess adapters, adopt ReleaseKit for Narratio, or add external
|
|
provider probes. Audita and workspace work retain their separate roadmaps.
|
|
|
|
## Required changes
|
|
|
|
### Dependency, toolchain, and notices
|
|
|
|
- Select the released `go.maximumdirect.net/configkit v1.2.1` module and migrate
|
|
all imports and exchanged named types together. Remove the old requirement;
|
|
use the released dependency rather than a filesystem replacement.
|
|
- Raise `go.mod` to Go 1.27.0 and select an available official Go 1.27 patch for
|
|
development/release validation and release builds. Align CI, runbook setup,
|
|
and incorporated Go attribution with that selection.
|
|
- Update ConfigKit source/license links to the reviewed commit and YAML v4
|
|
attribution to the selected stable release. Check the actual resulting module
|
|
graph. Maintainer tool dependencies are not automatically Narratio runtime
|
|
components. Narratio's separate direct YAML use need not be rewritten merely
|
|
because ConfigKit changed its internal parser dependency.
|
|
|
|
### Integration and simplification
|
|
|
|
- Apply the v1 API replacements actually used here, including
|
|
`LoadSecretsIntoEnvironment`, `LayerMain`, `RecordDerivedDefault`, and
|
|
`RecordNormalization`. Review other public names against the migration guide.
|
|
- Use keyed expanded options and definition literals. Keep metadata projections
|
|
compatible with current source evidence, native layer names, and safe errors.
|
|
- Replace broad list-record workarounds with supported field-level recording
|
|
when the application changes only fields. Record whole-list changes where
|
|
order or membership changes warrant them.
|
|
- Adopt native discovery, destination-based artifact authoring, authored
|
|
collection operations, and native final-domain rendering/comparison. Remove
|
|
replaced search, comparison, and redundant metadata machinery.
|
|
- Apply the agreed source-relative resource bases and secondary-document imports.
|
|
Share definitions and loading context between execution and inspection so an
|
|
operator validates the configuration the pipeline will actually consume.
|
|
- Review secondary-document input materialization and publication as well as
|
|
parsing. A copied root file can retain imports whose
|
|
companions are absent at its new location. Materialize the effective domain
|
|
document where a subprocess or restored session consumes it, retaining native
|
|
authoring evidence separately. Review placeholder checks against the effective
|
|
session input and preserve required external party/session formats.
|
|
|
|
### Documentation and maintained examples
|
|
|
|
Update the configuration reference, CLI reference, operations/bootstrap guidance,
|
|
configuration internals, contributor prerequisites, and release procedure for
|
|
the implemented semantics. Keep canonical facts in their respective owners.
|
|
Update project-specific architecture wording where selected path/bootstrap
|
|
decisions change it; preserve ADR 0003's ownership boundary.
|
|
|
|
Migrate the production/testing bundle to `config:`. Demonstrate destination-based
|
|
artifact and family files, a native directory import restricted to YAML, an
|
|
essential required import, native inheritance, and a useful selected-overlay
|
|
list customization. Keep illustrative optional choices labeled as choices.
|
|
Show how the declared artifact key is referenced by dependent artifacts.
|
|
|
|
Provide concise manual upgrade instructions for existing Narratio configurations:
|
|
control-key/API changes, optional import defaults, collection behavior, source
|
|
discovery, and selected path/bootstrap changes. Separately installed applications
|
|
own their configuration versions; Narratio's migration does not automatically
|
|
rewrite their files.
|
|
|
|
The eventual Narratio release requires a major version because existing valid
|
|
configuration control blocks change. The Go minimum increase alone is permitted
|
|
in a minor release under the adopted release policy. This roadmap does not
|
|
authorize tagging, publication, or retrospective release-note changes.
|
|
|
|
### Verification
|
|
|
|
Use temporary inputs and controlled synthetic credentials. Exercise native
|
|
discovery and the chosen bootstrap convention together, explicit file selection,
|
|
selected-source errors, optional and required imports, import destinations,
|
|
inherited artifact identities, native list customization, and final domain
|
|
validation. Add coverage at Narratio boundaries rather than duplicating
|
|
ConfigKit's complete structural test matrix.
|
|
|
|
Verify inherited presence/defaulting, party order, final campaign/family
|
|
expansion, resource paths, source attribution, and secret-safe inspection.
|
|
Diff/show must include derived members and publish rules; compare equal effective
|
|
configurations across different source arrangements. Credential rotation behind
|
|
the same reference name must remain absent from semantic diffs and digests.
|
|
|
|
Update tests that encode intentionally retired syntax or policies. Run focused
|
|
configuration/command/example/documentation checks and the established full
|
|
test, race, vet, build, and module checks on the adopted Go line. Preserve the
|
|
current supported build targets. No new live-service or CI-completion gate is
|
|
introduced by this migration.
|