2026-11-17 20:23:22 +00:00
# Adopt ConfigKit v1.2.1 and native configuration ergonomics
## Purpose
Adopt current ConfigKit as Narratio's configuration owner and make its native
authoring and inspection features useful to operators. Align configuration
semantics with other ConfigKit consumers. Existing Narratio assembly conventions
are migration inputs rather than compatibility requirements for this work.
This roadmap defines the target and required changes. A subsequent implementation
2026-11-17 20:27:47 +00:00
plan will define staged work for the agreed target. This pass adds documentation
only. The earlier [v0.5.0 roadmap ](config.md ) and
2026-11-17 20:23:22 +00:00
[implementation plan ](implementation.md ) describe the completed baseline.
The ownership boundary in [ADR 0003 ](../adr/0003-assign-configuration-ownership.md )
continues to apply: ConfigKit resolves configuration before its final handoff;
Narratio owns domain validation, application derivation, and runtime use.
Supported callbacks and native metadata provide the integration boundary.
## Reviewed baseline and releases
Narratio currently selects `gitea.maximumdirect.net/eric/ConfigKit v0.5.0` ,
commit `dc09959cca4e105f49ec316c6d4ba585bd2340bd` , with Go 1.26.0 as its
consumer minimum. Release builds currently select Go 1.26.7.
The adoption target is `go.maximumdirect.net/configkit v1.2.1` , commit
`51ef3144158db1a09fabdbd1c7ae59908c84923b` . The published tag and the sibling
checkout agree on that revision. ConfigKit requires Go 1.27.0 and uses
`go.yaml.in/yaml/v4 v4.0.0` . Its code license remains LGPL-3.0-or-later.
The pinned [configuration contract ](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md )
and [consumer guide ](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md )
own upstream semantics. The intervening releases supply the following relevant
changes:
| Release | Adoption relevance |
| --- | --- |
| v0.6.0 | Loading and metadata optimizations; Go minimum moves to 1.26.0. |
| v0.7.0 | Native safe effective-configuration `Diff` . |
| v0.7.1 | Maintainer-tool update; runtime behavior unchanged. |
| v0.8.0 | Input/import destinations, sequence-item placement, shared source-version checks, and initial identified-sequence composition. |
| v0.9.0 | Directory and expanded import selection; optional-by-default imports; identified-list replace/append/upsert and inheritance; CLI assignments move after decoding and inheritance. Its final collection APIs replace the initial v0.8 registrations. |
| v0.10.0 | Native ordered configuration discovery through `SearchConfig` . |
| v0.10.1 | Loading consolidation and import bookkeeping improvements. |
| v0.10.2 | Correctness fixes for decoding, callback/resource lifetime, discovery, inspection budgets, and collection operations. |
| v1.0.0 | Vanity module path, source-root `config` controls, public API cleanup, source lookup, provenance and sensitivity fixes, and Go 1.27.0 minimum. |
| v1.1.0 | Field-level list resolution records and identity-aware inspection across list movement. |
| v1.2.0 | Main-relative companion inputs and overlay context for layered loading; overlay filesystem fixes. |
| v1.2.1 | Stable go-yaml v4.0.0 dependency; maintainer release-tool update. |
The corresponding `docs/releases/<version>.md` files at the pinned revision
provide the migration details. Select v1.2.1 directly; intermediate releases
provide context rather than required deployment stops.
## Current Narratio integration
The [configuration reference ](../config.md ) and
[configuration internals ](../internal/configuration.md ) describe current behavior.
- Commands select a pipeline path through Narratio's search loop, bootstrap
credentials from an explicit directory or a sibling `secrets/` directory,
and then call ConfigKit.
- Pipeline loading enables explicit path/glob imports and one selected overlay.
Campaign, session, and canonical-party documents use separate loads. Session
loading also supports command-scoped downloaded input snapshots.
- Typed defaults, recorded normalization, domain validation, and native
provenance are integrated. Additional path/dependency resolution and
campaign-owned family expansion occur after the pipeline load.
- Artifact files currently repeat enclosing `scriptorium.artifacts` mappings.
Logical IDs come from map keys. Families expand against canonical party data.
- Show, diff, and digest use a safe finalized-domain projection. Narratio owns
the current mapping-diff algorithm. Native load metadata supplies source and
artifact-catalog evidence.
Some public import descriptions still state restrictions from older ConfigKit
behavior. Replace these with current native semantics during adoption rather
than recreating them as Narratio checks.
## Targeted end state
### Native assembly and operator choice
ConfigKit owns source discovery, acquisition, composition, inheritance, presence,
typed secrets, provenance, and safe native inspection. Narratio supplies its
schema, domain defaults and validation, supported application resolution, and
operator-selected inputs.
Use ordinary `Load` for an ordinary configuration. Use public native facilities
where they serve the operator workflow. Layered loading is appropriate when
actual lower-priority inputs are needed; typed defaults remain appropriate for
simple defaults. Main-relative companion inputs allow such underlays to follow
a discovered main without Narratio searching for it again.
Expose the library's authored `config.imports` and `config.sequences` semantics
where applicable. Imports use ConfigKit defaults, including optional absence,
unless the operator explicitly requests other supported behavior. An invalid
selected source still fails. Operators mark essential imports `required: true` .
Directory imports can select YAML with an explicit `pattern` ; unrestricted
filenames remain the library default.
Native destinations, nested imports, discovery filters, source-local relative
import paths, collection operations, and inheritance work under ConfigKit's
rules. Narratio validates the final effective domain entries. It adds no
fragment-count rule, mandatory fragment version, import-mode allowlist, or
parallel composition engine.
### Ergonomic artifact files and list customization
Support an artifact definition without repeated enclosing mappings through an
explicit import destination. For example:
```yaml
config:
imports:
- path: artifacts/session-recap.yml
destination: [scriptorium, artifacts, session_recap]
required: true
```
The imported file contains the artifact fields such as `prompt_id` , `inputs` ,
and `output_path` . The literal destination defines the reusable key
`session_recap` , with logical source `narratio.artifact.session_recap` .
The filename is an organizational choice. Keyed files mounted at
`[scriptorium, artifacts]` support declared directory or glob imports when many
definitions should be discovered together. Apply the same pattern to families.
Keep keyed artifact/family mappings and campaign-owned character data. They
already fit native inheritance and stable identity. Full-root and disjoint
definitions remain valid native inputs; a one-artifact-per-file organization is
guidance rather than a validity requirement.
Allow authored native collection declarations for existing list fields where
their final shapes satisfy Narratio's schema. Operators can choose replacement,
append, or upsert and identified-record inheritance. Explain that upsert
replaces whole matching records and that an empty append/upsert contribution
does not clear a list. Register application-owned sequence identities only
where Narratio has a real domain identity requirement; generic library
capabilities do not establish new uniqueness requirements for every list.
### Discovery, credentials, and resource paths
Replace Narratio-owned local candidate probing with ConfigKit search descriptions
2026-11-17 20:27:47 +00:00
and `SearchConfig` . Use the native `WorkingDirectory` , `UserConfigDirectory` ,
and `UnixSystemDirectories` presets in that order with application name
`narratio` . Use their documented filenames. Explicit `--config` selection
remains an exclusive `File` input. Report the source actually selected by the
library. This provides convenient development-local configuration and standard
ConfigKit naming. Existing `pipeline.yml` files remain explicitly selectable;
migration guidance explains the new discovery names and precedence.
2026-11-17 20:23:22 +00:00
Bootstrap uses `LoadSecretsIntoEnvironment` once per invocation before typed
credential resolution and concurrent work. Existing environment variables win,
2026-11-17 20:27:47 +00:00
including explicitly empty variables. Use the invocation-owned conventional
directory `/usr/local/etc/narratio/secrets` , with `--secrets-dir` selecting
another directory. A missing conventional directory remains optional; an
explicit selection follows the existing explicit-selection behavior. Retain
safe partial-report handling and exact-name forwarding to subprocesses.
The independent directory lets bootstrap precede ConfigKit's first and only
configuration selection. Service environments can provide credentials directly;
development invocations can choose a project-local secrets directory explicitly.
The implicit sibling `secrets/` convention retires with this cutover.
Resolve relative file-resource references from their effective defining source's
ConfigKit resource context. Inherited values follow their effective origin;
CLI-relative paths use the invocation directory. Application-derived paths
record their dependencies. Use the main source as a documented fallback for
origins without a distinct usable resource base. This makes imported definitions
portable across bundles.
Review actual path-bearing fields explicitly. Artifact destinations, command
names, session-relative products, and remote object keys retain their domain
meanings. This policy introduces no new target-existence or root-containment
checks. ConfigKit's consumer examples illustrate application policy; Narratio
owns the selected interpretation.
### Imports in secondary documents
Enable native explicit imports for locally sourced campaign, party, and session
documents as well as pipelines. This supports split rosters and shared metadata
with consistent library semantics. Campaign selection, party schema validation,
and session domain rules remain application-owned. Pipeline overlay selection
remains associated with the pipeline.
Materialize effective documents where raw root-file copying would leave
unresolved imports in downstream inputs or published session data. Remote
session snapshots retain their explicit source context; resolving imports
against an incidental download directory would give them an incorrect base.
Remote companion-file retrieval remains separate work.
2026-11-17 20:23:22 +00:00
### Final-domain inspection through native APIs
Keep validate, show, sources, artifact catalog, and diff useful for the operator's
fully resolved pipeline, including campaign-driven members and publish rules.
Use native effective rendering and `Diff` where they own the operation, with
Narratio formatting and generated-domain context at the presentation boundary.
A raw pipeline `Result` is a frozen load snapshot. Mutating its `Config` later
does not update rendering or diff. Integrate this distinction explicitly:
record domain resolution within a supported resolver when its inputs are
available there; where later campaign resolution remains appropriate, create
a completed native inspection result from the finalized safe domain projection
using the public mapping-source API. That result describes application-derived
values; retained authoring results supply actual file provenance.
Use native comparison rather than maintaining a second general mapping-diff
engine. Preserve final-domain coverage and credential identities. Narratio's
digest and stage-owned reuse evidence remain application responsibilities.
Source-layout changes alone do not change equivalent domain values. Source
inspection can report composite origins, empty optional-import selections, and
application derivation without inventing a unique defining-file contract.
### Scope and compatibility
This is a clean native cutover. Update source-root `configkit:` controls to
`config:` and remove old-module/API references in maintained code and fixtures.
Nested `config` data remains ordinary data under ConfigKit's source-root rules.
The `configkit` package name and `configkit:"secret"` Go tag retain their
upstream spelling. Mapping and identified-list `extends` follow current native
rules.
This roadmap deliberately permits revised configuration semantics. Update
tests and documentation for selected changes rather than preserving historical
acceptance rules through compatibility loaders or keyword aliases.
Application identity, artifact dependency meaning, runtime output destinations,
campaign-owned party authority, remote session retrieval, and stage reuse remain
Narratio domain concerns. This scope does not redesign the runtime workspace,
change subprocess adapters, adopt ReleaseKit for Narratio, or add external
provider probes. Audita and workspace work retain their separate roadmaps.
## Required changes
### Dependency, toolchain, and notices
- Select the released `go.maximumdirect.net/configkit v1.2.1` module and migrate
all imports and exchanged named types together. Remove the old requirement;
use the released dependency rather than a filesystem replacement.
- Raise `go.mod` to Go 1.27.0 and select an available official Go 1.27 patch for
development/release validation and release builds. Align CI, runbook setup,
and incorporated Go attribution with that selection.
- Update ConfigKit source/license links to the reviewed commit and YAML v4
attribution to the selected stable release. Check the actual resulting module
graph. Maintainer tool dependencies are not automatically Narratio runtime
components. Narratio's separate direct YAML use need not be rewritten merely
because ConfigKit changed its internal parser dependency.
### Integration and simplification
- Apply the v1 API replacements actually used here, including
`LoadSecretsIntoEnvironment` , `LayerMain` , `RecordDerivedDefault` , and
`RecordNormalization` . Review other public names against the migration guide.
- Use keyed expanded options and definition literals. Keep metadata projections
compatible with current source evidence, native layer names, and safe errors.
- Replace broad list-record workarounds with supported field-level recording
when the application changes only fields. Record whole-list changes where
order or membership changes warrant them.
- Adopt native discovery, destination-based artifact authoring, authored
collection operations, and native final-domain rendering/comparison. Remove
replaced search, comparison, and redundant metadata machinery.
2026-11-17 20:27:47 +00:00
- Apply the agreed source-relative resource bases and secondary-document imports.
2026-11-17 20:23:22 +00:00
Share definitions and loading context between execution and inspection so an
operator validates the configuration the pipeline will actually consume.
2026-11-17 20:27:47 +00:00
- Review secondary-document input materialization and publication as well as
parsing. A copied root file can retain imports whose
2026-11-17 20:23:22 +00:00
companions are absent at its new location. Materialize the effective domain
document where a subprocess or restored session consumes it, retaining native
authoring evidence separately. Review placeholder checks against the effective
session input and preserve required external party/session formats.
### Documentation and maintained examples
Update the configuration reference, CLI reference, operations/bootstrap guidance,
configuration internals, contributor prerequisites, and release procedure for
the implemented semantics. Keep canonical facts in their respective owners.
Update project-specific architecture wording where selected path/bootstrap
decisions change it; preserve ADR 0003's ownership boundary.
Migrate the production/testing bundle to `config:` . Demonstrate destination-based
artifact and family files, a native directory import restricted to YAML, an
essential required import, native inheritance, and a useful selected-overlay
list customization. Keep illustrative optional choices labeled as choices.
Show how the declared artifact key is referenced by dependent artifacts.
Provide concise manual upgrade instructions for existing Narratio configurations:
control-key/API changes, optional import defaults, collection behavior, source
discovery, and selected path/bootstrap changes. Separately installed applications
own their configuration versions; Narratio's migration does not automatically
rewrite their files.
The eventual Narratio release requires a major version because existing valid
configuration control blocks change. The Go minimum increase alone is permitted
in a minor release under the adopted release policy. This roadmap does not
authorize tagging, publication, or retrospective release-note changes.
### Verification
Use temporary inputs and controlled synthetic credentials. Exercise native
discovery and the chosen bootstrap convention together, explicit file selection,
selected-source errors, optional and required imports, import destinations,
inherited artifact identities, native list customization, and final domain
validation. Add coverage at Narratio boundaries rather than duplicating
ConfigKit's complete structural test matrix.
Verify inherited presence/defaulting, party order, final campaign/family
expansion, resource paths, source attribution, and secret-safe inspection.
Diff/show must include derived members and publish rules; compare equal effective
configurations across different source arrangements. Credential rotation behind
the same reference name must remain absent from semantic diffs and digests.
Update tests that encode intentionally retired syntax or policies. Run focused
configuration/command/example/documentation checks and the established full
test, race, vet, build, and module checks on the adopted Go line. Preserve the
current supported build targets. No new live-service or CI-completion gate is
introduced by this migration.