20 KiB
Adopt ConfigKit v1.2.1 and native configuration ergonomics
Purpose
Adopt current ConfigKit as Narratio's configuration owner and make its native authoring and inspection features useful to operators. Align configuration semantics with other ConfigKit consumers. Existing Narratio assembly conventions are migration inputs rather than compatibility requirements for this work.
This roadmap defines the target and required changes. A subsequent implementation plan will define staged work after the open decisions are resolved. This pass adds documentation only. The earlier v0.5.0 roadmap and implementation plan describe the completed baseline.
The ownership boundary in ADR 0003 continues to apply: ConfigKit resolves configuration before its final handoff; Narratio owns domain validation, application derivation, and runtime use. Supported callbacks and native metadata provide the integration boundary.
Reviewed baseline and releases
Narratio currently selects gitea.maximumdirect.net/eric/ConfigKit v0.5.0,
commit dc09959cca4e105f49ec316c6d4ba585bd2340bd, with Go 1.26.0 as its
consumer minimum. Release builds currently select Go 1.26.7.
The adoption target is go.maximumdirect.net/configkit v1.2.1, commit
51ef3144158db1a09fabdbd1c7ae59908c84923b. The published tag and the sibling
checkout agree on that revision. ConfigKit requires Go 1.27.0 and uses
go.yaml.in/yaml/v4 v4.0.0. Its code license remains LGPL-3.0-or-later.
The pinned configuration contract and consumer guide own upstream semantics. The intervening releases supply the following relevant changes:
| Release | Adoption relevance |
|---|---|
| v0.6.0 | Loading and metadata optimizations; Go minimum moves to 1.26.0. |
| v0.7.0 | Native safe effective-configuration Diff. |
| v0.7.1 | Maintainer-tool update; runtime behavior unchanged. |
| v0.8.0 | Input/import destinations, sequence-item placement, shared source-version checks, and initial identified-sequence composition. |
| v0.9.0 | Directory and expanded import selection; optional-by-default imports; identified-list replace/append/upsert and inheritance; CLI assignments move after decoding and inheritance. Its final collection APIs replace the initial v0.8 registrations. |
| v0.10.0 | Native ordered configuration discovery through SearchConfig. |
| v0.10.1 | Loading consolidation and import bookkeeping improvements. |
| v0.10.2 | Correctness fixes for decoding, callback/resource lifetime, discovery, inspection budgets, and collection operations. |
| v1.0.0 | Vanity module path, source-root config controls, public API cleanup, source lookup, provenance and sensitivity fixes, and Go 1.27.0 minimum. |
| v1.1.0 | Field-level list resolution records and identity-aware inspection across list movement. |
| v1.2.0 | Main-relative companion inputs and overlay context for layered loading; overlay filesystem fixes. |
| v1.2.1 | Stable go-yaml v4.0.0 dependency; maintainer release-tool update. |
The corresponding docs/releases/<version>.md files at the pinned revision
provide the migration details. Select v1.2.1 directly; intermediate releases
provide context rather than required deployment stops.
Current Narratio integration
The configuration reference and configuration internals describe current behavior.
- Commands select a pipeline path through Narratio's search loop, bootstrap
credentials from an explicit directory or a sibling
secrets/directory, and then call ConfigKit. - Pipeline loading enables explicit path/glob imports and one selected overlay. Campaign, session, and canonical-party documents use separate loads. Session loading also supports command-scoped downloaded input snapshots.
- Typed defaults, recorded normalization, domain validation, and native provenance are integrated. Additional path/dependency resolution and campaign-owned family expansion occur after the pipeline load.
- Artifact files currently repeat enclosing
scriptorium.artifactsmappings. Logical IDs come from map keys. Families expand against canonical party data. - Show, diff, and digest use a safe finalized-domain projection. Narratio owns the current mapping-diff algorithm. Native load metadata supplies source and artifact-catalog evidence.
Some public import descriptions still state restrictions from older ConfigKit behavior. Replace these with current native semantics during adoption rather than recreating them as Narratio checks.
Targeted end state
Native assembly and operator choice
ConfigKit owns source discovery, acquisition, composition, inheritance, presence, typed secrets, provenance, and safe native inspection. Narratio supplies its schema, domain defaults and validation, supported application resolution, and operator-selected inputs.
Use ordinary Load for an ordinary configuration. Use public native facilities
where they serve the operator workflow. Layered loading is appropriate when
actual lower-priority inputs are needed; typed defaults remain appropriate for
simple defaults. Main-relative companion inputs allow such underlays to follow
a discovered main without Narratio searching for it again.
Expose the library's authored config.imports and config.sequences semantics
where applicable. Imports use ConfigKit defaults, including optional absence,
unless the operator explicitly requests other supported behavior. An invalid
selected source still fails. Operators mark essential imports required: true.
Directory imports can select YAML with an explicit pattern; unrestricted
filenames remain the library default.
Native destinations, nested imports, discovery filters, source-local relative import paths, collection operations, and inheritance work under ConfigKit's rules. Narratio validates the final effective domain entries. It adds no fragment-count rule, mandatory fragment version, import-mode allowlist, or parallel composition engine.
Ergonomic artifact files and list customization
Support an artifact definition without repeated enclosing mappings through an explicit import destination. For example:
config:
imports:
- path: artifacts/session-recap.yml
destination: [scriptorium, artifacts, session_recap]
required: true
The imported file contains the artifact fields such as prompt_id, inputs,
and output_path. The literal destination defines the reusable key
session_recap, with logical source narratio.artifact.session_recap.
The filename is an organizational choice. Keyed files mounted at
[scriptorium, artifacts] support declared directory or glob imports when many
definitions should be discovered together. Apply the same pattern to families.
Keep keyed artifact/family mappings and campaign-owned character data. They already fit native inheritance and stable identity. Full-root and disjoint definitions remain valid native inputs; a one-artifact-per-file organization is guidance rather than a validity requirement.
Allow authored native collection declarations for existing list fields where their final shapes satisfy Narratio's schema. Operators can choose replacement, append, or upsert and identified-record inheritance. Explain that upsert replaces whole matching records and that an empty append/upsert contribution does not clear a list. Register application-owned sequence identities only where Narratio has a real domain identity requirement; generic library capabilities do not establish new uniqueness requirements for every list.
Discovery, credentials, and resource paths
Replace Narratio-owned local candidate probing with ConfigKit search descriptions
and SearchConfig. Explicit --config selection remains an exclusive File
input. Report the source actually selected by the library. The open decisions
below select search conventions and the independent bootstrap directory needed
before a discovered load resolves credentials.
Bootstrap uses LoadSecretsIntoEnvironment once per invocation before typed
credential resolution and concurrent work. Existing environment variables win,
including explicitly empty variables. Keep explicit --secrets-dir, safe
partial-report handling, and exact-name forwarding to subprocesses. The selected
bootstrap convention must allow native discovery without a second preliminary
configuration load or a duplicate file-selection loop.
Use native source/resource context for application path interpretation. The resource-base decision below applies to file references rather than artifact destinations, command names, or remote object keys. ConfigKit's consumer examples illustrate application policy; they do not impose it on Narratio.
Final-domain inspection through native APIs
Keep validate, show, sources, artifact catalog, and diff useful for the operator's
fully resolved pipeline, including campaign-driven members and publish rules.
Use native effective rendering and Diff where they own the operation, with
Narratio formatting and generated-domain context at the presentation boundary.
A raw pipeline Result is a frozen load snapshot. Mutating its Config later
does not update rendering or diff. Integrate this distinction explicitly:
record domain resolution within a supported resolver when its inputs are
available there; where later campaign resolution remains appropriate, create
a completed native inspection result from the finalized safe domain projection
using the public mapping-source API. That result describes application-derived
values; retained authoring results supply actual file provenance.
Use native comparison rather than maintaining a second general mapping-diff engine. Preserve final-domain coverage and credential identities. Narratio's digest and stage-owned reuse evidence remain application responsibilities. Source-layout changes alone do not change equivalent domain values. Source inspection can report composite origins, empty optional-import selections, and application derivation without inventing a unique defining-file contract.
Scope and compatibility
This is a clean native cutover. Update source-root configkit: controls to
config: and remove old-module/API references in maintained code and fixtures.
Nested config data remains ordinary data under ConfigKit's source-root rules.
The configkit package name and configkit:"secret" Go tag retain their
upstream spelling. Mapping and identified-list extends follow current native
rules.
This roadmap deliberately permits revised configuration semantics. Update tests and documentation for selected changes rather than preserving historical acceptance rules through compatibility loaders or keyword aliases.
Application identity, artifact dependency meaning, runtime output destinations, campaign-owned party authority, remote session retrieval, and stage reuse remain Narratio domain concerns. This scope does not redesign the runtime workspace, change subprocess adapters, adopt ReleaseKit for Narratio, or add external provider probes. Audita and workspace work retain their separate roadmaps.
Required changes
Dependency, toolchain, and notices
- Select the released
go.maximumdirect.net/configkit v1.2.1module and migrate all imports and exchanged named types together. Remove the old requirement; use the released dependency rather than a filesystem replacement. - Raise
go.modto Go 1.27.0 and select an available official Go 1.27 patch for development/release validation and release builds. Align CI, runbook setup, and incorporated Go attribution with that selection. - Update ConfigKit source/license links to the reviewed commit and YAML v4 attribution to the selected stable release. Check the actual resulting module graph. Maintainer tool dependencies are not automatically Narratio runtime components. Narratio's separate direct YAML use need not be rewritten merely because ConfigKit changed its internal parser dependency.
Integration and simplification
- Apply the v1 API replacements actually used here, including
LoadSecretsIntoEnvironment,LayerMain,RecordDerivedDefault, andRecordNormalization. Review other public names against the migration guide. - Use keyed expanded options and definition literals. Keep metadata projections compatible with current source evidence, native layer names, and safe errors.
- Replace broad list-record workarounds with supported field-level recording when the application changes only fields. Record whole-list changes where order or membership changes warrant them.
- Adopt native discovery, destination-based artifact authoring, authored collection operations, and native final-domain rendering/comparison. Remove replaced search, comparison, and redundant metadata machinery.
- Follow the decisions below for resource bases and secondary-document loading. Share definitions and loading context between execution and inspection so an operator validates the configuration the pipeline will actually consume.
- If secondary-document imports are adopted, review input materialization and publication as well as parsing. A copied root file can retain imports whose companions are absent at its new location. Materialize the effective domain document where a subprocess or restored session consumes it, retaining native authoring evidence separately. Review placeholder checks against the effective session input and preserve required external party/session formats.
Documentation and maintained examples
Update the configuration reference, CLI reference, operations/bootstrap guidance, configuration internals, contributor prerequisites, and release procedure for the implemented semantics. Keep canonical facts in their respective owners. Update project-specific architecture wording where selected path/bootstrap decisions change it; preserve ADR 0003's ownership boundary.
Migrate the production/testing bundle to config:. Demonstrate destination-based
artifact and family files, a native directory import restricted to YAML, an
essential required import, native inheritance, and a useful selected-overlay
list customization. Keep illustrative optional choices labeled as choices.
Show how the declared artifact key is referenced by dependent artifacts.
Provide concise manual upgrade instructions for existing Narratio configurations: control-key/API changes, optional import defaults, collection behavior, source discovery, and selected path/bootstrap changes. Separately installed applications own their configuration versions; Narratio's migration does not automatically rewrite their files.
The eventual Narratio release requires a major version because existing valid configuration control blocks change. The Go minimum increase alone is permitted in a minor release under the adopted release policy. This roadmap does not authorize tagging, publication, or retrospective release-note changes.
Verification
Use temporary inputs and controlled synthetic credentials. Exercise native discovery and the chosen bootstrap convention together, explicit file selection, selected-source errors, optional and required imports, import destinations, inherited artifact identities, native list customization, and final domain validation. Add coverage at Narratio boundaries rather than duplicating ConfigKit's complete structural test matrix.
Verify inherited presence/defaulting, party order, final campaign/family expansion, resource paths, source attribution, and secret-safe inspection. Diff/show must include derived members and publish rules; compare equal effective configurations across different source arrangements. Credential rotation behind the same reference name must remain absent from semantic diffs and digests.
Update tests that encode intentionally retired syntax or policies. Run focused configuration/command/example/documentation checks and the established full test, race, vet, build, and module checks on the adopted Go line. Preserve the current supported build targets. No new live-service or CI-completion gate is introduced by this migration.
Open Decisions
1. Default pipeline discovery conventions
Recommendation: use the native WorkingDirectory, UserConfigDirectory,
and UnixSystemDirectories presets in that order with application name
narratio. Use their documented filenames and retain --config for exclusive
selection. This makes development-local configuration convenient and matches
ConfigKit's standard naming. Existing pipeline.yml files remain explicitly
selectable; migration guidance explains the new discovery names and precedence.
Alternative: use ConfigKit SearchPaths for the current two system
pipeline.yml paths, or add those paths to a broader ordered native search.
This reduces deployment disruption while still removing Narratio's probing
loop, but preserves project-specific naming and gives local development less
convenience. Adding fallback paths also creates more precedence to explain.
2. Secrets bootstrap before native discovery
Recommendation: use an invocation-owned conventional directory
/usr/local/etc/narratio/secrets, with --secrets-dir selecting another
directory and existing environment variables taking priority. Missing
conventional secrets remain optional; an explicitly selected directory follows
the existing explicit-selection behavior. This separates bootstrap from main
file discovery and lets ConfigKit perform the first and only configuration
selection. Service environments can provide credentials directly; development
invocations can choose a project-local secrets directory explicitly.
Alternative: preserve implicit secrets/ beside the selected main file.
This is convenient for portable configuration bundles, but ConfigKit's discovery
source selects its file during loading. Preserving pre-load bootstrap would
require a coordinated upstream discovery/bootstrap facility or retaining
Narratio-owned preselection. The latter reduces native consolidation; moving
environment mutation into a hidden resolver would change the existing startup
boundary. An upstream request is viable if sibling-directory convenience is
more important than an immediately self-contained migration.
3. Relative resource paths in imported definitions
Recommendation: resolve relative file-resource references from their effective defining source's ConfigKit resource context. Inherited values follow their effective origin. CLI-relative paths use the invocation directory; application-derived paths record their dependencies. Use the main source as a documented fallback for origins without a distinct usable resource base. This makes imported artifact/integration definitions portable across bundles. Review actual path-bearing fields explicitly; session-relative products and remote keys retain their domain bases. Introduce no new target-existence or root-containment checks merely to adopt this policy.
Alternative: retain current per-field pipeline-root bases while using native metadata elsewhere. This minimizes path edits and remains a valid ConfigKit consumer policy, but moving reusable definitions still requires knowing the destination bundle's root. Absolute resource paths are another viable operator choice, with clear meaning but reduced portability.
4. Imports in campaign, party, and session documents
Recommendation: enable native explicit imports for locally sourced campaign, party, and session documents as well as pipelines. This supports split rosters and shared metadata with consistent library semantics. Keep campaign selection, party schema validation, and session domain rules application-owned. Remote session snapshots retain their explicit source context: remote companion-file retrieval needs its own supported input contract, rather than resolving relative imports against an incidental download directory. Pipeline overlay selection remains associated with the pipeline; it does not implicitly select overlays for every document. Adoption includes effective-document materialization where raw root-file copying would leave unresolved imports in downstream inputs or published session data.
Alternative: enable the richer facilities only for pipelines and retain single-source secondary documents. This bounds the migration and simplifies remote/local parity, but leaves an intentional project-specific difference in authoring ergonomics. If selected, document that scope explicitly rather than presenting it as a ConfigKit limitation.