Align configuration documentation with native loading
This commit is contained in:
@@ -1,210 +1,81 @@
|
||||
# Configuration Internals
|
||||
|
||||
User-visible fields, defaults, and selection behavior belong in the
|
||||
[Configuration Reference](../config.md). This document describes the internal
|
||||
pipeline-loading boundary implemented by `internal/config`.
|
||||
User-visible fields, defaults, discovery, and selection belong in the
|
||||
[Configuration Reference](../config.md). This document describes the
|
||||
`internal/config` loading and resolution boundary.
|
||||
|
||||
## Pipeline Loading
|
||||
|
||||
`LoadPipeline` assembles and validates a pipeline in this order:
|
||||
Commands bootstrap secrets once before loading, then select an explicit file or
|
||||
native required discovery. ConfigKit loads typed defaults, source-root imports,
|
||||
and the selected overlay, resolves inheritance, and strictly decodes the
|
||||
result. Narratio applies recorded defaults, normalization, resource resolution,
|
||||
and effective-domain validation. The selected root, source contexts, metadata,
|
||||
and authoring result are retained for provenance. Family expansion and checks
|
||||
requiring the campaign occur after campaign and party resolution.
|
||||
|
||||
1. Give ConfigKit a fresh typed `PipelineConfig` default value, the root file,
|
||||
enabled imports, and the caller's overlay selection. Omitted, empty, and
|
||||
exact `default` selections use the base; named overlays use the fixed
|
||||
`overlays/<name>.yml` convention.
|
||||
2. ConfigKit parses the root and declared imports, additively merges base
|
||||
sources, applies only the selected overlay, resolves native sibling
|
||||
inheritance, and strictly decodes the result.
|
||||
3. The definition resolver applies recorded entry-level and inherited defaults,
|
||||
runs deterministic domain normalization, records derived publish destinations,
|
||||
and resolves ordinary paths relative to the root pipeline file.
|
||||
4. The definition validator performs read-only checks that require only the
|
||||
pipeline file. Checks requiring canonical-party family expansion remain at
|
||||
the existing explicit pipeline/campaign resolution boundary.
|
||||
5. Narratio retains public ConfigKit metadata for source ownership, attaches
|
||||
runtime-only state, and computes the normalized effective digest. Artifact
|
||||
and family validation applies to the composed effective entries.
|
||||
|
||||
This ordering preserves monolithic configuration behavior. Moving a field to
|
||||
an imported fragment changes its source ownership, not its path base, default,
|
||||
or schema semantics.
|
||||
Resource resolution consults native entry origins and source contexts before
|
||||
normalization. An effective source supplies the base when usable; otherwise the
|
||||
main source, then the captured invocation directory, is the fallback. Explicit
|
||||
CLI file references use the invocation directory. Derived values record their
|
||||
dependencies. See the [configuration reference](../config.md#pipeline-imports-and-overlays)
|
||||
for the resource classifications and native authoring contract.
|
||||
|
||||
## Loaded Context Resolution
|
||||
|
||||
`LoadedPipelineCampaign` carries one already composed pipeline and its selected
|
||||
campaign into session resolution. `LoadSessionWithPipelineCampaignOptions`
|
||||
loads a local session against that context, while
|
||||
`ResolveLoadedPipelineCampaign` also accepts an already loaded remote session
|
||||
or no session while a caller retrieves one. Compatibility loaders route through
|
||||
these functions after their initial pipeline and campaign reads.
|
||||
`LoadedPipelineCampaign` carries the selected pipeline and campaign into local
|
||||
or remote session resolution. Local campaign, session, and party loads enable
|
||||
imports through their own ConfigKit definitions; only the pipeline selects an
|
||||
overlay. Native optional search distinguishes an absent local session from an
|
||||
invalid selected file. Remote session bytes have no incidental download-directory
|
||||
context, and retain object provenance. Effective session strings are checked for
|
||||
unresolved template tokens, and campaign/session identity and stable-input rules
|
||||
remain Narratio-owned.
|
||||
|
||||
Application commands own pipeline and campaign discovery, campaign-file versus
|
||||
registry selection, and the corresponding mutual-exclusion rules. Once they
|
||||
have a `LoadedPipelineCampaign`, local session discovery and remote-session
|
||||
download retain that exact pipeline object and its private provenance. Removing
|
||||
a temporary downloaded session file therefore cannot invalidate the resolved
|
||||
pipeline or campaign context.
|
||||
Campaign and session loaders retain detached document values after their own
|
||||
resolution. The session document is captured before campaign defaults are copied
|
||||
into runtime session state. Configuration-owned renderers produce standalone
|
||||
campaign and session YAML, preserving omitted session overrides. Prepare uses
|
||||
those renderers and canonical party rendering for materialization and resume
|
||||
comparison.
|
||||
|
||||
Campaign and session files use independent single-source ConfigKit definitions
|
||||
with imports and selected overlays disabled; native sibling inheritance remains
|
||||
available within each document. Canonical-party input follows the same
|
||||
single-source rule. Campaign files normally load directly from their source;
|
||||
session files are first read through the 8 MiB bounded-read boundary because the
|
||||
application must reject unresolved template placeholders before passing an exact
|
||||
byte snapshot to ConfigKit. Local and remotely downloaded session bytes then use
|
||||
the same typed loader and retain the existing post-load identity expectations,
|
||||
stable-input precedence, and source-relative path semantics.
|
||||
## Inspection And Provenance
|
||||
|
||||
The application also has a separate read-only inspection resolver for `config
|
||||
validate`, `config show`, `config sources`, and `config artifacts`. It uses the
|
||||
same production root, overlay, and campaign selection functions, but never routes
|
||||
through session discovery,
|
||||
remote-session download, adapter composition, workspace
|
||||
initialization, manifest access, or cleanup. A pipeline with retained artifact
|
||||
family declarations must resolve its selected campaign before ordinary pipeline
|
||||
validation, which expands its canonical-party members and generated publish
|
||||
rules. A pipeline without those declarations may be validated by itself.
|
||||
Secret-directory bootstrap remains invocation-scoped and happens before the
|
||||
ordinary ConfigKit load because explicit typed references must resolve even for
|
||||
inspection.
|
||||
Read-only inspection shares production pipeline and campaign selection without
|
||||
session discovery, workspace initialization, adapters, or manifest access. After
|
||||
applicable campaign and family resolution, a safe typed final-domain DTO contains
|
||||
concrete artifacts and generated publish rules. It stores credential reference
|
||||
names rather than typed secrets. A defaults-backed native result freezes this
|
||||
DTO without resolving credentials again; `WriteEffective` supplies show and
|
||||
prepared pipeline YAML, and deterministic JSON of the same DTO supplies the
|
||||
versioned semantic digest. Resolved resources participate in that digest, while
|
||||
source layout and secret values do not.
|
||||
|
||||
`MarshalEffectivePipeline` is the configuration-owned projection for `config
|
||||
show`. It serializes one explicit typed safe projection with the existing YAML
|
||||
encoder. The projection has no slots for resolution-only artifact family
|
||||
declarations or private provenance. The result has one trailing newline;
|
||||
commands do not marshal runtime objects directly.
|
||||
|
||||
`EffectivePipelineSources` and `EffectiveCampaignSources` provide the separate
|
||||
safe provenance projection for `config sources`. Pipeline ownership begins with
|
||||
the complete logical field paths projected from the schema and classifies
|
||||
each contributor as root, import, overlay, or centralized default. It reads
|
||||
ConfigKit's effective origins and inheritance locations when building each
|
||||
projection, so inherited values retain their contributors and controlling
|
||||
directives without a second cached leaf-ownership table. The
|
||||
projection replaces generated concrete member paths with paired family and
|
||||
canonical-party records, and does the same for generated publish rules.
|
||||
Campaign records identify campaign-owned fields and party inputs; canonical
|
||||
derived players point to the party source. The application command only joins these sorted
|
||||
records with selection metadata and never reparses configuration files.
|
||||
|
||||
`EffectiveArtifactCatalog` projects the same finalized concrete artifact map for
|
||||
`config artifacts`. It derives source identity from each map key, carries sorted
|
||||
effective dependencies, aggregates ConfigKit definition contributors, and adds
|
||||
family/character/party provenance for generated members. Family source evidence
|
||||
can name multiple contributing files. It does not inspect files named after
|
||||
artifacts or infer another runtime graph.
|
||||
|
||||
`config diff` uses a thin paired loader that performs two independent ConfigKit
|
||||
loads. When campaign
|
||||
resolution is needed, the command loads one selected campaign and party and
|
||||
expands both effective pipelines from that same party value. The configuration
|
||||
owner projects each normalized effective mapping directly from that same typed
|
||||
projection into sorted logical paths using JSON number preservation;
|
||||
mapping leaves are compared individually while sequence values remain atomic.
|
||||
Values are compact deterministic JSON representations for command output, not
|
||||
raw YAML fragments, ownership records, or secret material. A differing digest
|
||||
with no projected difference is treated as an internal consistency error.
|
||||
|
||||
Campaign context construction reads the campaign-owned party source through
|
||||
`ParseParty`. The required canonical party retains its raw bytes and normalized
|
||||
roster in runtime-only `ResolvedParty` provenance. Canonical resolution creates
|
||||
a deterministic `derived_from_party` players projection. The party is the only
|
||||
configured roster authority; player and session-party fields are unsupported.
|
||||
Authoring results separately supply native origins, source histories, and empty
|
||||
optional-import evidence to source and artifact inspection. Family and party
|
||||
context identifies generated entries without reparsing sources. Diff fully
|
||||
resolves each side against the same selected campaign and party, then uses
|
||||
ConfigKit `Diff` on the safe snapshots. The command presents native JSON Pointer
|
||||
paths and safe compact values.
|
||||
|
||||
## Canonical Party Domain
|
||||
|
||||
`ParseParty` is the package-owned boundary for parsing a party source. Its
|
||||
single-source ConfigKit definition disables imports and overlays, requires
|
||||
`schema_version` through `VersionRule`, and strictly decodes the
|
||||
character-keyed typed schema before applying Narratio's party-domain checks.
|
||||
Successful metadata coordinates recover authored character-map order without a
|
||||
second YAML parse; class and alias slice order comes directly from typed decode.
|
||||
The canonical value retains a separate exact byte copy of its source so
|
||||
consumers can materialize the authored party document without reserializing it.
|
||||
Its `PlayersYAML` method deterministically derives the ID-sorted, versioned
|
||||
players-only projection.
|
||||
|
||||
An unversioned source is rejected before resolution, family expansion, or
|
||||
prepare-stage materialization.
|
||||
File-backed party loading enables imports; `ParseParty` accepts contextless bytes.
|
||||
The shared definition checks the main document's version and Narratio validates
|
||||
the effective character roster. Native acquisition order and source coordinates
|
||||
establish roster order, favoring each child's own declaration. A deterministic
|
||||
renderer writes standalone canonical party YAML from that roster; the players
|
||||
projection and family expansion use the same resolved roster. The external format
|
||||
is defined in the [party contract](../integrations/party.md).
|
||||
|
||||
## Validation Ownership
|
||||
|
||||
Configuration validation remains within `internal/config`, with the public
|
||||
entry points and pipeline-domain sequencing separated from the private owners
|
||||
for core pipeline/session rules, external adapters, publish/storage, Notarius,
|
||||
Scriptorium, and shared scalar/path checks. These are package-local ownership
|
||||
boundaries rather than extension points or a general validation framework.
|
||||
|
||||
ConfigKit supplies composition, inheritance, presence, typed-secret, and
|
||||
provenance mechanics; Narratio supplies the domain rules applied to the
|
||||
effective decoded entries. There is no separate authoring-history acceptance
|
||||
gate: contributors and declaration history remain inspection evidence, not an
|
||||
additional validity condition.
|
||||
The [configuration reference](../config.md#pipeline-imports-and-overlays)
|
||||
documents sibling syntax, reserved-key migration, and the operator-visible
|
||||
inspection behavior.
|
||||
|
||||
Validation is read-only, fails fast, and does not define which of several
|
||||
independent invalid fields is reported first. Map-backed declarations are
|
||||
nevertheless traversed deterministically. Reusable domain normalization is an
|
||||
explicit preceding operation for both loaded and programmatically constructed
|
||||
pipelines. Notarius normalization commits canonical scalar, reference, and output
|
||||
values only after the complete normalization succeeds; its validator then checks
|
||||
that candidate without changing it. Scriptorium validates artifact declarations
|
||||
and outputs, dependencies and inputs, variables, referenced outputs, and the final
|
||||
dependency graph in deterministic artifact order.
|
||||
|
||||
Diagnostics identify the relevant semantic field. Merge conflicts also
|
||||
retain every claiming source needed to correct an additive ownership conflict.
|
||||
Complete prose, punctuation, formatting, and private validation choreography are
|
||||
not compatibility contracts.
|
||||
|
||||
## Diagnostics And Runtime Metadata
|
||||
|
||||
Syntax, duplicate-key, merge, overlay, and schema failures include the
|
||||
relevant source name and full field path. Additive conflicts report every
|
||||
claiming source so operators can repair the split without repeatedly
|
||||
rediscovering additional conflicts.
|
||||
|
||||
The loaded pipeline retains private runtime metadata for the absolute root
|
||||
path, ordered imports, selected overlay name, source, and path, original
|
||||
ConfigKit metadata and source snapshots, and effective digest. Leaf source
|
||||
records are projected from that retained native metadata when requested. This
|
||||
metadata does not participate in YAML decoding or alter the public
|
||||
configuration model.
|
||||
|
||||
The effective digest is SHA-256 over a versioned deterministic JSON encoding of
|
||||
the same typed safe projection used by show and diff. Runtime Notarius paths remain absolute for
|
||||
execution, but the digest substitutes their normalized logical values captured
|
||||
before root-relative resolution, so relocating an equivalent configuration
|
||||
bundle does not change provenance. Because ConfigKit and resolution metadata
|
||||
are private, the digest excludes source layout, overlay name, and ownership.
|
||||
Configuration stores credentials as ConfigKit typed references. A successful
|
||||
load resolves their values inside the opaque typed boundary, while this safe
|
||||
projection serializes only `EnvName()` identities. Credential-value changes are
|
||||
therefore never hashed; reference-name changes remain ordinary provenance.
|
||||
The optional Notarius credential follows the same path even for disabled or
|
||||
resumable extraction; normalization copies the opaque reference without reading
|
||||
or resolving it again. Direct YAML serialization has a configuration-owned safe
|
||||
projection so only the reference name can leave this boundary.
|
||||
`recomputePipelineEffectiveDigest` is the single package-owned refresh point
|
||||
for later runtime expansion.
|
||||
|
||||
## Test Surfaces
|
||||
|
||||
`pipeline_configkit_test.go` exercises real imports and fixed-path overlays,
|
||||
base selection, unselected-overlay isolation, source attribution, cross-file
|
||||
inheritance, and effective composition. Existing focused configuration
|
||||
tests protect defaults, explicit-value semantics, strict decoding, validation,
|
||||
metadata, and root-relative path behavior. Application configuration-loader tests protect the
|
||||
single-read boundary by changing the pipeline file after its initial load and
|
||||
confirming local session resolution retains the original pipeline. Other
|
||||
configuration tests continue to protect defaults and validation after assembly.
|
||||
`party_test.go` protects the versioned typed party schema, domain invariants,
|
||||
authored character order, retained bytes, and deterministic players projection
|
||||
without involving campaign or runtime wiring.
|
||||
`party_resolution_test.go` protects campaign-owned party loading, canonical
|
||||
input restrictions, source provenance, and virtual players input selection.
|
||||
ConfigKit owns composition, inheritance, presence, typed-secret resolution, and
|
||||
provenance. Narratio owns validation and normalization of effective domain
|
||||
values. Source histories are inspection evidence, not an additional acceptance
|
||||
gate. Focused configuration and application tests exercise imports, resource
|
||||
contexts, snapshots, secret-safe inspection, party resolution, and materialized
|
||||
inputs through the production boundaries.
|
||||
|
||||
## Artifact Family Resolution
|
||||
|
||||
|
||||
Reference in New Issue
Block a user