|
|
|
|
@@ -1,210 +1,81 @@
|
|
|
|
|
# Configuration Internals
|
|
|
|
|
|
|
|
|
|
User-visible fields, defaults, and selection behavior belong in the
|
|
|
|
|
[Configuration Reference](../config.md). This document describes the internal
|
|
|
|
|
pipeline-loading boundary implemented by `internal/config`.
|
|
|
|
|
User-visible fields, defaults, discovery, and selection belong in the
|
|
|
|
|
[Configuration Reference](../config.md). This document describes the
|
|
|
|
|
`internal/config` loading and resolution boundary.
|
|
|
|
|
|
|
|
|
|
## Pipeline Loading
|
|
|
|
|
|
|
|
|
|
`LoadPipeline` assembles and validates a pipeline in this order:
|
|
|
|
|
Commands bootstrap secrets once before loading, then select an explicit file or
|
|
|
|
|
native required discovery. ConfigKit loads typed defaults, source-root imports,
|
|
|
|
|
and the selected overlay, resolves inheritance, and strictly decodes the
|
|
|
|
|
result. Narratio applies recorded defaults, normalization, resource resolution,
|
|
|
|
|
and effective-domain validation. The selected root, source contexts, metadata,
|
|
|
|
|
and authoring result are retained for provenance. Family expansion and checks
|
|
|
|
|
requiring the campaign occur after campaign and party resolution.
|
|
|
|
|
|
|
|
|
|
1. Give ConfigKit a fresh typed `PipelineConfig` default value, the root file,
|
|
|
|
|
enabled imports, and the caller's overlay selection. Omitted, empty, and
|
|
|
|
|
exact `default` selections use the base; named overlays use the fixed
|
|
|
|
|
`overlays/<name>.yml` convention.
|
|
|
|
|
2. ConfigKit parses the root and declared imports, additively merges base
|
|
|
|
|
sources, applies only the selected overlay, resolves native sibling
|
|
|
|
|
inheritance, and strictly decodes the result.
|
|
|
|
|
3. The definition resolver applies recorded entry-level and inherited defaults,
|
|
|
|
|
runs deterministic domain normalization, records derived publish destinations,
|
|
|
|
|
and resolves ordinary paths relative to the root pipeline file.
|
|
|
|
|
4. The definition validator performs read-only checks that require only the
|
|
|
|
|
pipeline file. Checks requiring canonical-party family expansion remain at
|
|
|
|
|
the existing explicit pipeline/campaign resolution boundary.
|
|
|
|
|
5. Narratio retains public ConfigKit metadata for source ownership, attaches
|
|
|
|
|
runtime-only state, and computes the normalized effective digest. Artifact
|
|
|
|
|
and family validation applies to the composed effective entries.
|
|
|
|
|
|
|
|
|
|
This ordering preserves monolithic configuration behavior. Moving a field to
|
|
|
|
|
an imported fragment changes its source ownership, not its path base, default,
|
|
|
|
|
or schema semantics.
|
|
|
|
|
Resource resolution consults native entry origins and source contexts before
|
|
|
|
|
normalization. An effective source supplies the base when usable; otherwise the
|
|
|
|
|
main source, then the captured invocation directory, is the fallback. Explicit
|
|
|
|
|
CLI file references use the invocation directory. Derived values record their
|
|
|
|
|
dependencies. See the [configuration reference](../config.md#pipeline-imports-and-overlays)
|
|
|
|
|
for the resource classifications and native authoring contract.
|
|
|
|
|
|
|
|
|
|
## Loaded Context Resolution
|
|
|
|
|
|
|
|
|
|
`LoadedPipelineCampaign` carries one already composed pipeline and its selected
|
|
|
|
|
campaign into session resolution. `LoadSessionWithPipelineCampaignOptions`
|
|
|
|
|
loads a local session against that context, while
|
|
|
|
|
`ResolveLoadedPipelineCampaign` also accepts an already loaded remote session
|
|
|
|
|
or no session while a caller retrieves one. Compatibility loaders route through
|
|
|
|
|
these functions after their initial pipeline and campaign reads.
|
|
|
|
|
`LoadedPipelineCampaign` carries the selected pipeline and campaign into local
|
|
|
|
|
or remote session resolution. Local campaign, session, and party loads enable
|
|
|
|
|
imports through their own ConfigKit definitions; only the pipeline selects an
|
|
|
|
|
overlay. Native optional search distinguishes an absent local session from an
|
|
|
|
|
invalid selected file. Remote session bytes have no incidental download-directory
|
|
|
|
|
context, and retain object provenance. Effective session strings are checked for
|
|
|
|
|
unresolved template tokens, and campaign/session identity and stable-input rules
|
|
|
|
|
remain Narratio-owned.
|
|
|
|
|
|
|
|
|
|
Application commands own pipeline and campaign discovery, campaign-file versus
|
|
|
|
|
registry selection, and the corresponding mutual-exclusion rules. Once they
|
|
|
|
|
have a `LoadedPipelineCampaign`, local session discovery and remote-session
|
|
|
|
|
download retain that exact pipeline object and its private provenance. Removing
|
|
|
|
|
a temporary downloaded session file therefore cannot invalidate the resolved
|
|
|
|
|
pipeline or campaign context.
|
|
|
|
|
Campaign and session loaders retain detached document values after their own
|
|
|
|
|
resolution. The session document is captured before campaign defaults are copied
|
|
|
|
|
into runtime session state. Configuration-owned renderers produce standalone
|
|
|
|
|
campaign and session YAML, preserving omitted session overrides. Prepare uses
|
|
|
|
|
those renderers and canonical party rendering for materialization and resume
|
|
|
|
|
comparison.
|
|
|
|
|
|
|
|
|
|
Campaign and session files use independent single-source ConfigKit definitions
|
|
|
|
|
with imports and selected overlays disabled; native sibling inheritance remains
|
|
|
|
|
available within each document. Canonical-party input follows the same
|
|
|
|
|
single-source rule. Campaign files normally load directly from their source;
|
|
|
|
|
session files are first read through the 8 MiB bounded-read boundary because the
|
|
|
|
|
application must reject unresolved template placeholders before passing an exact
|
|
|
|
|
byte snapshot to ConfigKit. Local and remotely downloaded session bytes then use
|
|
|
|
|
the same typed loader and retain the existing post-load identity expectations,
|
|
|
|
|
stable-input precedence, and source-relative path semantics.
|
|
|
|
|
## Inspection And Provenance
|
|
|
|
|
|
|
|
|
|
The application also has a separate read-only inspection resolver for `config
|
|
|
|
|
validate`, `config show`, `config sources`, and `config artifacts`. It uses the
|
|
|
|
|
same production root, overlay, and campaign selection functions, but never routes
|
|
|
|
|
through session discovery,
|
|
|
|
|
remote-session download, adapter composition, workspace
|
|
|
|
|
initialization, manifest access, or cleanup. A pipeline with retained artifact
|
|
|
|
|
family declarations must resolve its selected campaign before ordinary pipeline
|
|
|
|
|
validation, which expands its canonical-party members and generated publish
|
|
|
|
|
rules. A pipeline without those declarations may be validated by itself.
|
|
|
|
|
Secret-directory bootstrap remains invocation-scoped and happens before the
|
|
|
|
|
ordinary ConfigKit load because explicit typed references must resolve even for
|
|
|
|
|
inspection.
|
|
|
|
|
Read-only inspection shares production pipeline and campaign selection without
|
|
|
|
|
session discovery, workspace initialization, adapters, or manifest access. After
|
|
|
|
|
applicable campaign and family resolution, a safe typed final-domain DTO contains
|
|
|
|
|
concrete artifacts and generated publish rules. It stores credential reference
|
|
|
|
|
names rather than typed secrets. A defaults-backed native result freezes this
|
|
|
|
|
DTO without resolving credentials again; `WriteEffective` supplies show and
|
|
|
|
|
prepared pipeline YAML, and deterministic JSON of the same DTO supplies the
|
|
|
|
|
versioned semantic digest. Resolved resources participate in that digest, while
|
|
|
|
|
source layout and secret values do not.
|
|
|
|
|
|
|
|
|
|
`MarshalEffectivePipeline` is the configuration-owned projection for `config
|
|
|
|
|
show`. It serializes one explicit typed safe projection with the existing YAML
|
|
|
|
|
encoder. The projection has no slots for resolution-only artifact family
|
|
|
|
|
declarations or private provenance. The result has one trailing newline;
|
|
|
|
|
commands do not marshal runtime objects directly.
|
|
|
|
|
|
|
|
|
|
`EffectivePipelineSources` and `EffectiveCampaignSources` provide the separate
|
|
|
|
|
safe provenance projection for `config sources`. Pipeline ownership begins with
|
|
|
|
|
the complete logical field paths projected from the schema and classifies
|
|
|
|
|
each contributor as root, import, overlay, or centralized default. It reads
|
|
|
|
|
ConfigKit's effective origins and inheritance locations when building each
|
|
|
|
|
projection, so inherited values retain their contributors and controlling
|
|
|
|
|
directives without a second cached leaf-ownership table. The
|
|
|
|
|
projection replaces generated concrete member paths with paired family and
|
|
|
|
|
canonical-party records, and does the same for generated publish rules.
|
|
|
|
|
Campaign records identify campaign-owned fields and party inputs; canonical
|
|
|
|
|
derived players point to the party source. The application command only joins these sorted
|
|
|
|
|
records with selection metadata and never reparses configuration files.
|
|
|
|
|
|
|
|
|
|
`EffectiveArtifactCatalog` projects the same finalized concrete artifact map for
|
|
|
|
|
`config artifacts`. It derives source identity from each map key, carries sorted
|
|
|
|
|
effective dependencies, aggregates ConfigKit definition contributors, and adds
|
|
|
|
|
family/character/party provenance for generated members. Family source evidence
|
|
|
|
|
can name multiple contributing files. It does not inspect files named after
|
|
|
|
|
artifacts or infer another runtime graph.
|
|
|
|
|
|
|
|
|
|
`config diff` uses a thin paired loader that performs two independent ConfigKit
|
|
|
|
|
loads. When campaign
|
|
|
|
|
resolution is needed, the command loads one selected campaign and party and
|
|
|
|
|
expands both effective pipelines from that same party value. The configuration
|
|
|
|
|
owner projects each normalized effective mapping directly from that same typed
|
|
|
|
|
projection into sorted logical paths using JSON number preservation;
|
|
|
|
|
mapping leaves are compared individually while sequence values remain atomic.
|
|
|
|
|
Values are compact deterministic JSON representations for command output, not
|
|
|
|
|
raw YAML fragments, ownership records, or secret material. A differing digest
|
|
|
|
|
with no projected difference is treated as an internal consistency error.
|
|
|
|
|
|
|
|
|
|
Campaign context construction reads the campaign-owned party source through
|
|
|
|
|
`ParseParty`. The required canonical party retains its raw bytes and normalized
|
|
|
|
|
roster in runtime-only `ResolvedParty` provenance. Canonical resolution creates
|
|
|
|
|
a deterministic `derived_from_party` players projection. The party is the only
|
|
|
|
|
configured roster authority; player and session-party fields are unsupported.
|
|
|
|
|
Authoring results separately supply native origins, source histories, and empty
|
|
|
|
|
optional-import evidence to source and artifact inspection. Family and party
|
|
|
|
|
context identifies generated entries without reparsing sources. Diff fully
|
|
|
|
|
resolves each side against the same selected campaign and party, then uses
|
|
|
|
|
ConfigKit `Diff` on the safe snapshots. The command presents native JSON Pointer
|
|
|
|
|
paths and safe compact values.
|
|
|
|
|
|
|
|
|
|
## Canonical Party Domain
|
|
|
|
|
|
|
|
|
|
`ParseParty` is the package-owned boundary for parsing a party source. Its
|
|
|
|
|
single-source ConfigKit definition disables imports and overlays, requires
|
|
|
|
|
`schema_version` through `VersionRule`, and strictly decodes the
|
|
|
|
|
character-keyed typed schema before applying Narratio's party-domain checks.
|
|
|
|
|
Successful metadata coordinates recover authored character-map order without a
|
|
|
|
|
second YAML parse; class and alias slice order comes directly from typed decode.
|
|
|
|
|
The canonical value retains a separate exact byte copy of its source so
|
|
|
|
|
consumers can materialize the authored party document without reserializing it.
|
|
|
|
|
Its `PlayersYAML` method deterministically derives the ID-sorted, versioned
|
|
|
|
|
players-only projection.
|
|
|
|
|
|
|
|
|
|
An unversioned source is rejected before resolution, family expansion, or
|
|
|
|
|
prepare-stage materialization.
|
|
|
|
|
File-backed party loading enables imports; `ParseParty` accepts contextless bytes.
|
|
|
|
|
The shared definition checks the main document's version and Narratio validates
|
|
|
|
|
the effective character roster. Native acquisition order and source coordinates
|
|
|
|
|
establish roster order, favoring each child's own declaration. A deterministic
|
|
|
|
|
renderer writes standalone canonical party YAML from that roster; the players
|
|
|
|
|
projection and family expansion use the same resolved roster. The external format
|
|
|
|
|
is defined in the [party contract](../integrations/party.md).
|
|
|
|
|
|
|
|
|
|
## Validation Ownership
|
|
|
|
|
|
|
|
|
|
Configuration validation remains within `internal/config`, with the public
|
|
|
|
|
entry points and pipeline-domain sequencing separated from the private owners
|
|
|
|
|
for core pipeline/session rules, external adapters, publish/storage, Notarius,
|
|
|
|
|
Scriptorium, and shared scalar/path checks. These are package-local ownership
|
|
|
|
|
boundaries rather than extension points or a general validation framework.
|
|
|
|
|
|
|
|
|
|
ConfigKit supplies composition, inheritance, presence, typed-secret, and
|
|
|
|
|
provenance mechanics; Narratio supplies the domain rules applied to the
|
|
|
|
|
effective decoded entries. There is no separate authoring-history acceptance
|
|
|
|
|
gate: contributors and declaration history remain inspection evidence, not an
|
|
|
|
|
additional validity condition.
|
|
|
|
|
The [configuration reference](../config.md#pipeline-imports-and-overlays)
|
|
|
|
|
documents sibling syntax, reserved-key migration, and the operator-visible
|
|
|
|
|
inspection behavior.
|
|
|
|
|
|
|
|
|
|
Validation is read-only, fails fast, and does not define which of several
|
|
|
|
|
independent invalid fields is reported first. Map-backed declarations are
|
|
|
|
|
nevertheless traversed deterministically. Reusable domain normalization is an
|
|
|
|
|
explicit preceding operation for both loaded and programmatically constructed
|
|
|
|
|
pipelines. Notarius normalization commits canonical scalar, reference, and output
|
|
|
|
|
values only after the complete normalization succeeds; its validator then checks
|
|
|
|
|
that candidate without changing it. Scriptorium validates artifact declarations
|
|
|
|
|
and outputs, dependencies and inputs, variables, referenced outputs, and the final
|
|
|
|
|
dependency graph in deterministic artifact order.
|
|
|
|
|
|
|
|
|
|
Diagnostics identify the relevant semantic field. Merge conflicts also
|
|
|
|
|
retain every claiming source needed to correct an additive ownership conflict.
|
|
|
|
|
Complete prose, punctuation, formatting, and private validation choreography are
|
|
|
|
|
not compatibility contracts.
|
|
|
|
|
|
|
|
|
|
## Diagnostics And Runtime Metadata
|
|
|
|
|
|
|
|
|
|
Syntax, duplicate-key, merge, overlay, and schema failures include the
|
|
|
|
|
relevant source name and full field path. Additive conflicts report every
|
|
|
|
|
claiming source so operators can repair the split without repeatedly
|
|
|
|
|
rediscovering additional conflicts.
|
|
|
|
|
|
|
|
|
|
The loaded pipeline retains private runtime metadata for the absolute root
|
|
|
|
|
path, ordered imports, selected overlay name, source, and path, original
|
|
|
|
|
ConfigKit metadata and source snapshots, and effective digest. Leaf source
|
|
|
|
|
records are projected from that retained native metadata when requested. This
|
|
|
|
|
metadata does not participate in YAML decoding or alter the public
|
|
|
|
|
configuration model.
|
|
|
|
|
|
|
|
|
|
The effective digest is SHA-256 over a versioned deterministic JSON encoding of
|
|
|
|
|
the same typed safe projection used by show and diff. Runtime Notarius paths remain absolute for
|
|
|
|
|
execution, but the digest substitutes their normalized logical values captured
|
|
|
|
|
before root-relative resolution, so relocating an equivalent configuration
|
|
|
|
|
bundle does not change provenance. Because ConfigKit and resolution metadata
|
|
|
|
|
are private, the digest excludes source layout, overlay name, and ownership.
|
|
|
|
|
Configuration stores credentials as ConfigKit typed references. A successful
|
|
|
|
|
load resolves their values inside the opaque typed boundary, while this safe
|
|
|
|
|
projection serializes only `EnvName()` identities. Credential-value changes are
|
|
|
|
|
therefore never hashed; reference-name changes remain ordinary provenance.
|
|
|
|
|
The optional Notarius credential follows the same path even for disabled or
|
|
|
|
|
resumable extraction; normalization copies the opaque reference without reading
|
|
|
|
|
or resolving it again. Direct YAML serialization has a configuration-owned safe
|
|
|
|
|
projection so only the reference name can leave this boundary.
|
|
|
|
|
`recomputePipelineEffectiveDigest` is the single package-owned refresh point
|
|
|
|
|
for later runtime expansion.
|
|
|
|
|
|
|
|
|
|
## Test Surfaces
|
|
|
|
|
|
|
|
|
|
`pipeline_configkit_test.go` exercises real imports and fixed-path overlays,
|
|
|
|
|
base selection, unselected-overlay isolation, source attribution, cross-file
|
|
|
|
|
inheritance, and effective composition. Existing focused configuration
|
|
|
|
|
tests protect defaults, explicit-value semantics, strict decoding, validation,
|
|
|
|
|
metadata, and root-relative path behavior. Application configuration-loader tests protect the
|
|
|
|
|
single-read boundary by changing the pipeline file after its initial load and
|
|
|
|
|
confirming local session resolution retains the original pipeline. Other
|
|
|
|
|
configuration tests continue to protect defaults and validation after assembly.
|
|
|
|
|
`party_test.go` protects the versioned typed party schema, domain invariants,
|
|
|
|
|
authored character order, retained bytes, and deterministic players projection
|
|
|
|
|
without involving campaign or runtime wiring.
|
|
|
|
|
`party_resolution_test.go` protects campaign-owned party loading, canonical
|
|
|
|
|
input restrictions, source provenance, and virtual players input selection.
|
|
|
|
|
ConfigKit owns composition, inheritance, presence, typed-secret resolution, and
|
|
|
|
|
provenance. Narratio owns validation and normalization of effective domain
|
|
|
|
|
values. Source histories are inspection evidence, not an additional acceptance
|
|
|
|
|
gate. Focused configuration and application tests exercise imports, resource
|
|
|
|
|
contexts, snapshots, secret-safe inspection, party resolution, and materialized
|
|
|
|
|
inputs through the production boundaries.
|
|
|
|
|
|
|
|
|
|
## Artifact Family Resolution
|
|
|
|
|
|
|
|
|
|
|