Document and validate configured secrets bootstrap

This commit is contained in:
2026-11-18 01:10:24 +00:00
parent ab6e610fb7
commit eeafc356e4
13 changed files with 172 additions and 46 deletions

View File

@@ -1019,7 +1019,7 @@ There are three separate tasks when working with credentials:
| --- | --- |
| Read a credential from an environment variable | A typed `Secret` field whose YAML value is the variable's name |
| Hide a configuration value in inspection output | A `configkit:"secret"` tag or `Definition.Sensitive` |
| Populate environment variables from local files before loading | An explicit `LoadSecretsIntoEnvironment` call |
| Populate environment variables from local files before loading | An explicit [secrets bootstrap](#explicit-secrets-directory-bootstrap) call |
For example, YAML `api_key: DEMO_API_KEY` names an environment variable. Its
credential is available to trusted application code through `Secret.Value()`
@@ -1051,8 +1051,8 @@ its effective source location for diagnostics. Each unique name is read with
`os.LookupEnv` once per load. Validation can inspect the credential through the
deliberately named `Value()` accessor and distinguish success with `IsResolved()`.
Later loads read the environment again, and cloned defaults are rebuilt from names
so a previously resolved result cannot retain a stale credential. `Load` never
writes to the environment.
so a previously resolved result cannot retain a stale credential. `Load` and
`LoadLayered` never write to the environment.
Credential bytes are held only in the per-load lookup state and the trusted returned
configuration, and are returned only by `Value`. Secret formatting uses a safe
@@ -1078,17 +1078,80 @@ reference state without printing its value.
## Explicit secrets-directory bootstrap
If an application stores credentials in files, it can call this helper before
loading configuration. A file named `DEMO_API_KEY` containing the dummy text
If an application stores credentials in files, it can explicitly bootstrap
them before loading configuration. A file named `DEMO_API_KEY` containing the dummy text
`example-value` supplies environment variable `DEMO_API_KEY`. YAML can then
refer to that variable with `api_key: DEMO_API_KEY`. Applications that already
receive environment variables do not need this file step.
Existing environment variables take priority over files. If `DEMO_API_KEY` is
already set to an empty string, its file is skipped and it stays empty. A later
By default existing environment variables take priority over files. If
`DEMO_API_KEY` is already set to an empty string, its file is skipped and it stays empty. A later
required `Secret` reference will still fail on that empty value. File bootstrap
accepting an empty file is different from a required reference resolving to an
empty credential.
empty credential. `LoadSecretsIntoEnvironmentWithOptions` accepts
`SecretsOptions{Overwrite: true}` to replace existing values explicitly.
For a directory selected by the main file, call
`LoadSecretsIntoEnvironmentFromConfig` with a host `File` or `SearchConfig`
source before `Load` or `LoadLayered`. The main document may declare:
```yaml
config:
secrets_dir: secrets
secrets_overwrite: false
```
`secrets_dir` is a nonempty relative host path, interpreted literally relative
to the selected main file's canonical directory. Its canonical target must be
inside the main filesystem root; a contained directory symlink is allowed.
Parent-relative paths can reach other directories inside an explicitly wider
root. Rooted, absolute, and host volume-qualified paths are invalid. `secrets_overwrite`
is a Boolean and defaults to false. A non-nil Go `Overwrite` option wins over
the YAML value, including an explicit false. Invalid declarations fail even
when Go supplies an override. Overwrite without a directory is a no-op.
Only the main document root owns these controls. Imports and underlays reject
them with source-local import diagnostics; selected overlays reject them as
overlay directives. Nested application `config` mappings and data assembled at
the application root by placement, defaults, CLI, or resolution remain ordinary
data. Normal `Load` and `LoadLayered` validate and consume main controls whether
imports are enabled or disabled, but never bootstrap or write the environment.
Each load resolves typed secret names from the environment at the usual point.
The companion accepts at most one `SecretsFromConfigOptions`, including an
optional host `FilesystemContext`. It selects and parses the main once, validates
the bootstrap controls, and completes directory bootstrap before returning a
prepared source. It does not run import acquisition, version checks, decoding,
resolution, or validation. It loads every eligible directory entry, including
names unused by the application's typed schema. A main with no directory still
returns a prepared source and empty report. `NoSource` or an exhausted optional
search returns `NoSource` and an empty report. Explicit absent-input context
continues to follow the normal absent-host-context rules. The helper also accepts
a previously prepared source, reusing its captured main and explicitly performing
bootstrap again.
The prepared source is a reusable snapshot of the selected main document,
including its unconsumed controls, byte count, canonical file identity, base,
and containment root. A later load neither rediscovers nor rereads the main
and does not repeat bootstrap; it reads imports and resources when accessed and
looks up secrets anew. The main's source record retains `SourceKindFile`. Each
load clones the document, creates its own source IDs
and records, and applies normal input accounting. No file handle or secret-file
contents remain in the prepared source. With no later context, loading uses the
captured context. A supplied later host context must resolve to the same root,
base, and main path; equivalent path spellings and root symlink aliases are
accepted. Native contexts, a file `BaseDir`, and different roots fail with
`options.invalid`. A caller needing another root prepares the original source
again. These rules apply to ordinary options and layered main inputs.
The helper returns a non-nil report on every attempt and a zero, unusable source
on error. Discovery, source, parse, control, and preparation-context failures
return `*LoadError` with available source records and source-local diagnostics.
Directory inspection, reads, and environment writes return `*SecretsError`.
An invalid directory declaration uses `secrets.invalid_path`; a non-Boolean
overwrite declaration uses `secrets.invalid_value`. Later loading failures do
not undo bootstrap effects or change its report. Multiple options fail before
acquisition or writes.
Read the report even when the helper returns an error:
@@ -1101,18 +1164,20 @@ Read the report even when the helper returns an error:
Render the returned `SecretsReport` separately from a later configuration load.
It contains filenames, paths, and statuses, not credential contents. The
[maintained startup example](../examples/consumers/consumers_test.go) shows how to
keep bootstrap and configuration reports together in the application.
[configured startup example](../examples/consumers/secrets.go) prepares a source
for either loading entry point; its [test](../examples/consumers/secrets_test.go)
shows both handoffs. The [direct-directory example](../examples/consumers/consumers_test.go)
shows how to keep bootstrap and configuration reports together.
### Preflight, limits, and partial writes
`LoadSecretsIntoEnvironment(directory)` is a separate, opt-in startup operation.
It is never called by `Load`, is not a YAML option, and does not participate in
configuration validation. Applications choose the native directory path and
should invoke the operation once before starting concurrent work. The selected
path may itself be a symlink; ConfigKit resolves it to an existing canonical
directory before examining
its immediate entries.
All bootstrap helpers are separate, opt-in startup operations. They are never
called by a configuration load and do not perform configuration validation.
Applications own startup ordering and should invoke bootstrap before concurrent
work or other process-environment mutation. The direct helpers accept a native
directory path; the configured helper resolves the declared path as described
above. The selected path may itself be a symlink; ConfigKit resolves it to an
existing canonical directory before examining its immediate entries.
The bootstrap enumerates at most 1,024 entries, including dot-prefixed entries.
Entries whose names begin with `.` (files, directories, or symlinks, such as `.git`
@@ -1120,16 +1185,18 @@ and `.gitignore`) are ignored before name or metadata validation, are never open
and are omitted from the report. Remaining entries are sorted by filename; each
must have an ASCII environment-variable name and be a regular, nonsymlink
file. Metadata for all remaining entries is validated before file content is read
or process state is changed. Existing environment variables win, including variables whose
value is empty; their files are not opened. Each eligible file is limited to 1 MiB,
and eligible content is limited to 8 MiB in total. ConfigKit removes exactly one
or process state is changed. In preservation mode existing environment variables
win, including variables whose value is empty; their files are not opened. In
overwrite mode their files are read and applied as well. Each eligible file is
limited to 1 MiB, and eligible content is limited to 8 MiB in total. ConfigKit removes exactly one
terminal LF or CRLF, preserves all other whitespace, accepts an empty value, and
rejects NUL bytes.
All eligible files are read successfully before application begins. Presence is
checked again immediately before each `os.Setenv`: a variable populated since
preflight is skipped. A write failure stops application without rollback, so prior
writes remain and later eligible entries are `not_applied`. This operation is not
All required files are read successfully before application begins. In
preservation mode presence is checked again immediately before each `os.Setenv`:
a variable populated since preflight is skipped. Overwrite mode applies the
staged value regardless of presence. A write failure stops application without
rollback, so prior writes remain and later eligible entries are `not_applied`. This operation is not
atomic and does not provide compare-and-set semantics for concurrent environment
mutation.
@@ -1137,9 +1204,11 @@ mutation.
records with `loaded`, `skipped_existing`, `not_applied`, or `failed` status.
`ExistingEmpty` distinguishes an empty existing winner. `SecretsReport.WriteTo`
renders only quoted names, canonical paths, statuses, and that empty-state flag;
it never retains or renders file contents. Failures return the non-nil partial
report with a `*SecretsError` containing detached `secrets.*` diagnostics and no
raw filesystem or environment error chain. Applications should retain the report
it never retains or renders file contents. Directory and environment failures
return the non-nil partial report with a `*SecretsError` containing detached
`secrets.*` diagnostics and no raw filesystem or environment error chain.
Configured main-preparation failures return `*LoadError` as described above.
Applications should retain the report
alongside the later configuration result and explicitly present blocked and failed
files when appropriate; ConfigKit stores no implicit global report.
@@ -1674,8 +1743,10 @@ rules below explain absence, patterns, placement, and ordering.
### Declaration and selection rules
Imports are opt-in. With `Options.Imports` enabled, a document may use its root
`config` mapping for `imports` and `sequences` controls. Ordinary `Load` may use
`sequences` without enabling imports, but an `imports` field still requires
`config` mapping for `imports` and `sequences` controls. The main root may also
declare the [secrets bootstrap controls](#explicit-secrets-directory-bootstrap).
Ordinary `Load` may use `sequences` and main bootstrap controls without enabling
imports, but an `imports` field still requires
`Options.Imports`. ConfigKit removes recognized root controls before application
decoding. Each imports sequence item is a mapping with exactly one `path`,
`dir`, or `glob` selector. `path` and `dir` accept a string or a list of strings;