# Main-relative secrets directories Status: planned, not implemented. [ADR 0023](../adr/0023-explicit-configured-secrets-bootstrap.md) supersedes ADR 0004 and records explicit configured bootstrap and optional overwrite. The decisions below preserve ordinary loading without environment writes and the existing direct-directory helper's Go signature. ## Motivation and baseline Applications can use `SearchConfig` to select their main configuration file during a load. They can also call `LoadSecretsIntoEnvironment` during startup, but that helper takes a native directory path before the load begins. A relative argument uses the process working directory. Using a directory beside the discovered file therefore requires the application to take over main-file selection or perform a preliminary load. [Configuration discovery](../config.md#configuration-discovery) · [Secrets bootstrap](../config.md#explicit-secrets-directory-bootstrap) ConfigKit already retains filesystem context for acquired main files and binds companion inputs to that context. Typed `Secret` references read the environment after application resolution and before final validation. A companion bootstrap helper can use the main's context before loading while preserving this processing order. [Main-relative contexts](../adr/0022-main-relative-context.md) · [Typed secret references](../config.md#typed-secret-references) ## Target behavior ### Declaration and path resolution An application explicitly calls a companion to `LoadSecretsIntoEnvironment` before `Load` or `LoadLayered`. The selected main document supplies the directory and overwrite policy through its root control mapping. For example, the following declaration in `/etc/demo/config.yml` selects `/etc/demo/secrets` and authorizes replacement of existing variables when the companion helper is called. ```yaml config: secrets_dir: secrets secrets_overwrite: true api_key: DEMO_API_KEY ``` Interpret these fields at the main document's root before placement and composition. They are bootstrap settings consumed as document controls. `Load` and `LoadLayered` recognize the controls and perform their usual environment lookup without invoking bootstrap. The companion helper call is the explicit authorization for environment writes. Only the main document selects the bootstrap directory and YAML overwrite policy. Diagnose these controls in unsupported source-control contexts. Nested application data follows the existing `config` rules. Imports, underlays, overlays, CLI overrides, and application resolution retain their ordinary effects on application configuration. They do not redirect this bootstrap operation. `secrets_overwrite` is an optional Boolean with a default of `false`. An explicitly supplied Go overwrite choice takes precedence over YAML, including when that choice is `false`. If the main supplies no directory, bootstrap is a no-op and the prepared main can still be loaded. An optional search that finds no main preserves the existing absent-main lifecycle. Resolve the directory from the selected main file's canonical base within its existing filesystem root. For a discovered symlink, the base is the selected file's canonical target directory. Parent-relative paths follow the same containment rule as other main-relative inputs. The declaration must be a nonempty relative path and requires a usable main-file context. Keep the declaration and any diagnostics in source-local coordinates. [Document-root controls](../adr/0020-document-root-controls.md) · [Layered loading](../config.md#layered-loading) ### Companion helper and prepared main source The companion accepts the main `Source`, including a `SearchConfig` source, and returns a secrets report and a prepared main source. Its options accept an explicit host `FilesystemContext` for main preparation and an optional overwrite choice. Represent overwrite as `*bool`: nil defers to YAML, while a supplied false or true takes precedence. The public names below are illustrative. ```go main := configkit.SearchConfig("demo", search) report, preparedMain, err := configkit.LoadSecretsIntoEnvironmentFromConfig(main) if err != nil { // Handle the bootstrap failure and inspect report. return err } result, err := configkit.Load[AppConfig](preparedMain) ``` For layered loading, supply the prepared source as the main `Input.Source`. The helper resolves discovery once, reads the selected main document, extracts the bootstrap controls, and performs bootstrap. It returns the captured main contents with the selected file's identity and filesystem context so the later load consumes that same main document. Keep its remaining controls, including imports and collection declarations, available to normal loading. Apply the supplied host context using the existing main-file root rules during preparation. Without an explicit context, derive the usual context from the selected main file. Capture the resolved canonical base and containment root with the document. Later loading uses that captured context when no explicit context is supplied. An explicit later context must resolve to the same context; diagnose an incompatible context rather than silently rebinding the prepared source. A caller needing a different root prepares the original source again. This rule applies to ordinary loading options and layered main `Input` alike. The helper prepares only the main document. Composition, import acquisition, inheritance, typed decoding, application resolution, secret lookup, and validation run through the subsequent ordinary or layered load. Main preparation must retain the applicable source limits and root constraints. Source records, diagnostics, placements, imports, and main-relative companions use the selected main's identity and context. Release library-owned file handles before the helper returns, including on error. Prepared sources are reusable snapshots of the main contents, with passive filesystem context and no shared mutable load state. Reusing one neither rediscovers nor rereads the main file and does not repeat bootstrap. This differs from reusing the original `File` or `SearchConfig` source. Later imports and resources observe the filesystem when accessed. A later load reads the environment at its normal secret-resolution step. Each load accounts for the captured main under its normal source limits and creates its own source and inspection records. [Complete-on-return loading](../adr/0008-complete-on-return-loading.md) · [Source resource contexts](../config.md#source-resource-contexts) ### Preparation failures and return values Use the existing error families at their respective boundaries. Discovery, main-file acquisition, YAML parsing, bootstrap-control validation, and preparation context failures return `LoadError` with applicable source records and source-local diagnostics. Secret-directory inspection, reads, and environment writes return `SecretsError` with the existing safe diagnostics. Do not introduce another error or reporting framework. Return a secrets report on every attempt. Before directory processing begins, it has no entries; after processing begins, it retains the entries and outcomes observed so far. On any error, return a zero, unusable prepared `Source`. Earlier environment writes remain reported and are not rolled back. A caller must handle the failure before attempting configuration loading. On success, return a reusable prepared source, including when no directory was declared. Optional search exhaustion returns the absent-main source and an empty report. Preparation validates YAML and the bootstrap settings needed for its operation; it does not claim the later configuration load will succeed. Normal version checks, composition, decoding, and application validation remain in that load. A later incompatible-context error is a loading error and does not undo bootstrap writes. ### Shared bootstrap and overwrite behavior Keep `LoadSecretsIntoEnvironment(string) (*SecretsReport, error)` unchanged, including its nonvariadic function type and preservation behavior. Add a separately named options-bearing helper for explicit overwrite. Changing the existing helper to accept variadic options would preserve ordinary calls but break assignments to its existing function type. The new public names below are provisional. ```go report, err := configkit.LoadSecretsIntoEnvironment("/etc/demo/secrets") report, err = configkit.LoadSecretsIntoEnvironmentWithOptions( "/etc/demo/secrets", configkit.SecretsOptions{Overwrite: true}, ) ``` The existing helper, options-bearing helper, and main-config companion use the same directory-bootstrap implementation. The direct helper's Boolean option defaults to false; it has no YAML policy to override. With overwrite disabled, existing variables win, including empty ones, with presence checked again immediately before application as today. With overwrite enabled, eligible files are read even for existing variables and their values replace those variables. Variables without a corresponding file are left alone. An applied empty file supplies an empty value in either mode. Bootstrap processes the directory's eligible entries, including credentials used by SDKs or subprocesses outside typed `Secret` fields. Reuse the existing dot-entry handling, file checks, content rules, limits, and filename ordering. Validate metadata and read all content needed by the chosen overwrite policy before beginning writes. Directory and file failures produce the existing safe reporting behavior. The secrets report identifies writes, skips, and partial failures without retaining credential bytes. Call bootstrap during startup before starting concurrent work. Successful bootstrap precedes configuration loading and validation. A later load failure leaves the completed environment writes in place. A bootstrap write failure stops further writes, leaves earlier writes in place, and reports remaining entries as `not_applied`. The existing lack of rollback and concurrent environment transaction guarantees continues to apply. Keep bootstrap reporting separate from the subsequent configuration's secret lookup evidence. Each operation describes what it observed and performed. [Secrets bootstrap](../config.md#explicit-secrets-directory-bootstrap) · [Explicit configured bootstrap](../adr/0023-explicit-configured-secrets-bootstrap.md) ### Filesystem scope Support discovered and explicitly selected host files, including their existing main-root options. This covers the requested deployment pattern and reuses the native secret-file checks. Caller-supplied filesystem support is a possible later extension requiring decisions about file-type and symlink capabilities. ## Work needed - Add `config.secrets_dir` and `config.secrets_overwrite` to the main-document control grammar and consume them during ordinary loading without triggering bootstrap. Retain source-local positions for diagnostics. - Factor main-file selection and capture for use by the companion helper and normal acquisition. Add a prepared-source handoff with retained main contents, identity, context, and ownership. Supply the preparation filesystem context and optional Go overwrite choice as described above. Preserve per-load accounting, diagnose incompatible later contexts, and implement the failure/return contract. - Preserve the current direct-directory signature and add a separately named options-bearing helper. Reuse directory bootstrap from all three entry points, with matching reports and failures. Finalize public names during implementation. - Verify discovery from a different working directory, canonical main paths, optional search exhaustion, declaration scope, default and explicit overwrite choices, Go-over-YAML precedence, and all helper interfaces. Include assignment of the unchanged helper to its existing function type. Cover empty values, dot entries, read failures before writes, and partial write failures. Verify bootstrap includes entries unused by typed fields and that later load failures preserve bootstrap effects. - Verify error types, empty and partial reports, and unusable prepared sources on failure. Cover absent-main success and failed later loads without rollback. - Verify prepared-source reuse after main-file changes and discovery changes, live later imports and environment lookup, equivalent and incompatible later contexts, handle cleanup, normal loading without environment writes, and agreement with selected-file provenance, placement, relative imports, and main-relative companions. Use disposable credential values and deterministic failure seams in line with the [testing policy](../policy/testing.md). - Document the startup workflow and add a consumer example. Implement the ownership decision in ADR 0023 and update the canonical configuration contract, public source documentation, and affected component documentation when the behavior lands. Retire the roadmap reference in architecture policy at that time.