219 lines
12 KiB
Markdown
219 lines
12 KiB
Markdown
# Field-Level Configuration Normalization
|
|
|
|
## Status And Purpose
|
|
|
|
Planned; implementation has not started. ConfigKit v1.1.0 is released, while
|
|
PromptKit currently depends on v1.0.0.
|
|
|
|
Move PromptKit's existing configuration-time normalization and derived defaults
|
|
into ConfigKit's recorded application resolution. Configuration inspection should
|
|
then show the normalized values used to construct execution settings, with
|
|
explanations of the changes, while unchanged resource references retain the
|
|
source context needed to read their files.
|
|
|
|
The benefit is a more useful inspection view and removal of the parallel
|
|
interpreted configuration maintained for execution. This is a focused loading
|
|
change, not a new configuration format, normalization policy, or execution API.
|
|
|
|
## Current Baseline
|
|
|
|
PromptKit already uses identified lists for backends, profiles, and prompts at
|
|
the root and within named collections. Six ConfigKit identity registrations
|
|
cover those destinations. This roadmap does not repeat that migration.
|
|
|
|
The configuration adapter's `Definition.Resolve` currently captures input
|
|
presence information. During validation, `interpretConfiguration` constructs
|
|
detached backend, profile, and prompt interpretations and applies normalization
|
|
and a conditional backend queue default. Runtime construction uses those
|
|
interpretations; configuration inspection retains ConfigKit's composed values.
|
|
Configured message roles and cache-control values are normalized later during
|
|
prompt materialization.
|
|
|
|
This separation avoided recording whole resource lists as normalized under
|
|
ConfigKit v1.0.0. Such a record would replace the effective origins of unchanged
|
|
fields, including `content_file`, whose origin selects the filesystem and
|
|
directory used to load message text.
|
|
|
|
The current behavior is documented in the [format reference](../formats.md) and
|
|
[source-loading implementation](../internal/sources.md). The
|
|
[configuration ownership decision](../adr/0001-configuration-boundary.md)
|
|
continues to govern the division of responsibility: ConfigKit owns composition;
|
|
PromptKit interprets effective values and supplies domain rules.
|
|
|
|
## Released Upstream Capability
|
|
|
|
[ConfigKit v1.1.0](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.1.0/docs/releases/v1.1.0.md)
|
|
allows individual field records inside lists through the existing `Normalize`,
|
|
`RecordNormalization`, and derived-default APIs:
|
|
|
|
- Equal-length ordinary lists are compared recursively by position. This covers
|
|
nested prompt messages and inputs without registering new identities for them.
|
|
- Registered collections with valid, unique entry and final identities support
|
|
matched field comparison. PromptKit already registers its resource lists.
|
|
- List length changes, and registered identity order or membership changes,
|
|
still need list or ancestor coverage. Matching is conservative when identities
|
|
are unsuitable. Final identity validation remains ConfigKit's responsibility.
|
|
- Entry queries retain their fixed resolution-entry meaning; records address
|
|
final positions. Public effective-configuration diffs remain atomic for lists.
|
|
|
|
These capabilities support the proposed in-place field transformations. No
|
|
additional ConfigKit API is a prerequisite. Historical source evidence retained
|
|
for a matched record is distinct from its effective origin: a whole-list
|
|
resolution record can still replace descendant effective origins. The migration
|
|
should use individual field records for individual changes.
|
|
|
|
## Target End State
|
|
|
|
### One Normalized Configuration For Loading And Inspection
|
|
|
|
PromptKit's `Definition.Resolve` captures the entry metadata needed for existing
|
|
presence/null semantics, then applies configuration-time transformations to the
|
|
working configuration and records their effects at the affected fields.
|
|
`Definition.Validate` examines the resulting values without mutating them.
|
|
Runtime indexes and prompt materialization consume that same normalized
|
|
configuration, producing the execution-owned domain values they need.
|
|
|
|
Remove `state.interpreted` and the detached interpretation copies once their
|
|
normalization work has moved. Retain copies needed for runtime ownership and
|
|
public detached results; removing the duplicate configuration is not a reason
|
|
to expose shared mutable runtime state.
|
|
|
|
PromptKit continues to own normalization rules. ConfigKit records their effects
|
|
and provenance. Composition, inheritance, precedence, and source acquisition
|
|
remain upstream responsibilities; this work adds no source-level validation or
|
|
history-based domain enforcement.
|
|
|
|
### Transformation Scope
|
|
|
|
Use the existing transformations as the migration inventory, rather than
|
|
introducing general trimming or case-folding across the configuration:
|
|
|
|
| Area | Existing behavior to move into recorded resolution |
|
|
| --- | --- |
|
|
| Backends | Normalize base endpoints and diagnostic endpoints; trim API-key environment names; trim and lowercase smoke-test authentication kinds. |
|
|
| Backend capacity | Apply the existing conditional queue-capacity default. |
|
|
| Profiles | Normalize configured nonempty endpoint overrides; trim service tier, reasoning effort, and API-key environment names. |
|
|
| Prompt metadata | Trim descriptions and session-ID templates. |
|
|
| Prompt inputs | Trim names and supplied content-type and description values. |
|
|
| Prompt output | Trim a supplied schema path. |
|
|
| Configured messages | Normalize roles and supplied cache-control type/TTL values using the shared domain rules. |
|
|
|
|
Preserve the rules' existing distinctions. For example, roles are lowercased,
|
|
but cache-control type normalization only trims; the move should not make new
|
|
spellings acceptable. Preserve optional pointers and existing empty/null
|
|
interpretation rather than creating fields merely to record an event.
|
|
|
|
The backend queue default currently applies when concurrency is enabled and
|
|
the decoded queue-capacity pointer is nil, including explicit null. ConfigKit's
|
|
`Default` helper skips explicit input, so substituting it mechanically would
|
|
change behavior. Retain the existing condition and record the resulting
|
|
assignment with `RecordDerivedDefault`, including its concurrency dependency.
|
|
|
|
Resource IDs, namespaces, and resource references keep their existing exact
|
|
matching rules. Message content and `content_file` values are not new
|
|
normalization targets. Schema paths continue to address the selected schema
|
|
source, rather than acquiring a configuration-file-relative meaning.
|
|
|
|
Request overrides, profile completion from backend settings, and caller-appended
|
|
messages still need their existing runtime processing. Share domain normalizers
|
|
where appropriate; avoid moving request-dependent work into loading merely to
|
|
make inspection look fully executable.
|
|
|
|
### Provenance And Resource Handling
|
|
|
|
Record changed scalar fields at their exact paths with concise, value-free
|
|
reasons. A role edit, for example, belongs at the role field within the message,
|
|
not at the message or prompt list. Avoid recording unchanged values needlessly.
|
|
|
|
Keep the current origin-based lookup for unchanged `content_file` values,
|
|
including values inherited or retained from another source. Normalizing a sibling
|
|
role or cache-control field should not change which file is opened.
|
|
|
|
A normalized field acquires a resolution origin; its earlier source evidence is
|
|
available for inspection. This does not promise that every normalized value
|
|
retains a file-backed effective origin. If implementation discovers a transformed
|
|
field that itself requires a source-relative resource lookup, preserve the
|
|
relevant entry context explicitly before changing that field, using ConfigKit's
|
|
public context APIs. Do not infer a runtime resource base from arbitrary history.
|
|
|
|
The planned normalizer does not need to reorder, add, or remove resource or
|
|
message entries. That describes this implementation's scope, not a new
|
|
restriction on operators' ConfigKit composition workflows.
|
|
|
|
### Observable Changes
|
|
|
|
Configuration inspection will show normalized configuration values and recorded
|
|
derived defaults. Explanations will expose the associated resolution operations
|
|
and retained source evidence through ConfigKit's existing inspection behavior.
|
|
This remains configuration inspection, not a rendering of a request after
|
|
profile completion, variable substitution, or caller overrides.
|
|
|
|
Configuration diffs will compare these normalized effective values. Two inputs
|
|
that differ only in a normalized spelling may no longer produce a value diff;
|
|
list changes that remain are still reported atomically. Existing redaction
|
|
continues to apply to inspection and explanations.
|
|
|
|
The intended execution behavior, accepted configuration values, resource
|
|
selection, and presence/null handling remain the same. Preserve public error
|
|
classification where applicable while integrating normalization failures with
|
|
the existing validation path. Do not add a promise of identical diagnostic
|
|
wording, ordering, or internal callback timing. No YAML syntax or collection
|
|
specification-version change is planned.
|
|
|
|
## Implementation Stages
|
|
|
|
### 1. Adopt ConfigKit v1.1.0
|
|
|
|
Update the released module dependency and sums, review applicable attribution,
|
|
and run the existing suite against the release. Use the published module rather
|
|
than a committed local replacement. Keep this dependency change separable from
|
|
the normalization refactor so failures can be attributed clearly.
|
|
|
|
### 2. Move Existing Configuration Transformations
|
|
|
|
Move backend, profile, and prompt transformations into recorded resolution,
|
|
including the conditional queue default. Preserve the entry metadata consumed
|
|
by presence-sensitive behavior. Update runtime construction to use the resolved
|
|
configuration and remove redundant interpretation state.
|
|
|
|
Keep domain validation on effective values. Account for ConfigKit's conservative
|
|
audit of lists with invalid identities so an ordinary invalid resource does not
|
|
accidentally become an unrecorded-mutation failure merely because a sibling
|
|
field needed normalization. This calls for integration coverage, not additional
|
|
identity rules or a separate source audit.
|
|
|
|
### 3. Include Configured Message Fields
|
|
|
|
Move configured role and cache-control normalization into the same lifecycle,
|
|
recording nested fields individually. Reuse the domain normalizers that serve
|
|
runtime messages. Simplify prompt materialization where work has become
|
|
redundant, while retaining template compilation, content-file reading, and
|
|
domain construction at their appropriate boundaries.
|
|
|
|
### 4. Verify The Boundary And Update Documentation
|
|
|
|
Add focused tests where behavior crosses the loading/inspection boundary:
|
|
|
|
- Normalized effective values and explanations for representative backend,
|
|
profile, prompt, and nested message fields at root and collection destinations.
|
|
- Relative content-file loading after sibling normalization, including inherited
|
|
values and values retained from a different contributing source.
|
|
- Existing absence, explicit null, empty, and zero handling, especially the queue
|
|
default and optional input/message metadata.
|
|
- Existing invalid-value and identity rejection through useful public errors,
|
|
without broadening accepted values or adding new restrictions.
|
|
- Execution-relevant equivalence for normalized definitions, including prompt
|
|
hashes, and continued normalization of caller-appended messages.
|
|
- Existing redaction, normalized-value diff behavior, and atomic list diffs.
|
|
|
|
Use targeted integration cases instead of reproducing ConfigKit's correspondence
|
|
test suite. Run the relevant configuration, inspection, and execution tests,
|
|
then the ordinary repository checks for the completed change.
|
|
|
|
Update the format reference, consumer inspection guidance, source-loading
|
|
documentation, and affected GoDoc when behavior lands. Explain the inspection
|
|
change and its limits in the release notes. Review architecture wording against
|
|
the implemented callback flow without converting implementation details into
|
|
new contracts. Retire this roadmap and its incoming catalog link once the work
|
|
is complete.
|