Recorded Field Normalization Implementation Plan

Implement field-normalization.md in the sequential stages below. Each stage is sized for one Luna coding-agent prompt. The feature roadmap owns intent and target behavior; this plan owns implementation steps.

Baseline And Sprint Rules

The baseline is fb7ebe6: ConfigKit v1.0.0, Go 1.27.0, six registered resource list identities, and execution-owned interpretations in internal/configuration. configurationDefinition captures presence in Resolve, then populates state.interpreted in Validate. load.go, targets.go, and prompts.go consume that second configuration. Message roles/cache controls are normalized in materializePrompt.

Read docs/development.md and the applicable agent, architecture, testing, documentation, and OSS policies. Follow ADR 0001. Use ConfigKit's released public API and documentation, including its v1.1.0 resolution-recording rules.

This is one end-to-end sprint. Intermediate stages need not compile or be coherent; do not add compatibility scaffolding, human approval gates, or external publication dependencies to make them independently deployable. Keep dependency adoption in a focused commit for diagnosis, not as a review gate. Run checks when their dependencies are present and record incomplete verification honestly. Reuse successful checks for unchanged content. Do not publish a release.

Implementation Decisions

Keep the existing identity registrations, configuration specification version, public APIs, and normalization rules. Do not normalize resource IDs, namespaces, references, model names, content, or content_file paths.
Capture existing backend/profile field states and prompt-input metadata from Resolution.EntryInfo before any assignment. Preserve omission, inherited explicitness, null, empty strings, false, and zero independently of normalized values. Do not retain callback contexts beyond their lifetime.
Perform changed-field assignments in Resolve and record exact final-index paths. Compare old/new scalar values before recording; preserve nil pointers and absent optional containers. Use concise value-free reasons. A small typed helper for optional strings is appropriate; a reflective traversal framework or a second configuration tree is not needed.
Resource lists inside Collections need map-entry copy/writeback. Modify the actual slices/pointers and write collection values back; helper targets and recorded paths must agree with the final typed document.
Visit records and nested fields directly. Build a uniqueness set once per resource list for the eligibility check below; avoid a metadata snapshot or full-resource search per scalar. Retain the existing lazy shared origin lookup for content materialization.
Before changing a registered list, check its final typed IDs once using the existing local-ID predicate and an exact-ID uniqueness set. If any ID is invalid or duplicated, leave that entire resource list unmodified by normalization and defaulting. ConfigKit's final identity validation and PromptKit's existing domain validation then reject it at their ordinary boundaries. This is an internal mutation-eligibility check on final values, not source prevalidation, identity repair, or a new acceptance rule. Test the invalid-ID-plus-padding combination so it cannot become an unrecorded-list-mutation error.
Successful endpoint normalization is assigned and recorded. If normalization fails, leave the original field unchanged; pure semantic validation reports the endpoint error using the existing validation boundary. Do not use a generic returned resolver error that hides the useful field diagnostic. Shared domain functions may be used for pure checks without assigning in validation.
Queue defaulting remains concurrency_limit != nil && > 0 && queue_capacity == nil. Assign 1024 and call RecordDerivedDefault at queue_capacity, with concurrency_limit as dependency. Do not substitute Default: explicit null currently receives this default too. Explicit zero remains zero.
Validate checks the resolved document without mutation. Domain construction, prompt compilation, and file loading remain at their current appropriate boundaries. Preserve public error identities where applicable, not exact text, diagnostic ordering, or callback timing.
Retain request-time normalization, backend-to-profile completion, runtime-owned copies, and detached public results. They serve different purposes from the duplicate interpreted configuration being removed.

Stage 1: Adopt Released ConfigKit v1.1.0

Scope: module metadata and applicable attribution only.

Update go.maximumdirect.net/configkit to v1.1.0 using the compatible local Go installation. Update sums without unrelated dependency upgrades or a committed filesystem replacement. Keep Go 1.27.0 unless the actual dependency metadata requires a change; no increase is planned.
Review the selected release's license and notices. Update its House Format entry in docs/oss/attribution.md, including version and commit-pinned links. The local upstream v1.1.0 tag currently resolves to 191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9; verify against the selected source.
Run go test ./... with caches prepared. Resolve only actual compatibility failures; do not start the normalization refactor in this dependency unit.

Acceptance: module and notices select the release, the existing suite works, and no local replacement or unrelated upgrade was introduced.

Completion: ConfigKit v1.1.0 and its commit-pinned notices are selected; the complete Go test suite passed with Go 1.27.0.

Stage 2: Record Backend And Profile Transformations

Scope: internal/configuration/load.go, focused normalization helpers if useful, and backend/profile loading tests.

Keep presence capture first in Resolve. Add the final-list eligibility check and in-place traversal for root and collection backends/profiles.
Replace detached backend interpretation with field records for base endpoint, API-key environment name, smoke reachability/authentication endpoints, and authentication kind. Preserve existing endpoint helper behavior, including diagnostic endpoints' distinct rules. Authentication kind trims/lowercases; environment names only trim.
Record the conditional queue assignment exactly as specified above. Do not manufacture absent smoke-test containers or fields.
Normalize profiles' nonempty endpoint overrides and supplied service-tier, reasoning-effort, and API-key environment strings. An exactly empty endpoint override remains distinct from whitespace which fails endpoint normalization.
Move endpoint failure checks into pure validation on effective fields. Temporarily remaining interpretation helpers must not repeat mutation or maintain a second backend/profile normalization policy.
Adapt TestLoadInterpretsBackendNormalizationAndNullDefault and the target case in interpretation_test.go: inspection now contains normalized values and defaulted capacity, while execution settings remain as before. Add exact field-operation assertions and one unchanged-field/no-new-event assertion.

Validation: focused go test ./internal/configuration, once the intermediate callback wiring supports it. Cover absent/null/zero capacity and missing/zero/ positive concurrency, optional endpoint overrides, and invalid endpoints.

Completion: backend and profile fields are normalized during resolution with field-level records and queue-default dependency evidence; endpoint validation is non-mutating. Focused configuration tests pass on Go 1.27.0.

Stage 3: Record Prompt Metadata And Configured Messages

Scope: prompt resolution helpers, prompts.go, and prompt loading tests.

Traverse root/collection prompt lists with the same eligibility handling. Move description/session-template trimming, input-name trimming, optional input content-type/description trimming, and optional output schema-path trimming from interpretPrompts into exact field records.
For configured messages, reuse domain.NormalizeMessageRole and domain.NormalizeCacheControl. Write back only changed supplied scalar fields at messages/<index>/role or cache_control/type and cache_control/ttl. Preserve required-field errors for absent role/type. Do not allocate an omitted cache control or TTL, lowercase cache-control types, trim content, or record an entire message/list merely to cover a scalar change.
If a message normalizer rejects a value, leave that field/group unchanged and let existing message validation/construction reject it with useful context. Keep shared domain checks where needed for validation; remove redundant configuration-specific assignment logic from materialization. Caller-appended messages continue using their runtime normalizers.
Keep content-file opening, template compilation, output-contract construction, duplicate normalized input-name checks, and hash construction in their existing owning paths. A trimmed schema path still addresses the selected schema source.
Adapt prompt interpretation tests to the new inspection view. Add focused cases for padded roles, trim-only cache-control spelling, optional/null input metadata, and normalized input-name collisions. Compare padded and canonical definitions with the same identities for equivalent materialized definitions and hashes; do not freeze a new hash encoding.

Validation: prompt/configuration tests plus existing domain-message tests. Invalid roles, cache-control values, and missing required message fields still fail; explicit empty inline content retains its existing behavior.

Completion: prompt metadata and configured message fields are normalized and recorded during resolution. Focused configuration and domain tests pass, including equivalent prompt hashes and retained input presence.

Stage 4: Use The Single Resolved Configuration

Scope: loader/target/materialization wiring and affected tests.

Change Validate to validate its supplied resolved document directly. Remove loaded.interpreted, interpretConfiguration, and obsolete detached interpretation helpers/copies after their transformations have moved.
Route backend inventory/index construction, execution-profile construction, and prompt materialization through state.result.Config. Keep the captured entry field-state maps and prompt metadata needed for runtime presence/null semantics. Do not derive those states from newly defaulted pointer values.
Preserve domain/runtime copying and public detachment. Keep the existing source-origin lookup for content_file; none of the selected transformations changes that field. No history scan or origin fallback is needed.
Remove or rename tests whose old assertion was preservation of authored whitespace in configuration inspection. Retain their meaningful execution, ownership, and rejection assertions rather than weakening coverage wholesale.
Check existing partial-profile completion, inherited endpoint/key settings, prepared execution, requirement metadata, and scoped inventories through the new single-configuration path.

Validation: go test ./internal/configuration ./internal/engine and the affected root API tests. Search for remaining interpreted-state consumers and confirm validation does not mutate the working configuration.

Completion: validation, inventories, execution profiles, and prompt construction use the single resolved configuration. The configuration, engine, and root API tests pass; no interpreted-state consumers remain.

Stage 5: Verify Provenance And Inspection Integration

Scope: focused configuration tests and existing public inspection/diff tests.

Add representative root and collection cases covering field normalization operations and queue-derived-default dependency evidence. Check operation, path, effective value, and relevant retained source evidence without freezing whole histories, reason wording, source ordering, or upstream internals.
Extend the inherited/imported content fixtures so a message role/cache-control edit occurs beside a content_file originating in another source directory or filesystem. Include later retained fields from a different contributor. Assert the expected file's captured content and unchanged effective origin; do not substitute a common directory that hides wrong-origin resolution.
Combine invalid/duplicate final resource identities with fields needing normalization/defaulting. Assert useful rejection and absence of an accidental unrecorded-mutation failure. Retain existing tests that replaced lower-layer invalid values are not independently validated.
Verify configuration diffs omit a value change when two otherwise identical definitions normalize to the same value, while genuinely changed list values still use ConfigKit's atomic list diff. Exercise the public passthrough rather than implementing PromptKit-specific diff equivalence.
Extend existing sensitive-provider inspection coverage to normalized records and explanations as relevant. Reasons must not interpolate configuration values. Keep exact input metadata presence distinctions and public detachment.
Reuse existing runtime/appended-message tests; add only missing boundary cases. Do not reproduce ConfigKit's generic identity-correspondence suite, add live provider tests, or introduce performance thresholds.

Validation: focused configuration, public inspection/diff, preparation, and execution tests. All new tests use deterministic offline sources/collaborators.

Stage 6: Align Documentation And Verify The Sprint

Scope: current contract/implementation documentation and final integration.

Update docs/formats.md, docs/consumers/pkg-promptkit.md, docs/consumers/configuration-migration.md, and docs/internal/sources.md where they describe authored/composed values excluding PromptKit normalization. Explain recorded effective normalization and defaults, retained provenance, normalized-value diff behavior, and the boundary with request completion.
Review affected public GoDoc and architecture wording against the new callback flow. Change only stale claims; no new public API, YAML version, normalizer policy, origin guarantee, or execution-readiness promise is needed.
Document the inspection/dependency change and its limits in the next release's docs/releases/v0.18.0.md, creating the note if needed under the release procedure. Keep canonical detail in the format/inspection guides. v0.17.0 has been published: preserve its note, including its historical two-view normalization description. Do not publish, retag, or edit published notes for this work.
Format changed Go files and run the integrated checks with the compatible local toolchain and prepared caches:
```
go test -race -vet=off ./...
go vet ./...
go run ./examples/go-library/prepare
go run ./examples/go-library/run
go run ./examples/go-library/observe
git diff --check
```
The examples' successful output semantics should remain unchanged. No release launcher/profile change is planned, so the separate Python release-wiring diagnostic need not be rerun merely for this refactor.
Review the full diff for presence loss, mutation outside resolution, broad list records, duplicate configuration state, stale docs, and unintended API changes. Record completion in the feature roadmap; retire it and its catalog pointer during normal completed-roadmap cleanup. Follow focused commit policy.

Completion means the one resolved configuration drives inspection and runtime construction; all selected transformations have field-level records; unchanged resource origins and runtime semantics are preserved; tests and documentation cover the intentional inspection change. There are no unresolved product decisions or upstream publication steps required by this plan.

16 KiB Raw Blame History