diff --git a/docs/internal/cli.md b/docs/internal/cli.md index 0a8bb947..a91aab0e 100644 --- a/docs/internal/cli.md +++ b/docs/internal/cli.md @@ -1,204 +1,157 @@ # CLI Internals -This document describes **internal/cli**, Notarius's production composition -root. The [CLI reference](../cli.md) owns command syntax and exit statuses; -[Configuration](../config.md) owns configuration values; and -[Operations](../operations.md) owns filesystem layout, recovery, and operator -procedures. +`internal/cli` is the production composition root. [CLI Reference](../cli.md) +owns syntax, reports, and outcome codes; [Configuration](../config.md) owns +settings; [Operations](../operations.md) owns physical state and recovery. +This guide explains declaration, loading, and orchestration owners, not another +public command manual. -## Inputs, Outputs, And Boundaries +## Native Application Construction -The CLI accepts process arguments, standard streams, and injectable options -used by tests and embedding code. It writes command results to the supplied -streams and returns a process exit status. For a run, it also creates the -production catalog and runtime collaborators, hands a prepared pipeline and -source bytes to the framework, and places the logical files returned by the -runner. +[NewApplication](../../internal/cli/application.go) returns a native +`*commandkit.App` from application identity and injectable options. Construction +declares commands without configuration discovery, engine creation, provider +calls, or run-storage allocation. The executable supplies build identity; +`RunWithOptions` is a convenience execution entry point, not a manual dispatcher. -It is the only boundary allowed to compose concrete registries, LLM clients, -cache/checkpoint collaborators, debug recorders, and physical output paths. -Pipeline modules receive interfaces and request data rather than CLI streams or -filesystem roots. The [Architecture](../policy/architecture.md) defines this -composition-root boundary; [Pipeline Internals](pipeline.md) owns resolution, -preparation, and runner mechanics after their inputs are supplied. +The constructor wires: -## Dispatch And Configuration Handoff +- native help/version and installed shell completion; +- application pipeline declarations and a native run alias; +- ckconfig whole-application and focused PromptKit scopes, sharing one loader; +- ckprompt resource/requirements inspection, concrete request validation, + prepared preview, namespace selection, and explicit smoke testing. -The root dispatcher handles help, version reporting, configuration validation, -effective-configuration inspection, explanation inspection, pipeline listing, -and a pipeline run. Version reporting resolves build -information through `internal/buildinfo` before production composition, so the -diagnostic remains available without configuration or runtime collaborators. -The public syntax, streams, exit classes, and version semantics are defined by -the [CLI reference](../cli.md). Other root commands normalize injectable -options before dispatch so that a missing production dependency fails as a -command error rather than reaching execution. +ckconfig adapters use the original application result and focused native subtree +view. ckprompt uses an engine loader through its standard builders rather than +application-owned copies of inspection DTOs. Ordinary resource inspection does +not become request preparation or an implicit probe. The +[native CLI reference][native-cli] owns parser/report mechanics. -Commands that need configuration use one shared ConfigKit-backed loader. The -CLI constructs an exclusive file source for an explicit flag or environment -selection, or a native required search source, then passes it to `LoadSource` -with the requested overlay and only explicit configuration-valued run overrides. -It receives a **LoadedConfig** and uses its retained main-source path for -invocation provenance without a second search. The -configured discovery and precedence contract is in -[Configuration](../config.md), while the loading and resolution mechanics are -in [Configuration Internals](configuration.md). +## Invocation-Local Loading -`config effective` and `config explain` render the loaded ConfigKit view by -default and do not resolve pipelines or construct providers. With explicit -`--scope promptkit`, they construct the PromptKit inspection engine from the -same captured source groups used by execution. Effective output is PromptKit's -redacted YAML; explanation output shows its source history. `config profile` -uses that engine to emit one neutral Notarius-owned JSON summary for the exact -requested profile ID. These inspection commands load no run pipeline, prepare -no request, resolve no credential value, and make no provider call. Ordinary -prompt and provider-parameter content is not generally secret-free even though -credential-shaped provider parameter keys are redacted. +[nativeLoaders](../../internal/cli/configkit_load.go) belongs to one invocation. +It selects the main source and assembles one layered ConfigKit application +result. Engine construction calls `promptkit.NewFromConfig` on that result, +with runtime-only client options; it does not load YAML again. -Configuration validation without a selected pipeline checks structural -configuration only. Validation with a selected pipeline also builds the -effective catalog, resolves the pipeline, and verifies every explicit effective -PromptKit profile. Selected LLM-backed input, chunk, lane, output, and validator -profiles are inspected against the configured PromptKit source and backend -registrations without preparing a request or performing generation. Engine -construction validates effective prompt definitions and captures message content; -an unknown or invalid profile fails before pipeline preparation. -Credential availability remains an execution-time concern. Pipeline listing -validates configuration before returning normalized, sorted identifiers. +The retained selection includes supplied-value state, overlay, and override +values. Repeated identical selections share a result/engine, while distinct +comparison selections cannot be memoized into one side. There is no +process-global loader cache. The selected main's native context anchors companion +imports; source edits do not mutate an existing captured engine. -## Production Composition +[Configuration Internals](configuration.md) owns definition, normalization, +origins, and conversion. Shared loaders allocate no output/cache/debug data. -The production composition helper allocates every framework registry and the -prompt-asset registry, then registers the generic, Seriatim, and D&D module -families in that order. The resulting registries provide both the module -catalog used for resolution and the concrete constructors used for preparation. -Tests may provide a catalog or registries instead; production code must not -silently merge an injected partial catalog with production registrations. +## Catalog And Preflight Boundaries -The production LLM factory builds one PromptKit-backed client from the native -source groups assembled from PromptKit catalogs, registered application prompt -and fallback assets, the optional operator file or recursively selected -directory, and the optional local backend registration. The same construction -path serves runtime, preflight, and configuration inspection. It attaches the -profile-provenance recorder, captures one immutable PromptKit configuration -identity, creates one scheduler from the effective global LLM limit, and wraps -the client before it reaches modules. Registration and LLM construction errors -are returned before a pipeline is prepared. Configuration fields remain in -[Configuration](../config.md#promptkit-profiles); source ownership, capture, -identity, and adapter mechanics remain in [LLM Runtime](llm.md). +[catalog.go](../../internal/cli/catalog.go) composes generic, Seriatim, and D&D +registrars into typed registries, codecs, evidence projectors, and validator +defaults. Registration does not build an asset/schema registry or an LLM service. +Injected registries/catalogs are alternative sources, not implicitly augmented +with production entries. -The factory also accepts `LLMRuntimeOverrides`, whose reasoning pointer -preserves inherit, replace, and clear states across the composition boundary. -Run orchestration constructs this value from the mutually exclusive -`--reasoning-effort` and `--clear-reasoning-effort` controls. Absence preserves -a nil pointer, replacement is trimmed, and clear uses a non-nil empty string. -The same override reaches the one shared production client, checkpoint -identity, and debug invocation metadata. Persistent reasoning configuration -remains owned by PromptKit profiles; Notarius configuration has no reasoning -field. +[pipeline_commands.go](../../internal/cli/pipeline_commands.go) separates: + +- listing: enumerate captured configured IDs without a catalog/engine; +- showing: domain resolution and composition description, without profile expansion; +- validating: domain resolution plus known-resource preflight, supplying the + same captured engine only when model roles require it. + +[Preflight](../../internal/preflight/preflight.go) accepts collaborators/data; +it does not load configuration, register/construct modules, read execution +inputs, render CLI reports, or allocate state. Partial profiles pass native +inspection, with concrete request readiness deferred until preparation. + +## Run Inputs And Native Overrides + +[run_input.go](../../internal/cli/run_input.go) binds typed application inputs +and repeated literal selections. [execution_overrides.go](../../internal/cli/execution_overrides.go) +uses reusable ckprompt resource/execution inputs, preserving omission versus +explicit empty/false/zero. Only profile/session and execution controls needed +by pipeline runs are selected; prompt inputs stay module-owned. + +The resolved run-wide profile selection reaches every LLM-backed role. Native +execution overrides travel through operation requests to producers, validators, +and continuation, without mutating loaded settings. The +[public reference](../cli.md#run-wide-llm-overrides) owns exact spellings/semantics. + +Reference selectors are expanded at the CLI boundary using registered target +specifications. Pipeline/lane actions become exact binding changes after +specificity/conflict handling. Generic resolution receives exact reference +bindings/unbinds, not CLI selector grammar. External/generated conflicts and +required-slot/type checks remain with domain resolution; materialization follows. + +[session.go](../../internal/cli/session.go) resolves the effective session from +explicit supplied input or the input-module/raw-byte policy before runner work. +Framework/model code transports it rather than deriving another session. ## Run Orchestration -After parsing and validating a run invocation, the CLI performs this ordered -handoff: +[pipeline_run.go](../../internal/cli/pipeline_run.go) performs the ordered handoff: -1. load and validate configuration with ConfigKit command overrides; -2. create and validate a safe run identity, then allocate a debug bundle only - when requested; -3. build the effective catalog, resolve requested reference changes, resolve - the effective pipeline, and inspect its explicit effective PromptKit - profiles; -4. materialize external or generated references and record redacted invocation - and resolution provenance when debug capture is enabled; -5. construct registries, the scheduled LLM client, and prepared modules; -6. read the source input once, resolve its effective session from the explicit - override or resolved input module and raw bytes, then construct requested - checkpoint collaborators and invoke the framework runner with that same - value; and -7. write the runner's logical output files only after a successful run, then - complete the command report and user-facing result. +1. Load/validate the selected application result and derive runtime configuration. +2. Generate a safe run identity and allocate requested capture. +3. Capture selected configuration, compose registries, expand reference actions, + and resolve the effective pipeline. +4. Obtain the shared engine when needed and perform offline preflight. +5. Materialize references, construct prepared typed modules, read input bytes, + and resolve the effective session. +6. Construct the accepted-result cache and optional recorder collaborator, + create one native execution group, and invoke the runner with independent + worker and execution selections. +7. Report process diagnostics, publish returned logical files only after runner + success, finalize requested capture, then submit the native completion report. -Preparation happens before source parsing, so module construction and -dependency failures cannot begin stage execution. The CLI also preserves the -framework's result and warning information when it writes summaries and the -final command result. Detailed state lifecycle, resume handling, and physical -path confinement are maintained in [Run State Internals](state.md) and -[Operations](../operations.md). +Modules receive the shared engine, operation data, and explicit collaborators, +not physical roots or CommandKit reporters. Detailed worker, assessment, +eligibility, and cache algorithms belong in [Pipeline Internals](pipeline.md); +physical publication/capture collaborators belong in [Run State Internals](state.md). -The CLI owns the versioned generated-session policy and resolves the sole -effective value before checkpoint construction. It records that value in the -final debug invocation summary when capture is enabled and passes it unchanged -to checkpoint identity and `pipeline.RunInput`. The public flag and stability -contract are defined by the [CLI reference](../cli.md#run); framework and LLM -packages only transport the supplied value. +Requested capture allocates before engine/preflight and finalizes once. +Primary operation causes and additional capture failures remain available. +Later report delivery failure does not rerun model work, remove valid artifacts, +or terminalize capture again. -The CLI also owns the scope grammar for reference flags. It enumerates the -selected chunk and lane targets from registered module specifications, expands -pipeline- and lane-scoped actions into exact stage-and-lane bindings, and -resolves overlapping bind and unbind actions by specificity before calling -configuration resolution. The generic pipeline therefore receives only exact -`ReferenceBinding` and `ReferenceUnbind` values and has no knowledge of CLI -selector syntax. Configuration resolution retains ownership of configured -external/generated conflicts, required slots, and module compatibility; file -materialization still occurs afterward. +## Native Reporting And Outcomes -For `run --json`, the CLI constructs and encodes its private run-result receipt -after a successful runner result is available, before it publishes logical -output files. It writes the prepared receipt to standard output only after -output publication and requested debug terminalization succeed. A receipt-write -failure exits with runtime status 1 and may leave partial standard-output bytes, -but the already-published output bundle remains complete and requested debug -reporting remains successfully terminalized. The CLI reports a bounded -command-owned error and does not repeat terminal reporting. The receipt remains -a CLI reporting concern rather than a framework or output-module responsibility; -its public contract is the -[native completion report](../cli.md#completion-report). +[run_result.go](../../internal/cli/run_result.go) submits one small application +value with a native text template. Native serializers render that same value. +Domain files, not a persisted report or ledger, are the deliverables. +Fallback qualification is independent of optional diagnostic visibility. -## Failure Mapping And Terminal Reporting +Warnings/advisories keep their domain classification and use the reporter's +diagnostic channel. Outcome helpers retain native configuration/preflight causes +and distinguish identifiable invalid input from operational/mixed failures. +CommandKit retains rendering/output/diagnostic delivery evidence separately from +the operation cause; quiet mode does not mask such failures. -Argument, flag, and invocation-combination failures are reported to standard -error before runtime composition and use the syntax error class. Once an -invocation is syntactically valid, configuration loading and validation, -resolution, registration, profile checks, reference materialization, module -construction, input reads, runner failures, output publication, and requested -debug handling use the runtime failure class. The public status numbers and -stream contract are defined in the [CLI reference](../cli.md#output-streams-and-exit-statuses). +The [completion report](../cli.md#completion-report) owns fields and +[stream/outcome contract](../cli.md#output-streams-and-exit-statuses) owns process +behavior. There is no universal Notarius envelope or parallel serializer. -When debug capture has been allocated, one command-state value records the -known run result. Guarded terminalization writes a success report once, or -attempts a failure report and error record once. A persistence failure is -reported in addition to the original failure and never replaces it. If a debug -path exists, failure output includes that path so the retained diagnostic data -is discoverable. +## Focused Verification -## Invariants To Preserve +- [Application tests](../../internal/cli/application_test.go) cover lazy built-ins, + shared scopes, selection evidence, and native diff outcomes. +- [Loader tests](../../internal/cli/native_load_test.go) and + [schema tests](../../internal/cli/native_schema_load_test.go) cover native + composition, captured contexts, and operator contracts. +- [Pipeline command tests](../../internal/cli/pipeline_commands_test.go) cover + listing/showing/preflight without execution inputs or state. +- [Override tests](../../internal/cli/execution_overrides_test.go) and + [run tests](../../internal/cli/pipeline_run_test.go) cover supplied values, + routing, references, sessions, and run ordering. +- [Reference tests](../../internal/cli/reference_contract_test.go) cover selector + specificity, generated conflicts, and materialization boundaries. +- [Native executable tests](../../internal/cli/native_process_test.go) cover the + real command routes and fake-provider presentations. +- [Completion tests](../../internal/cli/run_result_command_test.go) and + [recovery contracts](../../internal/cli/recovery_contract_test.go) cover retained + publication, capture/delivery failures, reuse, fallback, and native facts. -- Only the CLI composes production implementations and physical runtime roots. -- Configuration and resolved composition failures occur before module - preparation or source parsing. -- A runner's logical files are published only after a successful run. -- Production registries and a caller-supplied catalog or registries are - alternative composition sources, not an implicit mixture. -- A requested debug bundle has one terminal report attempt; its persistence - errors supplement rather than obscure the primary command error. -- User-facing flags, paths, exit codes, and configuration fields are defined - by their public documentation, not duplicated here. +Use focused CLI tests and pair with configuration/preflight tests when those +boundaries change. [Development](../development.md#validation) owns wider checks. -## Focused Tests - -- **internal/cli/command_contract_test.go** covers dispatch, help, syntax and - runtime error classes, discovery, validation, and listing. -- **internal/cli/run_contract_test.go** covers the run handoff, publication, - debug reporting, and command-owned state collaborators. -- **internal/cli/production_contract_test.go** covers registrar composition, - production catalog contents, assets, and representative configuration - validation. -- **internal/cli/reference_contract_test.go** covers scoped CLI reference - expansion, specificity, bind/unbind conflicts, generated-reference - protection, origin separation, and materialization boundaries. -- **internal/cli/state_hardening_test.go** covers safe run identity, state - roots, and failure ordering. - -Run **go test ./internal/cli** after changing command composition or command -behavior. Pair it with **go test ./internal/core/config** when the configuration -handoff changes. +[native-cli]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md diff --git a/docs/internal/configuration.md b/docs/internal/configuration.md index a92a3c76..5ea4b007 100644 --- a/docs/internal/configuration.md +++ b/docs/internal/configuration.md @@ -1,161 +1,148 @@ # Configuration Internals -This document describes the maintainer-facing configuration boundary in -**internal/core/config**. The [Configuration](../config.md) reference owns the -file format, fields, defaults, precedence contract, and selectable keys. The -[CLI reference](../cli.md) owns command syntax; this document does not redefine -either interface. +This guide explains the maintained configuration boundary and its composition-root +handoff. [Configuration](../config.md) owns source selection, syntax, fields, +defaults, and selectable modules; [CLI Reference](../cli.md) owns invocation. +[Architecture](../policy/architecture.md#configuration-and-provenance) defines +ownership. Do not infer operator contracts from private helper choices here. -## Boundary +## Owners And Handoffs -The configuration package turns a selected ConfigKit-composed YAML document -into a validated, independently owned configuration. It then resolves a -requested pipeline against a module catalog before the framework prepares or -runs anything. +| Owner | Responsibility | +| --- | --- | +| [core/config definition](../../internal/core/config/load.go) | Typed application document, native PromptKit configuration rules, application resolution/validation. | +| [CLI loaders](../../internal/cli/configkit_load.go) | Main-source selection, native underlays, invocation-local result/engine reuse. | +| [LoadedConfig](../../internal/core/config/loaded_config.go) | Retain the original result and derive independently owned runtime pipeline configuration. | +| [Config.Resolve](../../internal/core/config/effective_config.go) | Resolve a selected pipeline against the supplied catalog. | +| [preflight](../../internal/preflight/preflight.go) | Check known native resources from resolved composition and supplied engine. | +| Pipeline preparation | Construct typed execution collaborators from resolved, materialized inputs; see [Pipeline Internals](pipeline.md). | -| Boundary | Inputs | Outputs | Does not own | -| --- | --- | --- | --- | -| Loading | Selected ConfigKit source, overlay, overrides, and invocation directory | **LoadedConfig** with ConfigKit inspection metadata and a populated **Config** | Choosing the source or reporting a command result. | -| Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. | -| Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. | -| Inspection | **LoadedConfig** and an optional JSON Pointer | Redacted effective YAML or explanation text | Pipeline selection, module preparation, or provider work. | +The production flow keeps one application-owned +`*configkit.Result[config.Document]`. That result feeds runtime conversion, +whole-app/focused inspection, and `promptkit.NewFromConfig`. Rendered YAML is +an operator view, not an interchange format for constructing the engine. -The CLI describes the selected source, invokes this package, and supplies the -result to the framework. Configuration never reads an input file, constructs a -module, or creates output, cache, or debug paths. Those responsibilities remain -at their respective [CLI](cli.md), [pipeline](pipeline.md), and -[run-state](state.md) boundaries. +## Definition And Application Meaning -## Loading And Validation +`Document` contains native `promptkit.Settings` at its PromptKit field. +`Definition` composes application defaults, version policy, sensitivity, +identified step lists, resolution, and validation with `promptkit.Configure`. +Native resource rules therefore participate in the same load rather than being +validated by a separate application parser. -`Load` selects an exclusive file; `LoadSource` accepts the CLI's source and -enables imports through the same definition and callback path. ConfigKit composes -the main input, imports, and optional overlay, then expands native inheritance -before typed decoding. Explicit CLI assignments follow decoding and precede -application resolution. The document definition supplies typed -defaults and structural validation; Notarius's recorded resolution changes -normalize configured paths after decode. Reference paths resolve before key -normalization moves their source coordinates. Ordinary list leaf edits are -recorded at the leaf; a changed step identity requires a covering list record. -Actual map-key moves retain structural records. Path normalization follows the source -of the surviving value, including a value copied through inheritance, rather -than the inheritance directive's location. Inherited input remains explicit -for presence-sensitive behavior, including zero, false, and empty values; -existing field validation still governs whether the resulting values are valid. -ConfigKit's inspection history retains inheritance transfers alongside later -normalization, and effective configuration renders the normalized values used -by runtime conversion. The private typed document consumes the directive, so -no upstream inheritance type or unresolved directive reaches runtime pipeline -or module contracts. See [Configuration](../config.md#sibling-inheritance) -for the operator syntax and field rules. Only -`NOTARIUS_CONFIG` participates in discovery; retired field-level environment -variables do not participate in loading. Version 5 decoding rejects scalar -module bindings and scalar external references. ConfigKit retains redacted, -historical inspection metadata while **LoadedConfig** keeps that implementation -detail private behind effective and explanation writers. `MainSourcePath` reads -the non-imported main file from retained sources, and runtime conversion stays detached. +Application callbacks normalize pipeline identifiers and operational paths while +retaining presence/source history. The per-load state tracks original coordinates +as map keys move. Paths are resolved before normalization loses their source +coordinates. Inherited values retain their contributing origins; CLI path +assignments use invocation origin. See +[document paths](../../internal/core/config/document_paths.go) and +[pipeline document normalization](../../internal/core/config/pipeline_document.go). -**Config.Validate** checks configuration-only invariants before resolution. It -rejects incompatible profile sources, invalid state-surface values, unsupported -concurrency settings, malformed bindings and references, invalid retries, and -invalid pipeline, step, or lane structure. PromptKit local-backend validation -accepts only an absolute HTTP or HTTPS endpoint with a host and no user -information, query, or fragment, and rejects a negative local concurrency -limit. It also rejects a negative `promptkit.transient_retries` value; decoded -omission resolves to two, while explicit zero remains a distinct opt-out. Its -errors retain the closest known pipeline, lane, and binding context. -It deliberately does not require modules to be registered: that requires a -catalog and belongs to resolution. +ConfigKit owns parsing, imports, overlays, inheritance, typed composition, and +provenance. The application supplies meaning through supported rules. It does +not implement another inheritance resolver or translate retired inputs. +[The public reference](../config.md#imports-and-sibling-inheritance) owns exact +presence/null/list rules and processing order. -The exact user-selectable values and validation rules are defined in -[Configuration](../config.md). Keep document schema, source-aware normalization, -runtime conversion, and that reference aligned in the same change. +`Config.Validate` checks application-only state and pipeline structure without +requiring registered modules. Catalog-dependent compatibility belongs to +resolution. Configuration does not parse a source transcript, materialize +references, construct modules, or allocate run storage. -## Effective Resolution +## Production Source Assembly -**Config.Resolve** derives an absent extraction-worker value in an owned copy -and validates the configuration. A programmatically supplied worker value is -always explicit. It normalizes the requested pipeline ID, copies the selected -profile, and passes the non-empty command-level LLM profile override, requested -lane selection, and reference changes to the framework resolver. +`nativeLoaders.application` selects the main source and uses +`configkit.LoadLayered` with the application definition, native underlays, +selected overlay, and explicit overrides. Underlays come from bundled provider +resources, ordinary embedded application content, and main-relative operator +companions. Native source/filesystem contexts survive the load. -After module and validator selection, the resolver applies the effective -profile policy to LLM-backed bindings only: command override, binding profile, -pipeline profile, then the prompt default. Deterministic bindings remain -profile-free, and no second inheritance decision occurs during execution. The -public field definitions and precedence are owned by -[Configuration](../config.md#pipelines). +The embedded assets package supplies content, not registration behavior. +The operator companion uses a native main-relative context; it follows the +selected main rather than rediscovering the working directory. Strict peers, +optional absence, imports/destinations, and whole-record resource replacement +remain native composition. Source grammar and precedence belong in +[Configuration](../config.md#configuration-discovery-and-precedence). -The resolver retains configured `validation_policy` overrides and derives one -detached concrete terminal policy for the chunk producer and every lane's -extract, merge, and normalize producers. That field-by-field inheritance is -complete before preparation, and the effective values contribute to pipeline -and checkpoint identity; execution does not interpret configuration defaults. +Core `Load`/`LoadSource` entry points support selected sources and the same +definition/conversion for direct consumers and tests. They do not replace the +production composition root's underlay assembly. Do not accidentally introduce +a second production load by combining those paths. -The framework resolver supplies defaults, selects lanes, resolves validator -chains, checks registered module and artifact compatibility, validates module -options, and returns the fixed ordered pipeline shape. Positive validator retry -budgets require an LLM-backed selected validator; deterministic validators are -rejected during resolution. Eligible LLM-backed producer specifications also -contribute their declared correction protocol to the resolved metadata. The -resulting **EffectiveConfig** retains the selected ID, requested selection and reference -changes, a clone of the input configuration, and the resolved pipeline. -Callers may therefore retain or modify their input slices and maps without -changing the resolved result, and later consumers cannot mutate the original -configuration through the effective value. This ownership includes the nested -PromptKit local-backend value. +## Result Lifetime And Runtime Conversion -Resolution failures stop before module construction and source parsing. They -include an error path for an unconfigured pipeline, missing module, missing -capability, incompatible artifact variant, invalid option, invalid reference, -or invalid lane selection. CLI code maps these valid-invocation failures to the -runtime error class described in the [CLI reference](../cli.md#output-streams-and-exit-statuses). +`LoadedConfig.Result()` exposes the original typed result for intentional native +handoff. `FromResult` derives pipeline runtime settings without loading sources +or serializing settings again. `RuntimeConfig` returns a detached copy; +the native PromptKit configuration stays with its owning result/engine rather +than being copied into a private runtime wrapper. -## Resolved Identity And Redaction +`MainSourcePath` reads the retained non-imported main-file evidence. It does not +repeat discovery or assume the first acquired file is the main. -The framework assigns the resolved pipeline a deterministic SHA-256 digest -after defaults, lane selection, module bindings, reference bindings, validator -chains, selected correction protocols, effective LLM profiles, and artifact -schema identity have been resolved. The digest excludes -its own stored value. It identifies resolved composition rather than raw YAML -bytes, a debug payload, or all runtime state. The CLI records it as invocation -provenance before execution; cache and checkpoint identity have additional -owners in [Run State Internals](state.md). +Within one invocation, `nativeLoaders` retains results by complete +`ckconfig.Selection`, including overlay and override presence/values. Captured +override containers are copied. A repeated identical selection returns the +same result; distinct diff sides remain distinct selections. The engine is +constructed lazily once for its retained result. This is invocation state, +not a process-wide persistent loader cache. -Configuration publication uses ConfigKit's redacted effective YAML. Resolved -pipeline publication uses **RedactedResolvedPipelinePayload**, which copies -bindings and nested option containers, replaces credential-shaped values with -**[REDACTED]**, and omits materialized reference content while retaining safe -provenance. This redaction is deliberately narrow and does not authorize -recording arbitrary environment values or provider requests. +`NewFromConfig` consumes the completed result, preserving native contexts and +inspection evidence without rerunning callbacks or adding underlays. Runtime-only +HTTP/client settings and test provider options stay at engine construction. +[PromptKit's application-owned-load contract][handoff] owns the library API. -## Invariants To Preserve +## Effective Resolution And Preflight -- ConfigKit composition applies defaults, imports, an optional overlay, and - explicit overrides in one direction with retained origin metadata. -- A configuration is structurally valid before it is resolved, and a resolved - pipeline is compatible with the supplied catalog before preparation begins. -- Whitespace-normalized identifiers are unique wherever they identify a - pipeline, step, lane, worker, or reference slot. -- Runtime conversion and resolved-pipeline publication return detached data. -- The resolved PromptKit transient-retry budget is always passed to the shared - built-in client; document presence preserves an explicit zero. -- The resolved digest changes when resolved composition changes and never - includes itself. +`Config.Resolve` clones configuration, resolves an empty programmatic cache-root +selection, validates structure, and passes selected pipeline/lanes, profile/cache +overrides, and exact reference changes to `pipeline.ResolvePipeline`. +The framework resolves module/default/validator/type/reference compatibility and +concrete producer controls before execution. Input slices/maps and the original +configuration do not become mutable aliases of the effective result. -## Focused Tests +Presence-aware selections preserve explicit invocation profile clearing, cache +overrides, and zero semantic-turn controls rather than collapsing into omission. +Deterministic bindings remain +separate from LLM roles. A resolved-composition digest is not an accepted-artifact +cache key; [state ownership](state.md) is a different boundary. -- **internal/core/config/document_test.go** and - **operational_document_test.go** cover ConfigKit document loading, - normalization, imports, overlays, and overrides. -- **internal/core/config/validation_contract_test.go** covers configuration - invariants and contextual failures. -- **internal/core/config/effective_config_contract_test.go** covers defaults, - selections, overrides, resolution context, digest changes, and ownership. -- **internal/core/config/loaded_config_test.go** covers safe effective and - explanation rendering; **redaction_test.go** covers resolved-pipeline - redaction and reference-content exclusion. +Preflight consumes resolved data, the catalog, and the captured engine when model +roles are selected. It checks static initial/correction/validator dependencies and +applicable explicit/default profiles. It does not load configuration, construct +modules, read reference/input bytes, or manufacture a request to prove readiness. +Partial profiles remain valid; dynamic request selections are runtime checks. -Run **go test ./internal/core/config** after changing this boundary. Changes to -the handoff or resolved-composition semantics also need the focused framework -pipeline tests. +## Inspection And Sensitive Values + +Native whole-app and focused views use retained ConfigKit evidence. Existing +effective/explanation writers delegate to that result rather than walking YAML. +Application sensitivity rules and native resource sensitivity participate in the +same definition. Source/runtime option containers must not be independently +reserialized as an alternative secret-handling mechanism. + +Do not infer that arbitrary source, prompt, provider, or module-option strings +are safe to disclose. Capture uses native inspection and exact marked-secret +paths, not heuristic application-text rewriting. See [CLI composition](cli.md) +and [LLM integration](llm.md) for those consumers. + +## Focused Verification + +- [Document/native settings tests](../../internal/core/config/native_settings_test.go) + prove typed handoff, native rules, source-relative messages, and secret-aware views. +- [Path tests](../../internal/core/config/document_paths_test.go) and + [inheritance tests](../../internal/core/config/inheritance_test.go) cover origins, + explicit presence, overlays, and source/runtime ownership. +- [Control tests](../../internal/core/config/validation_policy_contract_test.go) + cover loaded omission/zero and resolved controls. +- [Effective resolution tests](../../internal/core/config/effective_config_contract_test.go) + cover catalog checks, selection, and detached results. +- [CLI loader tests](../../internal/cli/native_load_test.go) cover actual production + underlays, strict companions, imports, frozen messages, and distinct selections. + [Schema-load tests](../../internal/cli/native_schema_load_test.go) exercise native + operator schema preparation/execution. + +Use focused configuration and CLI tests when changing this handoff. The +[development guide](../development.md#validation) owns repository-wide checks. + +[handoff]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/pkg-promptkit.md#reuse-an-application-owned-load diff --git a/docs/internal/llm.md b/docs/internal/llm.md index 66fa74a6..be2dc4ff 100644 --- a/docs/internal/llm.md +++ b/docs/internal/llm.md @@ -1,363 +1,186 @@ -# LLM Runtime Internals +# Native LLM Integration Internals -`internal/framework/llm` is Notarius’s provider-independent structured -completion boundary. It adapts framework requests to PromptKit, bounds -provider calls, assembles registered prompt and schema assets, records selected -profiles, and redacts provider errors. The architectural boundary is defined in -[Architecture](../policy/architecture.md#llm-boundary); profile sources, -credentials, and concurrency settings belong in -[Configuration](../config.md#promptkit-profiles) and -[Configuration](../config.md#concurrency-output-cache-and-debug). +Model-executing code uses native PromptKit types directly. This guide explains +Notarius's integration points, not a private completion service or a copied +library manual. [Architecture](../policy/architecture.md#native-execution-boundary) +defines the boundary; [Configuration](../config.md#promptkit-profiles) owns +resources/settings; [CLI Reference](../cli.md#run-wide-llm-overrides) owns request +controls; [Operations](../operations.md) owns state and handling. -## Structured Completion Boundary +## Construction And Ownership -Modules and LLM-backed validators depend only on -`contracts.StructuredLLMClient`. A completion request supplies a prompt ID and -version, optional profile and session IDs, named input material, variables, and -a caller-owned decode target. The successful response returns the validated raw -structured bytes together with non-secret provider, model, profile, and token -metadata. +The [CLI loader](../../internal/cli/configkit_load.go) hands the original completed +ConfigKit application result to `promptkit.NewFromConfig`. One invocation shares +the resulting engine across initial producers, corrections, normalization, and +LLM validators. Runtime-only HTTP/client settings and an injected native +`promptkit.LLMClient` remain construction options, not application YAML. +There is no provider-neutral completion conversion or second engine per role. -The caller owns the domain behavior: it chooses the prompt, prepares inputs, -selects the private response schema, and interprets the decoded result. The -adapter does not own source evidence, artifact conversion, normalization, or -durable schemas. Those responsibilities remain with the module and its -[integration contract](../integrations/). +[Configuration Internals](configuration.md) owns native definition/source assembly +and invocation selection reuse. PromptKit owns captured resources, request +preparation, native validation/repair, provider transport, conversation state, +and backend admission. See the [upstream Go consumer contract][consumer] for those +APIs rather than reproducing their mechanics here. -The calling module also resolves contextual entity selections and attaches any -application identity; PromptKit and this adapter do not own entity identity. +Application contracts remain typed production, candidate assessment, supported +fallback, and artifact encoding. Pure domain models, codecs, source resolution, +and deterministic transforms do not need PromptKit merely because a surrounding +operation invokes a model. -`PromptKitClient` validates the request target and prompt identity, maps each -named material to a PromptKit inline artifact while preserving its origin URI, -passes the supplied request session through to PromptKit's direct per-run -session field, retains the same value as the `session_id` prompt variable for -maintained prompt compatibility, and forwards profile selection. It does not -derive or replace session values; the CLI owns that policy. It then creates one -frozen prepared execution, captures its caller-owned credential-redacted -details for debug material, and executes that exact snapshot through -PromptKit's prepared-execution boundary. The direct field -is authoritative for provider session behavior. A session ID is a stable, -non-secret correlation identifier and may be exposed to providers and provider -observability. The adapter returns PromptKit’s validated raw bytes rather than -re-encoding the decoded target. An empty optional material is represented as -one space so its named input is retained by PromptKit. +## Requests And Fixed Operation Inputs -When a request includes semantic correction material, the adapter validates and -defensively copies it before preparation, then appends exactly two messages -after the ordinarily rendered prompt: the prior response as an assistant -message and the correction guidance as a user message. Requests without a -correction do not add messages or introduce caller roles. Ordinary request -summaries record correction byte counts and digests only; complete messages are -available solely in an explicitly requested debug trace. +Model modules receive the shared engine during typed preparation. A module +prepares transcript, references, contextual identity projections, and any +normalization candidate mapping once. It constructs a `promptkit.RunRequest` +with its selected prompt, profile/session, native artifact inputs, and variables. +Response interpretation consumes `*promptkit.RunResult` and produces the +application's typed candidate. -The adapter leaves the ordinary rendered message prefix, named inputs, -variables, session, profile, execution overrides, prepared-execution path, and -PromptKit repair policy unchanged for a corrected request. It never imports a -PromptKit message type into a module or pipeline contract. PromptKit reports -actual structural repair count and cumulative token usage per completion; the -pipeline's safe terminal debug record projects those values without copying -message content. +[Spell extraction](../../internal/modules/dnd/extract/spells/extractor.go) is a +representative path: fixed transcript/catalog/NPC projections feed an initial +request; native output is decoded and canonically mapped into its durable type. +The correction builder selects a paired prompt and supplies current guidance, +not another copy of original source/reference material. -Client construction may also receive a run-wide reasoning-effort override from -the CLI factory boundary. The adapter copies the caller-owned pointer and -creates a fresh PromptKit execution override for each request: a nil pointer -inherits the selected profile, a non-empty value replaces it, and an empty -value clears inherited reasoning. The CLI's mutually exclusive -`--reasoning-effort` and `--clear-reasoning-effort` controls select those -states. With neither flag, profile behavior remains unchanged. Because -production constructs one shared client, the selected state applies uniformly -to module calls, retries, and LLM-backed validators for the whole run. +Domain identity stays deterministic. Contextual selectors and request-local +normalization handles resolve against fixed supplied records; models do not +acquire authority over durable IDs, citation meaning, or typed artifact shape. +[Module Internals](modules.md) and [D&D Internals](dnd.md) own extension/domain +recipes. -An empty request profile lets the prompt select its configured default. Before a -run begins, the CLI asks the adapter to inspect every explicit profile on the -resolved selected LLM-backed bindings and validators, including inherited -pipeline profiles. Inspection resolves the profile and its selected backend and -target without preparing a request, reading credentials, admitting capacity, or -contacting a provider. Engine construction validates effective prompt definitions -and captures message files and static templates. A missing or invalid profile fails before -stage execution while a valid `api_key_env` may remain unset. Calls record the -profile actually selected by PromptKit. The recorder trims and deduplicates -non-secret profile identity, provider, model, selected backend ID, and -effective reasoning values for manifest use. Entries that differ in backend or -reasoning remain distinct and deterministically ordered. Endpoint-only profiles -retain an empty backend ID, which the published JSON omits. Successful -completion responses and recorded profile manifests identify the adapter -provider as `promptkit`. +Native request overrides preserve supplied values without rewriting configured +profiles. The producer helper's request composition preserves operation-owned +prompt/input/capture fields and existing callbacks; validators pass native +execution options explicitly on their own requests. -The CLI's profile-inspection engine and production adapter use the same source -assembly: PromptKit catalogs, one application group containing registered -prompts and fallback profiles, the selected operator file or directory, and -the optional final `local` backend registration. Execution, preflight, and -inspection therefore capture the same effective definitions. PromptKit and -ConfigKit compose recursively across groups while retaining each input's file -context; inputs within a group compose strictly. Operator imports are enabled -for the selected operator input, and imports retain their declaring file's -relative resource roots. PromptKit validates final effective definitions at -engine construction. It does not run a separate audit that rejects a malformed -definition superseded by a later overlay. The D&D registrar contributes its -`dnd-extraction` fallback, and maintained D&D prompts select that logical ID -by default. A profile selecting `backend: local` can use an operator -`backends.local` declaration or the optional Notarius registration; see -[native source rules](../config.md#native-promptkit-sources-and-migration). +## Native Production And Application Assessment -This PromptKit integration uses the old-path ConfigKit v0.5.0 types exposed by -PromptKit v0.12.0. Application configuration uses the distinct new-path -ConfigKit package; those named types do not cross the adapter boundary. +[RunProducer](../../internal/framework/pipeline/producer.go) connects ordinary +module interpretation/correction/fallback functions to the pipeline assessor. +It uses native `Run`, or retained prepared execution on the cache path, then +checks errors and public validation status before interpretation. Failed native +validation can accompany nil Go error and is not a usable semantic candidate. -Each engine captures its definitions at construction. A later source-file edit -does not mutate an existing engine; constructing a new engine captures the -updated view. Before execution, the adapter computes one immutable, -versioned `promptkit_configuration` checkpoint identity from public captured -inspection APIs. The private digest covers captured effective configuration, -all effective profile targets and credential-policy metadata (including -provider parameters), and every effective prompt ID/version and captured -content hash. The identity also includes PromptKit and the selected OpenRouter -and Rakestrawhome catalog versions. Local-backend concurrency settings remain -excluded because they affect scheduling; its endpoint is represented only in -the private digest, never as checkpoint metadata. Neither the private digest -payload, raw sources, paths, endpoints, provider parameters, nor prompt bodies -are published. An existing engine retains its identity after files change; a -new engine gets the identity of its newly captured definitions. This replaces -the former live-file profile and local-endpoint fingerprints. Run manifests -remain independent and list only profile choices actually observed during LLM -calls. +Every usable initial/revised candidate receives the full applicable configured +assessment chain once, including unchanged candidates and the final permitted +response. Native coordination owns history and additional-turn iteration. +Its initial callback uses already collected findings instead of reassessing the +initial result. Module templates own correction assignments; detecting modules/ +validators own actionable guidance, separately from diagnostics. -## Shared Provider-Call Limit +Native execution success is not domain acceptance. Stopped/exhausted outcomes +are control facts. Result-less failed continuation does not recover an earlier +candidate, and an undelivered response is not assessed or accepted. Caller +cancellation stops without fallback. Native timeout/provider/capacity failures +remain distinguishable through context state and typed causes. -Production construction creates one PromptKit client and wraps it in one -scheduled client. The scheduler has a fixed, positive permit limit, serves -queued calls in FIFO order, and removes a queued call when its context is -cancelled. It rechecks the caller context after admission and before dispatch. -A granted permit is released exactly once on every completion path. +Resolved producer mode chooses terminal handling. A supported permissive fallback +must preserve intrinsic/source/downstream invariants without repeating ordinary +assessment or claiming semantic approval. Preparation/capture/encoding failures +are not reasons to extend model budgets. Detailed assessment/disposition and +handoff eligibility belong in [Pipeline Internals](pipeline.md). -The scheduled wrapper surrounds every `CompleteStructured` call, so concurrent -lanes, pipeline retries, and LLM-backed validators share the same provider-call -ceiling. This ceiling is independent of pipeline worker concurrency; changing -worker counts cannot exceed the configured LLM limit. The configuration field -and its effective default are owned by -[Configuration](../config.md#concurrency-output-cache-and-debug). +## LLM Validators -PromptKit applies a second, independent admission limit when the selected -profile names a limited backend. It sits beneath the Notarius scheduled client, -so it may narrow but cannot expand the application-wide limit. Built-in -OpenRouter profiles select PromptKit's reserved backend and its upstream -capacity policy. A positive configured local-backend limit bounds active local -generations inside PromptKit; zero leaves that backend unlimited there. -Endpoint-only profiles do not select a PromptKit backend and remain limited -only by the Notarius scheduler. +The [combat-semantics validator](../../internal/modules/dnd/validate/scenedescriptions/combat_semantics/validator.go) +builds one native request with its own profile and scene-local input. It uses +the same engine and run-wide native execution/session selections, then checks +the native error/status before decoding. + +Successful interpretation returns a domain judgment and actionable rejection +guidance where appropriate. Execution or interpretation failure is unavailable +assessment with retained causes, not a semantic defect or invented instructions. +Cancellation delivers no judgment. Validators do not coordinate another semantic +conversation or rerun themselves to extend structural repair. + +A completed producer/continuation releases native admission before assessment, +so a nested validator can use that same limited backend. Application workers +and native admission remain independent; there is no second scheduler or queue. +Capacity handling follows producer mode, not an additional application retry. ## Prompt And Schema Assets -An `AssetRegistry` collects prompt, schema, and optional fallback-profile -filesystems from production module families. It retains each registered root's -filesystem context when constructing PromptKit's application source group. -Schema files are supplied separately through the engine's schema filesystem; -PromptKit reads and validates a selected schema while preparing that request, -not while constructing the configuration snapshot. Prompt and fallback -definitions are validated after native source composition, and referenced -prompt content is captured with the engine. Checkpoint identity uses captured -prompt content hashes rather than rescanning live files or hashing only -content-file paths. Raw paths and bytes are never included. The framework’s -`promptfs` helper combines -module-selected prompt files with reusable domain fragments without making the -framework depend on D&D content. +[assets/notarius.yml](../../assets/notarius.yml) imports ordinary embedded +initial/correction/validator resources with retained filesystem contexts. +The assets package remains a content-only leaf. Prompt declarations own default +output format, validation, and schema selections; modules own prompt meaning +and small static resource-ID dependencies, not behavioral asset registration. -LLM-facing content is embedded once by the root `assets` package. Each consumer -uses only its scoped subtree, while the module retains ownership of its prompt -declaration, ordered manifest, private response-schema identity, and -registration. Shared D&D fragments are selected by D&D's shared implementation; -the detailed convention is in [D&D Module Internals](dnd.md). This physical -arrangement and its data-only boundary are defined by -[Architecture](../policy/architecture.md) and -[ADR-0011](../adr/0011-centralize-llm-assets.md), rather than by this runtime -guide. +Each producer pair requests a complete replacement response using its shared +schema. Enduring principles live in initial system context; the follow-up +supplies current findings and its domain assignment. Normalization applies +each complete proposal afresh to fixed originals, not a union of prior groups. -The generic registrar is the sole production registration owner for the -semantic-reconciliation default prompt and private response schema. The -domain-neutral reconciliation package also exposes only its mandatory protocol -and candidate/transcript presentation files for domain prompt manifests. D&D -registry normalizers mount those files while retaining ownership and hashing -of their D&D system message, semantic instructions, and complete prompt -declaration. The response schema is therefore registered once even though -several typed normalizers select it. +Native preparation loads selected inline/file/root schemas. Intrinsic durable +artifact schemas remain application-owned codec contracts and are not removed +with duplicate model-response schema machinery. Full source/root/presence rules +belong in [Configuration](../config.md#prompt-output-contracts-and-schemas) and +[PromptKit formats][formats]. -Mounted prompt assets determine a module's fingerprint. The fingerprint hashes -only the module and shared files explicitly selected by its manifest, so an -unrelated asset does not invalidate a checkpoint. Schema loaders validate JSON, -attach identity and digest metadata, make defensive copies, and expose -diagnostics without raw schema bytes. +Native structural repair and transient retries remain upstream-owned. Notarius +does not pin omitted counts or wrap them in blanket producer/validator retries. +Application semantic allowance is a separate resolved control, not another +counter around native coordination. -Semantic-reconciliation normalizers extend this identity with the shared -response-schema digest, framework policy version, and complete limit-policy -digest. Their manifest metadata records the same content-free prompt, schema, -policy, and limit identities together with domain identity and normalization -policies. Request-local handles, source material, proposal content, and raw -asset bytes are not checkpoint metadata. +## Prepared Execution And Reuse -Private response schemas validate a model transport envelope. They are not the -durable artifact schema and should not be documented as an external wire -contract. Durable formats and compatibility rules remain in the -[integration contracts](../integrations/). +For eligible cacheable producers, the helper prepares the initial request once, +retains its details and opaque message-content hash, and either discards the +handle on a usable hit or executes that same handle with `RunPrepared`. +Unused handles are discarded on exits. Continuation retains the initial key. -## Prompt Maintenance And Backend Caching +Hits decode the accepted application artifact/plan without semantic reassessment, +exchange restoration, native execution, or historical usage. Normalizers cache +applied artifacts, not proposals. Provider-side prompt-cache hints remain native +request metadata, distinct from accepted-result identity. State layout and +eligibility mechanics belong in [Run State Internals](state.md), not another +LLM cache abstraction. The [prepared execution contract][prepared] owns native +handle lifetime and admission behavior. -Prompt message order and shared asset bytes are runtime behavior. Backend cache -reuse depends on identical preceding roles, rendered bytes, and cache-control -metadata—not merely equivalent meaning. Keep reusable shared assets -byte-identical and preserve each prompt’s declared ordering and cache controls -when editing it. +## Native Evidence And Sensitive Data -For sibling prompts that can reuse the same source material, order universal -shared context first, request source material next, and module-specific -suffixes last. Put a cache boundary at a reusable prefix that is useful to the -backend. Redundant intermediate cache boundaries do not extend that reusable -prefix and add no value. +The runner supplies one invocation execution group. Actual initial, continuation, +and nested validator executions enroll once; group observations preserve native +identity, terminal coverage, and unknown usage without copied totals or a +durable profile inventory. -Prompt-family owners may choose a different sequence when their inputs and -reuse pattern differ. The D&D family’s extraction, scene-chunking, and NPC -normalization policies are maintained in [D&D Module Internals](dnd.md#prompt-construction). -Do not add tests that enforce prompt prose; prompt tests should verify the -meaningful input placement and cache controls of the prompt being changed. +[Native capture](../../internal/framework/pipeline/native_capture.go) composes +observers/preparation callbacks rather than replacing them. Retain prepared +details cheaply during callbacks, then write at normal boundaries. Cache paths +reuse existing details; debug never triggers a second preparation or model call. +Shared callback state is synchronized. Available result-less failure facts and +unassessed undelivered responses remain distinguishable. -## Validation, Repair, And Retries - -PromptKit performs prompt rendering, provider execution, and the prompt’s -structured-output validation. The adapter reports an empty result, validation -failure, empty structured body, or decode failure as -`ErrInvalidStructuredOutput`, while retaining the returned raw bytes and debug -material when they exist. Provider failures remain operational errors rather -than output-validation failures. Apart from documented context, capacity, and -invalid-output categories, provider error values and types do not cross the -adapter error chain; callers receive only a credential-redacted diagnostic. - -`LLMGenerationError` preserves the provider-neutral failure kind (`transport`, -`provider_status`, `malformed_response`, or generic `generation`) and any HTTP -status. When PromptKit exhausts an eligible transient-status retry budget, it -also records that exhaustion and the attempts in that generation. Those fields -do not include repair generations, and zero attempts means unavailable rather -than proof that no request was sent. - -Every execution that reaches PromptKit returns an optional request-local -`execution` summary. It contains only terminal aggregate attempts, retry and -repair counts, durations, outcome and failure classification, status, and -presence-aware token usage. Preparation failures have no summary; later local -decode failures retain the upstream terminal summary without changing it. - -When PromptKit rejects backend admission before generation, the adapter maps -`promptkit.ErrCapacityExceeded` to -`contracts.ErrLLMCapacityExceeded`, retaining prompt context and a redacted -upstream diagnostic without exposing the PromptKit sentinel or capacity-error -type as a framework contract. When supplied, the normalized selected backend -ID appears only in that safe application-owned diagnostic context. A canceled -caller context takes precedence. The adapter does not retry capacity failures; -the pipeline's existing binding attempt policy sees the operational error and -decides whether to rerun the complete operation. - -PromptKit executes structural repair within its structured-output flow. The -maintained production prompt manifests declare one additional repair attempt. -When a resolved binding supplies a repair value, the adapter inspects the -prompt, copies its complete output contract, changes only the repair limit, and -passes that complete replacement contract to PromptKit. This preserves the -prompt's output format, validation mode, schema, and provider structured-output -settings. - -A successful repair is an ordinary successful completion, not a warning. The -adapter reports PromptKit's actual repair count and its cumulative usage -directly, without adding the initial and corrective counts again. Debug prompt -material records the configured complete contract; debug response material -records the repaired response and actual validation result. If the repair -budget is exhausted, the adapter retains the final raw bytes and debug material -and reports `ErrInvalidStructuredOutput`. Generation failures during an initial -or corrective call remain provider-neutral operational errors with the same -redaction boundary. - -Structural repair does not replace pipeline retry behavior: a binding's -configured retry count reruns its complete stage attempt after an operational -or structural error, module-requested retry, or actionable semantic rejection. -An actionable module-requested retry and a validator rejection both use the -same correction payload when the producer exposes an exact latest response; -feedback-free module retries reconstruct the ordinary request without appended -messages. The pipeline owns attempt lifecycle, validation chains, and retry -diagnostics; see -[Pipeline Internals](pipeline.md#validation-retries-and-output) and the -[binding reference](../config.md#module-bindings-and-validators). - -## Timeout Ownership - -The caller context remains the outer cancellation authority. PromptKit applies -a positive effective generation timeout as an inner request deadline for each -built-in generation attempt; an explicit zero disables only that generation -deadline. The HTTP client timeout is a separate transport-wide cap. Notarius -forwards the caller context and does not install another timeout wrapper around -PromptKit. - -The selected PromptKit profile owns generation settings. The resolved global -`promptkit.transient_retries` configuration is passed to the shared PromptKit -engine and controls additional built-in attempts for eligible transient HTTP -failures in one generation. It defaults to two; explicit zero disables those -attempts. Notarius binding retries remain outside the adapter and repeat the -complete module operation and validation chain. -Operator-facing behavior is summarized in -[Operations](../operations.md#operational-limits), and the pinned upstream -contract is identified in -[PromptKit Integration](../integrations/pkg-promptkit.md). - -## Observability And Redaction - -When debug recording is enabled, the pipeline decorates the shared client. The -wrapper records prepared prompt and response material, timing, selected profile -and backend, effective model parameters, and call identifiers in the run’s -debug bundle, including material available from a failed structured completion. -The profile-inspection CLI projects copied values into its Notarius-owned -neutral JSON DTO; effective configuration and explanation remain PromptKit's -redacted YAML and source-history views. Credential names and `APIKeyRequired` -metadata are configuration metadata, not credential-availability checks. -Effective parameters use PromptKit's stable lower-case JSON field names and may -include `backend_id`. For a successful completion, a debug-write failure is -surfaced; when the completion already failed, its call error remains the -result. Debug-bundle location, retention, and handling are operational concerns -documented in [Operations](../operations.md#debug-bundles). - -The attempt-terminal summary is a separate safe trace record: it contains -attempt kinds, validator outcome counts and reason codes, effective policy, -terminal action, and repair/usage references. It excludes raw assistant -responses and correction text. Those values can appear only in the explicitly -requested detailed prompt and response artifacts, which require sensitive-data -handling. - -Run manifests receive selected profile summaries, including optional effective -backend and reasoning provenance, and component identities—not prompt, schema, -source, reference, or response content. The published field semantics belong -to the [JSON output contract](../integrations/json-output.md#manifestjson). -Provider error text is wrapped with prompt context and bearer credentials are -redacted before it crosses the runtime boundary. Known-secret redaction is -available to other runtime collaborators; it does not make prompt or response -contents safe for general logging. - -Generation failures expose an application-owned category and optional HTTP -status. Provider code, type, and message remain debug-only, after redaction. - -## Failure Boundaries - -- Construction fails for missing asset registries, mutually exclusive profile - sources, invalid asset registration, or a non-positive scheduler limit. -- Preparation failures, unavailable explicit profiles, provider failures, and - context cancellation propagate to the calling stage with context. -- Backend admission exhaustion is a provider-neutral operational error and is - not classified as invalid structured output or validator rejection. -- Malformed or schema-invalid provider output is classified separately as - invalid structured output so the module or pipeline can apply its own retry - and rejection policy. -- Domain semantic checks, evidence decisions, and deterministic normalization - run outside the provider adapter. +[Run capture](../../internal/cli/run_capture.go) uses secret-aware configuration +views and exact marked-secret omissions, while native snapshots preserve credential +omissions. Do not heuristically scrub arbitrary source/prompt/response strings. +Selected full evidence can be sensitive; private debug representations are not +a consumer serialization contract. Recording failures reach application boundaries +without retrying model work. [Operations](../operations.md#debug-bundles) owns +handling and [State Internals](state.md) owns recorder lifecycle. ## Focused Verification -Read the LLM adapter, scheduler, asset registry, schema loader, and redaction -tests when changing this boundary. Prompt changes also require the owning -module’s asset tests, and retry or debug changes require focused pipeline or -CLI coverage. The focused runtime and D&D checks are: +Use real kits with a fake native provider or local fake HTTP server, not a +replacement completion interface. Relevant owners include: -~~~sh -go test ./internal/framework/llm/... ./internal/modules/dnd/... -~~~ +- [Native loading](../../internal/cli/native_load_test.go) and + [schema overrides](../../internal/cli/native_schema_load_test.go). +- [Producer/continuation tests](../../internal/framework/pipeline/producer_test.go), + [ordered assessment](../../internal/framework/pipeline/candidate_assessment_test.go), + and [nested native validation](../../internal/framework/pipeline/native_assessment_test.go). +- [Accepted-result reuse](../../internal/framework/pipeline/producer_cache_test.go) + and [native capture](../../internal/framework/pipeline/native_capture_test.go). +- [Representative extraction](../../internal/modules/dnd/extract/spells/extractor_test.go) + and [validator execution](../../internal/modules/dnd/validate/scenedescriptions/combat_semantics/native_execution_test.go). +- [Assembled recovery](../../internal/cli/recovery_contract_test.go) and + [real executable checks](../../internal/cli/native_process_test.go). + +These verify native status/causes, complete assessment, fixed inputs/history, +override propagation, cancellation/admission, reuse, and evidence. They do not +claim production model quality. Wider checks belong in +[Development](../development.md#validation). + +[consumer]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/pkg-promptkit.md +[formats]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/formats.md +[prepared]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 319ef505..4c451aa5 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -2008,6 +2008,29 @@ old cache families, and legacy flags; retain necessary migration/history mention ## Stage 41: Refresh Configuration, CLI, And LLM Internal Guides +**Status:** Complete. Replaced the three internal guides with current native +ownership and source/test navigation. Configuration now explains the composed +definition, original typed result handoff, source contexts, detached runtime +conversion, complete-selection invocation reuse, focused inspection, and separate +catalog/preflight boundaries. CLI now explains lazy CommandKit construction, +standard suite wiring, registrars, supplied native overrides, ordered run +composition, and operation/capture/presentation outcomes. The retained LLM guide +describes direct native requests/results/continuation, fixed module inputs, +pipeline assessment, direct validator execution, prepared reuse, and native +observations without a replacement service or reporting ledger. + +Reviewed the named definitions/loaders, command composition/resolution, concrete +spell producer and combat-semantics validator, and their focused real-kit tests. +Removed obsolete adapter, scheduler, asset/schema registration, fingerprint, +profile-ledger, manual replay, and copied inspection descriptions. Public settings, +flags, defaults, state paths, and payloads link to canonical owners rather than +duplicated reference tables. All 85 local source/test/doc links and five pinned +upstream destinations/anchors resolve against repository/module sources. +Configuration, CLI, preflight, pipeline, spell extraction, and semantic-validator +package tests and vet pass, as does `git diff --check`. No production code or +operator contract changed; pipeline/state and module-specific guide rewrites +retain their next assigned stages. + **Goal:** explain the actual native composition and execution boundary. **Inspect/change:** `docs/internal/{configuration,cli,llm}.md`; the final