Document PromptKit configuration ownership and capture

This commit is contained in:
2026-10-04 14:12:11 +00:00
parent 169a94f7ea
commit 4aba4f5672
6 changed files with 215 additions and 62 deletions

View File

@@ -41,17 +41,26 @@ configured discovery and precedence contract is in
[Configuration](../config.md), while the loading and resolution mechanics are
in [Configuration Internals](configuration.md).
`config effective` and `config explain` only render the loaded ConfigKit view;
they do not resolve pipelines or construct providers. Configuration validation without a selected pipeline checks structural
`config effective` and `config explain` render the loaded ConfigKit view by
default and do not resolve pipelines or construct providers. With explicit
`--scope promptkit`, they construct the PromptKit inspection engine from the
same captured source groups used by execution. Effective output is PromptKit's
redacted YAML; explanation output shows its source history. `config profile`
uses that engine to emit one neutral Notarius-owned JSON summary for the exact
requested profile ID. These inspection commands load no run pipeline, prepare
no request, resolve no credential value, and make no provider call. Ordinary
prompt and provider-parameter content is not generally secret-free even though
credential-shaped provider parameter keys are redacted.
Configuration validation without a selected pipeline checks structural
configuration only. Validation with a selected pipeline also builds the
effective catalog, resolves the pipeline, and verifies every explicit effective
PromptKit profile. Selected LLM-backed input, chunk, lane, output, and validator
profiles are inspected
against the configured PromptKit source and backend registrations without
loading a prompt or performing generation, so an unknown or invalid profile
fails before pipeline preparation. Credential availability remains an
execution-time concern. Pipeline listing validates configuration before
returning normalized, sorted identifiers.
profiles are inspected against the configured PromptKit source and backend
registrations without loading a prompt or performing generation, so an unknown
or invalid final effective profile fails before pipeline preparation.
Credential availability remains an execution-time concern. Pipeline listing
validates configuration before returning normalized, sorted identifiers.
## Production Composition
@@ -62,14 +71,17 @@ catalog used for resolution and the concrete constructors used for preparation.
Tests may provide a catalog or registries instead; production code must not
silently merge an injected partial catalog with production registrations.
The production LLM factory builds one PromptKit-backed client from the resolved
**promptkit.profile_dir** or **promptkit.profile_file** source, attaches the
profile-provenance recorder, creates one scheduler from the effective global
LLM limit, and wraps the client before it reaches modules. Registration and LLM
construction errors are returned before a pipeline is prepared. Configuration
field definitions remain in [Configuration](../config.md#promptkit-profiles);
the D&D registrar's fallback profile assets and the adapter mechanics remain in
[LLM Runtime](llm.md).
The production LLM factory builds one PromptKit-backed client from the native
source groups assembled from PromptKit catalogs, registered application prompt
and fallback assets, the optional operator file or recursively selected
directory, and the optional local backend registration. The same construction
path serves runtime, preflight, and configuration inspection. It attaches the
profile-provenance recorder, captures one immutable PromptKit configuration
identity, creates one scheduler from the effective global LLM limit, and wraps
the client before it reaches modules. Registration and LLM construction errors
are returned before a pipeline is prepared. Configuration fields remain in
[Configuration](../config.md#promptkit-profiles); source ownership, capture,
identity, and adapter mechanics remain in [LLM Runtime](llm.md).
The factory also accepts `LLMRuntimeOverrides`, whose reasoning pointer
preserves inherit, replace, and clear states across the composition boundary.

View File

@@ -83,35 +83,38 @@ retain an empty backend ID, which the published JSON omits. Successful
completion responses and recorded profile manifests identify the adapter
provider as `promptkit`.
The CLI's profile-inspection engine and the production adapter use the same
profile-source construction to apply the configured profile directory or file,
the optional registered fallback profile assets, and the optional conventional
`local` backend. Preflight therefore resolves the same profile sources and
backend membership as runtime without performing generation. Fallback assets
are mounted only when at least one source is registered. The production D&D
registrar contributes its `dnd-extraction` fallback, and the maintained D&D
prompts select that logical ID by default. PromptKit owns source precedence and
profile parsing and inheritance: an operator-provided matching profile takes precedence over a
fallback profile without Notarius merging either document.
When the registration is absent, a profile selecting `backend: local` fails
inspection instead of falling back to a built-in or endpoint-only target.
The CLI's profile-inspection engine and production adapter use the same source
assembly: PromptKit catalogs, one application group containing registered
prompts and fallback profiles, the selected operator file or directory, and
the optional final `local` backend registration. Execution, preflight, and
inspection therefore capture the same effective definitions. PromptKit and
ConfigKit compose recursively across groups while retaining each input's file
context; inputs within a group compose strictly. Operator imports are enabled
for the selected operator input, and imports retain their declaring file's
relative resource roots. PromptKit validates final effective definitions at
engine construction. It does not run a separate audit that rejects a malformed
definition superseded by a later overlay. The D&D registrar contributes its
`dnd-extraction` fallback, and maintained D&D prompts select that logical ID
by default. A profile selecting `backend: local` requires the optional local
registration.
Before execution, the adapter also contributes a non-secret checkpoint
fingerprint for the effective PromptKit profile source. It combines the
identity of PromptKit's compiled-in profile catalog with a deterministic digest
of every YAML profile in the configured profile directory, or of the configured
profile file, and a deterministic digest of the flattened fallback profile
assets. The fingerprint contains neither profile content nor source paths. It
covers inherited pipeline profiles, explicit binding profiles, and
prompt-selected defaults, so changing a model or other profile setting cannot
reuse checkpoints created under the
prior profile source. This cache identity is independent of durable
profile provenance: run manifests continue to list only profiles actually
observed during LLM calls. When the local backend is registered, a second
fingerprint hashes its trimmed endpoint behind a stable marker. Changing that
semantic execution target invalidates checkpoint reuse. The raw endpoint is not
stored in checkpoint identity, and the local concurrency limit is excluded
because it changes scheduling rather than execution semantics.
Each engine captures its definitions at construction. A later source-file edit
does not mutate an existing engine; constructing a new engine captures the
updated view. Before execution, the adapter computes one immutable,
versioned `promptkit_configuration` checkpoint identity from public captured
inspection APIs. The private digest covers captured effective configuration,
all effective profile targets and credential-policy metadata (including
provider parameters), and every effective prompt ID/version and captured
content hash. The identity also includes PromptKit and the selected OpenRouter
and Rakestrawhome catalog versions. Local-backend concurrency settings remain
excluded because they affect scheduling; its endpoint is represented only in
the private digest, never as checkpoint metadata. Neither the private digest
payload, raw sources, paths, endpoints, provider parameters, nor prompt bodies
are published. An existing engine retains its identity after files change; a
new engine gets the identity of its newly captured definitions. This replaces
the former live-file profile and local-endpoint fingerprints. Run manifests
remain independent and list only profile choices actually observed during LLM
calls.
## Shared Provider-Call Limit
@@ -140,11 +143,16 @@ only by the Notarius scheduler.
## Prompt And Schema Assets
An `AssetRegistry` collects prompt, schema, and optional fallback-profile
filesystems from production module families. It flattens registered roots into
the corresponding PromptKit filesystems and rejects invalid roots, unreadable
assets, duplicate paths, and missing prompt or schema files during preparation.
Fallback assets receive a safe content digest for checkpoint identity; raw
paths and bytes are never included. The framework’s `promptfs` helper combines
filesystems from production module families. It retains each registered root's
filesystem context when constructing PromptKit's application source group.
Schema files are supplied separately through the engine's schema filesystem;
PromptKit reads and validates a selected schema while preparing that request,
not while constructing the configuration snapshot. Prompt and fallback
definitions are validated after native source composition, and referenced
prompt content is captured with the engine. Checkpoint identity uses captured
prompt content hashes rather than rescanning live files or hashing only
content-file paths. Raw paths and bytes are never included. The framework’s
`promptfs` helper combines
module-selected prompt files with reusable domain fragments without making the
framework depend on D&D content.
@@ -294,6 +302,10 @@ When debug recording is enabled, the pipeline decorates the shared client. The
wrapper records prepared prompt and response material, timing, selected profile
and backend, effective model parameters, and call identifiers in the run’s
debug bundle, including material available from a failed structured completion.
The profile-inspection CLI projects copied values into its Notarius-owned
neutral JSON DTO; effective configuration and explanation remain PromptKit's
redacted YAML and source-history views. Credential names and `APIKeyRequired`
metadata are configuration metadata, not credential-availability checks.
Effective parameters use PromptKit's stable lower-case JSON field names and may
include `backend_id`. For a successful completion, a debug-write failure is
surfaced; when the completion already failed, its call error remains the