Files
notarius/docs/internal/llm.md

187 lines
11 KiB
Markdown

# Native LLM Integration Internals
Model-executing code uses native PromptKit types directly. This guide explains
Notarius's integration points, not a private completion service or a copied
library manual. [Architecture](../policy/architecture.md#native-execution-boundary)
defines the boundary; [Configuration](../config.md#promptkit-profiles) owns
resources/settings; [CLI Reference](../cli.md#run-wide-llm-overrides) owns request
controls; [Operations](../operations.md) owns state and handling.
## Construction And Ownership
The [CLI loader](../../internal/cli/configkit_load.go) hands the original completed
ConfigKit application result to `promptkit.NewFromConfig`. One invocation shares
the resulting engine across initial producers, corrections, normalization, and
LLM validators. Runtime-only HTTP/client settings and an injected native
`promptkit.LLMClient` remain construction options, not application YAML.
There is no provider-neutral completion conversion or second engine per role.
[Configuration Internals](configuration.md) owns native definition/source assembly
and invocation selection reuse. PromptKit owns captured resources, request
preparation, native validation/repair, provider transport, conversation state,
and backend admission. See the [upstream Go consumer contract][consumer] for those
APIs rather than reproducing their mechanics here.
Application contracts remain typed production, candidate assessment, supported
fallback, and artifact encoding. Pure domain models, codecs, source resolution,
and deterministic transforms do not need PromptKit merely because a surrounding
operation invokes a model.
## Requests And Fixed Operation Inputs
Model modules receive the shared engine during typed preparation. A module
prepares transcript, references, contextual identity projections, and any
normalization candidate mapping once. It constructs a `promptkit.RunRequest`
with its selected prompt, profile/session, native artifact inputs, and variables.
Response interpretation consumes `*promptkit.RunResult` and produces the
application's typed candidate.
[Spell extraction](../../internal/modules/dnd/extract/spells/extractor.go) is a
representative path: fixed transcript/catalog/NPC projections feed an initial
request; native output is decoded and canonically mapped into its durable type.
The correction builder selects a paired prompt and supplies current guidance,
not another copy of original source/reference material.
Domain identity stays deterministic. Contextual selectors and request-local
normalization handles resolve against fixed supplied records; models do not
acquire authority over durable IDs, citation meaning, or typed artifact shape.
[Module Internals](modules.md) and [D&D Internals](dnd.md) own extension/domain
recipes.
Native request overrides preserve supplied values without rewriting configured
profiles. The producer helper's request composition preserves operation-owned
prompt/input/capture fields and existing callbacks; validators pass native
execution options explicitly on their own requests.
## Native Production And Application Assessment
[RunProducer](../../internal/framework/pipeline/producer.go) connects ordinary
module interpretation/correction/fallback functions to the pipeline assessor.
It uses native `Run`, or retained prepared execution on the cache path, then
checks errors and public validation status before interpretation. Failed native
validation can accompany nil Go error and is not a usable semantic candidate.
Every usable initial/revised candidate receives the full applicable configured
assessment chain once, including unchanged candidates and the final permitted
response. Native coordination owns history and additional-turn iteration.
Its initial callback uses already collected findings instead of reassessing the
initial result. Module templates own correction assignments; detecting modules/
validators own actionable guidance, separately from diagnostics.
Native execution success is not domain acceptance. Stopped/exhausted outcomes
are control facts. Result-less failed continuation does not recover an earlier
candidate, and an undelivered response is not assessed or accepted. Caller
cancellation stops without fallback. Native timeout/provider/capacity failures
remain distinguishable through context state and typed causes.
Resolved producer mode chooses terminal handling. A supported permissive fallback
must preserve intrinsic/source/downstream invariants without repeating ordinary
assessment or claiming semantic approval. Preparation/capture/encoding failures
are not reasons to extend model budgets. Detailed assessment/disposition and
handoff eligibility belong in [Pipeline Internals](pipeline.md).
## LLM Validators
The [combat-semantics validator](../../internal/modules/dnd/validate/scenedescriptions/combat_semantics/validator.go)
builds one native request with its own profile and scene-local input. It uses
the same engine and run-wide native execution/session selections, then checks
the native error/status before decoding.
Successful interpretation returns a domain judgment and actionable rejection
guidance where appropriate. Execution or interpretation failure is unavailable
assessment with retained causes, not a semantic defect or invented instructions.
Cancellation delivers no judgment. Validators do not coordinate another semantic
conversation or rerun themselves to extend structural repair.
A completed producer/continuation releases native admission before assessment,
so a nested validator can use that same limited backend. Application workers
and native admission remain independent; there is no second scheduler or queue.
Capacity handling follows producer mode, not an additional application retry.
## Prompt And Schema Assets
[assets/notarius.yml](../../assets/notarius.yml) imports ordinary embedded
initial/correction/validator resources with retained filesystem contexts.
The assets package remains a content-only leaf. Prompt declarations own default
output format, validation, and schema selections; modules own prompt meaning
and small static resource-ID dependencies, not behavioral asset registration.
Each producer pair requests a complete replacement response using its shared
schema. Enduring principles live in initial system context; the follow-up
supplies current findings and its domain assignment. Normalization applies
each complete proposal afresh to fixed originals, not a union of prior groups.
Native preparation loads selected inline/file/root schemas. Intrinsic durable
artifact schemas remain application-owned codec contracts and are not removed
with duplicate model-response schema machinery. Full source/root/presence rules
belong in [Configuration](../config.md#prompt-output-contracts-and-schemas) and
[PromptKit formats][formats].
Native structural repair and transient retries remain upstream-owned. Notarius
does not pin omitted counts or wrap them in blanket producer/validator retries.
Application semantic allowance is a separate resolved control, not another
counter around native coordination.
## Prepared Execution And Reuse
For eligible cacheable producers, the helper prepares the initial request once,
retains its details and opaque message-content hash, and either discards the
handle on a usable hit or executes that same handle with `RunPrepared`.
Unused handles are discarded on exits. Continuation retains the initial key.
Hits decode the accepted application artifact/plan without semantic reassessment,
exchange restoration, native execution, or historical usage. Normalizers cache
applied artifacts, not proposals. Provider-side prompt-cache hints remain native
request metadata, distinct from accepted-result identity. State layout and
eligibility mechanics belong in [Run State Internals](state.md), not another
LLM cache abstraction. The [prepared execution contract][prepared] owns native
handle lifetime and admission behavior.
## Native Evidence And Sensitive Data
The runner supplies one invocation execution group. Actual initial, continuation,
and nested validator executions enroll once; group observations preserve native
identity, terminal coverage, and unknown usage without copied totals or a
durable profile inventory.
[Native capture](../../internal/framework/pipeline/native_capture.go) composes
observers/preparation callbacks rather than replacing them. Retain prepared
details cheaply during callbacks, then write at normal boundaries. Cache paths
reuse existing details; debug never triggers a second preparation or model call.
Shared callback state is synchronized. Available result-less failure facts and
unassessed undelivered responses remain distinguishable.
[Run capture](../../internal/cli/run_capture.go) uses secret-aware configuration
views and exact marked-secret omissions, while native snapshots preserve credential
omissions. Do not heuristically scrub arbitrary source/prompt/response strings.
Selected full evidence can be sensitive; private debug representations are not
a consumer serialization contract. Recording failures reach application boundaries
without retrying model work. [Operations](../operations.md#debug-bundles) owns
handling and [State Internals](state.md) owns recorder lifecycle.
## Focused Verification
Use real kits with a fake native provider or local fake HTTP server, not a
replacement completion interface. Relevant owners include:
- [Native loading](../../internal/cli/native_load_test.go) and
[schema overrides](../../internal/cli/native_schema_load_test.go).
- [Producer/continuation tests](../../internal/framework/pipeline/producer_test.go),
[ordered assessment](../../internal/framework/pipeline/candidate_assessment_test.go),
and [nested native validation](../../internal/framework/pipeline/native_assessment_test.go).
- [Accepted-result reuse](../../internal/framework/pipeline/producer_cache_test.go)
and [native capture](../../internal/framework/pipeline/native_capture_test.go).
- [Representative extraction](../../internal/modules/dnd/extract/spells/extractor_test.go)
and [validator execution](../../internal/modules/dnd/validate/scenedescriptions/combat_semantics/native_execution_test.go).
- [Assembled recovery](../../internal/cli/recovery_contract_test.go) and
[real executable checks](../../internal/cli/native_process_test.go).
These verify native status/causes, complete assessment, fixed inputs/history,
override propagation, cancellation/admission, reuse, and evidence. They do not
claim production model quality. Wider checks belong in
[Development](../development.md#validation).
[consumer]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/pkg-promptkit.md
[formats]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/formats.md
[prepared]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later