Record simpler PromptKit-native application design
This commit is contained in:
@@ -26,9 +26,12 @@ and [LLM Runtime Internals](../internal/llm.md).
|
||||
|
||||
## 1. LLM API And Diagnostic Boundary
|
||||
|
||||
**Discussion / decision:** Direction selected: remove the Notarius PromptKit
|
||||
adapter and make the application fully PromptKit-native. Detailed design and
|
||||
migration consequences remain to be worked through.
|
||||
**Discussion / decision:** Direction agreed: remove the Notarius PromptKit
|
||||
adapter and compose the application directly from native PromptKit execution,
|
||||
typed artifact operations, and pipeline orchestration. A small application
|
||||
function connects generation to configured validation and acceptance. The design
|
||||
below incorporates items 2–6 and the Audita review; exact signatures and migration
|
||||
details remain open. No application changes have been implemented.
|
||||
|
||||
### Background And Intent
|
||||
|
||||
@@ -47,33 +50,103 @@ Missing reusable capabilities may justify upstream feature requests. The
|
||||
discussion is not constrained to what v0.23.2 can replace immediately; any
|
||||
resulting change to the adoption target remains to be selected.
|
||||
|
||||
### Design Consequences To Work Through
|
||||
### Application Structure
|
||||
|
||||
- Use native PromptKit APIs and contracts for LLM execution instead of preserving
|
||||
a parallel provider-neutral completion API or rebuilding the adapter under
|
||||
another name.
|
||||
- Review copied request, result, error, inspection, observation, and debug types
|
||||
for removal. Keep application-owned representations only where they serve a
|
||||
concrete application need, with compatibility decisions made explicitly.
|
||||
- Let PromptKit provide reusable execution mechanics. Determine how Notarius
|
||||
supplies domain inputs, interprets typed outputs, and expresses its acceptance
|
||||
decisions through the resulting design.
|
||||
- Review application composition, test seams, diagnostic storage, and disclosure
|
||||
handling without assuming they require an LLM abstraction layer.
|
||||
- Evaluate each upstream gap as a requirement to understand and potentially
|
||||
address in PromptKit, rather than automatically preserving local machinery.
|
||||
| Part | Responsibility |
|
||||
| --- | --- |
|
||||
| Application composition | Load configuration, assemble native prompt sources, and construct one shared PromptKit engine for producers, validators, and continuation runs. |
|
||||
| Artifact-family modules | Supply prompt inputs and response requirements, interpret responses, and apply domain rules. |
|
||||
| Pipeline execution | Connect generation to configured validation, decide acceptance or fallback, and manage dependencies and bounded workers. |
|
||||
| PromptKit | Prepare and execute requests, maintain conversations, enforce native budgets and admission, and report execution facts. |
|
||||
| Application records and CLI | Store artifacts and checkpoints, attach pipeline meaning to execution facts, and present results through CommandKit. |
|
||||
|
||||
Open questions:
|
||||
Engine construction becomes ordinary application wiring using item 2's embedded
|
||||
defaults and operator sources. Profile selection is expressed through native
|
||||
requests. The current `internal/framework/llm` package can shrink to engine
|
||||
assembly and focused helpers; whether those warrant a separate package remains
|
||||
an organizational choice, not a reason to retain an adapter.
|
||||
|
||||
- What application collaborators and helpers remain after the adapter is removed?
|
||||
- Which durable output or diagnostic contracts need deliberate migration?
|
||||
- Which existing behavioral differences should disappear through native adoption,
|
||||
and which reflect an application need that should be supplied to PromptKit?
|
||||
Use native requests, results, exchanges, errors, and observers directly in
|
||||
model-executing application code. Pure domain models, codecs, source-reference
|
||||
resolution, and deterministic transformations need no PromptKit dependency.
|
||||
Retain application interfaces for extracting artifacts, validating typed
|
||||
candidates, encoding output, and loading checkpoints. Deterministic stages
|
||||
continue to operate without conversation machinery.
|
||||
|
||||
**Follow-up:** Apply this direction to items 2–6, compare Audita under item 7,
|
||||
and reconcile the adoption roadmap. Record the significant boundary change in
|
||||
a successor ADR and update architecture and implementation documentation when
|
||||
the change lands. Do not treat the remaining topics as decided by this direction.
|
||||
### Connect Modules To Pipeline Assessment
|
||||
|
||||
The current [spell extractor](../../internal/modules/dnd/extract/spells/extractor.go)
|
||||
prepares inputs, invokes the private completion interface, interprets a response,
|
||||
and returns a copied raw model candidate to the
|
||||
[producer retry loop](../../internal/framework/pipeline/producer_attempts.go).
|
||||
Replace that arrangement with direct collaboration between module interpretation,
|
||||
pipeline assessment, and native execution:
|
||||
|
||||
1. The module prepares source material, references, and any candidate-handle
|
||||
mappings once, and constructs the native initial request.
|
||||
2. Application execution calls `Engine.Run`, capturing an exchange when
|
||||
continuation is enabled.
|
||||
3. The module interprets the structurally valid response into a typed candidate
|
||||
or actionable domain findings.
|
||||
4. The pipeline runs configured candidate validation and decides whether to
|
||||
accept, request correction, or apply terminal policy.
|
||||
5. If correction is needed, native `Coordinate` invokes the same interpretation
|
||||
and assessment for revised responses. The module supplies the repair request;
|
||||
PromptKit maintains history and enforces the turn budget.
|
||||
|
||||
A small shared application function can connect these responsibilities: the
|
||||
module supplies native request construction and response interpretation, while
|
||||
the pipeline supplies configured validation and terminal policy. It uses native
|
||||
PromptKit types and returns a typed application outcome. It does not introduce
|
||||
another completion API, conversation protocol, or retry engine.
|
||||
|
||||
Item 3 owns the detailed continuation and assessment rules, including final-turn
|
||||
assessment, failure handling, complete replacement responses, and safe fallback.
|
||||
For normalization, a revised reconciliation plan is applied to the original
|
||||
inputs to produce a fresh candidate. LLM-backed validators can call the same
|
||||
native engine and return application validation findings; they do not need the
|
||||
producer's conversation machinery.
|
||||
|
||||
### Removal, Records, And Testing
|
||||
|
||||
Retire `StructuredLLMClient`, the private structured-completion request/response
|
||||
family, adapter conversions, and opaque correction transport. Items 3, 4, and 6
|
||||
specify removal of the outer LLM retry loop, scheduler, debug completion wrapper,
|
||||
profile-recording plumbing, and duplicate execution accounting. Do not recreate
|
||||
that wrapper stack around native calls.
|
||||
|
||||
Attach pipeline scope through native callbacks and observers. Use native
|
||||
`ExecutionGroup` totals and small application projections for manifests and
|
||||
optional debug artifacts, following item 6. Checkpoint matching and durable
|
||||
acceptance provenance remain the application responsibilities selected in item 5.
|
||||
|
||||
Audita demonstrates shared engine assembly, native continuation, and CommandKit
|
||||
integration. Its `RoleClient`, structured-output interface, and opaque exchange
|
||||
wrapper are not part of this design; item 7 records the comparison's evidence
|
||||
and limits.
|
||||
|
||||
Test domain interpretation and assessment directly. Exercise model workflows
|
||||
with a real PromptKit engine and its injectable client or a fake HTTP provider.
|
||||
Pipeline orchestration can still use fake stage implementations. Testing does
|
||||
not require preserving a private completion interface.
|
||||
|
||||
### Remaining Implementation Work
|
||||
|
||||
- Select concrete module/pipeline function signatures that connect request
|
||||
construction, interpretation, configured validation, and terminal disposition.
|
||||
- Choose the location of engine assembly and focused application helpers without
|
||||
introducing a replacement LLM service layer.
|
||||
- Resolve item 2's deferred filesystem and schema choices and the remaining
|
||||
migration details recorded under items 3–6, including durable compatibility.
|
||||
- Evaluate any remaining upstream gap against an actual application requirement,
|
||||
rather than preserving historical machinery by default.
|
||||
|
||||
**Follow-up:** Reconcile the adoption roadmap with this design and items 2–6.
|
||||
Record the significant boundary change in successor ADRs. Update architecture
|
||||
and implementation documentation when the change lands, explicitly replacing
|
||||
the current transport-neutral completion boundary with PromptKit as the execution
|
||||
dependency of application workflows. Keep domain operations and application
|
||||
outcomes as the meaningful Notarius-owned boundaries.
|
||||
|
||||
## 2. Prompt Assets And Schema Assembly
|
||||
|
||||
|
||||
Reference in New Issue
Block a user