Add PromptKit architecture discussion notepad

This commit is contained in:
2026-11-18 18:52:49 +00:00
parent 0ef079610e
commit af250eb72f
2 changed files with 169 additions and 0 deletions

View File

@@ -0,0 +1,165 @@
# PromptKit Architecture Discussion
## Purpose And Status
Working notepad for reviewing the ownership boundaries in the
[PromptKit v0.23.2 adoption roadmap](promptkit.md). Work through the topics
below in order, recording questions, evidence, decisions, and follow-up work.
The six initial topics are proposals for discussion, not accepted changes.
The Audita comparison is a seventh topic and may inform earlier decisions.
The central question is how much reusable LLM machinery Notarius should retain
as PromptKit, ConfigKit, and CommandKit take on more responsibility. A working
principle to evaluate is that application ownership of a policy does not
necessarily require application ownership of its implementation.
No architectural changes have been agreed or implemented through this notepad.
The adoption roadmap remains the migration plan; reconcile it with decisions
from this discussion as they are made. Significant changes to accepted
architecture need successor ADRs under the
[documentation policy](../policy/documentation.md#architecture-decision-records).
Current boundaries are described in [Architecture](../policy/architecture.md)
and [LLM Runtime Internals](../internal/llm.md).
## 1. LLM API And Diagnostic Boundary
Consider using native PromptKit contracts in LLM-facing implementation packages
and reducing parallel completion, message, usage, execution-summary, and debug
types. Retain application contracts where they serve typed artifact handling,
stage context, durable output compatibility, or disclosure policy.
Questions:
- Which abstractions express Notarius behavior, and which merely copy PromptKit?
- Where should direct PromptKit dependencies be permitted?
- Can detailed debug material flow to an application-controlled writer without
being translated through generic pipeline contracts?
- Which persisted representations need compatibility or redaction adapters?
**Discussion / decision:** Pending. Start here.
**Follow-up:** Identify the smallest useful application boundary and any
architecture-policy changes it requires.
## 2. Prompt Assets And Schema Assembly
Consider replacing custom asset registration and virtual filesystem remounting
with native embedded sources, imports, collections, and source-relative assets.
Modules would continue to own prompt meaning and typed interpretation.
Questions:
- Can native sources represent the existing shared fragments and module assets?
- Which registry, remounting, and hashing helpers could disappear?
- What private schema identity is still needed for checkpoint reuse?
- How do we preserve message ordering and cache controls while simplifying
assembly?
**Discussion / decision:** Pending.
**Follow-up:** Separate private LLM schema plumbing from durable artifact
schemas; do not assume a captured prompt hash covers schema content.
## 3. Retry And Semantic Correction Ownership
Consider narrowing outer stage retries once PromptKit has exhausted its own
transport and structural-repair budgets. Notarius would retain application
assessment, semantic correction, fallback, and acceptance decisions.
Questions:
- Which operational failures justify repeating an entire stage?
- Should semantic correction preserve only the latest response or accumulate
conversation history?
- How should retries for non-LLM operations differ from LLM execution retries?
- Would a native orchestration API reduce machinery without obscuring typed
validation and terminal disposition?
**Discussion / decision:** Pending.
**Follow-up:** Evaluate behavior changes against
[ADR-0014](../adr/0014-feedback-aware-validation-retries.md) and
[ADR-0016](../adr/0016-feedback-aware-module-requested-retries.md).
PromptKit v0.23.2's `Coordinate` continues an exchange; it is not a direct
replacement for the existing latest-response-only policy or full stage loop.
## 4. Global Scheduling And Admission
Consider whether backend capacity limits and pipeline worker bounds meet the
operational need, or whether a separate cross-backend ceiling remains useful.
If it remains useful, consider implementing that reusable mechanism upstream.
Questions:
- What resource or operational budget does each limit protect?
- Should a global permit remain occupied during backend queueing and repair?
- Can the Notarius scheduler be removed, or should PromptKit gain engine-wide
admission support?
**Discussion / decision:** Pending.
**Follow-up:** Account for the existing global-limit contract. PromptKit
v0.23.2's per-backend limits do not directly replace it.
## 5. Execution Identity And Checkpoint Reuse
Consider moving identity for PromptKit-owned execution semantics upstream.
Notarius would combine opaque upstream identities with its source, module,
reference, schema, and policy identities and retain checkpoint eligibility and
storage decisions.
Questions:
- Which identity can be computed before a stage runs, and which needs a concrete
prepared request?
- Which changes should invalidate reuse, and at what granularity?
- Should a conservative application fingerprint be an interim implementation
rather than the architectural destination?
- Does this justify upstream work and a later adoption target?
**Discussion / decision:** Pending.
**Follow-up:** Review PromptKit's deferred semantic execution-target fingerprint
proposal. This capability is not supplied by the v0.23.2 adoption target.
## 6. Inspection And Execution Bookkeeping
Extend the roadmap's use of native inspection, `Run`, `OnPrepared`, observations,
and `ExecutionGroup` to reconsider surrounding wrappers and copied report types.
Notarius retains stage correlation, artifact provenance, storage, and disclosure
policy; CommandKit is the planned owner of standardized operator presentation.
Questions:
- Which reports and accounting mechanisms can use upstream results directly?
- Which application projections remain necessary for durable contracts?
- Can the PromptKit migration avoid building temporary machinery that the
CommandKit adoption would immediately replace?
**Discussion / decision:** Pending.
**Follow-up:** Coordinate this topic with item 1 and the roadmap's CommandKit
follow-up so report and adapter decisions are made once.
## 7. Compare Audita's PromptKit Workflow
Review the sibling Audita checkout at `../audita` relative to the Notarius
repository root. Audita uses PromptKit for a workflow the user identifies as
very similar, potentially identical. Verify the similarities and differences
before drawing architectural conclusions.
Questions:
- What are Audita's PromptKit version, execution flow, and ownership boundaries?
- How does it handle candidate assessment, correction, retries, scheduling,
configuration, inspection, and diagnostics?
- Which simpler patterns could Notarius adopt directly?
- Which differences arise from actual product requirements?
- Does duplication across the two applications reveal work that belongs in
PromptKit, ConfigKit, or CommandKit?
**Discussion / decision:** Pending; Audita has not yet been reviewed for this
comparison.
**Follow-up:** Record the reviewed Audita revision and evidence, then feed the
findings back into items 1–6 and the adoption roadmap.

View File

@@ -1,5 +1,9 @@
# PromptKit v0.23.2 Adoption
Open questions about the retained application machinery are tracked in the
[architecture discussion notepad](promptkit-architecture-discussion.md).
Those proposals remain pending until reconciled into this roadmap.
## Purpose
Replace Notarius's PromptKit v0.12 integration with a native v0.23.2