568 lines
38 KiB
Markdown
568 lines
38 KiB
Markdown
# PromptKit-Native Architecture Migration
|
||
|
||
## Purpose And Status
|
||
|
||
Make Notarius fully PromptKit-native: use upstream configuration, execution,
|
||
continuation, admission, inspection, and accounting directly, and retain the
|
||
application work of producing, assessing, storing, and reusing typed artifacts.
|
||
Historical duplication of PromptKit behavior is not a requirement to preserve it.
|
||
|
||
This is the implementation roadmap for the agreed directions in items 1–6 of
|
||
[the architecture discussion record](promptkit-architecture-discussion.md).
|
||
That document retains the discussion and Audita evidence. This roadmap owns the
|
||
selected target, unresolved choices, implementation sequence, and acceptance
|
||
criteria. All implementation phases below are pending; rewriting the roadmap
|
||
has not changed application behavior.
|
||
|
||
The core migration is a hard cutover for application configuration and PromptKit
|
||
resource definitions. Supply migration guidance and maintained examples; support
|
||
only the new formats in the resulting application. CommandKit adoption remains a
|
||
subsequent delivery phase with its ownership accounted for throughout this plan.
|
||
Release preparation and publication are separate work.
|
||
|
||
## Baseline And Dependency Target
|
||
|
||
At Notarius revision `12eeb6369e40746b2c18c55397a289a1e1b5908d`, the application
|
||
uses configuration version 5 and Go 1.27. Its dependencies include old-path
|
||
PromptKit v0.12.0 and ConfigKit v0.5.0, new-path ConfigKit v1.2.0, go-yaml
|
||
v4.0.0-rc.6, and two external PromptKit provider catalogs.
|
||
|
||
The initial dependency target remains `go.maximumdirect.net/promptkit@v0.23.2`
|
||
(tagged revision `382ccfb14b9630fd8f48c8364dbe2b6159637d71`), with shared
|
||
ConfigKit v1.2.1, stable go-yaml v4, and compatible validation dependencies.
|
||
The Go consumer minimum remains 1.27. The tagged [release notes][release],
|
||
[migration guide][migration], [format reference][formats], and
|
||
[Go consumer guide][consumer] supply the upstream baseline.
|
||
|
||
Confirm the final compatible tagged dependency set before implementation.
|
||
A demonstrated unmet application need may justify an upstream request and a
|
||
later target; the agreed design does not currently require a new PromptKit
|
||
feature. In particular, neither a cross-backend global limit nor a comprehensive
|
||
execution fingerprint is a prerequisite. Remove obsolete imports and catalog
|
||
dependencies when unused; do not retain a compatibility fork or permanent
|
||
`replace` directive. Update attribution under the [OSS policy](../policy/oss.md).
|
||
|
||
Audita's reviewed v0.23.2 integration demonstrates native source assembly,
|
||
continuation, observations, and CommandKit wiring. Adopt those mechanisms
|
||
without copying its `RoleClient`, structured-output interface, or opaque exchange
|
||
wrapper. Its per-proposal repair branches and lack of checkpoints do not define
|
||
Notarius's artifact and recovery policies.
|
||
|
||
## Target Architecture
|
||
|
||
### Application Structure And Ownership
|
||
|
||
| Part | Responsibility |
|
||
| --- | --- |
|
||
| Application composition | Load configuration and native sources; construct one shared PromptKit engine for producers, validators, and continuation runs. |
|
||
| Artifact-family modules | Prepare domain inputs, select prompt IDs and response requirements, interpret responses, and apply deterministic domain rules. |
|
||
| Pipeline execution | Connect generation to configured validation and terminal policy; manage ordered dependencies, typed handoffs, bounded workers, and stable result ordering. |
|
||
| PromptKit | Prepare and execute requests; maintain captured exchanges and conversations; enforce native retry, repair, turn, and admission budgets; report execution facts. |
|
||
| Application state and records | Store artifacts, apply checkpoint reuse policy, preserve acceptance provenance, and attach pipeline context to execution facts. |
|
||
| ConfigKit and CommandKit | ConfigKit owns configuration mechanics; CommandKit and ckconfig/ckprompt own standardized commands and presentation. Notarius supplies loaders, request builders, report content, and outcome mapping. |
|
||
|
||
Use native PromptKit requests, results, exchanges, errors, and observers in
|
||
model-executing application code. Remove the private provider-neutral completion
|
||
API. Pure domain models, codecs, source-reference resolution, and deterministic
|
||
transformations need no PromptKit dependency. Application interfaces continue to
|
||
express artifact production, typed validation, output encoding, and state access.
|
||
|
||
A small shared application function connects module request construction and
|
||
response interpretation to pipeline validation and terminal policy. It calls
|
||
native `Run` and `Coordinate` and returns a typed application outcome. It owns no
|
||
replacement completion API, conversation protocol, or general retry engine.
|
||
Exact signatures and helper placement remain open. The current
|
||
`internal/framework/llm` package may shrink to engine assembly and focused helpers
|
||
or cease to be a separate package.
|
||
|
||
Retain the fixed `input -> chunk -> extract -> merge -> normalize -> output`
|
||
shape, ordered steps and generated references, typed artifact-family boundaries,
|
||
and deterministic public ordering. Keep modules independent of physical output,
|
||
cache, and debug roots. Native adoption does not turn the pipeline into a general
|
||
workflow engine or redesign durable D&D artifacts.
|
||
|
||
### Configuration, Sources, And Prompt Assets
|
||
|
||
Introduce application configuration version 6 as the only supported version.
|
||
Preserve unrelated discovery, pipeline structure, and resolution behavior.
|
||
ConfigKit continues to own loading and composition under
|
||
[ADR-0020](../adr/0020-delegate-configuration-mechanics-to-configkit.md).
|
||
Retire `promptkit.profile_file`, `profile_dir`, and `local_backend` in favor of
|
||
native companion sources; do not introduce a separate `promptkit.config_file`.
|
||
Retire `concurrency.total_llm` and revise retry/worker settings as described below.
|
||
Their replacement defaults and detailed migration treatment remain open.
|
||
|
||
Use one source assembly for runtime, preflight, and inspection:
|
||
|
||
1. PromptKit's bundled OpenRouter underlay.
|
||
2. Notarius's embedded native prompts and profiles under `collections.notarius`.
|
||
3. An optional, higher-priority operator underlay containing `prompts.yml` and
|
||
recursive `.yml`/`.yaml` files under `prompts/`, beside the main application
|
||
configuration actually selected by ConfigKit.
|
||
|
||
Anchor companion discovery to that selected main source, whether explicit or
|
||
discovered. Configurations in the same directory share companions; there is no
|
||
working-directory fallback. The two companion locations are strict peers, so
|
||
filenames do not resolve conflicting operator records. Missing companions are
|
||
fine; loading errors do not silently fall back to embedded defaults. Preserve
|
||
native imports and source-relative resource paths.
|
||
|
||
Native declarations become the sole prompt assembly mechanism. Keep the embedded
|
||
content tree and module ownership of prompt meaning, response requirements, and
|
||
domain interpretation. Remove prompt/fallback-profile registration in
|
||
`AssetRegistry`, per-module `PromptAssetManifest` plumbing, `ModulePromptFS`,
|
||
shared-name maps, synthetic `sharedassets` mounts, and manual prompt enumeration
|
||
for checkpoint fingerprints. Schema-related registry responsibilities remain
|
||
subject to the explicit schema decision below.
|
||
|
||
Convert maintained resources, examples, and fixtures to native identified lists,
|
||
`config:` source controls, and sibling inheritance (`extends: {id: parent}`).
|
||
Same-ID replacement supplies a complete record; distinct inherited variants can
|
||
customize an existing resource. Explain native profile-null/backend fallback
|
||
semantics. The optional PromptKit source header is version 2, independently of
|
||
Notarius's application version. Do not translate legacy resource formats or
|
||
synthesize compatibility aliases.
|
||
|
||
Select prompts by ID alone; remove obsolete PromptKit version selectors and
|
||
related request/debug metadata. Retain current local resource IDs and use exact
|
||
qualified references, such as `notarius/dnd.spells` and
|
||
`notarius/dnd-extraction`. Keep the extraction profile's existing model/settings
|
||
and its reference to the bundled unqualified `openrouter` backend. Inherited
|
||
application profiles use sibling local IDs in the same collection. The bundled
|
||
inventory differs from the old external catalogs and selects no model for an
|
||
application request; document intentional replacements for removed catalog IDs.
|
||
The bundled backend supplies no concurrency limit. Application/operator limits
|
||
need explicit native configuration; application-supplied defaults remain U7.
|
||
|
||
Preserve prompt prose, message ordering, roles, and cache boundaries where their
|
||
meaning carries forward. Review and deliberately revise initial/repair prompts
|
||
for accumulated conversation history; preserving every old byte or retry message
|
||
is not a migration requirement. Shared content should use native filesystem
|
||
contexts and imports rather than reconstructed virtual trees.
|
||
|
||
PromptKit owns private model-response validation and structural repair. Native
|
||
schemas resolve against the configured schema root, separately from message-file
|
||
contexts, and v0.23.2 requires self-contained schemas loaded during preparation.
|
||
A request override replaces the complete output contract; it must retain the
|
||
intended schema, validation mode, and repair policy. A prompt hash alone does not
|
||
establish schema equality.
|
||
|
||
The custom read-only filesystem, schema root/path selection, module-supplied
|
||
request contracts versus prompt-declared contracts, wrapper removal, and useful
|
||
schema metadata remain open. An ordinary embedded schema filesystem, as used by
|
||
Audita, is a candidate rather than a selected requirement. Durable artifact
|
||
schemas and their validators remain distinct from private response schemas.
|
||
|
||
### Native Production, Continuation, And Assessment
|
||
|
||
Start with one conversation per producer operation: a chunk-plan generation,
|
||
one lane's extraction of one chunk, or one lane's normalization pass. Use
|
||
complete replacement model responses. Normalization may continue to return a
|
||
reconciliation plan that deterministic code applies to produce the artifact.
|
||
Per-entity branching is not part of this migration.
|
||
|
||
The application flow is:
|
||
|
||
1. Prepare source context, references, and candidate-handle mappings once. Build
|
||
the native initial request and call `Engine.Run` with exchange capture when
|
||
continuation is enabled. Capture is independent of detailed debug recording.
|
||
2. Check native structural validation, decode, and apply module interpretation
|
||
and safety checks. A module can supply actionable findings before it has a
|
||
materialized candidate. Run configured artifact validation on each candidate.
|
||
3. For actionable correction, call `Engine.Coordinate` with the captured exchange
|
||
and `AdditionalTurns`. Its callback selects a repair prompt and supplies
|
||
findings; PromptKit retains the initial exchange and subsequent turns.
|
||
4. Interpret every delivered revised response and rebuild from the fixed original
|
||
inputs. Run the complete configured validation chain again for changed
|
||
candidates, including validators that previously approved. Assess the final
|
||
permitted response before declaring exhaustion.
|
||
5. Stop on acceptance or a terminal condition. Apply application rejection,
|
||
failure, or supported safe-fallback policy after exhaustion. Native `stopped`
|
||
and `exhausted` outcomes do not themselves mean application acceptance.
|
||
|
||
Repair prompts supply a new user task; enduring principles belong in the original
|
||
system instructions. Keep source/reference and candidate-handle meanings stable
|
||
throughout the conversation. Avoid replaying the original task and constraints
|
||
that permit only an initial response. Accumulated history increases context use;
|
||
select the semantic turn budget accordingly. Separate repair profiles or schemas
|
||
are supported but not yet selected.
|
||
|
||
Remove `single_response_v1`, raw-response `ModelCandidate`/`SemanticCorrection`
|
||
transport, manual assistant/user replay, and correction-protocol capability
|
||
checks. Replace `runProducerAttempts`' LLM retry loop with native coordination and
|
||
application assessment. Adapt `NormalizeRetry` and `producerRetryDirective` to
|
||
findings and explicit fallback information. Retain useful ordered collection and
|
||
deduplication of actionable findings, rendered through a native repair prompt.
|
||
Reason codes and operator messages are not automatically model instructions.
|
||
|
||
Transport retries, structural repair, and semantic additional turns remain
|
||
separate controls. Preserve the ability to override native transport retries,
|
||
using the upstream default when omitted. Remove blanket outer LLM retries after
|
||
native transport/structural budgets are exhausted, including repeated LLM
|
||
validator invocations in `executeValidationChain`. Keep the ordered validator
|
||
chain and aggregation. A validator execution failure is distinct from rejection
|
||
of a producer candidate. Existing producer/validator `retries` settings cannot be
|
||
mechanically renamed; decide their migration and non-LLM failure handling
|
||
explicitly. Use native `OperationTimeout` when that operation scope is needed,
|
||
without adding an arbitrary default or replacing broader application deadlines.
|
||
|
||
Keep source grounding, reference resolution, candidate isolation, domain
|
||
invariants, and reconciliation safety checks. Unknown or overlapping handles
|
||
remain invalid. A supported safe fallback may retain safe merges while leaving
|
||
unsafe groups separate; it must pass configured artifact validation and terminal
|
||
policy. After structural-repair exhaustion it can be assessed directly without
|
||
restarting the same model operation. Whether an earlier safe candidate can
|
||
survive a later failed turn remains open; do not silently introduce best-so-far
|
||
acceptance.
|
||
|
||
Preserve rejection versus unavailable/incomplete validation. A structurally
|
||
valid candidate may advance under the existing `warn_continue` policy with
|
||
explicit incomplete-validation provenance, but it and its affected derivatives
|
||
cannot be loaded from or published to reusable checkpoint/cache state. A
|
||
completed rejection never advances. Cancellation cannot make an undelivered
|
||
response accepted. Keep conversation state in process, outside ordinary
|
||
manifests and checkpoints; explicitly requested detailed debug is separate.
|
||
|
||
### Admission And Pipeline Workers
|
||
|
||
Remove `Scheduler`, `scheduledClient`, and custom global queue/permit plumbing.
|
||
Retire the separate cross-backend ceiling and the default/maximum extraction
|
||
worker coupling to `concurrency.total_llm`. Keep bounded extraction workers and
|
||
bounded lane continuations with independent defaults to be selected.
|
||
|
||
Use one shared native engine across all roles. Native capacity belongs to the
|
||
engine and backend ID: profiles selecting the same ID share its pool; different
|
||
backend IDs or separate engines do not automatically share limits, even for the
|
||
same endpoint. Endpoint-only profiles and backends without a positive limit
|
||
remain unrestricted by native backend admission.
|
||
|
||
For limited backends, v0.23.2 bounds active model attempts by `concurrency_limit`
|
||
and admitted operations by that limit plus `queue_capacity`. Admission spans
|
||
preparation, retries, and repairs; active permits are released during retry waits.
|
||
A completed continuation turn releases admission before assessment, allowing a
|
||
nested validator to use the same backend. Configure native limits at the intended
|
||
capacity boundary and handle `ErrCapacityExceeded` through application policy.
|
||
|
||
Aggregate concurrency across independent backends may exceed the former global
|
||
ceiling. That change is intentional. Bounded pipeline workers are application
|
||
work controls, not a substitute cross-backend LLM ceiling. Defaults, retired-field
|
||
migration, and role-specific admission-failure dispositions remain open.
|
||
|
||
### Checkpoint Reuse For Successful Work
|
||
|
||
Checkpoints support recovery after a downstream failure: fixing a later stage
|
||
should not require rerunning earlier successful work. Use this conceptual key:
|
||
|
||
```text
|
||
logical stage identity + input-data identity + prompt ID, where applicable
|
||
```
|
||
|
||
Scope logical identity to the relevant step/lane/stage and artifact kind, without
|
||
hashing complete configuration or implementations. Inputs include actual source
|
||
or chunk data, reference material, and consumed upstream artifact content.
|
||
Deterministic stages need no synthetic prompt ID. Content digests can represent
|
||
inputs and verify payload integrity without fingerprinting the execution setup.
|
||
|
||
| Change | Intended reuse behavior |
|
||
| --- | --- |
|
||
| Fix a later failed stage | Reuse successful predecessors with matching logical work and inputs. |
|
||
| Edit prompt content while keeping its ID | Reuse; explicit recomputation applies the edit. |
|
||
| Change model, generation settings, implementation, or validation policy | Reuse otherwise usable completed work; explicit recomputation applies improvements. |
|
||
| Change the prompt ID or consumed source/reference/upstream data | No matching checkpoint. |
|
||
| Request explicit recomputation | Recompute the selected work and its dependents. |
|
||
|
||
Keep recording separate from reuse. The existing controls are
|
||
[`--resume` and `--recompute-step`](../cli.md#run); migrate their compatibility
|
||
behavior to this policy. A matching key does not make an unreadable, corrupted,
|
||
or incompatible artifact usable. Retain acceptance/completion status, integrity,
|
||
and decode/consumer compatibility checks. Exact schema hash equality is not
|
||
required for a compatible artifact.
|
||
|
||
Only accepted, completely validated work is reusable. Preserve original
|
||
execution and acceptance provenance; do not rerun semantic validators solely
|
||
because their configuration changed or claim the old artifact passed a new
|
||
policy. Retain the exclusion of rejected, structurally invalid, and
|
||
validation-incomplete work and its ineligible derivatives. The accepted tradeoff
|
||
is reuse of results produced by prompts, settings, or code that have improved.
|
||
|
||
Remove whole PromptKit configuration, catalog/library versions, model settings,
|
||
prompt-content hashes, and component-policy fingerprints as matching criteria.
|
||
Native hashes may remain diagnostic metadata. Revise storage/lookup scoping so
|
||
an invocation-wide pipeline/configuration digest cannot still block recovery.
|
||
There is no need for an upstream semantic-equality feature, persisted conversation,
|
||
native prepared-execution handle, or temporary conservative fingerprint.
|
||
|
||
Keep the separate source-addressed canonical chunk-plan cache policy, including
|
||
current validation of materialized chunks. Concrete checkpoint identifiers,
|
||
input representation, old-record transition, and incompatible-payload reporting
|
||
remain open.
|
||
|
||
### Inspection, Bookkeeping, And Presentation
|
||
|
||
Use native configuration/resource inspection rather than effective-YAML parsing,
|
||
a profile-inspector wrapper, or copied effective-model-parameter types. A profile
|
||
can be partial; its inspection is not the completed execution target. Use actual
|
||
prepared/result data for execution provenance, including request overrides, and
|
||
record the profiles/targets used rather than the available catalog.
|
||
|
||
Keep application pipeline preflight for modules, typed artifact compatibility,
|
||
reference bindings, and known prompt/profile selections. `config validate` checks
|
||
resolved application configuration; with a pipeline selection, load the same
|
||
native definitions used by execution, including a known prompt's default profile
|
||
when effective. Do not simulate stages to discover dynamic selections. Preflight
|
||
needs no transcript, credential values, or provider call. Use native requirement
|
||
inspection for explanation and concrete `Prepare`/`ValidateRequest` for completed
|
||
target, input, schema, and credential checks when requested. Partial profiles
|
||
alone are not preflight failures.
|
||
|
||
Use ordinary `Run` with `OnPrepared` for needed actual request details and optional
|
||
debug capture. Remove forced `PrepareExecution`/`RunPrepared` solely for recording.
|
||
`OnPrepared` runs after preparation/admission and before generation; it is absent
|
||
on preparation/admission failure and cannot be a terminal record. Native
|
||
observations supply terminal facts even when a failed call returns no `RunResult`.
|
||
Keep callbacks lightweight and synchronize shared observation state as required.
|
||
|
||
Use native `RunID`/`ParentRunID` plus application step/lane/chunk/validator scope.
|
||
Remove duplicate `LLMExecutionSummary`, usage types, enum conversions, cloning,
|
||
independent accumulators, debug completion wrappers, and redundant synthetic
|
||
call IDs. Replace adapter profile-recording/forwarding with only the projection
|
||
needed at application boundaries. Preserve native failure distinctions while
|
||
Notarius chooses domain outcomes and exit-status mapping.
|
||
|
||
Use `ExecutionGroup` for totals over selected executions. Enroll the initial run,
|
||
continuation runs, and nested validator executions explicitly as applicable.
|
||
Preserve terminal coverage and usage completeness: unknown usage is not zero.
|
||
Avoid counting both attempt and terminal aggregates or adding overlapping groups.
|
||
Provider execution success remains separate from domain acceptance.
|
||
|
||
Retain application records of acceptance, rejection, validation completeness,
|
||
fallback, generation versus reuse, and pipeline outcome. Reused artifacts retain
|
||
original provenance; the current invocation records reuse without counting old
|
||
token usage as current work. Keep small deliberate projections for the
|
||
[published manifest](../integrations/json-output.md) and
|
||
[run-result receipt](../integrations/run-result.md), with explicit compatibility
|
||
decisions. Do not serialize complete upstream runtime types as durable contracts.
|
||
|
||
Notarius retains debug placement, opt-in capture, retention, and disclosure
|
||
handling, including credential redaction. Native callbacks provide the data;
|
||
ordinary provenance retains only needed fields, and sensitive response/history
|
||
content stays out of ordinary records. Debug is not checkpoint input.
|
||
|
||
CommandKit with ckconfig/ckprompt is the selected owner of standardized
|
||
configuration/resource commands, requirements, validation, preview, opt-in smoke
|
||
diagnostics, formatting, visibility, and progress presentation. Notarius supplies
|
||
loaders, request builders, report content, pipeline meaning, and outcome mapping.
|
||
Keep machine-output receipts deliberate and presentation delivery failures
|
||
distinguishable from operation outcomes. Until that follow-up lands, make only
|
||
the changes needed to keep existing inspection commands usable through native
|
||
APIs; do not build temporary custom commands or a new reporting framework.
|
||
|
||
## Unresolved Choices
|
||
|
||
The directions above are selected. The following details are not. Resolve and
|
||
record each choice before implementing the work that depends on it; independent
|
||
work need not wait for every choice. Candidates here are not implicit defaults.
|
||
|
||
| ID | Choice still needed | Resolve for |
|
||
| --- | --- | --- |
|
||
| U1 | Final compatible tagged PromptKit/ConfigKit/dependency set; any specific unmet need requiring an upstream request. v0.23.2 is the initial target, not a restriction on useful upstream work. | Phase 1; revisit if the target or upstream needs change. |
|
||
| U2 | Remaining custom read-only filesystem responsibilities; schema root/path and assembly, prompt-declared versus per-request contracts, removable wrappers, and metadata useful for manifests/artifact compatibility. | Phase 2 source and schema assembly. |
|
||
| U3 | Concrete module/pipeline signatures and placement of engine assembly, request construction, interpretation, assessment, and focused helpers. | Phase 3 representative workflow, refined during Phase 4. |
|
||
| U4 | Initial and repair prompt definitions, response contracts, and profile selection within complete replacement and fixed operation inputs. | Phases 3–4 for each producer role. |
|
||
| U5 | Mapping producer/validator retry settings to native controls; names, defaults, compatibility, non-LLM failure handling, and any explicitly justified recovery beyond native exhaustion. | Phases 3–4 and configuration migration. |
|
||
| U6 | Detailed terminal/fallback dispositions, especially whether any earlier safe candidate survives a later failed turn. | Phases 3–4; no implicit best-so-far acceptance. |
|
||
| U7 | Independent worker defaults, native backend/queue defaults where supplied, retired `total_llm` migration, and admission-failure handling by role. | Phase 4 shared-engine cutover. |
|
||
| U8 | Logical checkpoint/prompt identifiers, consumed-input representation, storage scoping, old-record transition, and load-compatibility reporting. | Phase 5 recovery migration. |
|
||
| U9 | Minimal durable fields and their compatibility treatment; observer/group enrollment, application scope, and debug write/disclosure handling. | Design with Phase 3, complete in Phase 5. |
|
||
| U10 | CommandKit/ckconfig/ckprompt versions, command exposure and compatibility, and application wiring for the follow-up. | Phase 7; do not recreate its machinery earlier. |
|
||
|
||
## Implementation Sequence
|
||
|
||
Each phase is pending. Phases describe reviewable outcomes and dependencies,
|
||
not separate releases. The supported core result is the hard cutover; any
|
||
intermediate migration scaffolding must retire before that result is complete.
|
||
Update affected canonical documentation as behavior lands. Record significant
|
||
boundary decisions in successor ADRs without rewriting accepted decision text.
|
||
|
||
### Phase 1: Establish The Migration Baseline
|
||
|
||
- Confirm U1 against the tagged APIs and dependency graph. Carry forward the
|
||
Audita evidence without treating its wrappers as requirements.
|
||
- Map the changed architecture and ADRs, and identify concrete decisions needed
|
||
for the next phase. Keep unresolved entries visible until actually settled.
|
||
- Identify the production callers and old contracts covered by the removal
|
||
inventory below; use that inventory to track complete cutover coverage.
|
||
|
||
**Completion evidence:** selected dependency baseline and decision/implementation
|
||
scope are recorded; unresolved choices have a clear dependent phase. No new
|
||
upstream feature is assumed necessary without an identified requirement.
|
||
|
||
### Phase 2: Assemble Native Sources And A Shared Engine
|
||
|
||
- Resolve U2, update dependencies, and implement version 6 and companion-source
|
||
composition. Convert embedded resources and establish the schema loading path.
|
||
- Construct the shared native engine and use the same assembly for execution,
|
||
preflight, and inspection. Remove duplicate prompt assembly as callers migrate.
|
||
- Add maintained embedded-only and operator-override examples with the new format.
|
||
|
||
**Completion evidence:** offline preparation proves resource selection, shared
|
||
content/schema resolution, source precedence, and operator replacement. Invalid
|
||
sources fail rather than silently selecting defaults. Preflight needs no provider
|
||
credentials or synthetic stage execution.
|
||
|
||
### Phase 3: Prove Native Production And Assessment
|
||
|
||
- Resolve U3 and the relevant U4–U6/U9 choices for a representative extraction
|
||
workflow. Connect native generation, domain interpretation, configured
|
||
validation, and continuation through the small application function.
|
||
- Exercise a configured LLM validator on the same engine. Attach native observers,
|
||
application scope, and selected execution-group membership from the start.
|
||
- Exercise normalization's fixed candidate handles and fresh plan application,
|
||
including the selected terminal/fallback policy.
|
||
|
||
**Completion evidence:** fake-provider workflows demonstrate accumulated history,
|
||
full revalidation, final-turn acceptance, correct exhaustion/failure handling,
|
||
and typed application outcomes without the private completion/correction API.
|
||
|
||
### Phase 4: Migrate All Roles And Retire Runtime Machinery
|
||
|
||
- Apply the established flow to every LLM-backed chunker, extractor, normalizer,
|
||
and validator. Preserve deterministic stage paths and ordered validation.
|
||
- Complete U4–U7 across roles. Migrate retry controls, remove the global scheduler,
|
||
and give bounded pipeline workers independent configuration.
|
||
- Remove the adapter, copied completion contracts, correction-protocol checks,
|
||
outer LLM retry loops, and redundant asset registration once unused.
|
||
|
||
**Completion evidence:** all production roles share native execution and admission;
|
||
no application path still depends on the retired completion API, global ceiling,
|
||
or latest-response-only correction. Native failures and cancellation settle
|
||
through the selected role policy. Tests for obsolete machinery retire with it.
|
||
|
||
### Phase 5: Simplify Recovery And Execution Records
|
||
|
||
- Resolve U8–U9. Implement logical-work/input/prompt checkpoint matching and revise
|
||
storage scoping so unrelated configuration changes cannot defeat recovery.
|
||
- Implement and document old-checkpoint handling, artifact usability checks,
|
||
original acceptance provenance, and explicit recomputation behavior.
|
||
- Complete native inspection, observation/group accounting, minimal durable
|
||
projections, and optional debug capture; remove the duplicate bookkeeping.
|
||
|
||
**Completion evidence:** a downstream-failure/fix/resume scenario reuses successful
|
||
predecessors; settings changes do not spuriously invalidate them, consumed-data
|
||
changes do, and reused work does not count as current execution. Published records
|
||
follow the selected compatibility policy and preserve failure/usage distinctions.
|
||
|
||
### Phase 6: Verify And Document The Core Cutover
|
||
|
||
- Complete the acceptance checks below and the removal inventory. Verify the
|
||
dependency graph contains no unused old-path PromptKit/ConfigKit or catalogs.
|
||
- Finish migration guidance, maintained examples, attribution, successor ADRs,
|
||
and canonical documentation updates. Reconcile affected future-work notes.
|
||
- Keep existing inspection commands usable with native reports. Document that
|
||
the standardized CommandKit command surface remains the subsequent phase.
|
||
|
||
**Completion evidence:** the core native architecture, configuration migration,
|
||
checkpoint policy, diagnostics, and maintained offline workflows are verified;
|
||
remaining work is explicitly the CommandKit follow-up or separately identified
|
||
future work, not hidden adapter or retry compatibility machinery.
|
||
|
||
### Phase 7: Adopt CommandKit Presentation
|
||
|
||
- Resolve U10 and adopt CommandKit with ckconfig/ckprompt. Supply application and
|
||
engine loaders plus request builders for the selected diagnostic commands.
|
||
- Replace applicable Notarius-owned configuration/PromptKit commands and standard
|
||
presentation while preserving deliberate machine-output contracts and domain
|
||
outcome mapping. Keep provider smoke diagnostics explicitly opt-in.
|
||
|
||
**Completion evidence:** standard commands use upstream integrations and the same
|
||
native source assembly as execution; application preflight and published receipts
|
||
retain their intended meaning. This phase completes the presentation target and
|
||
is not a prerequisite for completing the core PromptKit cutover.
|
||
|
||
## Removal Inventory
|
||
|
||
| Retire or replace | Retained application responsibility |
|
||
| --- | --- |
|
||
| `StructuredLLMClient`, private structured-completion request/response types, adapter conversions | Native request construction and typed domain interpretation. |
|
||
| Prompt/profile `AssetRegistry` plumbing, `PromptAssetManifest`, `ModulePromptFS`, synthetic shared mounts, manual fingerprint enumeration | Embedded content and module semantics; schema responsibilities resolved under U2. |
|
||
| `single_response_v1`, raw `ModelCandidate`/`SemanticCorrection`, manual replay, correction capability checks | Typed candidates, actionable findings, native exchanges, and assessment. |
|
||
| LLM retry portions of `runProducerAttempts` and `executeValidationChain`, module rerun directives | Ordered validators, acceptance/rejection/fallback policy, and useful provenance. |
|
||
| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, worker coupling | Bounded pipeline work and native per-backend admission. |
|
||
| Broad configuration/prompt/catalog/implementation checkpoint fingerprints and invocation-wide lookup barriers | Logical work, consumed-input identity, load checks, and explicit recomputation. |
|
||
| Inspection wrappers, copied effective parameters, `LLMExecutionSummary` and usage conversions/accumulators, debug completion wrapper, redundant call IDs, adapter profile recorder | Native facts with application scope and selected durable projections. |
|
||
| Custom standard inspection commands/presentation in the CommandKit phase | Application loaders, request builders, report content, and outcome mapping. |
|
||
|
||
## Acceptance And Verification
|
||
|
||
Use focused behavioral tests with real deterministic collaborators, a native
|
||
engine, and injected clients or a fake HTTP provider. Test domain interpretation
|
||
and assessment directly; pipeline tests can use fake stages. Replace tests that
|
||
preserve removed mechanics rather than reproducing a private LLM interface for
|
||
testing. Default tests remain offline and independent of real credentials.
|
||
|
||
| Area | Required evidence |
|
||
| --- | --- |
|
||
| Configuration and assets | Version 6/new resources work; legacy formats are rejected intentionally; main-relative companions and native imports work independently of working directory; peer conflicts fail; same-ID replacement affects only the intended default; ordering/cache controls and selected schema contracts survive assembly. |
|
||
| Preflight and inspection | Runtime, inspection, and preflight share effective definitions; unknown statically selected IDs and invalid definitions fail offline; partial profiles alone pass; completed target checks occur on concrete requests; actual-run overrides appear in provenance. |
|
||
| Native artifact workflow | Representative extraction and normalization produce typed candidates, invoke ordered validators, and preserve source/reference identity, safe plan application, artifact-schema checks, and stable output ordering. |
|
||
| Continuation and failures | Multiple turns accumulate history with fixed input/handle meanings; every revised candidate and final allowed response is assessed; native transport/structural exhaustion is not silently retried outside PromptKit; validator failure differs from rejection; selected safe fallback and cancellation policy hold. |
|
||
| Admission and workers | All roles share the selected backend pool; independent backends remain independent; pipeline work stays bounded; queue exhaustion, nested validators, and cancellation settle without leaked capacity or a retained global ceiling. |
|
||
| Checkpoints | Successful predecessors survive downstream fixes and prompt/model/validation-setting changes; changed prompt IDs or consumed source/reference/upstream data miss; recompute forces selected work/dependents; corrupt/incompatible or unaccepted/incomplete artifacts and ineligible derivatives are unusable; compatible schema changes do not require exact hash equality. |
|
||
| Reuse provenance | Original acceptance and execution provenance survive resume without new semantic validation claims or historical token usage counted as current; recording remains separate from reuse; canonical chunk-plan caching retains its distinct policy. |
|
||
| Execution records | Initial/continuation/nested validator accounting preserves terminal coverage and unknown-versus-zero usage without double counting; failures without a result remain observable; native success is distinct from acceptance; debug is opt-in and ordinary records exclude sensitive response/history content. |
|
||
| Published behavior | Maintained minimal and complete D&D workflows, session routing, artifact output, manifest/receipt compatibility, and selected migration diagnostics work through the assembled CLI. CommandKit-phase tests cover its own command/presentation changes. |
|
||
|
||
Run focused checks while implementing and the repository-wide commands for the
|
||
completed application cutover, following the [testing policy](../policy/testing.md):
|
||
|
||
```sh
|
||
go test ./...
|
||
go vet ./...
|
||
go build ./cmd/notarius
|
||
```
|
||
|
||
Check maintained examples, dependency removal, links, and attribution alongside
|
||
those commands. Live provider checks and human model-quality evaluation can
|
||
inform deployment and prompt refinement; they are not requirements of the
|
||
deterministic offline suite. Planning/documentation completion does not claim
|
||
that these implementation acceptance checks have already passed.
|
||
|
||
## Migration Guidance And Documentation
|
||
|
||
Maintain examples that distinguish embedded-only operation from companion
|
||
configuration, using separate directories when needed. Explain moving/importing
|
||
old sources, qualified IDs, complete replacement versus inheritance, native local
|
||
backends, removed catalogs and prompt versions, and the separate application and
|
||
PromptKit source versions. Document worker/global-limit changes, retry-budget
|
||
migration, accumulated conversation costs, and the selected failure policy.
|
||
|
||
Explain successful-work checkpoint reuse, explicit recomputation to apply prompt
|
||
or setting improvements, consumed-data misses, original validation provenance,
|
||
and the chosen old-record transition. Document changed inspection and durable
|
||
fields deliberately rather than promising incidental old adapter output.
|
||
|
||
Update the canonical [configuration](../config.md), [CLI](../cli.md),
|
||
[operations](../operations.md), [internal overview](../internal/overview.md),
|
||
[LLM](../internal/llm.md), [pipeline](../internal/pipeline.md),
|
||
[modules](../internal/modules.md), [state](../internal/state.md), and affected
|
||
integration/consumer guides when their behavior changes. Update the
|
||
[architecture policy](../policy/architecture.md) to make native PromptKit an
|
||
explicit execution dependency and remove the superseded scheduler, retry,
|
||
asset-registration, and fingerprint constraints.
|
||
|
||
Record significant ownership and contract changes in successor ADRs, including
|
||
those affecting [asset assembly](../adr/0011-centralize-llm-assets.md),
|
||
[validation retries](../adr/0014-feedback-aware-validation-retries.md),
|
||
[module-requested retries](../adr/0016-feedback-aware-module-requested-retries.md),
|
||
and [PromptKit configuration](../adr/0019-adopt-native-promptkit-configuration.md).
|
||
Also record the changed admission and checkpoint matching decisions. Preserve
|
||
accepted decision text and update supersession status as appropriate under the
|
||
[documentation policy](../policy/documentation.md#architecture-decision-records).
|
||
The discussion record supplies rationale and evidence; this roadmap tracks
|
||
implementation status until the corresponding behavior lands.
|
||
|
||
[release]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/releases/v0.23.2.md
|
||
[migration]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/consumers/configuration-migration.md
|
||
[formats]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/formats.md
|
||
[consumer]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/consumers/pkg-promptkit.md
|