38 KiB
PromptKit-Native Architecture Migration
Purpose And Status
Make Notarius fully PromptKit-native: use upstream configuration, execution, continuation, admission, inspection, and accounting directly, and retain the application work of producing, assessing, storing, and reusing typed artifacts. Historical duplication of PromptKit behavior is not a requirement to preserve it.
This is the implementation roadmap for the agreed directions in items 1–6 of the architecture discussion record. That document retains the discussion and Audita evidence. This roadmap owns the selected target, unresolved choices, implementation sequence, and acceptance criteria. All implementation phases below are pending; rewriting the roadmap has not changed application behavior.
The core migration is a hard cutover for application configuration and PromptKit resource definitions. Supply migration guidance and maintained examples; support only the new formats in the resulting application. CommandKit adoption remains a subsequent delivery phase with its ownership accounted for throughout this plan. Release preparation and publication are separate work.
Baseline And Dependency Target
At Notarius revision 12eeb6369e40746b2c18c55397a289a1e1b5908d, the application
uses configuration version 5 and Go 1.27. Its dependencies include old-path
PromptKit v0.12.0 and ConfigKit v0.5.0, new-path ConfigKit v1.2.0, go-yaml
v4.0.0-rc.6, and two external PromptKit provider catalogs.
The initial dependency target remains go.maximumdirect.net/promptkit@v0.23.2
(tagged revision 382ccfb14b9630fd8f48c8364dbe2b6159637d71), with shared
ConfigKit v1.2.1, stable go-yaml v4, and compatible validation dependencies.
The Go consumer minimum remains 1.27. The tagged release notes,
migration guide, format reference, and
Go consumer guide supply the upstream baseline.
Confirm the final compatible tagged dependency set before implementation.
A demonstrated unmet application need may justify an upstream request and a
later target; the agreed design does not currently require a new PromptKit
feature. In particular, neither a cross-backend global limit nor a comprehensive
execution fingerprint is a prerequisite. Remove obsolete imports and catalog
dependencies when unused; do not retain a compatibility fork or permanent
replace directive. Update attribution under the OSS policy.
Audita's reviewed v0.23.2 integration demonstrates native source assembly,
continuation, observations, and CommandKit wiring. Adopt those mechanisms
without copying its RoleClient, structured-output interface, or opaque exchange
wrapper. Its per-proposal repair branches and lack of checkpoints do not define
Notarius's artifact and recovery policies.
Target Architecture
Application Structure And Ownership
| Part | Responsibility |
|---|---|
| Application composition | Load configuration and native sources; construct one shared PromptKit engine for producers, validators, and continuation runs. |
| Artifact-family modules | Prepare domain inputs, select prompt IDs and response requirements, interpret responses, and apply deterministic domain rules. |
| Pipeline execution | Connect generation to configured validation and terminal policy; manage ordered dependencies, typed handoffs, bounded workers, and stable result ordering. |
| PromptKit | Prepare and execute requests; maintain captured exchanges and conversations; enforce native retry, repair, turn, and admission budgets; report execution facts. |
| Application state and records | Store artifacts, apply checkpoint reuse policy, preserve acceptance provenance, and attach pipeline context to execution facts. |
| ConfigKit and CommandKit | ConfigKit owns configuration mechanics; CommandKit and ckconfig/ckprompt own standardized commands and presentation. Notarius supplies loaders, request builders, report content, and outcome mapping. |
Use native PromptKit requests, results, exchanges, errors, and observers in model-executing application code. Remove the private provider-neutral completion API. Pure domain models, codecs, source-reference resolution, and deterministic transformations need no PromptKit dependency. Application interfaces continue to express artifact production, typed validation, output encoding, and state access.
A small shared application function connects module request construction and
response interpretation to pipeline validation and terminal policy. It calls
native Run and Coordinate and returns a typed application outcome. It owns no
replacement completion API, conversation protocol, or general retry engine.
Exact signatures and helper placement remain open. The current
internal/framework/llm package may shrink to engine assembly and focused helpers
or cease to be a separate package.
Retain the fixed input -> chunk -> extract -> merge -> normalize -> output
shape, ordered steps and generated references, typed artifact-family boundaries,
and deterministic public ordering. Keep modules independent of physical output,
cache, and debug roots. Native adoption does not turn the pipeline into a general
workflow engine or redesign durable D&D artifacts.
Configuration, Sources, And Prompt Assets
Introduce application configuration version 6 as the only supported version.
Preserve unrelated discovery, pipeline structure, and resolution behavior.
ConfigKit continues to own loading and composition under
ADR-0020.
Retire promptkit.profile_file, profile_dir, and local_backend in favor of
native companion sources; do not introduce a separate promptkit.config_file.
Retire concurrency.total_llm and revise retry/worker settings as described below.
Their replacement defaults and detailed migration treatment remain open.
Use one source assembly for runtime, preflight, and inspection:
- PromptKit's bundled OpenRouter underlay.
- Notarius's embedded native prompts and profiles under
collections.notarius. - An optional, higher-priority operator underlay containing
prompts.ymland recursive.yml/.yamlfiles underprompts/, beside the main application configuration actually selected by ConfigKit.
Anchor companion discovery to that selected main source, whether explicit or discovered. Configurations in the same directory share companions; there is no working-directory fallback. The two companion locations are strict peers, so filenames do not resolve conflicting operator records. Missing companions are fine; loading errors do not silently fall back to embedded defaults. Preserve native imports and source-relative resource paths.
Native declarations become the sole prompt assembly mechanism. Keep the embedded
content tree and module ownership of prompt meaning, response requirements, and
domain interpretation. Remove prompt/fallback-profile registration in
AssetRegistry, per-module PromptAssetManifest plumbing, ModulePromptFS,
shared-name maps, synthetic sharedassets mounts, and manual prompt enumeration
for checkpoint fingerprints. Schema-related registry responsibilities remain
subject to the explicit schema decision below.
Convert maintained resources, examples, and fixtures to native identified lists,
config: source controls, and sibling inheritance (extends: {id: parent}).
Same-ID replacement supplies a complete record; distinct inherited variants can
customize an existing resource. Explain native profile-null/backend fallback
semantics. The optional PromptKit source header is version 2, independently of
Notarius's application version. Do not translate legacy resource formats or
synthesize compatibility aliases.
Select prompts by ID alone; remove obsolete PromptKit version selectors and
related request/debug metadata. Retain current local resource IDs and use exact
qualified references, such as notarius/dnd.spells and
notarius/dnd-extraction. Keep the extraction profile's existing model/settings
and its reference to the bundled unqualified openrouter backend. Inherited
application profiles use sibling local IDs in the same collection. The bundled
inventory differs from the old external catalogs and selects no model for an
application request; document intentional replacements for removed catalog IDs.
The bundled backend supplies no concurrency limit. Application/operator limits
need explicit native configuration; application-supplied defaults remain U7.
Preserve prompt prose, message ordering, roles, and cache boundaries where their meaning carries forward. Review and deliberately revise initial/repair prompts for accumulated conversation history; preserving every old byte or retry message is not a migration requirement. Shared content should use native filesystem contexts and imports rather than reconstructed virtual trees.
PromptKit owns private model-response validation and structural repair. Native schemas resolve against the configured schema root, separately from message-file contexts, and v0.23.2 requires self-contained schemas loaded during preparation. A request override replaces the complete output contract; it must retain the intended schema, validation mode, and repair policy. A prompt hash alone does not establish schema equality.
The custom read-only filesystem, schema root/path selection, module-supplied request contracts versus prompt-declared contracts, wrapper removal, and useful schema metadata remain open. An ordinary embedded schema filesystem, as used by Audita, is a candidate rather than a selected requirement. Durable artifact schemas and their validators remain distinct from private response schemas.
Native Production, Continuation, And Assessment
Start with one conversation per producer operation: a chunk-plan generation, one lane's extraction of one chunk, or one lane's normalization pass. Use complete replacement model responses. Normalization may continue to return a reconciliation plan that deterministic code applies to produce the artifact. Per-entity branching is not part of this migration.
The application flow is:
- Prepare source context, references, and candidate-handle mappings once. Build
the native initial request and call
Engine.Runwith exchange capture when continuation is enabled. Capture is independent of detailed debug recording. - Check native structural validation, decode, and apply module interpretation and safety checks. A module can supply actionable findings before it has a materialized candidate. Run configured artifact validation on each candidate.
- For actionable correction, call
Engine.Coordinatewith the captured exchange andAdditionalTurns. Its callback selects a repair prompt and supplies findings; PromptKit retains the initial exchange and subsequent turns. - Interpret every delivered revised response and rebuild from the fixed original inputs. Run the complete configured validation chain again for changed candidates, including validators that previously approved. Assess the final permitted response before declaring exhaustion.
- Stop on acceptance or a terminal condition. Apply application rejection,
failure, or supported safe-fallback policy after exhaustion. Native
stoppedandexhaustedoutcomes do not themselves mean application acceptance.
Repair prompts supply a new user task; enduring principles belong in the original system instructions. Keep source/reference and candidate-handle meanings stable throughout the conversation. Avoid replaying the original task and constraints that permit only an initial response. Accumulated history increases context use; select the semantic turn budget accordingly. Separate repair profiles or schemas are supported but not yet selected.
Remove single_response_v1, raw-response ModelCandidate/SemanticCorrection
transport, manual assistant/user replay, and correction-protocol capability
checks. Replace runProducerAttempts' LLM retry loop with native coordination and
application assessment. Adapt NormalizeRetry and producerRetryDirective to
findings and explicit fallback information. Retain useful ordered collection and
deduplication of actionable findings, rendered through a native repair prompt.
Reason codes and operator messages are not automatically model instructions.
Transport retries, structural repair, and semantic additional turns remain
separate controls. Preserve the ability to override native transport retries,
using the upstream default when omitted. Remove blanket outer LLM retries after
native transport/structural budgets are exhausted, including repeated LLM
validator invocations in executeValidationChain. Keep the ordered validator
chain and aggregation. A validator execution failure is distinct from rejection
of a producer candidate. Existing producer/validator retries settings cannot be
mechanically renamed; decide their migration and non-LLM failure handling
explicitly. Use native OperationTimeout when that operation scope is needed,
without adding an arbitrary default or replacing broader application deadlines.
Keep source grounding, reference resolution, candidate isolation, domain invariants, and reconciliation safety checks. Unknown or overlapping handles remain invalid. A supported safe fallback may retain safe merges while leaving unsafe groups separate; it must pass configured artifact validation and terminal policy. After structural-repair exhaustion it can be assessed directly without restarting the same model operation. Whether an earlier safe candidate can survive a later failed turn remains open; do not silently introduce best-so-far acceptance.
Preserve rejection versus unavailable/incomplete validation. A structurally
valid candidate may advance under the existing warn_continue policy with
explicit incomplete-validation provenance, but it and its affected derivatives
cannot be loaded from or published to reusable checkpoint/cache state. A
completed rejection never advances. Cancellation cannot make an undelivered
response accepted. Keep conversation state in process, outside ordinary
manifests and checkpoints; explicitly requested detailed debug is separate.
Admission And Pipeline Workers
Remove Scheduler, scheduledClient, and custom global queue/permit plumbing.
Retire the separate cross-backend ceiling and the default/maximum extraction
worker coupling to concurrency.total_llm. Keep bounded extraction workers and
bounded lane continuations with independent defaults to be selected.
Use one shared native engine across all roles. Native capacity belongs to the engine and backend ID: profiles selecting the same ID share its pool; different backend IDs or separate engines do not automatically share limits, even for the same endpoint. Endpoint-only profiles and backends without a positive limit remain unrestricted by native backend admission.
For limited backends, v0.23.2 bounds active model attempts by concurrency_limit
and admitted operations by that limit plus queue_capacity. Admission spans
preparation, retries, and repairs; active permits are released during retry waits.
A completed continuation turn releases admission before assessment, allowing a
nested validator to use the same backend. Configure native limits at the intended
capacity boundary and handle ErrCapacityExceeded through application policy.
Aggregate concurrency across independent backends may exceed the former global ceiling. That change is intentional. Bounded pipeline workers are application work controls, not a substitute cross-backend LLM ceiling. Defaults, retired-field migration, and role-specific admission-failure dispositions remain open.
Checkpoint Reuse For Successful Work
Checkpoints support recovery after a downstream failure: fixing a later stage should not require rerunning earlier successful work. Use this conceptual key:
logical stage identity + input-data identity + prompt ID, where applicable
Scope logical identity to the relevant step/lane/stage and artifact kind, without hashing complete configuration or implementations. Inputs include actual source or chunk data, reference material, and consumed upstream artifact content. Deterministic stages need no synthetic prompt ID. Content digests can represent inputs and verify payload integrity without fingerprinting the execution setup.
| Change | Intended reuse behavior |
|---|---|
| Fix a later failed stage | Reuse successful predecessors with matching logical work and inputs. |
| Edit prompt content while keeping its ID | Reuse; explicit recomputation applies the edit. |
| Change model, generation settings, implementation, or validation policy | Reuse otherwise usable completed work; explicit recomputation applies improvements. |
| Change the prompt ID or consumed source/reference/upstream data | No matching checkpoint. |
| Request explicit recomputation | Recompute the selected work and its dependents. |
Keep recording separate from reuse. The existing controls are
--resume and --recompute-step; migrate their compatibility
behavior to this policy. A matching key does not make an unreadable, corrupted,
or incompatible artifact usable. Retain acceptance/completion status, integrity,
and decode/consumer compatibility checks. Exact schema hash equality is not
required for a compatible artifact.
Only accepted, completely validated work is reusable. Preserve original execution and acceptance provenance; do not rerun semantic validators solely because their configuration changed or claim the old artifact passed a new policy. Retain the exclusion of rejected, structurally invalid, and validation-incomplete work and its ineligible derivatives. The accepted tradeoff is reuse of results produced by prompts, settings, or code that have improved.
Remove whole PromptKit configuration, catalog/library versions, model settings, prompt-content hashes, and component-policy fingerprints as matching criteria. Native hashes may remain diagnostic metadata. Revise storage/lookup scoping so an invocation-wide pipeline/configuration digest cannot still block recovery. There is no need for an upstream semantic-equality feature, persisted conversation, native prepared-execution handle, or temporary conservative fingerprint.
Keep the separate source-addressed canonical chunk-plan cache policy, including current validation of materialized chunks. Concrete checkpoint identifiers, input representation, old-record transition, and incompatible-payload reporting remain open.
Inspection, Bookkeeping, And Presentation
Use native configuration/resource inspection rather than effective-YAML parsing, a profile-inspector wrapper, or copied effective-model-parameter types. A profile can be partial; its inspection is not the completed execution target. Use actual prepared/result data for execution provenance, including request overrides, and record the profiles/targets used rather than the available catalog.
Keep application pipeline preflight for modules, typed artifact compatibility,
reference bindings, and known prompt/profile selections. config validate checks
resolved application configuration; with a pipeline selection, load the same
native definitions used by execution, including a known prompt's default profile
when effective. Do not simulate stages to discover dynamic selections. Preflight
needs no transcript, credential values, or provider call. Use native requirement
inspection for explanation and concrete Prepare/ValidateRequest for completed
target, input, schema, and credential checks when requested. Partial profiles
alone are not preflight failures.
Use ordinary Run with OnPrepared for needed actual request details and optional
debug capture. Remove forced PrepareExecution/RunPrepared solely for recording.
OnPrepared runs after preparation/admission and before generation; it is absent
on preparation/admission failure and cannot be a terminal record. Native
observations supply terminal facts even when a failed call returns no RunResult.
Keep callbacks lightweight and synchronize shared observation state as required.
Use native RunID/ParentRunID plus application step/lane/chunk/validator scope.
Remove duplicate LLMExecutionSummary, usage types, enum conversions, cloning,
independent accumulators, debug completion wrappers, and redundant synthetic
call IDs. Replace adapter profile-recording/forwarding with only the projection
needed at application boundaries. Preserve native failure distinctions while
Notarius chooses domain outcomes and exit-status mapping.
Use ExecutionGroup for totals over selected executions. Enroll the initial run,
continuation runs, and nested validator executions explicitly as applicable.
Preserve terminal coverage and usage completeness: unknown usage is not zero.
Avoid counting both attempt and terminal aggregates or adding overlapping groups.
Provider execution success remains separate from domain acceptance.
Retain application records of acceptance, rejection, validation completeness, fallback, generation versus reuse, and pipeline outcome. Reused artifacts retain original provenance; the current invocation records reuse without counting old token usage as current work. Keep small deliberate projections for the published manifest and run-result receipt, with explicit compatibility decisions. Do not serialize complete upstream runtime types as durable contracts.
Notarius retains debug placement, opt-in capture, retention, and disclosure handling, including credential redaction. Native callbacks provide the data; ordinary provenance retains only needed fields, and sensitive response/history content stays out of ordinary records. Debug is not checkpoint input.
CommandKit with ckconfig/ckprompt is the selected owner of standardized configuration/resource commands, requirements, validation, preview, opt-in smoke diagnostics, formatting, visibility, and progress presentation. Notarius supplies loaders, request builders, report content, pipeline meaning, and outcome mapping. Keep machine-output receipts deliberate and presentation delivery failures distinguishable from operation outcomes. Until that follow-up lands, make only the changes needed to keep existing inspection commands usable through native APIs; do not build temporary custom commands or a new reporting framework.
Unresolved Choices
The directions above are selected. The following details are not. Resolve and record each choice before implementing the work that depends on it; independent work need not wait for every choice. Candidates here are not implicit defaults.
| ID | Choice still needed | Resolve for |
|---|---|---|
| U1 | Final compatible tagged PromptKit/ConfigKit/dependency set; any specific unmet need requiring an upstream request. v0.23.2 is the initial target, not a restriction on useful upstream work. | Phase 1; revisit if the target or upstream needs change. |
| U2 | Remaining custom read-only filesystem responsibilities; schema root/path and assembly, prompt-declared versus per-request contracts, removable wrappers, and metadata useful for manifests/artifact compatibility. | Phase 2 source and schema assembly. |
| U3 | Concrete module/pipeline signatures and placement of engine assembly, request construction, interpretation, assessment, and focused helpers. | Phase 3 representative workflow, refined during Phase 4. |
| U4 | Initial and repair prompt definitions, response contracts, and profile selection within complete replacement and fixed operation inputs. | Phases 3–4 for each producer role. |
| U5 | Mapping producer/validator retry settings to native controls; names, defaults, compatibility, non-LLM failure handling, and any explicitly justified recovery beyond native exhaustion. | Phases 3–4 and configuration migration. |
| U6 | Detailed terminal/fallback dispositions, especially whether any earlier safe candidate survives a later failed turn. | Phases 3–4; no implicit best-so-far acceptance. |
| U7 | Independent worker defaults, native backend/queue defaults where supplied, retired total_llm migration, and admission-failure handling by role. |
Phase 4 shared-engine cutover. |
| U8 | Logical checkpoint/prompt identifiers, consumed-input representation, storage scoping, old-record transition, and load-compatibility reporting. | Phase 5 recovery migration. |
| U9 | Minimal durable fields and their compatibility treatment; observer/group enrollment, application scope, and debug write/disclosure handling. | Design with Phase 3, complete in Phase 5. |
| U10 | CommandKit/ckconfig/ckprompt versions, command exposure and compatibility, and application wiring for the follow-up. | Phase 7; do not recreate its machinery earlier. |
Implementation Sequence
Each phase is pending. Phases describe reviewable outcomes and dependencies, not separate releases. The supported core result is the hard cutover; any intermediate migration scaffolding must retire before that result is complete. Update affected canonical documentation as behavior lands. Record significant boundary decisions in successor ADRs without rewriting accepted decision text.
Phase 1: Establish The Migration Baseline
- Confirm U1 against the tagged APIs and dependency graph. Carry forward the Audita evidence without treating its wrappers as requirements.
- Map the changed architecture and ADRs, and identify concrete decisions needed for the next phase. Keep unresolved entries visible until actually settled.
- Identify the production callers and old contracts covered by the removal inventory below; use that inventory to track complete cutover coverage.
Completion evidence: selected dependency baseline and decision/implementation scope are recorded; unresolved choices have a clear dependent phase. No new upstream feature is assumed necessary without an identified requirement.
Phase 2: Assemble Native Sources And A Shared Engine
- Resolve U2, update dependencies, and implement version 6 and companion-source composition. Convert embedded resources and establish the schema loading path.
- Construct the shared native engine and use the same assembly for execution, preflight, and inspection. Remove duplicate prompt assembly as callers migrate.
- Add maintained embedded-only and operator-override examples with the new format.
Completion evidence: offline preparation proves resource selection, shared content/schema resolution, source precedence, and operator replacement. Invalid sources fail rather than silently selecting defaults. Preflight needs no provider credentials or synthetic stage execution.
Phase 3: Prove Native Production And Assessment
- Resolve U3 and the relevant U4–U6/U9 choices for a representative extraction workflow. Connect native generation, domain interpretation, configured validation, and continuation through the small application function.
- Exercise a configured LLM validator on the same engine. Attach native observers, application scope, and selected execution-group membership from the start.
- Exercise normalization's fixed candidate handles and fresh plan application, including the selected terminal/fallback policy.
Completion evidence: fake-provider workflows demonstrate accumulated history, full revalidation, final-turn acceptance, correct exhaustion/failure handling, and typed application outcomes without the private completion/correction API.
Phase 4: Migrate All Roles And Retire Runtime Machinery
- Apply the established flow to every LLM-backed chunker, extractor, normalizer, and validator. Preserve deterministic stage paths and ordered validation.
- Complete U4–U7 across roles. Migrate retry controls, remove the global scheduler, and give bounded pipeline workers independent configuration.
- Remove the adapter, copied completion contracts, correction-protocol checks, outer LLM retry loops, and redundant asset registration once unused.
Completion evidence: all production roles share native execution and admission; no application path still depends on the retired completion API, global ceiling, or latest-response-only correction. Native failures and cancellation settle through the selected role policy. Tests for obsolete machinery retire with it.
Phase 5: Simplify Recovery And Execution Records
- Resolve U8–U9. Implement logical-work/input/prompt checkpoint matching and revise storage scoping so unrelated configuration changes cannot defeat recovery.
- Implement and document old-checkpoint handling, artifact usability checks, original acceptance provenance, and explicit recomputation behavior.
- Complete native inspection, observation/group accounting, minimal durable projections, and optional debug capture; remove the duplicate bookkeeping.
Completion evidence: a downstream-failure/fix/resume scenario reuses successful predecessors; settings changes do not spuriously invalidate them, consumed-data changes do, and reused work does not count as current execution. Published records follow the selected compatibility policy and preserve failure/usage distinctions.
Phase 6: Verify And Document The Core Cutover
- Complete the acceptance checks below and the removal inventory. Verify the dependency graph contains no unused old-path PromptKit/ConfigKit or catalogs.
- Finish migration guidance, maintained examples, attribution, successor ADRs, and canonical documentation updates. Reconcile affected future-work notes.
- Keep existing inspection commands usable with native reports. Document that the standardized CommandKit command surface remains the subsequent phase.
Completion evidence: the core native architecture, configuration migration, checkpoint policy, diagnostics, and maintained offline workflows are verified; remaining work is explicitly the CommandKit follow-up or separately identified future work, not hidden adapter or retry compatibility machinery.
Phase 7: Adopt CommandKit Presentation
- Resolve U10 and adopt CommandKit with ckconfig/ckprompt. Supply application and engine loaders plus request builders for the selected diagnostic commands.
- Replace applicable Notarius-owned configuration/PromptKit commands and standard presentation while preserving deliberate machine-output contracts and domain outcome mapping. Keep provider smoke diagnostics explicitly opt-in.
Completion evidence: standard commands use upstream integrations and the same native source assembly as execution; application preflight and published receipts retain their intended meaning. This phase completes the presentation target and is not a prerequisite for completing the core PromptKit cutover.
Removal Inventory
| Retire or replace | Retained application responsibility |
|---|---|
StructuredLLMClient, private structured-completion request/response types, adapter conversions |
Native request construction and typed domain interpretation. |
Prompt/profile AssetRegistry plumbing, PromptAssetManifest, ModulePromptFS, synthetic shared mounts, manual fingerprint enumeration |
Embedded content and module semantics; schema responsibilities resolved under U2. |
single_response_v1, raw ModelCandidate/SemanticCorrection, manual replay, correction capability checks |
Typed candidates, actionable findings, native exchanges, and assessment. |
LLM retry portions of runProducerAttempts and executeValidationChain, module rerun directives |
Ordered validators, acceptance/rejection/fallback policy, and useful provenance. |
Scheduler, scheduledClient, concurrency.total_llm, worker coupling |
Bounded pipeline work and native per-backend admission. |
| Broad configuration/prompt/catalog/implementation checkpoint fingerprints and invocation-wide lookup barriers | Logical work, consumed-input identity, load checks, and explicit recomputation. |
Inspection wrappers, copied effective parameters, LLMExecutionSummary and usage conversions/accumulators, debug completion wrapper, redundant call IDs, adapter profile recorder |
Native facts with application scope and selected durable projections. |
| Custom standard inspection commands/presentation in the CommandKit phase | Application loaders, request builders, report content, and outcome mapping. |
Acceptance And Verification
Use focused behavioral tests with real deterministic collaborators, a native engine, and injected clients or a fake HTTP provider. Test domain interpretation and assessment directly; pipeline tests can use fake stages. Replace tests that preserve removed mechanics rather than reproducing a private LLM interface for testing. Default tests remain offline and independent of real credentials.
| Area | Required evidence |
|---|---|
| Configuration and assets | Version 6/new resources work; legacy formats are rejected intentionally; main-relative companions and native imports work independently of working directory; peer conflicts fail; same-ID replacement affects only the intended default; ordering/cache controls and selected schema contracts survive assembly. |
| Preflight and inspection | Runtime, inspection, and preflight share effective definitions; unknown statically selected IDs and invalid definitions fail offline; partial profiles alone pass; completed target checks occur on concrete requests; actual-run overrides appear in provenance. |
| Native artifact workflow | Representative extraction and normalization produce typed candidates, invoke ordered validators, and preserve source/reference identity, safe plan application, artifact-schema checks, and stable output ordering. |
| Continuation and failures | Multiple turns accumulate history with fixed input/handle meanings; every revised candidate and final allowed response is assessed; native transport/structural exhaustion is not silently retried outside PromptKit; validator failure differs from rejection; selected safe fallback and cancellation policy hold. |
| Admission and workers | All roles share the selected backend pool; independent backends remain independent; pipeline work stays bounded; queue exhaustion, nested validators, and cancellation settle without leaked capacity or a retained global ceiling. |
| Checkpoints | Successful predecessors survive downstream fixes and prompt/model/validation-setting changes; changed prompt IDs or consumed source/reference/upstream data miss; recompute forces selected work/dependents; corrupt/incompatible or unaccepted/incomplete artifacts and ineligible derivatives are unusable; compatible schema changes do not require exact hash equality. |
| Reuse provenance | Original acceptance and execution provenance survive resume without new semantic validation claims or historical token usage counted as current; recording remains separate from reuse; canonical chunk-plan caching retains its distinct policy. |
| Execution records | Initial/continuation/nested validator accounting preserves terminal coverage and unknown-versus-zero usage without double counting; failures without a result remain observable; native success is distinct from acceptance; debug is opt-in and ordinary records exclude sensitive response/history content. |
| Published behavior | Maintained minimal and complete D&D workflows, session routing, artifact output, manifest/receipt compatibility, and selected migration diagnostics work through the assembled CLI. CommandKit-phase tests cover its own command/presentation changes. |
Run focused checks while implementing and the repository-wide commands for the completed application cutover, following the testing policy:
go test ./...
go vet ./...
go build ./cmd/notarius
Check maintained examples, dependency removal, links, and attribution alongside those commands. Live provider checks and human model-quality evaluation can inform deployment and prompt refinement; they are not requirements of the deterministic offline suite. Planning/documentation completion does not claim that these implementation acceptance checks have already passed.
Migration Guidance And Documentation
Maintain examples that distinguish embedded-only operation from companion configuration, using separate directories when needed. Explain moving/importing old sources, qualified IDs, complete replacement versus inheritance, native local backends, removed catalogs and prompt versions, and the separate application and PromptKit source versions. Document worker/global-limit changes, retry-budget migration, accumulated conversation costs, and the selected failure policy.
Explain successful-work checkpoint reuse, explicit recomputation to apply prompt or setting improvements, consumed-data misses, original validation provenance, and the chosen old-record transition. Document changed inspection and durable fields deliberately rather than promising incidental old adapter output.
Update the canonical configuration, CLI, operations, internal overview, LLM, pipeline, modules, state, and affected integration/consumer guides when their behavior changes. Update the architecture policy to make native PromptKit an explicit execution dependency and remove the superseded scheduler, retry, asset-registration, and fingerprint constraints.
Record significant ownership and contract changes in successor ADRs, including those affecting asset assembly, validation retries, module-requested retries, and PromptKit configuration. Also record the changed admission and checkpoint matching decisions. Preserve accepted decision text and update supersession status as appropriate under the documentation policy. The discussion record supplies rationale and evidence; this roadmap tracks implementation status until the corresponding behavior lands.