Record simpler checkpoint reuse policy
This commit is contained in:
@@ -5,7 +5,7 @@
|
||||
Working notepad for reviewing the ownership boundaries in the
|
||||
[PromptKit v0.23.2 adoption roadmap](promptkit.md). Work through the topics
|
||||
below in order, recording questions, evidence, decisions, and follow-up work.
|
||||
Items 1–4 record selected architectural directions. Detailed design,
|
||||
Items 1–5 record selected architectural directions. Detailed design,
|
||||
read-only filesystem and schema decisions, and the remaining topics are still
|
||||
under discussion.
|
||||
Item 7 records the completed Audita comparison and its implications for the
|
||||
@@ -19,9 +19,9 @@ features where PromptKit does not yet meet an application need.
|
||||
|
||||
No application changes have been implemented through this notepad. The adoption
|
||||
roadmap still needs reconciliation with the selected direction, including its
|
||||
adapter-preservation, latest-response-only, and cross-backend scheduler
|
||||
preservation language. Significant changes to accepted architecture need
|
||||
successor ADRs under the
|
||||
adapter-preservation, latest-response-only, cross-backend scheduler preservation,
|
||||
and broad checkpoint-fingerprint language. Significant changes to accepted
|
||||
architecture need successor ADRs under the
|
||||
[documentation policy](../policy/documentation.md#architecture-decision-records).
|
||||
Current boundaries are described in [Architecture](../policy/architecture.md)
|
||||
and [LLM Runtime Internals](../internal/llm.md).
|
||||
@@ -102,8 +102,9 @@ Candidates for removal:
|
||||
individual modules; schema responsibilities remain under review.
|
||||
- `ModulePromptFS`, `PromptAssetManifest`, shared-name lookup tables, synthetic
|
||||
`sharedassets` mounts.
|
||||
- Manual prompt-file enumeration for fingerprints, replaced by captured native
|
||||
prompt identities subject to the broader identity design in item 5.
|
||||
- Manual prompt-file enumeration for checkpoint fingerprints, retired under
|
||||
item 5's simpler reuse policy. Native prompt hashes remain available for
|
||||
inspection and diagnostics.
|
||||
|
||||
Keep prompt prose, ordered messages, cache controls, workload profiles, private
|
||||
response schema documents, and domain interpretation. Durable artifact schemas
|
||||
@@ -123,22 +124,24 @@ Deferred decisions:
|
||||
- Schema assembly, root/path selection, wrapper removal, and schema metadata.
|
||||
Supplying the embedded filesystem as a native schema root remains a candidate,
|
||||
not a selected design.
|
||||
- How schema identity contributes to manifests and checkpoint reuse.
|
||||
- Schema metadata for manifests and artifact compatibility. Item 5 settles that
|
||||
checkpoint matching does not require exact schema identity.
|
||||
|
||||
Implementation questions:
|
||||
|
||||
- Can native sources represent the existing shared fragments and module assets?
|
||||
- Which registry, remounting, and hashing helpers could disappear?
|
||||
- What private schema identity is still needed for checkpoint reuse?
|
||||
- What schema metadata remains useful for inspection and artifact loading under
|
||||
item 5's simpler reuse policy?
|
||||
- How do we preserve message ordering and cache controls while simplifying
|
||||
assembly?
|
||||
|
||||
**Follow-up:** Continue with item 3, then revisit the deferred filesystem and
|
||||
schema decisions with any new clarity. Confirm native source contexts for shared
|
||||
fragments and operator replacements. Resolve metadata and reuse identity with
|
||||
items 5–6. Verify the resulting definitions through native preparation and
|
||||
fake-provider execution; filesystem remounting tests can retire with the removed
|
||||
implementation. The agreed direction has not yet been implemented.
|
||||
fragments and operator replacements. Apply item 5's reuse policy and resolve
|
||||
remaining metadata with item 6. Verify the resulting definitions through native
|
||||
preparation and fake-provider execution; filesystem remounting tests can retire
|
||||
with the removed implementation. The agreed direction has not yet been implemented.
|
||||
|
||||
## 3. Retry And Semantic Correction Ownership
|
||||
|
||||
@@ -260,8 +263,8 @@ and the native continuation evidence in item 7.
|
||||
candidate may survive a later failed turn.
|
||||
- Failure handling for non-LLM operations and any justified recovery beyond
|
||||
native execution budgets.
|
||||
- Item 2's deferred filesystem and schema assembly choices, and items 5–6's
|
||||
execution identity and durable bookkeeping design.
|
||||
- Item 2's deferred filesystem and schema assembly choices, item 5's remaining
|
||||
checkpoint implementation details, and item 6's durable bookkeeping design.
|
||||
|
||||
**Follow-up:** Resolve the remaining details above, then reconcile the adoption
|
||||
roadmap. Record the changed architecture in a successor to
|
||||
@@ -346,24 +349,99 @@ with item 5's execution identity and checkpoint reuse discussion.
|
||||
|
||||
## 5. Execution Identity And Checkpoint Reuse
|
||||
|
||||
Consider moving identity for PromptKit-owned execution semantics upstream.
|
||||
Notarius would combine opaque upstream identities with its source, module,
|
||||
reference, schema, and policy identities and retain checkpoint eligibility and
|
||||
storage decisions.
|
||||
**Discussion / decision:** Direction agreed: resuming preserves successful stage
|
||||
results for the same logical work and data; explicit recomputation applies
|
||||
changed execution behavior. Use a simple Notarius-owned matching policy instead
|
||||
of comprehensive execution fingerprints. No application changes have been
|
||||
implemented.
|
||||
|
||||
Questions:
|
||||
### Purpose And Matching Rule
|
||||
|
||||
- Which identity can be computed before a stage runs, and which needs a concrete
|
||||
prepared request?
|
||||
- Which changes should invalidate reuse, and at what granularity?
|
||||
- Should a conservative application fingerprint be an interim implementation
|
||||
rather than the architectural destination?
|
||||
- Does this justify upstream work and a later adoption target?
|
||||
Checkpoints preserve a successful producer's result when a dependent stage fails.
|
||||
After fixing the failed stage, an operator should be able to retry it without
|
||||
paying to rerun the successful producer. Matching every configuration parameter
|
||||
or implementation revision would undermine that recovery purpose.
|
||||
|
||||
**Discussion / decision:** Pending.
|
||||
The agreed conceptual key is:
|
||||
|
||||
**Follow-up:** Review PromptKit's deferred semantic execution-target fingerprint
|
||||
proposal. This capability is not supplied by the v0.23.2 adoption target.
|
||||
```text
|
||||
logical stage identity + input-data identity + prompt ID, where applicable
|
||||
```
|
||||
|
||||
Scope the checkpoint to the relevant step/lane/stage and artifact kind so that
|
||||
shared prompt IDs do not mix unrelated results. These are logical identifiers,
|
||||
not hashes of implementations or complete configuration. Deterministic stages
|
||||
need no synthetic prompt identity.
|
||||
|
||||
Input data includes the source or chunk being processed, reference material,
|
||||
and upstream artifacts actually consumed. Matching only the original source
|
||||
would miss changes to generated dependencies. Content digests can compactly
|
||||
identify these inputs and check stored payload integrity without requiring an
|
||||
exact fingerprint of the execution environment.
|
||||
|
||||
| Change | Reuse behavior |
|
||||
| --- | --- |
|
||||
| Fix the implementation of a later failed stage | Preserve earlier successful stages with matching work and inputs. |
|
||||
| Edit prompt text while retaining its ID | Reuse; explicit recomputation applies the edit. |
|
||||
| Change model, generation settings, or implementation/validation policy | Reuse otherwise compatible completed work; explicit recomputation applies the change. |
|
||||
| Change the prompt ID or data consumed by the stage | No matching checkpoint under this rule. |
|
||||
| Explicitly request recomputation | Recompute the selected work and its dependents. |
|
||||
|
||||
The existing controls are `--resume` and `--recompute-step`, documented in the
|
||||
[CLI reference](../cli.md#run) and
|
||||
[selective recompute workflow](../operations.md#selective-recompute). Their
|
||||
current strict compatibility checks still need migration to this agreed policy.
|
||||
The accepted tradeoff is that a checkpoint may preserve a successful result
|
||||
produced by code, prompts, or settings that have since improved.
|
||||
|
||||
### Matching, Loading, And Provenance
|
||||
|
||||
A matching key selects a checkpoint; it does not make an unreadable or
|
||||
incompatible artifact usable. Retain completion/acceptance status, integrity
|
||||
checks, and the ability to decode and consume the stored artifact. Exact schema
|
||||
hash equality is not required when the artifact remains compatible.
|
||||
|
||||
Only accepted, completely validated results are reusable. Rejected, structurally
|
||||
invalid, or validation-incomplete results and their ineligible derivatives
|
||||
remain outside reusable state. Acceptance on resume refers to the earlier
|
||||
execution: do not automatically rerun semantic validators merely because their
|
||||
configuration changed, or claim the artifact passed the new validation policy.
|
||||
Preserve original provenance; explicit recomputation applies changed behavior.
|
||||
|
||||
Useful execution metadata can remain available for explanation without becoming
|
||||
a matching requirement. Neither complete conversation history nor native
|
||||
prepared-execution handles are needed to locate and reuse a completed artifact.
|
||||
The separate source-addressed canonical chunk-plan policy remains distinct.
|
||||
|
||||
### Simplification And Ownership
|
||||
|
||||
- Remove the broad PromptKit configuration fingerprint from checkpoint matching,
|
||||
including effective-configuration parsing and whole-catalog enumeration.
|
||||
- Remove library/catalog versions, model settings, prompt-content hashes, and
|
||||
component-policy fingerprints as automatic invalidators under this policy.
|
||||
- Keep straightforward input/dependency identity, successful-stage records,
|
||||
artifact loading checks, storage, and explicit recomputation in Notarius.
|
||||
- Keep PromptKit hashes available for native inspection and diagnostics without
|
||||
making them checkpoint prerequisites.
|
||||
|
||||
Comprehensive upstream execution identity is no longer required for this
|
||||
design. It does not require a later PromptKit release or a temporary conservative
|
||||
configuration fingerprint. Item 5 supersedes the earlier proposal to make an
|
||||
upstream semantic-equality feature part of the migration's checkpoint design.
|
||||
|
||||
### Remaining Implementation Work
|
||||
|
||||
Define the concrete logical stage/prompt identifiers and consumed-input
|
||||
representation. Revise checkpoint lookup and storage scoping so that an
|
||||
invocation-wide pipeline/configuration digest cannot still prevent reuse of
|
||||
successful predecessors. Decide how existing checkpoint records transition to
|
||||
the simpler format and how incompatible payloads are reported.
|
||||
|
||||
**Follow-up:** Reconcile the adoption roadmap and record the changed reuse
|
||||
contract in an ADR. Update current-behavior documentation when implemented.
|
||||
Verify recovery after a downstream fix, reuse across prompt/settings changes,
|
||||
misses after consumed-data changes, explicit recomputation, and rejection of
|
||||
unusable or unaccepted stored artifacts. Continue with item 6.
|
||||
|
||||
## 6. Inspection And Execution Bookkeeping
|
||||
|
||||
@@ -542,7 +620,8 @@ the comparison alone did not settle those details.
|
||||
|
||||
Item 4 now records agreement to use native admission with bounded application
|
||||
workers. Item 6 can draw on Audita's observations and CommandKit integration.
|
||||
Item 5 remains independent upstream/application identity work.
|
||||
Item 5 now selects simple application-owned recovery matching and removes the
|
||||
upstream execution-fingerprint prerequisite.
|
||||
The two requested capabilities already work with v0.23.2; no new PromptKit
|
||||
feature is needed merely to provide replaceable defaults or domain-directed
|
||||
continuation. Any remaining upstream request should identify the specific unmet
|
||||
|
||||
Reference in New Issue
Block a user