Resolve completion reporting and publication contracts
This commit is contained in:
@@ -927,14 +927,18 @@ independent accumulators, debug completion wrappers, synthetic call IDs, and
|
||||
adapter profile-recording/forwarding. Native snapshots serve actual-target
|
||||
inspection; do not reconstruct a durable profile inventory.
|
||||
|
||||
Use `ExecutionGroup` for selected diagnostic totals, explicitly enrolling initial,
|
||||
continuation, and nested validator executions without overlapping double counts.
|
||||
Preserve terminal coverage and unknown-versus-zero usage. Reuse adds no historical
|
||||
usage to current totals. These facts support diagnostics and capture, not a
|
||||
mandatory consumer execution ledger. Notarius's application outcome supplies
|
||||
publication and fallback qualifications for the completion summary; assessment,
|
||||
failure, and reuse details support optional diagnostics and debug capture without
|
||||
independently maintained reporting state.
|
||||
Use one native `ExecutionGroup` per invocation for diagnostic totals. Explicitly
|
||||
enroll every actual initial, continuation, and nested LLM validator execution once,
|
||||
with application step/lane/chunk/validator context attached to observations.
|
||||
Use native aggregation directly, without application-maintained totals or a
|
||||
hierarchy of overlapping groups. Preserve terminal coverage and unknown-versus-zero
|
||||
usage. Cache hits create no execution and add no historical usage to current totals.
|
||||
Concrete observer composition, enrollment plumbing, and context attachment remain
|
||||
implementation choices within this contract. These facts support diagnostics and
|
||||
capture, not a mandatory consumer execution ledger. Notarius's application outcome
|
||||
supplies publication and fallback qualifications for the completion summary;
|
||||
assessment, failure, and reuse details support optional diagnostics and debug
|
||||
capture without independently maintained reporting state.
|
||||
|
||||
### Artifact Delivery And Completion Output
|
||||
|
||||
@@ -946,6 +950,28 @@ need no run manifest, metadata bundle, persisted receipt, or execution history t
|
||||
use the artifacts. Retire reporting-only metadata and mandatory manifests instead
|
||||
of rebuilding them with native values.
|
||||
|
||||
Preserve the existing run-specific destination and domain artifact paths:
|
||||
|
||||
```text
|
||||
<output-root>/<run-id>/
|
||||
lanes/<safe-lane-id>.json
|
||||
chunk-map.json # when explicitly enabled and available
|
||||
evidence-context.json # when explicitly enabled and available
|
||||
```
|
||||
|
||||
Keep the existing lane filename mapping and intrinsic payload schemas. Remove
|
||||
the mandatory `index.json`, `manifest.json`, and reporting-only `rejected.json`,
|
||||
`warnings.json`, and `diagnostics.json` sidecars. Their relevant diagnostic content
|
||||
belongs in ordinary diagnostics and optional debug capture; retain intrinsic domain
|
||||
content in the artifacts themselves. `output_directory` identifies the absolute
|
||||
run-specific directory, not the configured output root.
|
||||
|
||||
Consumers check process status, optionally decode the small completion report, and
|
||||
read documented artifact paths beneath the published directory. For configurable
|
||||
pipelines, the pipeline configuration identifies expected lanes. Migration guidance
|
||||
replaces receipt/index traversal with those paths, without introducing a replacement
|
||||
discovery manifest or a payload-schema change.
|
||||
|
||||
Submit one small structured value through `Reporter.Report`:
|
||||
|
||||
| Field | Meaning |
|
||||
@@ -954,6 +980,7 @@ Submit one small structured value through `Reporter.Report`:
|
||||
| `artifact_count` | Deliverable artifacts, including reused artifacts; excludes intermediates, cache entries, and debug files. |
|
||||
| `output_directory` | Absolute published artifact directory. |
|
||||
| `debug_directory` | Optional absolute completed run-specific capture directory; omitted when capture is disabled. |
|
||||
| `fallback_count` | Number of producer operations that emitted a fallback; omitted when zero. |
|
||||
|
||||
Supply a native text template for the ordinary successful case:
|
||||
|
||||
@@ -970,10 +997,19 @@ or validation-history collection is required.
|
||||
|
||||
When a completed run used a fallback, qualify the human summary and structured
|
||||
value with that fact, even if native diagnostic verbosity suppresses individual
|
||||
warnings. For example, a small fallback count can accompany the completion fields;
|
||||
exact field names remain implementation details. Do not present the run as if all
|
||||
operations obtained normally validated results. Detailed reasons belong in optional
|
||||
diagnostics/debug capture, without per-producer histories in the completion report.
|
||||
warnings. Count each producer operation that actually emitted a fallback once;
|
||||
do not count rejected attempts, individual validator failures, or downstream
|
||||
artifacts affected by that fallback. JSON/YAML include `fallback_count` only when
|
||||
positive. The native text template can render:
|
||||
|
||||
```text
|
||||
Success with fallback: dnd-session produced 6 artifacts in /work/results/run-123
|
||||
2 operations used fallback.
|
||||
```
|
||||
|
||||
Do not present the run as if all operations obtained normally validated results.
|
||||
Detailed reasons belong in optional diagnostics/debug capture, without per-producer
|
||||
histories in the completion report.
|
||||
Quiet mode continues to suppress stdout.
|
||||
|
||||
Wire results to stdout and diagnostics to stderr:
|
||||
@@ -1071,9 +1107,12 @@ never cause model retries. Finalize before submitting the success report.
|
||||
|
||||
Replace separate summary/trace categories with a compact run overview,
|
||||
configuration/composition snapshots, parsed source, and per-operation records.
|
||||
Use native execution IDs with application scope for navigation. `latest` is the
|
||||
selected link name; other filenames and serialized application fields remain
|
||||
implementation choices.
|
||||
Organize operation directories by meaningful pipeline scope, with native execution
|
||||
IDs within those operations. Provide a readable run overview and clearly named
|
||||
structured snapshots and text views. `latest` is the selected link name; exact
|
||||
debug filenames, directory nesting, and serialized application fields are explicitly
|
||||
delegated to implementation. Evaluate them by ease of investigating an operation;
|
||||
do not freeze them as a public format or introduce another debug framework.
|
||||
|
||||
| Capture | Selected contents and depth |
|
||||
| --- | --- |
|
||||
@@ -1128,7 +1167,10 @@ reopen the architecture. IDs retain continuity with the earlier roadmap.
|
||||
| ID | Remaining choice or dependency | Resolve for |
|
||||
| --- | --- | --- |
|
||||
| U1 | Review the message-content hash API/release documented upstream in v0.24.0, including its ConfigKit dependency; verify the final compatible dependency set against the selected baseline. | Track in Phase 1; required to finish cache integration in Phase 5. |
|
||||
| U9 | Observer/group enrollment and application scope; concrete conditional report fields and debug filenames/serialization within the agreed contracts; publication-layout/consumer migration details preserving intrinsic artifact schemas. | Implementation details in Phases 3–5; no new receipt/manifest inventory or capture-policy decision required. |
|
||||
|
||||
U1 is the only remaining unresolved dependency decision. Observer wiring, debug
|
||||
filenames, and other routine implementation details are delegated under the
|
||||
resolved designs below; all implementation phases remain pending.
|
||||
|
||||
**U2 resolved:** Ordinary embedded assets, prompt-declared output contracts,
|
||||
native schema-source selection, and removal of duplicate schema/filesystem
|
||||
@@ -1186,6 +1228,17 @@ explicit `--cache`, module-binding `cache`, pipeline `cache`, then `auto`.
|
||||
results without reading. Cache I/O is best effort; legacy records remain untouched
|
||||
and are not imported. Implement in Phases 3–5, subject to U1's hash dependency.
|
||||
|
||||
**U9 resolved:** [Execution bookkeeping](#inspection-and-execution-bookkeeping)
|
||||
uses one native execution group per invocation, with all actual initial,
|
||||
continuation, and LLM validator executions enrolled once and application scope
|
||||
attached. [Completion output](#artifact-delivery-and-completion-output) adds only
|
||||
optional positive `fallback_count`, counting producer operations that emitted
|
||||
fallbacks. Preserve run-specific domain artifact paths and schemas; retire index,
|
||||
manifest, and reporting-only sidecars and migrate consumers to documented paths.
|
||||
Observer wiring and [debug filenames/layout](#inventory-and-depth) are explicitly
|
||||
implementation choices, not unresolved architecture or stable debug contracts.
|
||||
Implement in Phases 3–5.
|
||||
|
||||
**U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight,
|
||||
output, debug policy, and coordinated cutover sequencing are selected above.
|
||||
The former later CommandKit phase is retired. CLI compatibility is a non-goal;
|
||||
@@ -1206,9 +1259,8 @@ major boundary changes in successor ADRs without rewriting accepted decisions.
|
||||
Keep the cache dependency explicit without blocking independent work.
|
||||
- Map affected ADRs, current callers, contracts, and removal inventory. Carry
|
||||
forward Audita evidence without adopting its application wrappers.
|
||||
- Preserve U9 with its dependent phases; do not reopen resolved module
|
||||
execution, prompt/guidance ownership, retry controls/defaults, assets/schemas,
|
||||
worker/admission policy, CommandKit, debug, cache, or output ownership.
|
||||
- Carry resolved U2–U10 designs into their dependent phases without reopening
|
||||
ownership. Leave observer wiring and debug filenames to implementation.
|
||||
|
||||
**Completion evidence:** reviewed dependency baseline, upstream dependency, scope,
|
||||
and outstanding choices are explicit. Upstream availability alone does not
|
||||
@@ -1243,11 +1295,12 @@ without credentials or synthetic requests. `config diff` handles two selections.
|
||||
Resolve `max_additional_turns` with omission/zero distinctions and pass it to
|
||||
native coordination; inherit native transport and structural-repair defaults.
|
||||
- Exercise an LLM validator on the same engine and normalization with fixed
|
||||
handles and fresh plan application. Select actual observer/group enrollment.
|
||||
handles and fresh plan application. Wire U9's single invocation execution group
|
||||
with application scope for initial, continuation, and validator executions.
|
||||
- Run through the new command surface and alias, using the small completion
|
||||
report, native outcomes/streams, and useful debug capture from the outset.
|
||||
Begin the resolved U8 cache integration and U9 details without a provisional
|
||||
local message hash.
|
||||
Begin the resolved U8 cache integration and U9 reporting/capture implementation
|
||||
without a provisional local message hash.
|
||||
|
||||
**Completion evidence:** fake-provider workflows exercise accumulated history,
|
||||
collection of independent validator findings without rejection short-circuiting,
|
||||
@@ -1286,7 +1339,8 @@ requirements remain.
|
||||
- Resolve U1's supplying release and implement U8's unified lookup at initial
|
||||
preparation, accepted-artifact/plan storage under that hash, directory and mode
|
||||
precedence, and best-effort cache I/O. Retire both legacy cache mechanisms.
|
||||
- Complete the artifact-first publication/report contract and U9 details. Remove
|
||||
- Complete U9's artifact-first publication/report contract. Preserve domain paths
|
||||
and schemas, add `fallback_count`, and retire index/reporting sidecars. Remove
|
||||
checkpoint fingerprint graphs, mandatory manifests/receipts, durable execution
|
||||
ledgers, and duplicate accounting; do not retain them to populate summaries.
|
||||
- Complete capture inventory, per-operation evidence, directory/empty-value
|
||||
@@ -1299,8 +1353,11 @@ affected downstream work skip cache lookup/storage; unaffected accepted work
|
||||
remains reusable. Chunk plans use the same cache, rematerialize against current
|
||||
source, and no longer pin one partition across changed messages. Mode precedence,
|
||||
cache I/O failures, and leaving both legacy stores untouched are verified.
|
||||
Artifacts need no metadata bundle. Machine output and debug capture obey the
|
||||
selected contracts, including failures after publication and quiet-mode capture.
|
||||
Artifacts need no metadata bundle and consumers use documented domain paths.
|
||||
Machine output and debug capture obey the selected contracts, including omitted
|
||||
zero/accurate positive fallback counts, failures after publication, and quiet-mode
|
||||
capture. Observer wiring and debug organization satisfy the selected scope and
|
||||
inventory without becoming separate public contracts.
|
||||
|
||||
### Phase 6: Verify And Document The Coordinated Cutover
|
||||
|
||||
@@ -1333,7 +1390,7 @@ Release preparation/publication is governed separately by release policy.
|
||||
| Separate source-keyed chunk-plan store, canonical-plan provenance/publication policy, nested legacy cache configuration, `--resume`, `--recompute-step`, separate chunk-cache flag | Unified message-content lookup/storage, domain chunk-plan interpretation/materialization, `cache_directory`, pipeline/module `cache`, and invocation-wide `--cache`. |
|
||||
| Inspection wrappers, copied effective parameters, usage summaries/conversions/accumulators, debug completion wrapper, synthetic IDs, profile recorder | Native facts with application scope for diagnostics. |
|
||||
| Custom dispatch/help/version/parsing, command-specific loading, copied inspection, legacy aliases/flags, parallel encoders and verbosity handling | Native declarations/loaders, pipeline handlers, domain input checks. |
|
||||
| Mandatory run manifests, reporting-only metadata bundles, extensive receipts and durable execution ledgers | Domain artifact files, one small native completion report, optional debug evidence. |
|
||||
| Mandatory `index.json`/`manifest.json`, reporting-only `rejected.json`/`warnings.json`/`diagnostics.json`, extensive receipts and durable execution ledgers | Preserved domain artifact paths and schemas, one small native completion report with optional `fallback_count`, ordinary diagnostics, and optional debug evidence. |
|
||||
| Summary/trace hierarchy, heuristic debug-content scrubbing, custom transcript/deduplication machinery | Convenient run/operation capture and native secret/credential omissions. |
|
||||
|
||||
## Acceptance And Verification
|
||||
@@ -1360,8 +1417,8 @@ retry tests or freeze complete help text and native debug snapshots.
|
||||
| Accepted-artifact cache | Downstream recovery; initial-message keys; corrected final artifact stored late under the initial key; changed messages miss; settings/ID-only changes can hit; no model/revalidation on hits; unusable entries miss. Failed/rejected/incomplete work is not stored; fallbacks and affected downstream work skip lookup/storage while unaffected accepted predecessors remain reusable; no required history. |
|
||||
| Chunk-plan reuse | Final accepted plan stored under the initial request hash after materialization/validation/correction; hits rematerialize with structural/source-reference usability checks and no renewed semantic validation. Changed messages can select a new partition; deterministic chunkers run without synthetic cache requests. |
|
||||
| Cache controls and migration | Root-level directory default/explicit selection; mode precedence CLI, module binding, pipeline, then `auto`. `auto` reads/writes; `bypass` does no I/O or directory creation; `refresh` only replaces entries after new acceptance. Unusable entries miss; cache I/O failures preserve successful pipeline results. Both legacy stores remain untouched and unread; no conversion or separate canonical-plan mechanism. |
|
||||
| Output/outcomes | Artifact files independent of report format; native text/JSON/YAML summary and qualified outcomes; no mandatory metadata bundle; quiet suppression, verbose stderr, debug path; native nonzero failures and useful explanations. Publication survives later capture/report failure without model reruns. |
|
||||
| Execution observations | Initial/continuation/validator totals preserve unknown-versus-zero usage without double counts; result-less failures observable; reuse does not count historical work; native execution success remains distinct from acceptance. |
|
||||
| Output/outcomes | Preserved run-specific domain paths, lane filename mapping, and payload schemas; optional chunk-map/evidence-context publication remains. No index/manifest/reporting sidecars. Native text/JSON/YAML summary includes positive `fallback_count` and omits zero; count emitting producer operations, not attempts, validator failures, or affected derivatives. Quiet suppression, verbose stderr, debug path, and useful nonzero failures hold. Publication survives later capture/report failure without model reruns. |
|
||||
| Execution observations | One native invocation group enrolls actual initial/continuation/LLM validator executions once, with application scope; native totals preserve unknown-versus-zero usage without application accumulators or overlapping-group aggregation. Result-less failures remain observable; cache hits add no executions/history; native execution success remains distinct from acceptance. |
|
||||
| Debug controls/lifecycle | Default/CLI/config/explicit-empty paths and native mode ordering; quiet capture; allocation before engine/preflight; start-time relative `latest`, no completion retarget; disabled runs leave it untouched; capture failures preserve primary causes and publication facts. |
|
||||
| Debug evidence | Available input/source/native snapshots, readable text, all visible turns, assessments, normalization and deterministic outputs, cache hits, and failed-run evidence; no invented provider transcript; ConfigKit secrets/native credentials omitted, no heuristic content scrubbing. |
|
||||
| End-to-end migration | Maintained minimal and complete D&D workflows, session routing, reference behavior, raw artifacts, reports, and migration examples work through the new executable. No postponed CommandKit phase. |
|
||||
@@ -1408,8 +1465,11 @@ Provide practical release guidance for operators and subprocess callers:
|
||||
Explain refresh/bypass, best-effort I/O, fallback/affected-work exclusion, removal
|
||||
of both legacy mechanisms without importing/deleting records, and retirement of
|
||||
canonical-source partition reuse.
|
||||
- Artifact delivery without mandatory manifests/receipts; preserve intrinsic
|
||||
artifact schemas and document any explicitly selected format change.
|
||||
- Artifact delivery through preserved run-specific domain paths and lane filename
|
||||
mapping, without index/manifest/reporting sidecars or receipt traversal. Document
|
||||
`fallback_count` and its omission/counting rules; consumers use the published
|
||||
directory and pipeline-defined lanes. Preserve intrinsic artifact schemas and
|
||||
document any explicitly selected format change.
|
||||
- Debug root/default/empty behavior, `latest`, native mode ordering, failure
|
||||
behavior, and ordinary operator responsibility for application data.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user