Record debug bundle organization and latest symlink
This commit is contained in:
@@ -24,8 +24,9 @@ PromptKit v0.23.3 in place of the master roadmap's initial v0.23.2 target.
|
|||||||
Item 2 selects the application command structure and input conventions. Item 3
|
Item 2 selects the application command structure and input conventions. Item 3
|
||||||
selects native composition, shared loaders, and application preflight. Item 4
|
selects native composition, shared loaders, and application preflight. Item 4
|
||||||
selects native reporting, machine output, and outcomes. Item 5 selects conditional
|
selects native reporting, machine output, and outcomes. Item 5 selects conditional
|
||||||
debug capture through the shared `--debug` flag and native diagnostics. Debug
|
debug capture through the shared `--debug` flag, native diagnostics, and a run/
|
||||||
content/organization, exact run-report fields, and verification remain open below.
|
operation bundle with a `latest` symlink. Capture depth, exact run-report fields,
|
||||||
|
and verification remain open below.
|
||||||
|
|
||||||
The PromptKit architecture and checkpoint decisions remain in force. This
|
The PromptKit architecture and checkpoint decisions remain in force. This
|
||||||
discussion develops the CommandKit portion of that migration and its interactions
|
discussion develops the CommandKit portion of that migration and its interactions
|
||||||
@@ -646,9 +647,10 @@ retains the bookkeeping boundary; U9 still owns concrete durable record choices.
|
|||||||
**Discussion / decision:** Controls, native capture hooks, lifecycle, failure
|
**Discussion / decision:** Controls, native capture hooks, lifecycle, failure
|
||||||
policy, and optional diagnostics agreed. Use `--debug` for native verbosity and
|
policy, and optional diagnostics agreed. Use `--debug` for native verbosity and
|
||||||
durable capture with `/tmp/notarius-debug` as the default root. Operator-owned
|
durable capture with `/tmp/notarius-debug` as the default root. Operator-owned
|
||||||
data handling and recording each fact once are agreed. The summary/trace
|
data handling and recording each fact once are agreed. Replace the summary/trace
|
||||||
distinction, concrete capture inventory, and bundle organization remain under
|
hierarchy with a compact run index, configuration/composition snapshots, and
|
||||||
discussion. Implementation is pending.
|
operation records; maintain a `latest` symlink at the debug root. Capture depth
|
||||||
|
and concrete serialized fields remain under discussion. Implementation is pending.
|
||||||
|
|
||||||
### Shared Debug Control With Conditional Capture
|
### Shared Debug Control With Conditional Capture
|
||||||
|
|
||||||
@@ -731,6 +733,15 @@ and the destination and run identity are known, before engine construction and
|
|||||||
pipeline preflight. Argument and configuration-load failures use native
|
pipeline preflight. Argument and configuration-load failures use native
|
||||||
diagnostics and may have no bundle. Loaders remain free of storage allocation.
|
diagnostics and may have no bundle. Loaders remain free of storage allocation.
|
||||||
|
|
||||||
|
After creating the run directory, update `<debug-root>/latest` to a relative
|
||||||
|
symlink targeting that run ID before engine construction or preflight starts.
|
||||||
|
This points to the newly started debug session, including one that later fails;
|
||||||
|
it is not a last-successful-run selector. Runs without capture leave it alone.
|
||||||
|
Do not update it again at completion, so an older overlapping run cannot take
|
||||||
|
the link back when it finishes. Concurrent starts use the last successful link
|
||||||
|
update; individual run directories remain the actual storage locations.
|
||||||
|
Treat link-update failures under the requested-capture failure policy below.
|
||||||
|
|
||||||
One application-owned recorder records initial context, accepts native execution
|
One application-owned recorder records initial context, accepts native execution
|
||||||
evidence and application decisions, and finalizes once with the known pipeline,
|
evidence and application decisions, and finalizes once with the known pipeline,
|
||||||
publication, and capture outcomes. Debug remains separate from output/cache,
|
publication, and capture outcomes. Debug remains separate from output/cache,
|
||||||
@@ -793,25 +804,66 @@ Neither debug verbosity nor enabled capture initiates additional provider probes
|
|||||||
Smoke reports retain native support, applicability, and result distinctions;
|
Smoke reports retain native support, applicability, and result distinctions;
|
||||||
status zero alone does not establish that a check passed.
|
status zero alone does not establish that a check passed.
|
||||||
|
|
||||||
### Open Discussion: Summary And Trace
|
### Selected Bundle Organization
|
||||||
|
|
||||||
**Agreed principle:** Record each fact once and reference it where needed. Use
|
**Agreed principle:** Record each fact once and reference it where needed. Use
|
||||||
shared application outcome facts and native execution evidence rather than
|
shared application outcome facts and native execution evidence rather than
|
||||||
parallel summary/trace accounting or repeated snapshots.
|
parallel summary/trace accounting or repeated snapshots.
|
||||||
|
|
||||||
The current `summary/` and `trace/` layout is not selected as the new design.
|
Organize the bundle around the run and its operations, replacing `summary/` and
|
||||||
Discuss what diagnostic questions the bundle should answer and which application
|
`trace/` as separate categories. Use a compact overview for navigation and keep
|
||||||
records and native evidence it needs. Summary/trace are not separate security
|
detailed evidence with its operation. Summary/trace are not security classes.
|
||||||
classes. Keep these choices open:
|
|
||||||
|
|
||||||
- Whether a separate summary is useful, and whether it is a compact index/final
|
Illustrative layout: `latest` is the selected link name; other file names and
|
||||||
outcome rather than another copy of execution records.
|
exact serialized fields remain implementation choices:
|
||||||
- Whether summary/trace are useful conceptual categories or directory boundaries.
|
|
||||||
- Which prepared details, outputs, conversation snapshots, and application
|
```text
|
||||||
decisions to persist, and at what granularity.
|
<debug-root>/
|
||||||
- How self-contained a failed-run bundle should be, given that it may have no
|
latest -> <run-id>/
|
||||||
published output bundle, and when to reference existing artifacts instead of
|
<run-id>/
|
||||||
copying them.
|
run.json
|
||||||
|
config/
|
||||||
|
application.yml
|
||||||
|
promptkit.yml
|
||||||
|
pipeline.json
|
||||||
|
operations/
|
||||||
|
<operation-id>/
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
| Component | Purpose |
|
||||||
|
| --- | --- |
|
||||||
|
| Run overview | Identify the invocation, outcome, published output, and available diagnostic records. |
|
||||||
|
| Configuration snapshots | Preserve effective settings through native inspection. |
|
||||||
|
| Resolved pipeline | Describe selected steps, lanes, modules, validators, and reference relationships. |
|
||||||
|
| Operation records | Connect diagnostic evidence with its stage, chunk, lane, or validator. |
|
||||||
|
|
||||||
|
The overview is an index rather than a second execution ledger. Its terminal
|
||||||
|
facts come from the same application outcome used for the CLI report, and it
|
||||||
|
references detailed records instead of duplicating requests, responses, findings,
|
||||||
|
or error history. Capture configuration and resolved composition once when
|
||||||
|
available. Use native execution IDs with application scope to connect records.
|
||||||
|
Reused work has a checkpoint/reuse decision and reference, not a fabricated model
|
||||||
|
execution.
|
||||||
|
|
||||||
|
Retain enough available context to investigate a run that fails before publishing
|
||||||
|
output. Do not depend entirely on a successful output manifest or automatically
|
||||||
|
copy every original input and published artifact. Select a conversation
|
||||||
|
representation that avoids repeatedly saving the complete accumulated history
|
||||||
|
after each turn or maintaining an independent competing transcript.
|
||||||
|
|
||||||
|
### Open Discussion: Capture Depth
|
||||||
|
|
||||||
|
The organization is selected; precise evidence and granularity remain open:
|
||||||
|
|
||||||
|
- Which prepared request details, outputs, intermediate artifacts, validation
|
||||||
|
decisions, and correction content to retain.
|
||||||
|
- How to represent conversation history without repeated prefixes while using
|
||||||
|
PromptKit's native evidence.
|
||||||
|
- How much failed-run context to retain independently, and when references to
|
||||||
|
existing inputs, checkpoints, or published artifacts suffice.
|
||||||
|
- Which desired evidence is available through native public hooks and which
|
||||||
|
gaps would justify an upstream feature request.
|
||||||
|
|
||||||
Evidence: [native shared debug binding](../../../commandkit/docs/consumers/guide.md#invocation-reporting),
|
Evidence: [native shared debug binding](../../../commandkit/docs/consumers/guide.md#invocation-reporting),
|
||||||
[reporting-mode ordering](../../../commandkit/docs/cli.md#modes-and-visibility),
|
[reporting-mode ordering](../../../commandkit/docs/cli.md#modes-and-visibility),
|
||||||
|
|||||||
Reference in New Issue
Block a user