Record debug bundle organization and latest symlink
This commit is contained in:
@@ -24,8 +24,9 @@ PromptKit v0.23.3 in place of the master roadmap's initial v0.23.2 target.
|
||||
Item 2 selects the application command structure and input conventions. Item 3
|
||||
selects native composition, shared loaders, and application preflight. Item 4
|
||||
selects native reporting, machine output, and outcomes. Item 5 selects conditional
|
||||
debug capture through the shared `--debug` flag and native diagnostics. Debug
|
||||
content/organization, exact run-report fields, and verification remain open below.
|
||||
debug capture through the shared `--debug` flag, native diagnostics, and a run/
|
||||
operation bundle with a `latest` symlink. Capture depth, exact run-report fields,
|
||||
and verification remain open below.
|
||||
|
||||
The PromptKit architecture and checkpoint decisions remain in force. This
|
||||
discussion develops the CommandKit portion of that migration and its interactions
|
||||
@@ -646,9 +647,10 @@ retains the bookkeeping boundary; U9 still owns concrete durable record choices.
|
||||
**Discussion / decision:** Controls, native capture hooks, lifecycle, failure
|
||||
policy, and optional diagnostics agreed. Use `--debug` for native verbosity and
|
||||
durable capture with `/tmp/notarius-debug` as the default root. Operator-owned
|
||||
data handling and recording each fact once are agreed. The summary/trace
|
||||
distinction, concrete capture inventory, and bundle organization remain under
|
||||
discussion. Implementation is pending.
|
||||
data handling and recording each fact once are agreed. Replace the summary/trace
|
||||
hierarchy with a compact run index, configuration/composition snapshots, and
|
||||
operation records; maintain a `latest` symlink at the debug root. Capture depth
|
||||
and concrete serialized fields remain under discussion. Implementation is pending.
|
||||
|
||||
### Shared Debug Control With Conditional Capture
|
||||
|
||||
@@ -731,6 +733,15 @@ and the destination and run identity are known, before engine construction and
|
||||
pipeline preflight. Argument and configuration-load failures use native
|
||||
diagnostics and may have no bundle. Loaders remain free of storage allocation.
|
||||
|
||||
After creating the run directory, update `<debug-root>/latest` to a relative
|
||||
symlink targeting that run ID before engine construction or preflight starts.
|
||||
This points to the newly started debug session, including one that later fails;
|
||||
it is not a last-successful-run selector. Runs without capture leave it alone.
|
||||
Do not update it again at completion, so an older overlapping run cannot take
|
||||
the link back when it finishes. Concurrent starts use the last successful link
|
||||
update; individual run directories remain the actual storage locations.
|
||||
Treat link-update failures under the requested-capture failure policy below.
|
||||
|
||||
One application-owned recorder records initial context, accepts native execution
|
||||
evidence and application decisions, and finalizes once with the known pipeline,
|
||||
publication, and capture outcomes. Debug remains separate from output/cache,
|
||||
@@ -793,25 +804,66 @@ Neither debug verbosity nor enabled capture initiates additional provider probes
|
||||
Smoke reports retain native support, applicability, and result distinctions;
|
||||
status zero alone does not establish that a check passed.
|
||||
|
||||
### Open Discussion: Summary And Trace
|
||||
### Selected Bundle Organization
|
||||
|
||||
**Agreed principle:** Record each fact once and reference it where needed. Use
|
||||
shared application outcome facts and native execution evidence rather than
|
||||
parallel summary/trace accounting or repeated snapshots.
|
||||
|
||||
The current `summary/` and `trace/` layout is not selected as the new design.
|
||||
Discuss what diagnostic questions the bundle should answer and which application
|
||||
records and native evidence it needs. Summary/trace are not separate security
|
||||
classes. Keep these choices open:
|
||||
Organize the bundle around the run and its operations, replacing `summary/` and
|
||||
`trace/` as separate categories. Use a compact overview for navigation and keep
|
||||
detailed evidence with its operation. Summary/trace are not security classes.
|
||||
|
||||
- Whether a separate summary is useful, and whether it is a compact index/final
|
||||
outcome rather than another copy of execution records.
|
||||
- Whether summary/trace are useful conceptual categories or directory boundaries.
|
||||
- Which prepared details, outputs, conversation snapshots, and application
|
||||
decisions to persist, and at what granularity.
|
||||
- How self-contained a failed-run bundle should be, given that it may have no
|
||||
published output bundle, and when to reference existing artifacts instead of
|
||||
copying them.
|
||||
Illustrative layout: `latest` is the selected link name; other file names and
|
||||
exact serialized fields remain implementation choices:
|
||||
|
||||
```text
|
||||
<debug-root>/
|
||||
latest -> <run-id>/
|
||||
<run-id>/
|
||||
run.json
|
||||
config/
|
||||
application.yml
|
||||
promptkit.yml
|
||||
pipeline.json
|
||||
operations/
|
||||
<operation-id>/
|
||||
...
|
||||
```
|
||||
|
||||
| Component | Purpose |
|
||||
| --- | --- |
|
||||
| Run overview | Identify the invocation, outcome, published output, and available diagnostic records. |
|
||||
| Configuration snapshots | Preserve effective settings through native inspection. |
|
||||
| Resolved pipeline | Describe selected steps, lanes, modules, validators, and reference relationships. |
|
||||
| Operation records | Connect diagnostic evidence with its stage, chunk, lane, or validator. |
|
||||
|
||||
The overview is an index rather than a second execution ledger. Its terminal
|
||||
facts come from the same application outcome used for the CLI report, and it
|
||||
references detailed records instead of duplicating requests, responses, findings,
|
||||
or error history. Capture configuration and resolved composition once when
|
||||
available. Use native execution IDs with application scope to connect records.
|
||||
Reused work has a checkpoint/reuse decision and reference, not a fabricated model
|
||||
execution.
|
||||
|
||||
Retain enough available context to investigate a run that fails before publishing
|
||||
output. Do not depend entirely on a successful output manifest or automatically
|
||||
copy every original input and published artifact. Select a conversation
|
||||
representation that avoids repeatedly saving the complete accumulated history
|
||||
after each turn or maintaining an independent competing transcript.
|
||||
|
||||
### Open Discussion: Capture Depth
|
||||
|
||||
The organization is selected; precise evidence and granularity remain open:
|
||||
|
||||
- Which prepared request details, outputs, intermediate artifacts, validation
|
||||
decisions, and correction content to retain.
|
||||
- How to represent conversation history without repeated prefixes while using
|
||||
PromptKit's native evidence.
|
||||
- How much failed-run context to retain independently, and when references to
|
||||
existing inputs, checkpoints, or published artifacts suffice.
|
||||
- Which desired evidence is available through native public hooks and which
|
||||
gaps would justify an upstream feature request.
|
||||
|
||||
Evidence: [native shared debug binding](../../../commandkit/docs/consumers/guide.md#invocation-reporting),
|
||||
[reporting-mode ordering](../../../commandkit/docs/cli.md#modes-and-visibility),
|
||||
|
||||
Reference in New Issue
Block a user