Record native reporting machine output and outcomes
This commit is contained in:
@@ -22,8 +22,9 @@ this discussion.
|
||||
Item 1 selects the dependency versions and standard capabilities, including
|
||||
PromptKit v0.23.3 in place of the master roadmap's initial v0.23.2 target.
|
||||
Item 2 selects the application command structure and input conventions. Item 3
|
||||
selects native composition, shared loaders, and application preflight. Detailed
|
||||
reporting, debug capture, and verification choices remain open below.
|
||||
selects native composition, shared loaders, and application preflight. Item 4
|
||||
selects native reporting, machine output, and outcomes. Exact run-report fields,
|
||||
debug capture, and verification choices remain open below.
|
||||
|
||||
The PromptKit architecture and checkpoint decisions remain in force. This
|
||||
discussion develops the CommandKit portion of that migration and its interactions
|
||||
@@ -81,9 +82,9 @@ spending migration effort recreating existing CLI conventions.
|
||||
| Receipt content, debug artifact lifecycle, and application outcome meaning | Notarius, expressed through CommandKit's reporting and outcome conventions. |
|
||||
|
||||
Adopt the standard capability suite selected in item 1, including upstream
|
||||
commands and recommendations beyond the existing Notarius surface. Items 2–3
|
||||
select application inputs and composition; the remaining discussion determines
|
||||
result presentation, debug capture, and verification.
|
||||
commands and recommendations beyond the existing Notarius surface. Items 2–4
|
||||
select application inputs, composition, and reporting; the remaining discussion
|
||||
determines debug capture, verification, and concrete record details.
|
||||
|
||||
Pipeline and domain code should remain independent of CommandKit. Native
|
||||
PromptKit observations supply execution facts; Notarius attaches application
|
||||
@@ -91,10 +92,10 @@ meaning and selects report content. CommandKit supplies presentation mechanics.
|
||||
|
||||
Two application needs already identified for deliberate treatment are:
|
||||
|
||||
- The current `run --json` publishes a receipt after output publication. Determine
|
||||
the needed result information and express it through an appropriate native
|
||||
output facility. The old flag, envelope, formatting, and stream conventions are
|
||||
not compatibility requirements; reports must still describe the actual outcome.
|
||||
- The current `run --json` publishes a receipt after output publication. Item 4
|
||||
replaces its presentation with one compact application report rendered natively
|
||||
for humans and machines. Exact fields/versioning remain with U9; reports must
|
||||
describe the actual publication and domain outcome.
|
||||
- The current `--debug` requests durable artifacts. Retain the distinction between
|
||||
artifact capture and reporting verbosity, while choosing native controls and
|
||||
meanings rather than requiring that flag to retain its old behavior.
|
||||
@@ -499,24 +500,144 @@ Evidence: [native suite and loader ownership](../../../commandkit/docs/consumers
|
||||
|
||||
## 4. Reports, Machine Output, And Outcomes
|
||||
|
||||
**Discussion / decision:** Pending.
|
||||
**Discussion / decision:** Agreed. Use native CommandKit reports for command
|
||||
results, native outcomes for command completion, and a compact application-owned
|
||||
run report for artifact discovery and domain status. Distinguish pipeline results,
|
||||
command completion, and presentation delivery. Implementation is pending.
|
||||
|
||||
- How should native reports and output modes express application results? Where
|
||||
does an application data requirement justify raw output rather than a standard
|
||||
report, independently of the old CLI's output conventions?
|
||||
- How should syntax, configuration, pipeline, and publication outcomes map to
|
||||
native outcome/status conventions and user-facing explanations? How should
|
||||
presentation failures remain distinguishable from the operation outcome?
|
||||
- Which application scope and minimal projections connect native observations
|
||||
to progress, terminal summaries, manifests, and receipts without duplicate
|
||||
execution accounting?
|
||||
### One Report Value For Human And Machine Output
|
||||
|
||||
Review the current [run-result contract](../integrations/run-result.md) and
|
||||
Application commands submit ordinary values through `Reporter.Report` and use
|
||||
native `--format text|yaml|json` rendering. Execution supplies one run-report
|
||||
value with a concise text template for default human output; JSON and YAML render
|
||||
the same structured value. Do not maintain separate human-summary and JSON-receipt
|
||||
construction paths.
|
||||
|
||||
```sh
|
||||
notarius run dnd-session --input-file transcript.json
|
||||
notarius run dnd-session --input-file transcript.json --format json
|
||||
```
|
||||
|
||||
Standard configuration and PromptKit commands retain their upstream report shapes
|
||||
and behavior. Do not wrap them in a universal Notarius result envelope. No current
|
||||
run-report need requires raw output; reserve native `Output`/`RenderOutput` for an
|
||||
operation whose actual product needs a byte stream.
|
||||
|
||||
### Compact Application Run Report
|
||||
|
||||
CommandKit owns presentation; Notarius owns the meaning and deliberate contract
|
||||
of the application run report. Retain these content categories:
|
||||
|
||||
| Content | Purpose |
|
||||
| --- | --- |
|
||||
| Application report version | Identify the report contract for consumers. |
|
||||
| Run and pipeline identity | Identify completed work. |
|
||||
| Published bundle directory and applicable index path | Locate artifacts without guessing filenames. |
|
||||
| Accepted/normalized and rejected output counts | Summarize production results. |
|
||||
| Validation completeness/status | Distinguish fully validated results from permitted incomplete validation. |
|
||||
| Process-warning counts | Expose degradation independently of reporting verbosity. |
|
||||
| Debug location, when applicable | Locate requested diagnostic artifacts. |
|
||||
|
||||
Detailed per-producer validation records, rejection reasons, and execution
|
||||
provenance belong in the published bundle rather than duplicated collections in
|
||||
the command report. Do not serialize the entire internal pipeline result or
|
||||
complete upstream runtime types as a durable contract. U9 will select exact
|
||||
fields, versioning, and durable record formats. The old CLI receipt's shape is
|
||||
migration evidence, not a requirement to reproduce it.
|
||||
|
||||
### Publication, Streams, And Machine Consumption
|
||||
|
||||
The run command executes the pipeline, publishes the output bundle, completes
|
||||
required run finalization, submits its successful report, and returns a native
|
||||
outcome. The report describes an actual published result. Item 5 will settle the
|
||||
treatment of requested debug-finalization failures.
|
||||
|
||||
Use one primary run-result report, with progress and diagnostics separate. Wire
|
||||
CommandKit's result destination to stdout and its diagnostic destination to
|
||||
stderr. Subprocess consumers request `--format json`, capture streams separately,
|
||||
check the exit status, and decode the successful run report. A report-write
|
||||
failure may leave partial stdout even though publication already succeeded.
|
||||
These run-consumption rules do not replace the native report behavior of other
|
||||
commands, including configuration commands that can report failed validation.
|
||||
|
||||
### Native Visibility And Message Categories
|
||||
|
||||
| Native facility | Application use |
|
||||
| --- | --- |
|
||||
| `Report` | Requested command result. |
|
||||
| `Progress` | Pipeline progress. |
|
||||
| `Warn` | Actionable process degradation. |
|
||||
| `Info` | Supplementary operational information. |
|
||||
| `Debug` | Diagnostic detail, distinct from durable artifact capture. |
|
||||
| Outcome explanation | Terminal failure or usage guidance. |
|
||||
|
||||
Adopt native visibility rules directly: ordinary mode suppresses warning messages,
|
||||
and quiet mode suppresses command results. Include validation status and
|
||||
process-warning counts in the primary run report and its human summary so those
|
||||
facts do not depend on warning-message visibility. Operators request `--verbose`
|
||||
for warning details; machine callers expecting a report do not request `--quiet`.
|
||||
Use native progress selection and failure visibility without custom bypasses.
|
||||
Quiet suppression is not a delivery failure; native rendering errors can still
|
||||
fail a quiet invocation.
|
||||
|
||||
Artifact-quality advisories retain their application classification. Do not
|
||||
reclassify them as process warnings merely to make them visible.
|
||||
|
||||
### Native Outcomes And Domain Acceptance
|
||||
|
||||
Adopt the standard process-status conventions:
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
| `0` | The command completed successfully. |
|
||||
| `1` | Operational failure, cancellation, publication failure, or presentation failure. |
|
||||
| `2` | Invalid invocation or identifiable invalid configuration input. |
|
||||
| `3` | Native `config diff --check` found differences. |
|
||||
|
||||
Use ckconfig's structured configuration-error classification. Do not classify
|
||||
every configuration failure as operational or every preflight failure as invalid
|
||||
input. Invalid operator selections and missing bundled application resources can
|
||||
have different causes; preserve that distinction and native error identity.
|
||||
|
||||
Command completion remains separate from domain acceptance. A successfully
|
||||
published run can contain recorded rejections or validation-incomplete results
|
||||
allowed by application policy; those facts appear in reports and artifacts and
|
||||
do not automatically fail the command. A terminal pipeline failure does fail the
|
||||
command. Native status zero also does not replace reading standard PromptKit
|
||||
reports for partial discovery or unsupported smoke checks.
|
||||
|
||||
Application handlers supply useful native outcome explanations and retain the
|
||||
underlying error through native cause handling. CommandKit presents the terminal
|
||||
result. Remove duplicate terminal-error printing and custom outcome wrappers.
|
||||
|
||||
### Delivery Failures, Progress, And Accounting
|
||||
|
||||
Use CommandKit's native separation of operation causes from output and diagnostic
|
||||
delivery failures. If publication succeeds but stdout fails, the final command
|
||||
status reflects presentation failure while the published artifacts remain a
|
||||
fact. Do not rewrite pipeline history as failed execution or automatically rerun
|
||||
model work because report delivery failed.
|
||||
|
||||
Notarius owns stage/lane/chunk progress, domain acceptance, and checkpoint reuse.
|
||||
PromptKit supplies native execution observations and `ExecutionGroup` totals;
|
||||
CommandKit presents the selected facts. Explicitly enroll the relevant initial,
|
||||
continuation, and validator executions without double counting. Checkpoint reuse
|
||||
does not add historical token usage to current execution totals, and unavailable
|
||||
usage remains distinguishable from zero.
|
||||
|
||||
Remove bespoke format selection, parallel receipt encoders, custom verbosity
|
||||
handling, duplicate execution accounting, and superseded terminal-reporting
|
||||
machinery. Retain only application report content, publication facts, domain
|
||||
decisions, and small deliberate durable projections.
|
||||
|
||||
Evidence: [native reporting](../../../commandkit/docs/consumers/guide.md#invocation-reporting),
|
||||
[visibility and delivery](../../../commandkit/docs/cli.md#reporting), and
|
||||
[outcome classification](../../../commandkit/docs/cli.md#outcome-status).
|
||||
The current [run-result contract](../integrations/run-result.md) and
|
||||
[CLI failure mapping](../internal/cli.md#failure-mapping-and-terminal-reporting)
|
||||
to understand application needs and caller migration impact. Their existing CLI
|
||||
shapes and status mappings do not override the native-cutover direction.
|
||||
Item 6 of the [PromptKit discussion record](promptkit-architecture-discussion.md#6-inspection-and-execution-bookkeeping)
|
||||
provides the bookkeeping boundary; U9 still owns concrete durable record choices.
|
||||
document migration impact. Item 6 of the
|
||||
[PromptKit discussion record](promptkit-architecture-discussion.md#6-inspection-and-execution-bookkeeping)
|
||||
retains the bookkeeping boundary; U9 still owns concrete durable record choices.
|
||||
|
||||
## 5. Debug Capture And Optional Diagnostics
|
||||
|
||||
|
||||
Reference in New Issue
Block a user