From 09d4d7992a627cab1b098aca0044be093db1c422 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 23 Oct 2026 14:08:10 +0000 Subject: [PATCH] Draft output policy and invocation reporting roadmap --- docs/roadmap/reporting.md | 178 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 178 insertions(+) create mode 100644 docs/roadmap/reporting.md diff --git a/docs/roadmap/reporting.md b/docs/roadmap/reporting.md new file mode 100644 index 0000000..9280f85 --- /dev/null +++ b/docs/roadmap/reporting.md @@ -0,0 +1,178 @@ +# Output policy and invocation reporting + +Status: initial planning. The three output modes and uniform treatment of help +and version are selected. Consumer API, flag details, and delivery mechanics +remain open. Implementation has not started. + +## Purpose + +Provide consistent handling of `--quiet` and `--verbose` across application +commands and optional integrations. Reduce repeated output classification, +diagnostic presentation, and delivery bookkeeping where CommandKit can provide +a meaningful benefit. + +Ordinary Go writers and formatting functions already make application output +straightforward. Design reporting conveniences around the shared output policy +and invocation lifecycle. Applications continue to own their report content, +domain behavior, and operation outcomes. + +## Current baseline + +[`Streams` and `Invocation`](../../identity.go) expose caller-selected output +and diagnostic writers. [`Outcome` and `Diagnostic`](../../outcome.go) carry +operation causes, diagnostic writers, and delivery state. Handlers can return +`Failure(err)` or attach diagnostics to an outcome for presentation after the +handler returns. `MessageDiagnostic` currently writes the supplied message with +a newline; severity does not add a label automatically. + +CommandKit coordinates final diagnostic delivery and retains delivery failures +separately from the operation cause. Custom handlers can also write directly to +the supplied streams and retain responsibility for their output. The existing +outcome and channel ownership decisions are recorded in +[ADR 0005](../adr/0005-invocation-outcomes.md). + +Integration adapters already translate upstream diagnostic records into +CommandKit diagnostics while preserving native rendering and source context. +Some adapters wrap multiple records in one diagnostic writer and classify the +batch by its highest severity. This matters when designing selective visibility. + +There is currently no shared quiet/verbose policy or invocation reporter API. + +## Selected output policy + +The operator chooses among quiet, ordinary, and verbose presentation: + +| Output category | `--quiet` | No flag | `--verbose` | +| --- | --- | --- | --- | +| Substantive application output | Suppressed | Shown | Shown | +| Warnings and informational messages | Suppressed | Suppressed | Shown | +| Error diagnostics for a failed run | Shown | Shown | Shown | + +Substantive output includes requested reports and command results. Progress and +completion chatter are informational messages. The category of a message +determines its visibility; stream destination alone cannot distinguish a +warning from an error. + +Apply the same policy to help and version. There are no command-specific +exceptions for successful output: + +```text +app help --quiet # succeeds silently +app version --quiet # succeeds silently +``` + +Usage and operational failures still report their errors in quiet mode. +The feature must distinguish successful output suppression from failure +diagnostic delivery. + +## Tentative direction + +### Flag and invocation behavior + +Model quiet, ordinary, and verbose as mutually exclusive presentation modes. +The proposed rule when both flags occur is last occurrence wins, consistent +with existing repetition conventions. Exact boolean forms, resets, short +spellings, placement, and collision handling remain to be specified. + +Keep verbosity a presentation choice: a quiet invocation still performs its +requested operation and returns the ordinary outcome. Apply the policy through +CommandKit-managed reporting and streams. Output written directly to process +streams or external destinations remains application-owned. + +### Streaming and failure diagnostics + +The proposed approach applies visibility when output is emitted. A verbose run +may show progress before subsequently failing, and an ordinary run may write +part of its substantive output before failing. Previously written bytes cannot +be withdrawn. Whole-run buffering solely to make failed runs contain only error +output is not part of the proposed direction. + +Keep errors understandable in every mode, including source locations and +related explanatory context. Unrelated warnings and informational messages +remain subject to the selected mode. Retain suppressed diagnostics for +programmatic inspection rather than equating suppression with loss of evidence. +The exact representation of retained and delivered diagnostics remains open. + +### Consumer reporting API + +An invocation-scoped facility could classify messages, coordinate immediate and +end-of-handler presentation, and retain delivery failures. Earlier illustrative +API shapes included `inv.Reporter()`, `Info`, `Warn`, `Diagnostic`, and an +`AtEnd()` view. These names and shapes are exploratory, not selected contracts. +Reassess the necessary API around the output modes before adopting them. + +Continue to support standard `io.Writer` usage for report bodies and native +report writers. Reuse existing diagnostic and outcome concepts where practical. +Any reporting facility should coordinate with returned `Outcome.Diagnostics` +and operation causes so already-presented messages are not repeated. + +Whether reporting methods return write errors, retain them for finalization, +or expose an explicit delivery-error accessor remains open. Emitting an +error-severity message and deciding the operation's outcome need clearly +defined responsibilities. + +### Upstream diagnostic ownership + +ConfigKit and PromptKit remain authoritative for their diagnostic semantics, +severity, source context, and disclosure rules. Integration adapters expose +those capabilities through the shared CommandKit presentation policy. + +Support consistent reporting of ConfigKit errors and warnings as its adopted +public API evolves. This roadmap does not assume that a future warning API +already exists or prescribe its upstream shape. Preserve useful native +diagnostic detail and supported error identity, and coordinate returned errors +with diagnostics that already explain them. + +## Work needed + +1. Settle the open decisions with examples covering ordinary application output, + immediate and deferred diagnostics, failure after partial output, help and + version, and integration diagnostics in every mode. +2. Review current diagnostic batching, parsing, help, outcome presentation, and + writer ownership to identify the smallest shared changes. Include invocation + failures that occur before a handler receives its `Invocation`. +3. Specify the public API and flag contracts, including interactions with command + and flag aliases. Assess existing flag collisions and compatibility impact. +4. Implement the selected behavior across core and the integrations. Preserve + upstream rendering and semantics while applying the chosen visibility policy. +5. Verify classification, flag selection, suppression, retained diagnostics, + ordering, duplicate prevention, and write-failure handling with focused + behavioral tests and maintained consumer examples. +6. Update GoDoc, the CLI reference, consumer guidance, and release documentation + when implemented. Review whether any resulting architectural change needs an + ADR; exact reporting APIs and flag syntax remain API/CLI contracts. + +## Open Decisions + +### OD-1: Flag grammar and scope + +How are quiet and verbose selected, repeated, combined, or explicitly reset? +Are short spellings built in? Where may the flags appear, including on help +requests? How are canonical and alias spelling collisions handled, and how is +the mode determined for invocations that fail during routing or parsing? + +### OD-2: Consumer API and output classification + +What is the smallest API for substantive output, informational messages, +warnings, and errors? How do direct writes through `Invocation.Streams` interact +with it? How can helpers report through the same invocation? Is a dedicated +immediate/deferred reporter needed, and how does it compose with existing +outcome diagnostics? + +### OD-3: Failure presentation and diagnostic retention + +What identifies an error that must be shown, particularly for application domain +outcomes and error-severity diagnostics accompanying a successful outcome? +Which contextual records belong to an error explanation? How are mixed upstream +diagnostic batches filtered while preserving useful native rendering? What is +retained when presentation is suppressed, and how is duplicate presentation +prevented without hiding unrelated failures? + +### OD-4: Delivery lifecycle and quiet execution + +How are immediate and deferred messages ordered and finalized, including when +the operation fails? How do callers observe and react to write failures? What +delivery state represents intentional suppression? Does quiet mode discard +rendered bytes or skip report rendering, and how does that choice preserve +operation semantics and failure detection? Which streaming behavior becomes +part of the public contract?