commandkit/docs/roadmap/reporting.md

Output policy and invocation reporting

Status: initial planning. Message categories, all four output modes, uniform treatment of help and version, and OD-1's flag grammar and scope are selected. Consumer API and delivery mechanics remain open. Implementation has not started.

Purpose

Provide consistent handling of --quiet, --verbose, and --debug across application commands and optional integrations. Reduce repeated output classification, diagnostic presentation, and delivery bookkeeping where CommandKit can provide a meaningful benefit.

Ordinary Go writers and formatting functions already make application output straightforward. Design reporting conveniences around the shared output policy and invocation lifecycle. Applications continue to own their report content, domain behavior, and operation outcomes.

Current baseline

Streams and Invocation expose caller-selected output and diagnostic writers. Outcome and Diagnostic carry operation causes, diagnostic writers, and delivery state. Handlers can return Failure(err) or attach diagnostics to an outcome for presentation after the handler returns. MessageDiagnostic currently writes the supplied message with a newline; severity does not add a label automatically.

CommandKit coordinates final diagnostic delivery and retains delivery failures separately from the operation cause. Custom handlers can also write directly to the supplied streams and retain responsibility for their output. The existing outcome and channel ownership decisions are recorded in ADR 0005.

Integration adapters already translate upstream diagnostic records into CommandKit diagnostics while preserving native rendering and source context. Some adapters wrap multiple records in one diagnostic writer and classify the batch by its highest severity. This matters when designing selective visibility.

There is currently no shared verbosity policy or invocation reporter API.

Selected output policy

Distinguish substantive application output from diagnostic message categories:

• Substantive output (Normal): The requested report or command result.
• Error: The invocation failed and will return a non-zero status code.
• Warning: Something went wrong or deserves attention, but the invocation can still succeed. A recovered problem is a warning.
• Info: Progress or useful explanation, including completion messages.
• Debug: Troubleshooting detail, including intermediate decisions and execution-path information.

Debug mode adds troubleshooting detail to verbose output:

Output category	--quiet	No flag	--verbose	--debug
Substantive application output	Suppressed	Shown	Shown	Shown
Error	Shown	Shown	Shown	Shown
Warning	Suppressed	Suppressed	Shown	Shown
Info	Suppressed	Suppressed	Shown	Shown
Debug	Suppressed	Suppressed	Suppressed	Shown

Info belongs in verbose mode; debug includes verbose messages. Normal output remains a substantive result category, separate from diagnostic severity. The category of a message determines its visibility; stream destination alone cannot distinguish a warning from an error.

Error presentation is tied to invocation failure. Prefer deriving error diagnostics from a failed outcome, which may carry detailed explanations. Avoid an unrestricted Error(...) reporting method that permits an error message followed by a successful outcome. The application still controls process exit under the existing outcome contract; the returned outcome must consistently represent the failure and its non-zero status.

Apply the same policy to help and version. There are no command-specific exceptions for successful output:

app help --quiet       # succeeds silently
app version --quiet    # succeeds silently

Usage and operational failures still report their errors in quiet mode. The feature must distinguish successful output suppression from failure diagnostic delivery.

Selected flag grammar and scope

Mode selection and resets

Model quiet, ordinary, verbose, and debug as mutually exclusive presentation modes. Each invocation starts in ordinary mode. Process flag occurrences in invocation order; the last enabled mode wins. Repetition does not increment verbosity, and debug is selected explicitly.

Accept presence and the existing explicit boolean syntax, including --quiet=true and --quiet=false. A false occurrence clears its mode only if that mode is currently selected, returning to ordinary mode. It does not restore an earlier selection or disable a different mode. Every occurrence is converted; invalid boolean values remain usage errors even when a later flag would replace the selection.

Flag sequence	Final mode
--quiet --verbose	Verbose
--debug --quiet	Quiet
--verbose --verbose	Verbose
--verbose --verbose=false	Ordinary
--quiet --verbose=false	Quiet
--verbose --quiet --quiet=false	Ordinary
--quiet=false --debug	Debug

Keep verbosity a presentation choice: a quiet invocation still performs its requested operation and returns the ordinary outcome. Apply the policy through CommandKit-managed reporting and streams. Output written directly to process streams or external destinations remains application-owned.

Placement, availability, and aliases

Core supplies --quiet, --verbose, and --debug for application commands, integrations, the root, canonical groups, and alias-only discovery help. Consumers do not register them individually. Preserve the existing placement grammar: flags follow the complete command path and may be interspersed with operands after that path. For example:

app config show --quiet
app export destination --verbose
app help config show --quiet
app version --quiet

Options before or between command-path components are outside this grammar. Tokens consumed as values and tokens following -- retain their ordinary meaning.

Reserve the three long spellings. Applications may add -q, -v, or -d through the alias API; no built-in short spellings are selected. An application flag declaration that defines a reserved long name fails installation with a collision error. Aliases may target these flags but may not overload canonical spellings. Apply the existing collision and atomic installation rules, including later registration.

Aliased and canonical occurrences participate in the same ordering. With -q → --quiet, app export --verbose -q selects quiet mode. Document the newly reserved canonical names as a compatibility change.

Routing and parsing failures

Determine the mode through the normal parser before execution or successful help/report presentation. Do not add a separate verbosity scanner. This keeps token interpretation consistent and avoids reinterpreting values or operands.

Routing and parsing failures report their errors regardless of verbosity. For example, app export --quiet --unknown still reports the unknown flag. Error visibility follows the selected output policy independently of how the mode is determined; avoiding a separate scanner is an implementation simplification.

CommandKit's own diagnostics

Use the shared reporting system for CommandKit's invocation diagnostics so parser, integration, and application messages follow the same visibility and delivery policy. Installation errors occur before an invocation exists and continue to be returned to the application without automatic printing.

Supported repetition and last-occurrence overrides are routine decisions. Repeated singleton flags, including --verbose --verbose, may produce debug detail; repetition alone does not warrant a warning. Repeatable flags retain their intended accumulation behavior. Add warnings for concrete concerns an operator can act on, such as a deprecated spelling if deprecation is introduced. Unknown flags, invalid values, and invalid command paths remain invocation failures with error diagnostics.

Start debug instrumentation with routing, alias resolution, verbosity selection, and invocation completion. Other useful candidates can be added where they explain a consequential decision:

Event	Useful detail
Command-alias resolution	Entered path, intermediate redirects, and final canonical path.
Flag-alias resolution	Entered spelling and canonical flag name.
Scoped alias selection	A command-scoped mapping took precedence over an application-wide mapping.
Verbosity selection	A later occurrence selected a different mode; for --verbose --debug, debug wins because it appears last.
Help-route selection	A canonical command or alias-only discovery context was selected.
Repeated singleton flag	A later occurrence replaced an earlier occurrence, without printing their values.
Input preparation	Canonical input validation completed and the handler is starting.
Invocation completion	Canonical command, outcome classification, and status.

Use the final selected mode to present earlier parser decisions. Collect the relevant events during parsing and apply visibility after mode selection: --verbose --debug shows the selection explanation, while --debug --quiet suppresses it. This collection of parser events does not require whole-run output buffering.

Debug messages should explain operator-relevant decisions independently of incidental implementation steps. Use command paths, flag names, and decision descriptions rather than raw argument or configuration-value dumps. Upstream libraries retain ownership of their diagnostic detail and disclosure rules.

Tentative direction

Streaming and failure diagnostics

The proposed approach applies visibility when output is emitted. A verbose or debug run may show progress before subsequently failing, and an ordinary run may write part of its substantive output before failing. Previously written bytes cannot be withdrawn. Whole-run buffering solely to make failed runs contain only error output is not part of the proposed direction.

Keep errors understandable in every mode, including source locations and related explanatory context. Unrelated warnings and informational messages remain subject to the selected mode. Retain suppressed diagnostics for programmatic inspection rather than equating suppression with loss of evidence. The exact representation of retained and delivered diagnostics remains open.

Consumer reporting API

An invocation-scoped facility could classify messages, coordinate immediate and end-of-handler presentation, and retain delivery failures. Earlier illustrative API shapes included inv.Reporter(), Info, Warn, Diagnostic, and an AtEnd() view. These names and shapes are exploratory, not selected contracts. Reassess the necessary API around the output modes and expanded message categories before adopting them. Debug messages need a corresponding classification; its exact API remains open.

Continue to support standard io.Writer usage for report bodies and native report writers. Reuse existing diagnostic and outcome concepts where practical. Any reporting facility should coordinate with returned Outcome.Diagnostics and operation causes so already-presented messages are not repeated.

Whether reporting methods return write errors, retain them for finalization, or expose an explicit delivery-error accessor remains open. The API must enforce the selected relationship between Error diagnostics and failed outcomes, including diagnostics supplied through generic or existing entry points.

Upstream diagnostic ownership

ConfigKit and PromptKit remain authoritative for their diagnostic semantics, severity, source context, and disclosure rules. Integration adapters expose those capabilities through the shared CommandKit presentation policy.

Support consistent reporting of ConfigKit errors and warnings as its adopted public API evolves. This roadmap does not assume that a future warning API already exists or prescribe its upstream shape. Preserve useful native diagnostic detail and supported error identity, and coordinate returned errors with diagnostics that already explain them.

Debug output follows the same upstream disclosure and safe-rendering contracts. Enabling debug does not authorize secret disclosure or bypass owning-library rules.

Work needed

1. Settle the open decisions with examples covering ordinary application output, immediate and deferred diagnostics, failure after partial output, help and version, and integration diagnostics in every mode.
2. Review current diagnostic batching, parsing, help, outcome presentation, and writer ownership to identify the smallest shared changes. Include invocation failures that occur before a handler receives its Invocation.
3. Specify the public API and flag contracts, including interactions with command and flag aliases, following the selected OD-1 rules above. Assess existing flag collisions and compatibility impact.
4. Implement the selected behavior across core and the integrations. Preserve upstream rendering and semantics while applying the chosen visibility policy.
5. Verify classification, flag selection, suppression, retained diagnostics, ordering, duplicate prevention, Error/non-zero-outcome consistency, debug visibility, and write-failure handling with focused behavioral tests and maintained consumer examples. Cover mode resets, mixed canonical/alias ordering, help and discovery contexts, and core parser events presented according to the final mode.
6. Update GoDoc, the CLI reference, consumer guidance, and release documentation when implemented. Review whether any resulting architectural change needs an ADR; exact reporting APIs and flag syntax remain API/CLI contracts.

Open Decisions

OD-2: Consumer API and output classification

What is the smallest API for substantive output, debug and informational messages, warnings, and failure diagnostics? How do direct writes through Invocation.Streams interact with it? How can helpers report through the same invocation? Is a dedicated immediate/deferred reporter needed, and how does it compose with existing outcome diagnostics?

OD-3: Failure presentation and diagnostic retention

How does the API enforce that Error diagnostics accompany a failed invocation with a non-zero status, including application domain outcomes? How should inconsistent caller-supplied combinations be handled through existing diagnostic and outcome APIs? How are recovered upstream problems represented as warnings while preserving their native context? Which contextual records belong to an error explanation? How are mixed upstream diagnostic batches filtered while preserving useful native rendering? What is retained when presentation is suppressed, and how is duplicate presentation prevented without hiding unrelated failures?

OD-4: Delivery lifecycle and quiet execution

How are immediate and deferred messages ordered and finalized, including when the operation fails? How do callers observe and react to write failures? What delivery state represents intentional suppression? Does quiet mode discard rendered bytes or skip report rendering, and how does that choice preserve operation semantics and failure detection? Which streaming behavior becomes part of the public contract?