Files

Eric Rakestraw 0b70f37267 Add detailed validation evidence

2026-11-07 04:52:58 +00:00

49 KiB

Raw Blame History

Command line reference

CommandKit owns command routing and syntax for commands registered with its App. The caller supplies argv without the executable name, a context, and the output and diagnostic writers. App.Execute returns an Outcome; it does not exit the process.

Help

The root help command lists registered commands and their summaries. Help is available through a bare invocation, --help, -h, and help:

example
example --help
example -h
example help
example help config show
example config --help

help [COMMAND...] resolves an exact registered path. A command group without an execution handler displays its help when invoked by itself. A runnable command with children runs its handler when invoked without help; add --help to inspect that command and its children.

Help uses registered descriptions, command summaries, local flags, defaults, required and repeatable flag metadata, and operand names and counts. It does not construct the selected command's input or invoke its handler. Supplied flag and operand values are still parsed so malformed values and unknown flags remain usage errors. Missing execution-required values and input-combination checks are skipped for help.

Version

version reports the application identity by default:

example version
example version --scope commandkit
example version --format yaml

The supported flags are:

Flag	Values	Default	Meaning
`--scope`	`app`, `commandkit`	`app`	Select the caller's application identity or the embedded CommandKit module identity.
`--format`	`text`, `yaml`, `json`	automatic	Select the output format; automatic uses text for version.

Text output is NAME VERSION, followed by (REVISION) when the selected identity has a revision. An empty application version is displayed as dev; the revision remains empty when omitted.

CommandKit identity uses the version of its module in build information. A versioned module replacement supplies the effective version. A local or versionless replacement, an untagged module, or missing build information displays dev. The CommandKit identity name is CommandKit; dependency build metadata does not supply a revision.

YAML and JSON output use these fields, including an empty revision:

name: ExampleApp
version: dev
revision: ""

Shell completion

Applications may install shell completion explicitly. When installed, generate and load the script for the executable name chosen by the application:

# Bash, after initializing bash-completion
source <(app completion bash)
# Zsh, after initializing its completion system with compinit
source <(app completion zsh)
# Fish
app completion fish | source

Bash requires the bash-completion package and its helpers in the current shell. If your startup files do not initialize it, source the package's initialization file before loading the generated script. For example, on systems that install it at the following path:

source /usr/share/bash-completion/bash_completion
source <(app completion bash)

Use the path provided by your Bash completion installation.

The application selects the executable basename at installation; use that same name to invoke the generated script. Completion offers canonical command and flag names, static values, and file or directory paths where declarations provide hints. A fully entered alias is resolved to its canonical command; alias spellings and partial alias paths are not suggested. Completion queries do not invoke input factories, providers, or the selected command; application startup remains application-owned. See the consumer guide for installation and hint declarations. Script generation follows the reporting policy, including quiet mode.

Reporting

CommandKit applies one shared reporting policy to application handlers, integrations, help, version, and CommandKit's own invocation diagnostics. The caller passes result and diagnostic writers to App.Execute; substantive results go to Streams.Out, and categorized messages, terminal explanations, and usage guidance go to Streams.Err.

The shared --format flag selects text, yaml, or json for reports submitted through the core reporter. When omitted, a report's text template selects text; otherwise scalar values and nonempty string lists use text, while structured values and empty collections use YAML. Explicit text without a template uses YAML for structured values. Invalid or empty format values are usage errors. Raw output, help, completion, and diagnostics are not encoded as reports.

Modes and visibility

Each invocation starts in ordinary mode. --quiet, --verbose, and --debug are mutually exclusive; the last enabled occurrence selects the mode. They accept presence or an explicit boolean value. A false occurrence clears the selected mode only when it names the current mode, returning to ordinary mode; it does not restore an earlier selection. Invalid boolean values are usage errors, even if a later flag would select another mode.

Category	`--quiet`	No mode flag	`--verbose`	`--debug`
Substantive result	Suppressed	Shown	Shown	Shown
Terminal failure	Shown	Shown	Shown	Shown
Warning	Suppressed	Suppressed	Shown	Shown
Info	Suppressed	Suppressed	Shown	Shown
Debug	Suppressed	Suppressed	Suppressed	Shown
Progress, unless enabled explicitly	Suppressed	Suppressed	Suppressed	Shown

--progress independently enables Progress; --progress=false disables it. Both accept the existing boolean syntax, and the last occurrence wins. An explicit setting overrides the mode default, except quiet always suppresses Progress. Without an explicit setting, debug enables Progress; ordinary and verbose modes leave it disabled. Progress does not enable Info, Warning, or Debug.

Placement and aliases

Reporting flags are available on application commands, the root, canonical groups, help, version, and alias-only discovery help. Place them after the complete command path; they may be interspersed with operands after that path. For example:

example export --quiet
example config show --verbose
example promptkit test all --debug --progress=false
example help config show --quiet
example version --quiet

Reporting flags do not go before or between command-path components. Tokens consumed as values and tokens after -- retain their normal meaning.

The reserved long spellings are --quiet, --verbose, --debug, --progress, and --format; application declarations cannot reuse them. There are no built-in short spellings. Applications may define aliases such as -q → --quiet, -v → --verbose, and -d → --debug; canonical and aliased occurrences share the same ordering. Thus example export --verbose -q selects quiet mode.

Channels and messages

Substantive reports use the result destination. Warnings, Info, Progress, Debug, terminal explanations, and usage guidance use the diagnostic destination. Simple Warning, Debug, and terminal failure messages begin with WARN:, DEBUG:, and ERROR: respectively. Info, Progress, and substantive results have no prefix. For multiline messages, the prefix appears on the opening line; native structured details keep their own formatting without duplicate labels. Progress is streamed as unprefixed lines; terminal redraw and progress bars are not provided. A failed outcome with no message or detail uses the fallback ERROR: Command failed..

Quiet mode suppresses successful help and version output, as it does other substantive results. Usage guidance and routing, parsing, and operational failures remain visible in every mode. A recovered problem is reported only if the application chooses to submit a Warning; an error cause by itself does not create a message.

Presentation failures

CommandKit continues rendering substantive output under quiet mode through a discard destination. This suppresses writes to the caller's result writer while still detecting report preparation, encoding, or rendering failures. An intentional suppression is not a delivery failure. A failure while rendering or writing requested content is retained in the returned Outcome and makes an otherwise successful or usage invocation operational with status 1. A domain classification and its nonzero status are preserved; a zero-status domain outcome becomes status 1 when presentation fails. Earlier bytes already written remain visible.

Registered command syntax

Options are parsed after the complete command path and may be interspersed with declared operands. Flags are local to the selected command. Use -- to end option parsing; all later tokens are operands.

Value flags accept --name value and --name=value. Boolean flags accept presence as true and explicit values such as --enabled=false. A singleton flag accepts repeated occurrences; the last successfully parsed occurrence supplies its value. Every supplied occurrence is converted, so an earlier malformed value remains an error. Repeatable flags preserve occurrence order, with each occurrence treated as one value, including strings containing commas.

Typed inputs keep supplied state separate from defaults. An explicit empty, false, or zero value is supplied. Help and version are available without constructing application command inputs.

Application-owned aliases

Applications can install command and flag aliases alongside their canonical commands. Aliases route an invocation to existing behavior; they do not add handlers, commands, or flags to a canonical command family.

Command alias paths are sequences of command tokens relative to the application root. The executable name, flags, flag values, and operands are not part of an alias path. Matching uses complete tokens and chooses the longest matching path. A group alias preserves its remaining command tokens, and aliases can compose through chains. For example, list collections can route to promptkit list collections, while pk can route to the promptkit group. The target must resolve to a canonical command or group.

Aliases do not rewrite operands or supply arguments. In particular, promptkit test KIND remains one canonical command whose KIND is an operand; all is an operand value, not a child command. smoke → promptkit test is a valid shortcut because it preserves the caller's KIND value.

Flag aliases can be application-wide or scoped to a canonical command path. An application-wide alias is effective wherever its canonical target flag is available and is ignored elsewhere. A scoped alias requires its canonical command and target flag to be installed. The selected canonical command determines applicable scoped aliases, so the alias also works when that command is reached through another command path. A scoped mapping takes precedence over an application-wide mapping with the same spelling.

Only tokens parsed as flag names participate in flag aliasing. Values, operands, and tokens following -- remain unchanged. This includes values that look like flags, such as --message=--hlep. Aliased flags use the target's canonical binding, conversion, default, required-value check, presence, and repetition behavior. Mixed spellings preserve occurrence order: singleton flags use the last successfully parsed occurrence, every supplied occurrence is converted, and repeatable flags accumulate values in order.

Flag aliases accept one long spelling or one individual short spelling. Short aliases use the canonical short-option cluster and attached-value grammar:

Input	Interpretation
`-vq`	Two boolean flags, `--verbose` and `--quiet`.
`-cblue` or `-c blue`	Value `blue` for a value-taking `-c` flag.
`-vqcblue`	`-v`, `-q`, then `--color=blue`.
`-cvq`	Value `vq` for a value-taking `-c` flag.
`-v=false`	Explicit false for the boolean flag `-v`.
`-c=`	An explicitly supplied empty string.

Boolean flags use presence as true; an explicit boolean value requires =. The first value-taking short flag consumes the remaining cluster suffix or the next argument. A consumed value is never interpreted as an alias. The canonical short-option empty-value form -c= supplies an empty value, consistently with --color= and -c "".

Help reached through an alias describes the canonical command's flags and operands while using the entered route in usage. It identifies the canonical target, lists applicable aliases separately, and supports global help aliases such as --hlep → --help. Help for an alias-only prefix lists its routes without creating a command group; the bare incomplete prefix points to that help. Help and alias registration do not construct command inputs or invoke acquisition callbacks.

Usage diagnostics preserve the command path and flag spelling the caller used. When a command alias resolved successfully, usage identifies its canonical target while retaining the entered path in the diagnostic. An unresolved route receives no invented canonical label. A flag conversion error identifies the entered flag spelling and the failing occurrence while preserving the original cause.

Configuration scopes

An application installs the declaration returned by ckconfig.Command(ckconfig.Suite[Input, State]{...}) through commandkit.New or App.Register. For the common single application scope, ckconfig.AppCommand returns a convenience declaration. The installed config scopes command lists scope names and the configuration operations currently available for those scopes:

example config scopes
example config scopes --format yaml

Text output uses one scope: operation, operation line per scope and lists any declared optional features. Structured output uses a scopes sequence with name, operations, and any supported features. Scope order is app, then promptkit, then other names alphabetically. A metadata-only scope can be listed without an acquisition callback. Discovery does not construct a runtime context or acquire configuration. It reports the intersection between declared scope capabilities and installed handlers, so an operation is listed only when the corresponding command is available.

Scopes can opt in to transient overrides. For an opted-in scope, acquisition commands accept repeatable --set POINTER=JSON; comparison also accepts --against-set POINTER=JSON for the right side. Each assignment is passed to the loader in order, with --set or --against-set as its source label. The pointer and value are split at the first =, so pointer keys containing = cannot be addressed by these flags; JSON strings can contain =. Values use JSON types, including null, false, zero, and empty containers. ConfigKit owns override path validation and conflict handling. A selected scope must opt in before receiving overrides; for --scope all, every selected scope must support them.

Applications can add ckprompt.ConfigScope(loadEngine) to the ConfigKit suite to expose PromptKit engine configuration through these handlers. The scope declares validate, show, sources, explain, and native diff. It does not install PromptKit prompt or profile definition commands.

Validate configuration

config validate loads and validates the selected scope. The default scope is app; --scope all validates every registered scope and is available when each scope declares validation. The command checks all selected capabilities before calling scope providers. Automatic output is concise text; structured formats emit a scopes sequence with each reached scope and its outcome (valid, failed, or canceled).

example config validate
example config validate --scope promptkit --format yaml
example config validate --scope all
example config validate --scope all --details --severity warning --code schema.version

Each successful scope is reported as SCOPE: valid. Aggregate validation stops at the first load failure or cancellation, then submits one report containing the reached scopes. Output errors occur after acquisition has stopped. Provider warnings go to the diagnostic stream and do not alone make validation fail. --details includes native findings and the source references needed to interpret them. Each scope's detail state is unavailable, empty, filtered, or present. Repeat --severity error|warning or --code CODE to filter findings: values within a filter match any, and the two filters narrow together. Filters require --details and never change the validation outcome. Detailed output suppresses ordinary warnings associated with its native findings; other messages and terminal failure explanations remain visible. --config and --overlay preserve whether an empty value was explicitly supplied, so a scope callback can map those inputs to its own ConfigKit load policy. Help and invalid syntax do not call scope providers.

Show effective configuration

config show displays the provider's safe effective configuration. The default scope is app. The native inspection YAML is converted to a logical report value, preserving meaningful scalar types, empty collections, and redaction. Automatic formatting uses text for a single scalar or nonempty string list and YAML for structured values; --format selects text, YAML, or JSON.

example config show
example config show --scope promptkit
example config show --scope all
example config show --path /server/addr --format json

--scope all acquires every selected scope before producing one mapping keyed by scope name, including when only one scope is registered. A single selected scope is unwrapped. A provider failure prevents report output. Conversion or output failures are recorded in Outcome.Presentation.Output.

For a scope that supports path-selected show, --path POINTER displays only that safe effective value. An explicitly empty pointer selects the root, and a present null remains a value. Missing and historical-only paths fail; path selection cannot be combined with --scope all.

List configuration sources

config sources reports the source inventory captured during the selected scope's acquisition. It reads the source facet once and does not reload configuration. The default scope is app; --scope all is unsupported. Text is the automatic format, and --format yaml or --format json returns a structured report. --source-id ID selects one entry by the ID from that acquisition; an unknown ID fails. If acquisition fails but returns a captured source inventory, the command reports that evidence and preserves the acquisition error. Unavailable inventory is reported as such.

scope: app
sources:
  - id: source-1
    label: "application config"
    path: /etc/example/config.yml
    layer: main
    kind: file
    underlay_index: 0
    imported_from: null
    destination: /server
    placement: mapping
    destination_is_fallback: false
    version_evidence:
      path: /version
      version:
        present: true
        null: false
        text: "1"
      location:
        source_id: source-1
        line: 2
        column: 1
      redacted: false
    empty_import_selections: []

Each source includes id, label, path, layer, kind, underlay_index, and imported_from, plus the native destination pointer, placement, destination_is_fallback, nullable version_evidence, and empty_import_selections. Placement is mapping or sequence_item. The import location is null when absent; otherwise it has source_id, line, and column. Version evidence preserves path, present, null, text, optional source location, and redacted as returned by ConfigKit. Owning values and source order are preserved, including empty fields, zero coordinates, false and null observations, and synthetic sources without paths. Absent version evidence is null. Each empty-import record has a source-local path, a location mapping (source_id, line, column), and a selector; records preserve their returned order. No records are empty_import_selections: []. An empty inventory is sources: []. Text output begins with the scope heading and prints the same source fields, quoting labels and paths. These fields project the selected ConfigKit source records; CommandKit does not reload files or rerun version checks. See the ConfigKit v1.1.0 source records for native evidence definitions.

Explain a configuration path

config explain --path POINTER explains the effective and historical value at a JSON Pointer path. An explicitly empty path selects the root. config explain --list discovers paths when the selected scope supports path discovery. An invocation with neither mode displays help; the two modes cannot be combined. The default scope is app; --format yaml or --format json selects structured output, and automatic output uses text. --scope all is unsupported.

example config explain --path /server/addr
example config explain --path /profiles/team~1summary --format yaml
example config explain --path= --format yaml
example config explain --list --under /profiles --historical

Listing normally includes current paths. --historical adds paths that only occurred in history; --under POINTER selects a subtree, and --explicit=true or --explicit=false filters current paths by native explicitness. Historical-only paths do not match an explicitness filter. These options require --list. The report contains the scope and each selected path's current presence and, when present, its explicitness. Path discovery does not include values or history.

Pointer tokens use JSON Pointer escaping: ~1 represents / and ~0 represents ~. A malformed pointer is a usage error with status 2 and is rejected before acquisition. A well-formed path that is unknown in the selected configuration is an operational error with status 1. Text output includes the scope's native explanation and available field metadata. Structured output includes metadata when supplied by the scope.

Structured output uses these root fields:

scope: app
path: /server/addr
effective:
  present: true
  redacted: false
  yaml: "localhost:8080\n"
origin:
  layer: main
  locations:
    - source_id: source-1
      line: 4
      column: 9
  operation: ""
  inheritance: []
history: []
secret_reference: null

Snapshot yaml values remain strings, including null and empty fragments. Origin and history retain source locations, operation names, inheritance paths, ordered reasons and dependencies. Secret references contain only the owning environment-variable name; effective and historical secret fragments remain redacted. Empty location, inheritance, history, reason, and dependency lists are serialized as [].

Compare configuration selections

config diff prepares two selections for one scope that declares comparison and invokes its comparison provider after input checks pass. The default scope is app; --scope all is unsupported. When an application installs ckprompt.ConfigScope(loadEngine), the PromptKit scope declares comparison and delegates it to PromptKit's captured configuration inspection API. PromptKit owns the meaning of that native comparison; CommandKit uses the existing config diff preparation and report path. --config and --overlay select the left side. --against-config and --against-overlay select the right side. An omitted field keeps that side's ordinary application loading behavior and unsupplied state; an explicit empty string remains supplied. Application selectors affect the left side, while declared --against-* counterparts affect the right. Equal selections are allowed and do not cause CommandKit to skip the provider.

Automatic output is YAML. The shared --format also accepts YAML, JSON, and text; text uses the same structured representation. For example:

example config diff --config before.yml --against-config after.yml
example config diff --config base.yml --against-overlay candidate
example config diff --config base.yml --overlay current --against-overlay ""
example config diff --scope promptkit --config before.yml --against-config after.yml
example config diff --config before.yml --against-config after.yml --details --check

--details replaces the concise report with {scope, differences}, containing one entry per native difference. Each entry has path, before, after, and hidden_changed; each snapshot has present, redacted, and value. An absent snapshot has present: false and a null value, distinct from a present null. Empty comparisons contain differences: [].

--check returns status 3 when native differences exist and 0 when they do not. Without it, a completed comparison succeeds even when differences exist. Usage errors retain status 2, and operational failures retain status 1. The report is still delivered in check mode; a presentation failure remains separate evidence and does not replace a nonzero difference outcome.

For ConfigKit-backed scopes, ConfigKit compares frozen effective typed values after defaults and resolution; source or provenance differences alone do not create a change. PromptKit delegates its native comparison to ConfigKit. The report contains additions, changes, and deletions. Additions and changes preserve semantic paths and contain right-side values provided by the native comparison. Deletions contain canonical JSON Pointer paths. Explicit null is a value; it is not a deletion. Changed sequences and native structural replacement regions stay atomic.

ConfigKit pairs sensitivity across both loads. A concealed change can therefore appear even when its two safe snapshots are equal. For those ConfigKit-backed comparisons, the report omits separate redaction and hidden-change metadata: a reported masked region changed, but that does not mean every member inside it changed. Marker-looking public strings stay ordinary strings. An empty comparison succeeds with this shape:

additions: {}
changes: {}
deletions: []

The reduced report is a partial safe view, not a reloadable configuration or an automatically applicable patch. Exceptional whole-root empty/null transitions can be ambiguous because the report has no root envelope; ordinary field-level null values and deletions retain their distinction.

Successful comparisons return status 0, including empty results. Unsupported scopes and format errors are usage errors; input checks retain the existing usage/operational classification. Load and native comparison failures are operational and preserve supported error identity.

The bundled FromConfigKit and AppCommand adapters invoke the supplied loader for each reached side, left before right, then call native comparison. They label native diagnostics with their left/right side and source legend, retain left diagnostics if right loading fails, and gather both loads' diagnostics before native comparison. Custom pair providers own their loading and diagnostics; application loaders and caches determine whether a loader call reads sources again.

The complete report is prepared before writing. Report preparation failures and cancellation observed before report delivery starts leave CommandKit's report output empty. Once writing starts, CommandKit does not interrupt the supplied writer or roll back delivered bytes. A write error or short write is recorded as incomplete delivery in Outcome.Presentation.Output and can leave partial output.

PromptKit definition inspection

Applications can install prompt, profile, and backend inspection independently with ckprompt.Command(suite). ckprompt.FromEngine(loadEngine) supplies these callbacks alongside requirements discovery and the five catalogs. Its loader receives invocation context, configuration selection, typed input, and shared state. Direct callbacks retain their own lifecycle.

example promptkit show prompt customer/summary
example promptkit show profile customer-support --format json
example promptkit show backend local --format yaml

Each command takes one resource ID and accepts --config and --overlay. Reports use the native PromptKit inspection value through the shared reporting formats; automatic and explicit text use YAML for these structured values. Native JSON field names and omission rules apply. Absent optional fields are omitted; explicit zero values and empty collections remain present. Prompt messages are captured content, not rendered requests, and the prompt hash is opaque.

Profile reports describe the selected profile before backend completion or request overrides. Backend reports describe configured backend fields. An api_key_env value names an environment variable; it is not a credential value or a claim that one is available. Neither report resolves references or performs a network probe. See the PromptKit resource inspection contracts. A missing ID is a usage error; native lookup errors and cancellation retain supported cause identity.

Resource namespace selection

An application can enable resource namespace flags with ckprompt.WithNamespaces and provide prompt, profile, and backend defaults. Omitted flags retain those defaults; an explicitly empty value clears the corresponding default. Qualified resource IDs pass through unchanged. Without the option, the application loader controls the engine view and these flags are unavailable.

Operation	Available namespace flags
`show prompt`	`--llm-prompt-namespace`
`show profile`	`--llm-profile-namespace`
`show backend`	`--llm-backend-namespace`
`requirements show`	`--llm-prompt-namespace`, `--llm-profile-namespace`
`requirements validate`, `show prepared`, `test`	All three, including `--llm-backend-namespace`

Catalog namespace filters remain separate; configuration inspection has no resource namespace selectors.

PromptKit convenience aliases

Applications that install ckprompt.Declarations(suite) provide scoped convenience spellings for the enabled PromptKit commands. Each applicable --llm-NAME flag also accepts --NAME; --llm-input-text additionally accepts --llm-input and --input for literal text. Use --llm-input-file or --input-file for a file reference. Canonical and alias spellings share the same binding, including their occurrence order and explicit-value semantics.

Applications that install ckprompt.Command(suite) receive the canonical spellings only. The aliases do not apply to application-owned commands unless the application installs them separately.

PromptKit prepared-request preview

When Suite.PrepareRequest is present, ckprompt.Command(suite) installs promptkit show prepared PROMPT_ID independently of validation and other inspection commands. It accepts the same configuration, selector, profile, session, variable, input, and execution flags as request validation:

example promptkit show prepared briefing --llm-input-text topic=roadmap --format json

The command returns PromptKit's native prepared-run report using the shared formats; automatic and explicit text use YAML. Preparation can read resources and report rendered application content. It makes no model call and does not guarantee a subsequent execution will succeed. Application callbacks and loaders retain their own effects. Native preparation errors retain supported cause identity. The PromptKit Prepare API owns the prepared value and upstream preparation behavior.

PromptKit requirements

When Suite.InspectRequirements is present, ckprompt.Command(suite) installs promptkit requirements show. It accepts one prompt ID and uses PromptKit's native selection and discovery rules:

example promptkit requirements show briefing --campaign spring
example promptkit requirements show briefing --llm-profile editorial --llm-var mode=full
example promptkit requirements show briefing --llm-session-id session-42 --llm-var note= --format yaml

The command accepts:

Flag	Values	Default	Meaning
`--config`	path string	omitted	Select the configuration input; explicit presence is retained.
`--overlay`	path string	omitted	Select one overlay; explicit presence is retained. Repeated occurrences use the last value.
Suite selectors	application-defined	per selector	The flags captured from `Suite.Selectors`.
`--llm-profile`	string	empty	Pass the profile ID to PromptKit; empty uses its native selection rule.
`--llm-session-id`	string	empty	Pass session context to PromptKit.
`--llm-var`	repeatable `KEY=VALUE`	none	Supply caller-variable context for native discovery.
`--format`	`text`, `yaml`, `json`	automatic	Select the requirements report format; automatic and text use YAML for this structured report.

The promptkit namespace identifies the operation, so there is no built-in --scope flag. An explicitly empty profile or session value retains PromptKit's native empty-string meaning. Profile selection establishes the reported profile identity rather than changing the discovered requirements. An enabled profile namespace qualifies an explicitly supplied short profile ID; it does not rewrite the prompt's stored default reference. Requirements inspection does not require the selected profile to be execution-ready. See the PromptKit v0.20.0 consumer guide for native configuration, selection, and requirements behavior. The built-in flags other than --llm-var are singleton bindings; when repeated, the last value is used. Suite selectors retain their declared binding behavior.

Each --llm-var occurrence must contain =. CommandKit splits at the first separator and preserves the exact key and remaining value. Empty keys and values, whitespace, commas, and additional = characters are retained. All occurrences are parsed; duplicate keys use the last value, while any malformed occurrence is a usage error even if a later assignment is valid.

The report contains selected identities, native completeness, and input, variable, and issue inventories. False values and empty inventories are retained. Input metadata preserves null versus an empty string. Variable entries contain name, kind, supplied, and explanation. supplied reports native exact-key presence, including an explicitly empty value, independently of kind and complete. Explanations and issue descriptions are upstream-owned human-readable summaries, not machine-readable details.

For a prompt with the template {{if .mode}}{{input "notes"}}{{end}} and no input declarations, omitting mode can produce a partial report:

prompt_id: briefing
profile_id: editorial
complete: false
inputs:
  - name: notes
    kind: conditional
    declared: false
    content_type: null
    description: null
    explanation: This reference is not known to be reached on every execution path.
variables:
  - name: mode
    kind: required
    supplied: false
    explanation: ""
issues:
  - code: missing_variable
    input_name: ""
    variable_name: mode
    description: A missing variable prevents selecting a dependency-bearing branch.

PromptKit owns the inventory, classifications, explanations, completeness, and structured issues. Successfully acquired and delivered partial reports return status 0; structured issues do not independently fail the command. Provider errors, native selection errors, cancellation, and report delivery failures use the operational path (status 1) with supported cause identity. Syntax, operand, variable-assignment, and format errors use usage status 2.

Concrete request validation

When Suite.ValidateRequest is present, ckprompt.Command(suite) also installs promptkit requirements validate. It can be installed without discovery, and discovery can remain installed without validation. The command accepts one prompt ID and passes a concrete request to PromptKit's native Engine.ValidateRequest preparation path:

example promptkit requirements validate briefing --campaign spring --llm-input-file topic=topic.txt
example promptkit requirements validate briefing --llm-profile editorial --llm-session-id session-42 --llm-var mode=full --llm-input-file notes=notes.txt

The command accepts:

Flag	Values	Default	Meaning
`--config`	path string	omitted	Select the configuration input; explicit presence is retained.
`--overlay`	path string	omitted	Select one overlay; explicit presence is retained. Repeated occurrences use the last value.
Suite selectors	application-defined	per selector	The flags captured from `Suite.Selectors`.
`--llm-profile`	string	empty	Pass the profile ID to PromptKit.
`--llm-session-id`	string	empty	Pass session context to PromptKit.
`--llm-var`	repeatable `KEY=VALUE`	none	Supply caller variables for the concrete request.
`--llm-input-text`	repeatable `NAME=TEXT`	none	Bind a literal PromptKit artifact.
`--llm-input-file`	repeatable `NAME=PATH`	none	Bind a file-backed PromptKit artifact.
`--format`	`text`, `yaml`, `json`	automatic	Select the success confirmation format; automatic and text use YAML for this structured report.

Validation also accepts the execution controls from --llm-endpoint through --llm-extra-params listed under smoke testing. The built-in flags other than --llm-var, --llm-input-text, and --llm-input-file are singleton bindings; when repeated, the last value is used. Explicit empty --config or --overlay values keep their supplied-presence information. The profile and session fields use native empty-string semantics. PromptKit owns their interpretation and selection errors.

Each --llm-var, --llm-input-text, and --llm-input-file occurrence must contain =. CommandKit splits at the first separator and preserves the exact key/name and remaining value. Empty names and values are passed to native validation. Text becomes a promptkit.Inline reference; file paths become promptkit.File references without being read, normalized, expanded, or rebased during argument binding. Every occurrence is parsed; for each input name the last text or file occurrence wins, regardless of form.

On success, the confirmation contains only the requested prompt ID and valid: true:

prompt_id: briefing
valid: true

Validation delegates concrete request preparation to PromptKit. Native preparation may resolve prompt/profile resources, load artifact and schema resources, render messages and session context, and perform applicable configuration and credential checks. It does not make a model call or retain a prepared execution snapshot. A successful report describes this preparation invocation; it does not certify provider acceptance or guarantee later execution. See PromptKit's Engine.ValidateRequest API.

Builder, engine-loading, and native validation errors use the operational path (status 1) with supported error identity and acquired diagnostics retained. Malformed syntax, missing operands, and unsupported formats use usage status 2. Successful diagnostics use the common reporting policy. Report delivery failures are recorded in Outcome.Presentation.Output, separately from the operation cause.

PromptKit catalogs

An installed ckprompt.Command(suite) exposes each catalog whose callback is present:

example promptkit list prompts
example promptkit list profiles --config settings.yml --campaign spring
example promptkit list backends --namespace team
example promptkit list namespaces --kind prompt --kind profile
example promptkit list collections

All catalogs accept --config, --overlay, applicable flags from Suite.Selectors, and the shared --format flag. Automatic output uses text for nonempty string lists and YAML for other inventories; explicit text uses the same rendering for structured inventories. Prompt, profile, and backend listings accept namespace selection:

Binding	Effect
omitted `--namespace` and `--all-namespaces`	Forward PromptKit's native default namespace filter.
`--namespace team`	Forward the named namespace filter.
`--namespace ""`	Select the unqualified namespace.
`--all-namespaces`	Forward the all-namespaces filter.
`--all-namespaces=false`	Do not select an override; a supplied namespace still applies, otherwise native default is used.

The namespace binding is a singleton and its last occurrence is used. Supplying a namespace while --all-namespaces=true is active is a usage error before provider work. Namespace inventory accepts repeatable --kind KIND; omitting --kind forwards PromptKit's native default kind selection. Repeated values are forwarded in order. Collections have no namespace or kind filter.

Each operation reports one bare sequence, retaining provider order and native identities. YAML examples follow:

Prompt entries are {id}.
Profiles, backends, and namespaces are strings.
Collection entries are {namespace, version?}, where version projects PromptKit's optional release version.
Nil and allocated-empty inventories both render as [].

- id: team/briefing

- autumn
- spring

A catalog result is an inventory, not a readiness check or execution result. The owning provider controls membership and order. Namespace filters are passed to PromptKit without local membership interpretation. See the PromptKit v0.18.0 consumer guide for native resource and selection definitions.

PromptKit smoke testing

When Suite.SmokeTest is present, ckprompt.Command(suite) installs promptkit test KIND. The special kind all runs the four concrete operations sequentially in this order: reachability, authentication, text_inference, and structured_output. Both forms use the same single-operation callback; each aggregate call changes the seed kind before invoking it. The command accepts:

example promptkit test text_inference --campaign spring --llm-prompt briefing --llm-profile spring
example promptkit test all --campaign spring --llm-prompt briefing --llm-profile spring
example promptkit test authentication --llm-backend local

Flag	Values	Default	Meaning
`--config`, `--overlay`	path strings	omitted	Select PromptKit configuration; supplied presence is retained.
Suite selectors	application-defined	per selector	Captured flags from `Suite.Selectors`.
`--llm-backend`	string	empty	Native backend ID.
`--llm-prompt`	string	empty	Native prompt ID.
`--llm-profile`	string	empty	Native profile ID.
`--llm-endpoint`	string	omitted	Override the execution endpoint.
`--llm-model`	string	omitted	Override the execution model.
`--llm-temperature`	float	omitted	Override execution temperature.
`--llm-max-tokens`	integer	omitted	Override execution maximum tokens.
`--llm-top-p`	float	omitted	Override execution top-p.
`--llm-timeout-seconds`	integer	omitted	Override the execution timeout.
`--llm-service-tier`	string	omitted	Override the execution service tier.
`--llm-reasoning-effort`	string	omitted	Override execution reasoning effort.
`--llm-api-key-env`	string	omitted	Supply an API-key environment-variable name to native configuration.
`--llm-transient-retries`	integer	omitted	Set the native inference transient retry count.
`--llm-extra-params`	JSON object	omitted	Supply provider parameters as a whole map.
`--format`	`text`, `yaml`, `json`	automatic	Select the report format; automatic and text use YAML for these structured reports.

The execution flags populate PromptKit's native execution override. The --llm-api-key-env value is a variable name, not a credential value. The extra-parameters flag accepts one JSON object, including an empty object; PromptKit owns its native replacement semantics and validation. Integer JSON numbers use signed or unsigned 64-bit integers where representable; decimal and exponent numbers use floating point.

Application selectors, loader policy, credentials, and client wiring remain application owned. The WithSmokeTesting helper accepts an optional typed builder. A nil builder forwards the seed; a supplied builder's complete native request is authoritative. The helper retains a native report when PromptKit returns both a report and an error.

A single report projects the native smoke report. It preserves native false, unknown/null, and incomplete values and does not add a readiness score. The report includes kind, supported, applicable, completed, reason, target, facts, timing, attempts, usage, and failure. Its target contains selected IDs, selection source, endpoint, effective settings, and settings-presence flags. Attempt entries retain their own facts, timing, failure, and token-usage evidence. Durations use Go duration strings; token counts preserve value, known, and complete.

Here is a native projection as rendered by the command:

kind: reachability
supported: false
applicable: null
completed: true
reason: http_diagnostics_unavailable
target:
  backend_id: local
  profile_id: spring
  prompt_id: briefing
  default_source_backend_id: ""
  selection_source: profile
  endpoint: https://example.invalid/v1
  effective_settings:
    backend_id: local
    endpoint: https://example.invalid/v1
    model: spring-model
    temperature: 0
    max_tokens: 0
    top_p: 0
    timeout_seconds: 600
    service_tier: ""
    reasoning_effort: ""
    api_key_env: ""
    extra_params:
      campaign: spring
  settings_presence:
    temperature: false
    max_tokens: false
    top_p: false
    timeout_seconds: false
facts:
  credentials_supplied: null
  endpoint_reachable: null
  credentials_accepted: null
  response_received: null
  generation_completed: null
  text_returned: null
  expected_text_matched: null
  structured_output_requested: null
  valid_json: null
  schema_matched: null
timing:
  started_at: 2026-10-17T19:05:36Z
  duration: 35.685µs
  capacity_wait: null
  request_duration: null
  retry_delay: null
  evaluation_duration: null
attempts: []
usage:
  prompt_tokens: {value: 0, known: true, complete: true}
  completion_tokens: {value: 0, known: true, complete: true}
  total_tokens: {value: 0, known: true, complete: true}
  cached_tokens: {value: 0, known: true, complete: true}
  cache_write_tokens: {value: 0, known: true, complete: true}
failure: ""

Aggregate output is one mapping with ordered tests entries. Each entry has selected kind, nullable report, and optional error text. The inner report keeps its native kind. This cancellation example shows two attempted operations with no reports; later kinds are absent because cancellation stops the sequence:

tests:
  - kind: reachability
    report: null
    error: "reachability operation failed"
  - kind: authentication
    report: null
    error: "context canceled"

A successful delivered single operation returns status 0. For all, all four callbacks must return without error for status 0. Native false, unknown, unsupported, or incomplete observations do not independently change the outcome. Operation errors use operational status 1; aggregate runs continue after individual errors, retain their reports and ordered error text, and join causes for supported errors.Is/errors.As access. Cancellation stops later operations and preserves results already acquired. Report encoding or delivery failures are recorded in Outcome.Presentation.Output, separately from operation causes. Syntax, operand, unsupported format, and conflicting namespace selection are usage status 2.

Native configuration, selection, and diagnostic meanings remain owned by PromptKit. See the v0.18.0 consumer guide, smoke API, and engine API.

Outcome status

CommandKit usage failures use status 2; operational failures use status 1. Successful operations use status 0. Application handlers may return a domain classification and status of their choice, which Execute preserves. The caller chooses how to map an outcome to process exit. A rendering or delivery failure changes a successful or usage outcome to operational status 1. A domain outcome keeps its classification and nonzero status; a zero-status domain outcome becomes 1 if presentation fails. The returned Outcome.Presentation records output and diagnostic delivery observations separately from its operation Cause.

49 KiB Raw Blame History