Record the documentation consistency audit

docs/roadmap/audit/13-documentation.md

@@ -0,0 +1,258 @@
+# Documentation consistency audit
+
+Status: five open P3 documentation findings, plus cross-references to previously
+recorded implementation discrepancies. Reviewed source:
+`efb5f774176526bc6de74eca5f7472d078ea1704`.
+
+## Architecture and scope
+
+The [internal overview](../../internal/overview.md) describes the implemented
+owners. CommandKit consists of three independently versioned Go modules:
+
+- Core owns captured command declarations, typed bindings and presence,
+  installation, alias routing, dispatch, completion, help/version, shared
+  text/YAML/JSON rendering, and invocation reporting and outcomes. Private
+  command metadata, path and alias packages connect the facade to an internal
+  Cobra/pflag adapter.
+- ckconfig depends on core and ConfigKit v1.1.0. It assembles six standard
+  configuration commands from declared scope capabilities: scopes, validate,
+  show, sources, explain, and diff. Loaders supply configuration and evidence;
+  the integration selects report content and core renders it.
+- ckprompt depends on core, ckconfig, ConfigKit v1.1.0, and PromptKit v0.20.0.
+  It provides optional catalogs, resource inspection, requirements discovery
+  and validation, prepared-request preview, smoke testing, reusable request
+  flags, and a configuration scope. Applications explicitly install these
+  capabilities and retain engine/client construction and executable lifecycle.
+
+Core has no ConfigKit or PromptKit dependency. Upstream libraries own domain
+semantics; applications own contexts, streams, runtime effects, credentials,
+and final exit mapping. The release launchers delegate source-only module
+publication and optional hosted-page delivery to ReleaseKit.
+
+The review covered the 47 tracked Markdown files, public package/API comments,
+maintained examples and fixtures, module metadata, release profiles and
+launchers, and relevant implementation and tests. The
+[documentation policy](../../policy/documentation.md), architecture, testing,
+release, and applicable attribution rules supplied the review boundaries.
+Codebase-memory-mcp discovery and source inspection were supplemented with
+direct reads of documentation, examples, and tooling.
+
+The current shared reporting and PromptKit v0.20.0 features are implemented.
+The ckprompt v0.3.0 candidate remains unpublished according to the current
+release records; implementation and publication are different states.
+Older release notes were evaluated as historical records against their local
+tags, not as current CLI instructions. Retired roadmaps, completed remediation
+descriptions, and explicitly deferred features were not treated as promises of
+current functionality.
+
+## Validation evidence
+
+Validation used the cached Go 1.27.0 toolchain with `GOTOOLCHAIN=local`,
+`GOPROXY=off`, `-mod=readonly`, and a temporary writable build cache. The initial
+attempt to use the default cache encountered the sandbox's read-only filesystem;
+the temporary cache resolved that environment limitation.
+
+- All default suites passed in the aligned workspace:
+  `go test -mod=readonly ./... ./integrations/ckconfig/... ./integrations/ckprompt/...`.
+  Each module's independent `GOWORK=off go test -mod=readonly ./...` also passed.
+- All 45 `go run` examples in the three example READMEs and all 20 developer-guide
+  example invocations were exercised using binaries built from the aligned
+  workspace. Their selected arguments and working directories were preserved.
+  The documented invalid-input examples failed as intended, and the differing
+  `config diff --check` example returned status 3. Representative reports and
+  help were compared with the CLI reference.
+- Temporary external-consumer probes substantiated D1 and D2 below. No
+  production files, existing documentation, or repository tests were changed.
+- The local-link pass checked 334 Markdown link occurrences. The only apparent
+  missing path was the intentional `COMMIT_URL` attribution-format placeholder
+  in the OSS policy. Repository-hosted links were also checked against locally
+  available Git objects: their file paths existed; unpublished ckprompt footer
+  targets and the release URL template were excluded. Checking 11 referenced
+  section fragments in those objects found D4.
+- Independent module graphs agreed with the versions listed in their attribution
+  entries. The declared Go requirements, release profiles, launchers, and
+  release-note dependency descriptions were checked against local metadata and
+  tags. No publication, delivery, credentialed network operation, or live model
+  call was attempted. External upstream URL availability was not tested.
+- Default tests skip interactive shell completion. The consumer fixture's
+  script-generation step was run separately and reproduced the already recorded
+  reserved-flag failure C6; the full interactive shell matrix was not run.
+
+Passing default suites do not establish that the previously recorded exceptional
+paths below are correct. They also do not validate every possible consumer
+provider or upstream configuration schema.
+
+## D1 — P3: Configuration provider GoDoc omits the effective-output contract
+
+**Documentation:** [contracts.go](../../../integrations/ckconfig/contracts.go),
+EffectiveWriter at lines 66–69 and View at 82–87. EffectiveWriter promises a
+native effective representation without specifying its encoding. View still
+describes exposing facets without reconstructing their reports.
+
+**Implementation:** [show.go](../../../integrations/ckconfig/show.go),
+runConfigShow at lines 68–101, buffers WriteEffective, passes its bytes to
+[DecodeYAMLReport](../../../report_value.go), and submits the decoded value to
+the shared reporter. The
+[CLI reference](../../cli.md#show-effective-configuration) already describes
+conversion to a logical report value.
+
+**Discrepancy and consequence:** a custom scope implementer reading the owning
+GoDoc is not told that WriteEffective must supply one supported YAML document,
+or that its formatting is not passed through. Valid YAML is not sufficient in
+every case: the converter rejects multiple documents, unsupported tags, and
+non-string mapping keys. This is an undocumented provider requirement, rather
+than evidence that all native YAML must be accepted.
+
+**Observed behavior:** a temporary custom EffectiveWriter emitting
+`answer: 42` produced a JSON mapping under `config show --format=json` with
+status 0. A two-document YAML stream and a mapping containing an explicitly
+tagged `!!timestamp` both returned status 1, empty result output, and a rendering
+error identifying the unsupported input. No acquisition error was needed.
+
+**Correct information and potential remediation:** describe WriteEffective as
+supplying safe effective YAML for conversion through DecodeYAMLReport. Link that
+helper as the common owner of the conversion contract, and make its supported
+value boundary sufficiently explicit there. Update View's comment to describe
+evidence used to build reports, rather than imply native report passthrough.
+Do not promise preservation of comments, styles, or original framing, or broaden
+YAML support merely to accommodate the old wording.
+
+The related unused WriteExplanation requirement is already tracked as
+[M6](03-complexity.md#m6--p3-explanation-providers-must-implement-an-unused-rendering-capability).
+Clarifying the current read-versus-render boundary does not require resolving
+that API simplification in this documentation change.
+
+## D2 — P3: Public flag metadata omits enforced declaration constraints
+
+**Documentation:** [command.go](../../../command.go), FlagMeta.Name and Short
+at lines 91–103, describe a name without leading dashes and an optional one-byte
+short spelling. The [registered CLI syntax](../../cli.md#registered-command-syntax)
+describes invocation grammar, not the additional declaration grammar.
+
+**Implementation:** validateBindings and validFlagName in the same file,
+lines 477–545. Long names must be nonempty segments of Unicode lowercase
+letters or digits separated by single hyphens. Leading, trailing, and repeated
+hyphens are rejected. Core also rejects reserved help/reporting names and `h`
+as a short spelling; `-` is not a valid short spelling. Declaration errors are
+retained until installation.
+
+**Discrepancy and consequence:** the exact exported metadata contract leaves
+consumers to discover ordinary naming restrictions through installation errors
+or private implementation. Names such as `foo_bar` and `fooBar` satisfy the
+published field description but cannot be installed. RenameFlag and flag-alias
+declarations also depend on the long-name grammar.
+
+**Observed behavior:** temporary public-API constructions accepted `foo-bar`,
+`123`, and `équipe`, but rejected `foo_bar`, `fooBar`, and `foo--bar` during New
+with the lowercase/hyphen-separated-name error.
+
+**Correct information and potential remediation:** document the enforced naming
+and reserved-spelling rules once with FlagMeta, and refer related public helpers
+to that contract. Keep the CLI reference focused on operator syntax. Do not
+accidentally introduce an ASCII-only or leading-letter restriction: neither is
+enforced by the implementation. If the intended naming policy differs, surface
+that as a deliberate API decision rather than silently changing validation.
+
+## D3 — P3: Reporting ADR still directs readers to a planned implementation
+
+**Documentation:** [ADR 0006](../../adr/0006-invocation-reporting.md), lines 5–7
+say shared formatting is implemented; line 54 leaves implementation status and
+format choices in the now-retired roadmap; lines 76–77 still describe the
+extension as planned until implementation and canonical contracts land.
+
+**Implementation and current contracts:** [report.go](../../../report.go)
+exports Report, Format, and RenderReport;
+[reporter.go](../../../reporter.go) integrates Report with invocation delivery;
+the [CLI reporting contract](../../cli.md#reporting) and
+[consumer guidance](../../consumers/guide.md#invocation-reporting) describe the
+implemented formats. The default suites exercise these APIs.
+
+**Discrepancy:** the ADR mixes current implementation status with pre-implementation
+wording and points to a historical roadmap for information now owned by current
+API and CLI documentation. This can make a completed prerequisite look pending.
+
+**Correct information and potential remediation:** keep the accepted ownership
+decision intact. A status clarification can explicitly identify the remaining
+future-tense passages as historical and direct current behavior to GoDoc and
+the CLI reference. If editorial cleanup of those passages is approved, remove
+only stale lifecycle wording; the documentation policy's accepted-decision
+immutability rule does not justify rewriting the architectural decision or
+creating a new ADR for an unchanged decision.
+
+## D4 — P3: A historical release note links to a nonexistent section
+
+**Documentation:** [ckprompt v0.2.0 release note](../../../integrations/ckprompt/docs/releases/v0.2.0.md),
+line 32, links to the consumer guide at the immutable
+`integrations/ckprompt/v0.2.0` tag with fragment
+`#promptkit-configuration-and-definitions`.
+
+**Evidence:** `git show integrations/ckprompt/v0.2.0:docs/consumers/guide.md`
+contains the heading `PromptKit configuration, definitions, and requirements`
+at line 244. The correct fragment is
+`#promptkit-configuration-definitions-and-requirements`. The linked document
+exists, but the requested section does not. The tagged modules' dependency
+versions agree with the note; this is a navigation defect, not a claim that
+historical dependencies should match current source.
+
+**Correct information and potential remediation:** provide the
+[correct pinned section link](https://forge.maximumdirect.net/base-layer/commandkit/src/tag/integrations/ckprompt/v0.2.0/docs/consumers/guide.md#promptkit-configuration-definitions-and-requirements)
+in a forward-only documentation correction or erratum where useful. Preserve
+the published tag, note, and hosted release contents. This defect does not
+warrant republishing the old version or backfilling hosted pages as part of the
+audit. Future note preparation should check fragments against the target
+revision as well as checking that the linked file exists.
+
+## D5 — P3: The root README duplicates the compatibility matrix
+
+**Documentation:** [README.md](../../../README.md), lines 25–30, repeats the
+published core/integration pairings, the incompatible ckconfig v0.3.0 and
+ckprompt v0.2.0 combination, and the current candidate dependency selection.
+The [consumer guide](../../consumers/guide.md#module-selection), lines 18–29,
+maintains those same details alongside installation guidance.
+
+**Implementation and policy:** the module requirements in
+[core](../../../go.mod), [ckconfig](../../../integrations/ckconfig/go.mod), and
+[ckprompt](../../../integrations/ckprompt/go.mod), plus locally available
+release tags, corroborate the current descriptions. However, the
+[one-owner rule](../../policy/documentation.md#one-canonical-owner) permits only
+a short stable summary in a non-owning document and requires links for volatile
+contract details. README owns orientation and current status, not a second
+historical module compatibility reference.
+
+**Discrepancy:** the facts currently agree; the finding is duplicated ownership
+under the adopted policy. The README's existing link does not remove the need
+to update its separate multi-version matrix whenever compatibility guidance
+changes.
+
+**Potential remediation:** retain a brief current-status statement and the module
+overview in README, and make the consumer guide the single maintained source of
+exact version combinations and installation choices. This requires no dependency
+change and no invented compatibility guarantee. Historical release notes remain
+historical records rather than duplicate current reference documents.
+
+## Previously recorded discrepancies affecting documented behavior
+
+These remain relevant to this audit but are not additional documentation
+findings. Their detailed triggers and remediation belong in the earlier audits.
+Current implementation paths were inspected; the completion-fixture failure was
+also reproduced here. Do not weaken the selected documentation contracts merely
+to make them describe these defects.
+
+| Current documentation or promised behavior | Existing implementation finding |
+| --- | --- |
+| Reusable parsed inputs and per-kind smoke seeds preserve the operator's selected values. | [C1](01-correctness.md#c1--p2-cli-provider-parameter-objects-leak-mutations-between-requests): mutation of parsed provider-parameter maps can cross replay/aggregate boundaries. |
+| Concurrent reporter submissions and deferred delivery follow the documented invocation lifecycle. | [C2](01-correctness.md#c2--p2-finalization-can-discard-a-successfully-queued-deferred-message) and [N2](04-concurrency.md#n2--p2-deferred-message-acceptance-remains-open-after-the-final-batch-detaches): a deferred submission can return success during finalization and be discarded. |
+| Prepared-request preview returns the prepared request; aggregate smoke success covers all requested kinds. | [C3](01-correctness.md#c3--p2-prepared-request-inspection-reports-success-without-a-result) permits null successful preview output; [N1](04-concurrency.md#n1--p2-cancellation-can-stop-aggregate-smoke-testing-with-a-success-outcome) can return success after cancellation stops an aggregate. |
+| Operation cancellation and presentation failures remain separately observable. | [C4](01-correctness.md#c4--p2-show-preparation-labels-ordinary-cancellation-as-a-rendering-error) and [N3](04-concurrency.md#n3--p2-buffered-configuration-show-cancellation-becomes-a-presentation-failure): a buffered-show cancellation is also recorded as a rendering failure. |
+| The developer guide's optional completion consumer check can reach shell validation. | [C6](01-correctness.md#c6--p3-the-interactive-completion-consumer-cannot-construct-its-app): the fixture declares the now-reserved `format` flag. Direct script generation still fails during App construction, before any interactive shell check. |
+
+## Assessment
+
+The principal current CLI descriptions, maintained README commands, module
+requirements, report defaults, and release-launcher instructions are consistent
+with the reviewed implementation on the exercised paths. The new findings are
+documentation clarifications and maintenance issues, not reasons to redesign
+the command suite or extend upstream functionality. Prioritize the public
+provider and naming contracts, then reconcile status/navigation and duplicated
+compatibility guidance. The previously recorded runtime defects remain separate
+implementation work.