diff --git a/docs/roadmap/registration-backend.md b/docs/roadmap/registration-backend.md new file mode 100644 index 0000000..9e57f39 --- /dev/null +++ b/docs/roadmap/registration-backend.md @@ -0,0 +1,482 @@ +# Registration and execution internals + +Status: proposed internal design for the finalized consumer API; no implementation +has begun. Two public composition seams identified below need resolution before +the staged implementation plan is finalized. + +## Purpose and authority + +Provide the [registration API](registration-api.md) through one coherent internal +system. Application commands, optional integration families, and built-in core +commands should use the same declaration, installation, routing, and execution +machinery. Command-specific handlers remain different where their responsibilities +are different. + +[ADR 0011](../adr/0011-captured-declarations-and-installation.md) owns the accepted +capture, installation, namespace, and input/state decisions. +[ADR 0010](../adr/0010-optional-integration-modules.md) owns module boundaries, and +the [architecture policy](../policy/architecture.md) governs implementation. +The [module roadmap](modules.md) owns the broader extraction and release work. +This document proposes internal mechanics and identifies gaps; it neither changes +the finalized API silently nor establishes additional consumer guarantees. + +The prototype has no compatibility priority. Replace its parallel paths and +bookkeeping where they obstruct the selected design. Preserve deliberately +selected grammar, presence, outcomes, reports, and callback lifetimes. Private +names below describe responsibilities rather than required exported symbols or a +mandated file layout. This is design work, not a staged implementation plan. + +## Baseline findings + +The following findings come from the current implementation. They identify reuse +opportunities and structural risks, not measured performance regressions. + +| Evidence | Pain point | Proposed solution | +| --- | --- | --- | +| `New` in [app.go](../../app.go) inserts help/version placeholders; `installBuiltins` in [builtins.go](../../builtins.go) replaces them directly | Built-ins bypass ordinary registry installation and have separate reserved-node bookkeeping | Construct real built-in declarations and install them with caller declarations in one batch | +| `Register` in [registry.go](../../registry.go) already stages roots before committing | Useful atomic installation exists, but privileged writers can bypass it | Make this the sole registry mutation boundary for every command source | +| `RegisterConfig` in [config.go](../../config.go) checks the root namespace and `App.configRegistered` before calling `Register` | The configuration installer duplicates collision policy and carries integration state in the core | Integration factories return root-only declarations; core installation decides collisions | +| `RegisterPromptKit` in [promptkit.go](../../promptkit.go) returns early when both inspection callbacks are nil | An absence shortcut can hide construction or metadata errors | Retain errors before representing absence; installation checks errors before skipping a no-op | +| `Group`, `validateCommandDeclaration`, and `cloneCommands` in [command.go](../../command.go) recursively validate/copy child trees; registration repeats this | Nested assembly can revisit a subtree at each ancestor and again at installation | Capture immutable child handles and local metadata once; perform contextual installation checks in one traversal | +| `normalizeConfigSuite` and `normalizePromptKitSuite` repeat flag-name, short-name, duplicate, and wiring rules | Moving integrations into modules could create several competing implementations of core binding policy | Validate generated command bindings in the core; integrations validate only their own suite semantics | +| `adaptContextSelector` and `contextsForScopes` in [selectors.go](../../selectors.go) apply selectors to mixed context storage, then copy/project it again | Selector storage and shared state have tangled lifetimes; pointer contexts can alias | Capture converted values once; initialize each scope input independently; create shared state after all input checks | +| Configuration `CheckInput` calls preflight, while `runConfigShow` and `runConfigValidation` select scopes again | Metadata selection and scope setup can be repeated between validation and execution | Carry an invocation-local prepared selection from `CheckInput` into `Run` | +| `Execute` in [execute.go](../../execute.go) identifies `help` by its path and asserts its private input type | A built-in name controls a separate execution path | Let the help declaration's typed runner receive a private execution context through the common compiler | +| `bridgeTree` in [parser.go](../../parser.go) caches neutral metadata; [internal/cobra/parse.go](../../internal/cobra/parse.go) creates a session per invocation | The useful metadata/session split should survive the refactor | Share captured metadata, refresh it after registry changes, and keep mutable backend values invocation-local | +| `commandAt` traverses names again after the adapter has already resolved a command | Duplicate route lookup can become another source of dispatch disagreement | Return a private command key from the adapter and resolve the captured runner directly | + +## Proposed internal ownership + +Keep most cohesive core machinery private in the root package. A facade does not +require forwarding-only files or duplicate representations for every public type. +Use an internal package where it protects an actual dependency boundary. + +| Owner | Internal responsibility | Does not own | +| --- | --- | --- | +| Core declarations | Captured command records, child handles, local declaration errors, root placement, explicit absence | Registry occupancy, provider acquisition, Cobra objects | +| Core bindings | Binding identity, typed conversion, presence/defaults, converted-value application, binding metadata validation | Scope names, ConfigKit/PromptKit types, application object cloning | +| Core registry | Validate/stage/commit installation batches and invalidate derived metadata | Integration-specific installed flags, command factories, report rendering | +| Core execution | Invoke the selected compiled runner, share the current presentation context, normalize/present outcomes | Application startup, provider schemas, process exit | +| `internal/commandmeta` (proposed) | Narrow backend-neutral routing/help DTOs, private command keys, parse result shapes | Public contracts, typed handlers, registry policy, upstream library types | +| `internal/cobra` | Translate neutral metadata into one invocation session; route, collect raw occurrences, render help | Typed value conversion, declaration validation, family ownership, acquisitions | +| `ckconfig` private implementation | Capture scopes, compile applicable selectors, prepare selected inputs, acquire views, perform each configuration operation | Core registry state or private binding access | +| `ckprompt` private implementation | Capture definition providers, adapt engine inspection, prepare definition input/state, render definition reports | A second dispatcher/parser or configuration-suite installation | + +The neutral metadata package is justified by the existing core/adapter exchange: +today its neutral structures live inside `internal/cobra`. Move those structures +as a unit rather than introduce an additional copy of the same DTOs. Only the core +execution coordinator needs to call the concrete adapter. Declaration and binding +logic can depend on the neutral shapes without depending on adapter operations. +This is not a backend-selection interface or a new public package. + +Integrations use supported public core contracts. Do not solve extraction by +letting them import core private assembly helpers. Small local selection-flag or +report-formatting helpers are acceptable; a fourth utility module or a universal +provider framework is not justified merely by a few similar declarations. + +## One installation path for every command + +```mermaid +flowchart TD + A[Application command factories] --> D[Captured core declarations] + B[Integration family factories] --> D + C[Core help and version factories] --> D + D --> V[Shared batch validation and staging] + V --> R[Instance registry commit] + R --> M[Cached neutral routing and help metadata] + M --> S[Fresh invocation adapter session] + S --> E[Selected typed runner and common outcome delivery] +``` + +### Construction and later registration + +Recommend one private `installBatch` operation used by both `New` and +`App.Register`: + +1. `New` validates application identity and creates an unpublished empty app. +2. Core factories construct real help/version declarations, using the same typed + command compiler as application declarations. +3. `New` submits built-ins and caller declarations together to `installBatch`. +4. `App.Register` submits its supplied declarations to that same operation. + +There are no placeholder commands, direct built-in registry writes, privileged +installation switches, or separate family installers. The app exists before its +built-in callbacks are captured, but those callbacks do not execute during +construction. A failed initial batch exposes no usable app. + +Help/version names are occupied by real built-in nodes. Optional family names are +occupied only by installed real roots. The ordinary collision check consequently +implements reservation without a parallel list of known integration names. +Root-only placement is captured metadata, not a check for a particular family +name or the package declaring it. + +### Batch validation and commit + +The shared installer should: + +- Inspect retained errors and zero/absence state. Report attached errors before + skipping an absent declaration, including absence nested in a group. +- Traverse real declarations with their installation path; enforce root placement + and report contextual construction/structural errors. Read captured binding + validation results rather than invoking parsers or factories. +- Check real root names against the existing registry and other roots in the + pending batch. Sibling collisions inside a declaration belong to the same core + structural-validation rules. +- Stage additions in a local pending map. Commit only after the entire batch is + valid. Existing captured nodes need not be cloned to stage new roots. +- Invalidate derived routing/help metadata once after a successful nonempty + commit. A failed batch or an entirely absent batch leaves registry and cached + metadata intact. + +All mutation is confined to the commit portion. Do not implement atomicity through +temporary insertion followed by rollback or through copying the whole existing +registry for every call. This preserves the selected per-call registry guarantee; +it adds no transaction for callback effects, allocation failures, or concurrency. + +Preserve retained causes with ordinary wrapping/joining where needed. Do not +flatten them into strings. The implementation need not promise exhaustive error +enumeration, stable error text, or a public aggregate-error type. + +## Capture once; keep installation state separate + +### Immutable declaration records + +Recommend an opaque `Command` handle referring to a private captured record. +That record contains descriptions, child handles, an optional compiled typed +payload, retained errors, root-only placement, and explicit absence state. + +Copy caller-owned metadata slices at their ownership boundary, including binding +lists, child-handle lists, scope operations, and selector applicability. Once +captured, those records are read-only. `Group` copies its child-handle slice; +it does not recursively clone already captured child records. `WithError` returns +a new wrapper/record retaining the existing content and added cause without +mutating the original handle. `RootGroup` reuses ordinary group capture and adds +placement metadata. A zero handle remains distinguishable from explicit absence. + +This permits declaration reuse without copying entire trees at every assembly +level. It does not clone application values, callbacks, or their object graphs. +Do not write registry paths, app pointers, converted values, or invocation caches +into reusable declarations. Core built-ins capture their own app-specific context +when each app is constructed. + +Keep payload and children independent in the private model. Do not introduce a +new prohibition on runnable commands with children merely because separate leaf +and group constructors are convenient. + +### Divide validation by responsibility and lifetime + +Use one implementation of each core metadata rule: + +- Command/binding constructors capture local metadata and retain local errors. +- Group capture checks its immediate composition without recursively rerunning + every child's local binding validator. +- Installation checks contextual placement and registry/batch collisions while + reporting retained errors through one tree traversal. +- Integrations check scope identifiers, applicability references, declared + operations, adapter issues, and enabled provider wiring. They hand generated + flags and operands to core command construction for generic binding checks. + +This prevents ancestor-by-ancestor validation and copying from growing with both +tree size and nesting depth. Exact allocations remain implementation choices; +do not add a global intern table, reflection-driven compiler, or mutable +"validated" cache shared between applications. + +Core name/short-name/help-reservation rules also check the combined standard and +selector flags on generated commands. Integration code should not reproduce +those rules with handwritten reserved-name lists. Error checking for selectors +in an absent definition family is the unresolved seam described below. + +## One typed conversion and application mechanism + +### Prepared typed values + +Retain the useful separation in the prototype's `prepareInput`: convert raw +occurrences before initializing input. Refactor its prepared setter closures into +one private typed prepared-value mechanism usable by ordinary bindings and +`CaptureFlags`/`FlagValues`. + +A prepared entry needs original binding identity, supplied state, and a typed +application closure. The closure captures the converted/default value and applies +it to initialized input storage. A repeatable binding copies its core-owned slice +when applying it to another input; it does not clone arbitrary elements. + +Scalar, boolean, repeated-flag, and operand constructors remain distinct where +their parsing and cardinality differ. Their execution glue should feed the same +typed preparation/application machinery, rather than create another conversion +system for integrations. + +`CaptureFlags` adapts each original binding to an outer command binding that stores +the prepared typed entry in `FlagValues`. It does not apply selectors to a shared +template, initialize suite input, or invoke a parser a second time. +`FlagValues.ApplyTo` selects the original identities and applies their prepared +values to the supplied destination. + +Recommend a private identity token per original binding and an invocation-local +lookup from token to prepared entry. Handle copies preserve identity; lookalike +bindings get a different token. A private nonzero-sized token object avoids +relying on addresses of zero-sized Go objects. No global registry or public ID is +needed. The lookup belongs to the invocation store, not the reusable binding. + +### Required values and errors + +Separate required metadata used for help from the point where execution enforces +it. Ordinary flags enforce their requirement in ordinary input preparation. +Captured inner flags enforce it when `ApplyTo` selects applicable bindings. Do not +erase required metadata or reimplement missing-value policy in each integration. + +Apply the API's existing `ErrFlagInput`/`ErrFlagWiring` categories through one core +classification helper. The command's input-validation path should recognize +wrapped wiring failures as operational; ordinary validation and missing input use +usage outcomes. An integration may wrap those causes with scope/family context +without parsing error messages or inventing another outcome protocol. + +Help still converts supplied occurrences and rejects malformed values through the +same preparation logic, but does not materialize command/suite input or enforce +execution-required fields. Presence, singleton repetition, and repeated-value +semantics remain those of the finalized API and selected grammar. + +## One compiled execution path, including help + +Recommend that the private typed command compiler produce a prepared invocation +closure. After raw conversion, this closure materializes `I`, applies values, +runs `CheckInput`, and invokes the typed handler. Keep `I` inside that closure +rather than passing an `any` input through separate factory/check/run calls and +repeated type assertions. A heterogeneous registry can store the resulting +non-generic private runner interface/function. + +The private runner receives the current execution context, containing streams, +cancellation, the app, and access to the current help/presentation session. For an +application or integration handler, the compiler passes only the public +`Invocation` and typed input. Built-in help uses a narrow private typed runner +adapter to render targets through that same current execution context. + +That private adapter shares declaration capture, metadata validation, binding +preparation, registry installation, and outcome presentation with every other +command. It changes only the final callback's access to core services. Version +can use ordinary typed handler wiring with captured identity. Do not add a +second built-in compiler or a dispatcher branch keyed on `path == "help"`. + +The adapter's local `--help` mechanism is still common syntax machinery, not a +separately registered application command. Bare root/group help and explicit +help commands use the same presentation service. Cobra must not add hidden +command families independently of captured registrations. + +### Metadata and session lifetime + +Preserve the existing cache/session split: + +- Registry changes invalidate one derived neutral routing/help snapshot. Sorting + and backend metadata projection occur when that snapshot is rebuilt, not on + every parse or help operation. +- A fresh mutable adapter session is built for each execution and reused for + its parsing, help, and usage presentation. Mutable Cobra values are not cached + across invocations. +- Give each snapshot a private key-to-runner table. The adapter returns the + selected neutral command key along with its path/raw values. Core execution + uses the table associated with that session rather than resolving the same + path again through `commandAt`. +- Command keys are meaningful only within their snapshot. Do not expose them, + depend on persistent numbering, or use a newly rebuilt table to interpret a + result from an older session. +- If a callback registers additional commands and subsequent presentation needs + current metadata, refresh the presentation session on demand. Do not reparse + argv or rerun factories, validators, or handlers merely to refresh help. + +This removes duplicated route resolution and preserves useful existing behavior +without promising concurrent dispatch, registration during another goroutine's +execution, or a general snapshot-isolation contract. + +## Integration preparation without parallel execution frameworks + +### Configuration suite + +At family construction, capture a private prepared suite containing ordered scope +records, lookup by scope name, supported operations, discovery metadata, and the +applicable original selector bindings for each scope. Those lists are a natural +output of composing scopes and selectors; no separate mutable membership cache +or public capability framework is needed. + +Each inspection command uses an invocation-local outer input containing standard +selectors, `FlagValues[I]`, and its prepared operation selection. Recommend a +pointer-shaped private outer input initialized by the command's own `NewInput`. +Its `CheckInput` can populate the prepared selection and `Run` receives that same +object. This matters because the public `CheckInput` receives its generic input +by value: assigning a plan to a field in a copied value struct would lose it. +This outer allocation is distinct from the suite's input and shared-state +factories and is not performed on help paths. + +The shared configuration preparation routine should: + +1. Resolve scopes, operation availability, formats, and applicable operand rules + from metadata before suite factories or acquisition. +2. For each selected scope, obtain a fresh `I` through `NewInput` or its zero + value, apply only applicable captured bindings, and call the scope-aware + `CheckInput` with scope identity and selection. +3. Retain the successfully prepared scope/input records in selected order on the + outer invocation input. On failure, return the supported error and stop; + later inputs need not be initialized. +4. In `Run`, after input validation succeeds and execution reaches acquisition, + obtain shared `S` once through `NewState` or its zero value. Pass that state + and each prepared input explicitly to the selected provider. + +Do not recompute selection in every operation's `Run`, put the plan in a shared +declaration closure, project a shallow copy of one input pointer, or clear +non-applicable fields after initialization. Non-applicable bindings should leave +the scope input's own initialized fields untouched. + +Share this preparation across validate/show/sources/explain. Keep acquisition and +report loops operation-owned: `show` acquires all selected views before report +output, while other operations retain their selected timing. A universal +"acquire everything then render" wrapper would alter behavior unnecessarily. +Discovery reads captured metadata without initializing suite input or state. + +`AppCommand` should build the ordinary scope and suite internally and call the +same family factory. It is convenience assembly, not a second config installer, +handler set, or execution cache. + +### PromptKit suite and scope adapter + +Definition commands use the same core captured bindings, typed runner, and outcome +delivery. A small private definition-input helper applies the entire selected +flag set, validates the typed input, and retains it for the runner. It needs no +configuration-scope projection or inherited mixed-context selector model. + +Within `ckprompt`, share engine loading/inspection and diagnostic adaptation +between scope and named-definition adapters where their upstream contract is the +same. Inspect the original loader before wrapping it. Provider lookup failures +still retain diagnostics and error identity. Keep report models and native +configuration views with their proper owners. + +`ConfigScope` constructs a public `ckconfig.Scope` and delegates configuration +command generation to `ckconfig`. It must not construct another config family or +reach into `ckconfig`'s private prepared suite. Its construction-error handoff is +the second unresolved seam below. + +### Share mechanics where ownership supports them + +Core owns outcome classification and diagnostic delivery once; integration +handlers return public outcomes and diagnostic envelopes. Within an integration, +share mundane writing/encoding and provider-error helpers where appropriate, +while preserving each report's actual timing and incomplete-delivery semantics. + +Some small helpers may remain similar between independent modules. Do not expose +generic encoders, upstream severity converters, standard-flag getters, or a +generic pipeline merely to eliminate a few lines. Important duplication to remove +is competing registration, validation, conversion, acquisition-lifetime, and +presentation policy—not every structurally similar field assignment. + +## Efficiency and implementation review evidence + +The intended cost profile is straightforward: + +- Capture local metadata once and share read-only subtrees; avoid recursively + copying/revalidating a child at every ancestor. +- Stage only added roots, with one contextual tree walk and ordinary name + lookups; avoid cloning the existing registry or installing commands one by one + inside a supposedly atomic batch. +- Derive routing/help metadata once per changed registry state, and use the + adapter's selected command key rather than repeating route traversal. +- Convert each supplied occurrence once per invocation. Apply prepared values + only to selected inputs, copying only core-owned repeated collections. +- Prepare scope selection once per invocation and shared state once when + acquisition is reached. No process-global provider cache is introduced. + +These are algorithmic recommendations, not latency/allocation guarantees. Existing +benchmarks can inform the implementation sprint; this audit has not run them or +measured performance. Profile before adding pools, segmented trees, bitsets, +concurrent registries, or mutable backend reuse. + +Focused implementation evidence should demonstrate the common path with all +three command sources: built-ins, application declarations, and integrations. +Include invalid bindings, conflicting roots in either order, an invalid final +member of a batch, nested root-only declarations, absence with retained errors, +source metadata edits after capture, and declaration reuse across apps. + +For execution, check wrapped wiring versus input failures, fresh repeatable +collections, pointer-shaped scope inputs with shared state, scope-aware validation, +help without factories/providers, and metadata refresh without callback replay. +Evidence for the internal path itself can inspect the shared installer/compiler; +do not turn a private function name or exact allocation count into a public +contract. These are review targets for the future implementation, not a request +to add or run tests during this planning pass. + +## Open Questions + +The main internal design can use the existing finalized API. The following two +seams need an explicit API clarification or small approved extension. Do not +resolve them with private-module imports, dummy providers, reflection, or hidden +global state. + +### OQ-1: Retaining core binding errors when a family is absent + +The API says suite validation precedes absence and invalid wiring must not be +swallowed. It does not explicitly define the validation boundary for selector +bindings when both definition capabilities are absent. If those bindings must +still be validated, `ckprompt` can retain its own adapter errors, but `Flag` +validation state is private to the core. An integration can construct a temporary +command containing its selector bindings, yet the finalized API offers no way to +transfer that +command's retained errors to `NoCommand()`. Discarding it loses evidence; +installing it adds a command that should be absent. Validating through a temporary +`App` would work mechanically but construct built-ins, routing ownership, and a +registry merely to inspect metadata. + +**Recommendation:** approve a narrow extension of the existing absence factory, +schematically `NoCommand(validation ...Command) Command`. A no-op can retain +local construction/structural errors from eagerly captured validation declarations +without installing their commands or claiming their namespaces. The empty +definition factory builds a metadata-only validation declaration using the same +generated flag assembly, then returns absence carrying that evidence. + +This reuses the common declaration validator and keeps one installation-time +error boundary. Validation sources are captured data, not preparation callbacks; +they do not acquire providers or introduce registry collision checks for roots +that will not be installed. Specify the evidence-only behavior explicitly before +changing the API. Plain `NoCommand()` retains its current meaning. + +**Alternative:** add a core declaration-validation function returning `error` for +local metadata/structure, and attach its result through the existing `WithError`. +That also avoids a temporary app and duplicated flag policy. It adds a public +validation entry point and a second observation point for errors; ensure it uses +the same validator rather than evolving independent rules. It is viable if +explicit validation is useful beyond absent families. + +**Alternative requiring a contract clarification:** validate only the bindings of +actual generated commands. An absent family has no installed bindings, so unused +selectors would not undergo core validation; retained adapter errors still fail. +This needs no new API and is proportionate to installed behavior, but limits the +meaning of suite validation before absence to integration metadata and retained +adapter issues rather than all unused bindings. Choose it deliberately, +rather than silently making empty capabilities swallow invalid declarations. + +### OQ-2: Retaining errors in a scope constructed by another integration + +The selected `Scope[I, S]` includes private adapter-construction issues owned by +`ckconfig`. `ckprompt.ConfigScope` returns that scope type, but cannot attach its +original nil-loader error through supported public contracts. An `Acquire == nil` +check alone is insufficient: narrowing `Operations` to empty makes a scope +metadata-only, while the original invalid loader must remain an error. Wrapping +the nil loader in a non-nil acquisition closure would incorrectly defer it to +invocation. A core `WithError` only accepts a `Command`, not a scope. + +**Recommendation:** approve an integration-owned operation, schematically +`ckconfig.WithScopeError[I, S](scope Scope[I, S], err error) Scope[I, S]`. +It returns a scope retaining the original issue, preserving supported error +identity; a nil error preserves the scope. The family factory collects those +issues into its command declaration before capability selection or absence can +discard them. `ConfigScope` uses this for its original loader validation and +otherwise fills the ordinary public scope fields. + +This mirrors the accepted core retained-error model at the boundary that needs +it. It does not add scope installation, expose private representations, or move +PromptKit types into the core. Scope capture still occurs at the existing family +factory; this operation is evidence attachment rather than a new lifecycle. + +**Alternative:** give the public scope declaration an explicit construction-error +field, consumed by the family factory alongside provider metadata. This is simple +and permits application-created adapters to supply errors directly. It expands +mutable public representation and requires defining preservation/combination of +multiple errors; the opaque attachment operation keeps those details private. + +Neither seam requires a general plugin system. Confirm their resolutions before +turning this design and the module roadmap into ordered implementation stages.