diff --git a/docs/roadmap/commandkit-migration-discussion.md b/docs/roadmap/commandkit-migration-discussion.md index 05720698..3e40a7f8 100644 --- a/docs/roadmap/commandkit-migration-discussion.md +++ b/docs/roadmap/commandkit-migration-discussion.md @@ -21,7 +21,8 @@ this discussion. Item 1 selects the dependency versions and standard capabilities, including PromptKit v0.23.3 in place of the master roadmap's initial v0.23.2 target. -Application-specific commands, wiring, and reporting details remain open below. +Item 2 selects the application command structure and input conventions. Loader +wiring and detailed reporting choices remain open below. The PromptKit architecture and checkpoint decisions remain in force. This discussion develops the CommandKit portion of that migration and its interactions @@ -98,8 +99,8 @@ Two application needs already identified for deliberate treatment are: ### Intended Sequencing -1. Incorporate item 1's selected dependency baseline and standard capabilities, - and resolve native application command design, bringing U10 forward. +1. Incorporate items 1–2's selected dependency baseline, standard capabilities, + and native application command design, bringing U10 forward. 2. Introduce CommandKit composition alongside native source assembly. Connect shared loaders and standard inspection while retaining application orchestration responsibilities. @@ -183,21 +184,149 @@ wiring and presentation details explicitly. ## 2. Native Command Surface And Inputs -**Discussion / decision:** Full native cutover agreed; concrete command and input -choices pending. Preserving the existing CLI is not a selection criterion. +**Discussion / decision:** Command structure and input conventions agreed. Use a +small application-owned `pipeline` family alongside the standard `config` and +`promptkit` families. These are migration targets, not implemented commands. -- How should Notarius workflows be exposed through native standard commands and - application handlers, including upstream naming and flag recommendations? -- Which application input semantics are needed, such as reference selection and - reasoning-effort inheritance or clearing, and how should native typed inputs - express them, including omitted versus explicitly supplied values? -- Which existing commands and custom parsing can disappear once standard - help/version, argument handling, and invalid-invocation behavior are adopted? +### Command Structure -Use the [CLI reference](../cli.md) to identify current workflows and write -migration guidance, not as a compatibility constraint. Coordinate native inputs -with the master roadmap's retry and worker policy decisions; their domain meaning -is separate from how the CLI expresses them. +```text +notarius pipeline list +notarius pipeline show +notarius pipeline validate +notarius pipeline run --input-file + +notarius config scopes +notarius config validate|show|sources|explain|diff ... + +notarius promptkit list ... +notarius promptkit show ... +notarius promptkit requirements show|validate ... +notarius promptkit test + +notarius version +notarius completion bash|zsh|fish +notarius help [command...] +``` + +The alternatives separated by `|` above summarize the installed operations; +they are not literal shell commands. + +| Application command | Responsibility | +| --- | --- | +| `pipeline list` | Discover configured pipelines. | +| `pipeline show` | Describe the resolved ordered steps, lanes, modules, reference slots, and configured selections. | +| `pipeline validate` | Perform offline application preflight using the intended selections and overrides. | +| `pipeline run` | Execute the selected pipeline and publish artifacts. | + +Move pipeline-specific validation out of `config validate --pipeline ...`. +Configuration validation, pipeline composition checks, and concrete PromptKit +request validation remain distinct. Pipeline preflight does not synthesize stage +requests or require provider credentials. PromptKit's concrete validation may +read inputs and schemas and check credentials without making a model call. + +**Naming clarification:** CommandKit supports both a root `run` command and a +nested `pipeline run` command. Neither is more native. The selected grouping +keeps pipeline discovery, inspection, validation, and execution together; it is +an application usability choice, not an upstream naming requirement. A shorter +root command could be selected on usability grounds without changing the native +architecture. Do not introduce a second route solely for legacy compatibility. + +### Input Conventions + +Use a required pipeline-ID operand and a required `--input-file PATH` for runs. +Use native typed declarations and supplied-value semantics, including explicit +empty, false, and zero values. Apply defaults only when the relevant value is +omitted; application validation still decides whether a supplied value is valid. + +| Concern | Selected convention | +| --- | --- | +| Configuration | Native `--config` and `--overlay`; name the application scope `app`. | +| Lane selection | Repeatable `--lane ID`; omission selects the normal pipeline lane set. Replace comma-separated `--only`. | +| Output placement | Application-owned `--output-dir PATH`. | +| Chunk-plan caching | `--chunk-cache auto\|bypass\|refresh`. | +| Checkpoint reuse | Keep `--resume` and `--recompute-step ID` with the agreed recovery policy. | +| Presentation | Native `--format`, `--quiet`, `--verbose`, `--debug`, and `--progress`. | +| Configuration-valued tuning | Supported application `--set POINTER=JSON` overrides, including worker and semantic-budget settings once their configuration design is settled. | + +Adopt native parsing rules: flags follow the complete command path and may be +interspersed with operands; `--` ends option parsing. Singleton flags use the +last successfully parsed occurrence, with every occurrence checked. Repeatable +flags preserve occurrence order and treat commas as ordinary value content. +Use filesystem and choice completion hints where applicable. + +Keep worker counts and semantic continuation budgets in configuration rather +than introducing additional dedicated CLI controls now. Their field names, +defaults, and domain meaning remain master-roadmap decisions. Item 3 will settle +shared loading and override wiring across application and standard commands. + +### Native LLM Controls + +Use ckprompt's reusable flag declarations for applicable profile, session, and +execution controls on `pipeline run`, including model, temperature, timeout, +reasoning effort, and transient retries. Run-wide overrides reach all LLM-backed +producers and validators, including continuation calls. Without an override, +normal binding/pipeline/prompt selection applies. + +| Reasoning input | Meaning | +| --- | --- | +| Omitted | Inherit the selected settings. | +| `--llm-reasoning-effort high` | Replace the inherited setting. | +| `--llm-reasoning-effort ""` | Clear the inherited setting. | + +Remove the separate `--clear-reasoning-effort` control. Native transient retries +remain distinct from application semantic continuation budgets. Individual +prompt selection and named prompt input injection belong on standard PromptKit +commands; pipeline modules construct their own requests and response contracts. +Use native `--llm-profile` and `--llm-session-id` spellings, without inventing +another LLM input parser. + +### Application Reference Inputs + +Retain repeatable `--reference SELECTOR=PATH` and `--without-reference SELECTOR`. +Keep useful pipeline-wide, lane, chunk, and stage-specific selectors, such as +`party=party.txt` and `npc-registry.extract.party=npc-party.txt`. CommandKit owns +repeatable input parsing and typed conversion; Notarius owns selector expansion, +specificity, required slots, and external/generated-reference conflict checks. +These selectors describe pipeline relationships and are not interchangeable +with PromptKit's individual-request `--llm-input-file` bindings. + +### Preparing Pipeline Inspection Content + +**Follow-up clarification:** Leverage ConfigKit for the composed, decoded +configuration, defaults, imports, overlay, inheritance, overrides, and native +provenance/redacted inspection. Feed that loaded configuration through the same +application resolution used by validation and execution to obtain selected +lanes, registered modules, validator chains, and reference relationships. +CommandKit renders a small application report of that resolved composition. + +ConfigKit does not know Notarius's module catalog or pipeline semantics. Reuse +the existing resolver rather than writing a second resolution path for `show`. +Generic effective configuration and source explanations remain available through +`config show` and `config explain`. ConfigKit's safe inspection facilities do not +automatically redact an arbitrary application report; select appropriate report +content rather than dumping the raw typed configuration. Detailed report content +and loader wiring remain with items 3–4. + +### Removals And Remaining Work + +Retire custom dispatch, argument-count handling, assembled help, version-command +parsing, standard configuration/LLM flag parsing, comma-separated lane parsing, +and the separate reasoning-clearing machinery. Replace `config effective` with +native `config show`, `config profile` with `promptkit show profile`, and bespoke +`--json` switches with `--format json`. + +Application handlers still validate meaningful input combinations and translate +typed inputs into pipeline selections. Native `--debug` supplies reporting +behavior; a distinct artifact-capture control remains to be selected in item 5. +Items 3–4 own loader behavior, exact report content, and outcome presentation. + +Evidence: [CommandKit command/input conventions](../../../commandkit/docs/cli.md#registered-command-syntax), +[reusable PromptKit flags](../../../commandkit/docs/consumers/guide.md#request-builders-and-loading), +[ConfigKit loading and inspection](../../../ConfigKit/README.md), and the current +[Notarius configuration boundaries](../internal/configuration.md). +Use the current [CLI reference](../cli.md) for migration guidance, not as a +compatibility constraint. ## 3. Application Composition, Loaders, And Preflight @@ -208,8 +337,9 @@ is separate from how the CLI expresses them. - How should native command selections and overrides reach shared loaders while retaining the selected ConfigKit source composition and main-source-relative companion behavior across execution and inspection? -- Where should pipeline-specific validation attach to standard configuration - validation, and what state should be acquired only when a command needs it? +- How should `pipeline show`, `pipeline validate`, and `pipeline run` share + application resolution alongside standard configuration validation, and what + state should be acquired only when a command needs it? Retain the selected offline preflight boundary: it checks application composition and known native resources without synthesizing a stage request or requiring