From 8078c6ecc7b8d0d114a4f9f8078542970aa1b043 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Thu, 19 Nov 2026 18:16:28 +0000 Subject: [PATCH] Align Narratio requirements with native CommandKit reports --- docs/roadmap/commandkit-notepad.md | 54 ++--- docs/roadmap/narratio-feature-request.md | 251 ++++++++++++----------- 2 files changed, 153 insertions(+), 152 deletions(-) diff --git a/docs/roadmap/commandkit-notepad.md b/docs/roadmap/commandkit-notepad.md index 1d89ddc..bd1abca 100644 --- a/docs/roadmap/commandkit-notepad.md +++ b/docs/roadmap/commandkit-notepad.md @@ -208,6 +208,32 @@ qualified IDs and explicit namespace flags when selecting namespaced resources, while retaining upstream behavior for unqualified IDs. Configurable defaults can be considered later if a need arises. +### Native inspection reports and consumer requirements + +Decision recorded 2026-11-19: adopt native inspection reports and the native +`promptkit requirements show` interface for Narratio's early dependency +discovery. Its consumer needs and acceptance scenarios are recorded in the +[aligned Narratio proposal](narratio-feature-request.md). This direction is +accepted; Scriptorium integration remains to be implemented. + +Consume native report fields without a Scriptorium-specific schema-version +envelope. Use native nonzero outcomes and diagnostics for invocation failures; +JSON report selection does not require JSON errors. Neither a separately +versioned envelope nor programmatic invocation-failure categories blocks this +migration. A future concrete need for either belongs in an upstream discussion. + +PromptKit owns requirement classification, discovery completeness, and structured +discovery issues. Completeness is distinct from satisfied bindings or execution +readiness; Narratio owns comparison against its planned inputs. Keep early +discovery separate from concrete request validation. Use current exact resource +identities and native input/variable metadata, without restoring obsolete prompt +versions or inventing input schemas or variable-default semantics. + +Standard prompt inspection includes captured messages; prepared preview includes +rendered content. Adopt these native reports without rebuilding Scriptorium's +previous summary projections, and explain the changed presentation in migration +guidance. Requirements discovery retains its distinct upstream content boundary. + ## Upstream baseline The initial review covered CommandKit **v0.7.0**, ckconfig **v0.9.0**, and @@ -252,33 +278,7 @@ responsibilities. Decide their mapping to the CommandKit invocation context and PromptKit operation timeout, separately from the standard per-attempt timeout flag. -### 2 Native reports and executable consumer needs - -Adopt native inspection reports as the starting point. Standard prompt -inspection includes captured prompt messages, unlike the current Scriptorium -summary. Document that change and confirm how native inspection and preparation -reports meet operator needs without rebuilding custom projections. - -Reconcile the proposed [Narratio requirements request](narratio-feature-request.md) -with native `promptkit requirements show`. Its central need remains valid: -discover input requirements before expensive upstream work and before input -files exist. Preserve the distinction between successful discovery and complete -discovery; a successful partial report does not establish all requirements. - -The proposal's legacy syntax and prompt-version assumptions need revision. Its -illustrative versioned JSON envelope and structured-error request also need a -decision: native reports do not add that Scriptorium envelope, and `--format -json` does not automatically turn diagnostics into a JSON error protocol. -Determine which consumer requirements are essential and whether any genuine -gap belongs upstream. Avoid treating the existing proposal as a mandate for a -new compatibility facade. - -Keep discovery distinct from concrete request validation: validation prepares a -real request and may read input artifacts and check relevant credentials; -discovery must work before those artifacts exist. Neither guarantees a future -generation will succeed. - -### 3 Application commands and standard request flags +### 2 Application commands and standard request flags Decide the final generation command operands and application-specific flags around the reusable ckprompt request flag groups. Use their standard names, diff --git a/docs/roadmap/narratio-feature-request.md b/docs/roadmap/narratio-feature-request.md index 36cd342..fffd847 100644 --- a/docs/roadmap/narratio-feature-request.md +++ b/docs/roadmap/narratio-feature-request.md @@ -1,8 +1,9 @@ -# Feature request: inspect prompt input requirements +# Prompt input requirements for Narratio -Status: proposed. This document requests an external Scriptorium API for -Narratio and other executable consumers. It does not describe implemented -behavior or prescribe implementation structure. +Status: accepted direction, pending implementation through the CommandKit +migration. This proposal records Narratio's consumer needs and their mapping +to the native interface; it does not describe shipped Scriptorium behavior. +The [CommandKit notepad](commandkit-notepad.md) records the migration decisions. ## Consumer need @@ -13,153 +14,153 @@ identities: for example, the source `narratio.transcript.final_trimmed_markdown` might be supplied under the prompt input name `transcript`. -Narratio needs to discover the selected prompt's expected input names before -running expensive upstream stages. It can then show operators how prompt inputs -map to pipeline products and report missing bindings before transcription or -generation begins. At that point, the eventual input files often do not exist. +Narratio needs to discover expected input names before running expensive +upstream stages and before the eventual input files exist. It can then show +operators how prompt inputs map to pipeline products and identify missing +bindings without reading prompt definitions or reproducing PromptKit's analysis. -The requested API must expose requirements without requiring callers to read -prompt definitions or reproduce Scriptorium's interpretation of them. +## Selected native operation -## Requested operation +Expose `scriptorium promptkit requirements show PROMPT_ID` through ckprompt. +Subprocess consumers select JSON with the standard `--format json` flag. +Default and explicit text output use the native YAML presentation. +The [upstream CLI contract][requirements-cli] owns syntax and report details. -Provide a read-only CLI operation that reports the effective input requirements -for a selected prompt and execution context. A proposed invocation is: +Use the same application configuration and framework source selection as +execution, including `--config`, `--overlay`, and applicable application +selectors. The native `--llm-profile`, `--llm-var`, `--llm-session-id`, and +prompt/profile namespace flags supply the relevant discovery context. +Preserve upstream selection semantics, including the prompt's default profile +when an explicit profile is omitted. Qualified IDs identify installed resources; +there is no separate prompt-version selector or reported prompt-version field. -```text -scriptorium inspect requirements --prompt ID [--prompt-version VERSION] - [--profile ID] [--var name=value ...] - [--config PATH] [--config-overlay NAME] - [--promptkit-source PATH ...] [--schema-dir DIR] - [--format text|json] -``` +The selected profile establishes identity, not execution readiness. Selecting +another profile does not itself change the prompt's discovered requirements. +Scriptorium's loading and request mapping must preserve the same effective +selection and applicable caller context across discovery and execution. -Command naming is open for agreement. The required capability is distinct from -the currently documented [prompt definition inspection](../cli.md#scriptorium-inspect-prompt): -the result describes the caller-facing requirements of the selected execution -context. Extending an existing public operation is acceptable if this contract -remains explicit and machine-readable. +Discovery requires no input bindings or output-artifact path. PromptKit performs +no artifact loading, schema loading, credential resolution, message rendering, +capacity reservation, or model call for this operation. Reading configuration +and prompt resources needed to build the captured engine is expected. The +[PromptKit discovery contract][requirements-api] owns the analysis and its +limits. -Selection must follow the same public configuration, source, prompt-version, -profile, and variable semantics as an equivalent execution request. Report the -resolved prompt ID/version and effective profile identity, including defaults -when the caller omits a selector. Unknown or ambiguous selections must produce -an actionable failure consistent with execution selection. +## Native response and consumer interpretation -Input paths, input contents, and an output-artifact path are not required. -Inspection must work before upstream products are created. It must not call a -model, require generation credentials, reserve generation capacity, or create -execution artifacts. Reading configuration and prompt resources needed for -inspection is expected. +Consume the native JSON report directly, without a Scriptorium-specific wrapper +or schema-version field. Successful report delivery writes one JSON document to +stdout, with diagnostics on stderr under CommandKit's standard reporting rules. +Consumers requesting a report should not enable quiet mode. -## Response contract +| Field | Consumer meaning | +| --- | --- | +| `prompt_id`, `profile_id` | Exact selected resource identities. | +| `inputs` | Declared and recognizable template-referenced inputs, including exact `name`, requirement `kind`, `declared`, `content_type`, `description`, and `explanation`. | +| `variables` | Recognizable caller-variable keys with `name`, requirement `kind`, `supplied`, and `explanation`. | +| `complete` | Whether the supported dependency analysis is complete for the supplied context. | +| `issues` | Structured discovery limitations with a `code`, associated input or variable names where known, and a human-readable description. | -Offer a readable text view for operators and a versioned JSON response for -subprocess consumers. JSON stdout must contain one response document; incidental -diagnostics belong on stderr. Stable ordering should make repeated inspection -of unchanged effective inputs easy to compare. +PromptKit sorts input and variable entries by exact name, and deduplicates and +orders issues by their code and associated names. Preserve empty inventories +and false values. Input metadata distinguishes absent/null values from an +explicitly empty string. Explanations and issue descriptions are for display; +Narratio must use structured fields rather than parse their prose. -The response should expose: +The native input metadata includes content type and description. Current +PromptKit inputs have no per-input schema declaration to project. Do not infer +constraints from names or treat the output-validation schema as an input schema. +Variable `supplied` records exact-key presence, including an explicitly empty +value; this interface does not introduce a separate variable-default system. -- Resolved prompt and profile identities. -- Every discoverable caller-supplied input name used by the selected prompt, - including template references rather than only explicit input declarations. -- Whether each input is required, optional, or conditional for this request. -- Any declared format or schema constraint that affects accepted input, using - public identifiers. Distinguish an unspecified constraint from a known one; - do not infer a format from an input's name. -- A description or requirement explanation where available, sufficient for an - operator to understand a missing-binding diagnostic. -- Caller-supplied variable requirements separately from file input requirements, - including whether supplied values or defaults satisfy them. Resolved variable - values and prompt contents need not be returned. -- Whether requirement discovery is complete for the supplied context, with - explicit reasons for unresolved requirements. +Requirements reports exclude artifact payloads, prompt contents, and unrelated +supplied variable values. A supplied value used to resolve an input identifier +can appear as that identifier. Keep this native behavior without adding a custom +projection of the report. -For example, this is an illustrative proposed JSON shape, not an existing -Scriptorium schema: +The upstream report contract and adopted dependency versions govern the shape. +Migration documentation must identify relevant consumer changes; this proposal +adds no independent response-versioning mechanism or stronger compatibility +guarantee. -```json -{ - "schema_version": "scriptorium.requirements.v1", - "prompt": {"id": "session_recap", "version": "1"}, - "profile": {"id": "default"}, - "complete": true, - "inputs": [ - {"name": "glossary", "requirement": "optional", "format": null}, - {"name": "transcript", "requirement": "required", "format": null} - ], - "variables": [], - "unresolved": [] -} -``` +## Conditional and incomplete discovery -The final field names and schema identifier are open for agreement. Publish -their meanings and compatibility rules so Narratio can consume the result -without parsing human-readable text. Input names must retain the exact spelling -used by execution. +PromptKit classifies dependencies as required, optional, or conditional using +the supplied context and its supported template analysis. Recognizable references +only on known skipped paths can be optional. Explicit required declarations +still apply. Unavailable input contents, unresolved variable values or dynamic +names, and unsupported dependency-relevant constructs can leave discovery +incomplete. Valid templates are not rejected merely because the analysis cannot +fully resolve them. -## Conditional and incomplete requirements +Narratio must inspect both the requirements and discovery completeness: -Use supplied variables and effective defaults when they determine whether an -input is needed. An input used only by an inactive conditional branch must not -be presented as unconditionally required. +- `complete: true` establishes discovery coverage, not satisfied bindings, + valid future input bytes, or execution readiness. Required inputs or missing + required variables may still be present. +- `complete: false` preserves useful findings and structured issues. An empty + partial inventory does not prove that no inputs are needed. +- A partial report can return status zero. An issue is not itself an invocation + failure. Narratio owns its policy for proceeding with incomplete dependency + information. -When a requirement depends on unavailable input contents, an unspecified -variable, or a dynamically selected name, report that limitation explicitly. -Return any requirements that are known, but do not present a partial list as a -complete contract. Conditional entries should explain their condition where it -can be expressed through the public API; otherwise report an unresolved reason. +Once input artifacts exist, native `promptkit requirements validate` can check +a concrete request through preparation. It is a separate operation and must not +replace early discovery. Neither operation guarantees later generation success. +Changes to configuration or resources between invocations can change findings. -An empty, complete input list means the selected context needs no caller-supplied -file inputs. An empty, incomplete list does not establish that conclusion. -Successfully returning an incomplete inspection result may exit zero, provided -the response clearly identifies incompleteness. Narratio can then distinguish -inspection success from a completed dependency check. +## Failures -## Errors and externally observable guarantees +Use CommandKit's native nonzero outcomes and diagnostic messages for selection, +configuration, operational, and report-delivery failures. JSON report selection +does not turn diagnostic messages into a categorized JSON error response. +Narratio can attach the diagnostic to its own artifact or pipeline context; +it must not depend on parsing the message to recover a stable error category. -Invalid selectors, unreadable required configuration/resources, and invalid -definitions that prevent inspection must return nonzero. Provide a structured -error with a stable category and safe human-readable message in JSON mode. -Known incomplete analysis should use the response's incompleteness contract -rather than masquerade as an unknown prompt or a successful complete result. +Structured discovery issue codes belong to successful inspection reports and +are distinct from invocation failures. Follow PromptKit's distinction between +partial discovery and failures that return no inspection. Absence of a future +input artifact is not a discovery failure because discovery does not load it. -Inspection must not fail merely because an eventual input artifact is absent. -It must not expose credentials, input payloads, or rendered prompt content in -its output or errors. - -The reported requirements must agree with execution for the same effective -configuration and selectors. This is a description of input requirements at -inspection time, not a guarantee of generation success or validation of future -input bytes. Changes to prompt resources or configuration after inspection may -change the requirements of a later execution. +The accepted consumer requirement is a nonzero return code plus failure +information. If programmatic failure categories or separate schema versioning +become necessary, discuss them upstream rather than introducing local wrappers. +Neither is a prerequisite for this migration. ## Consumer acceptance scenarios 1. A prompt requires `transcript`, but Narratio binds only `session_transcript`. - Inspection reports the exact required name, allowing Narratio to identify the - missing binding before producing either file. -2. A template references an undeclared input. Inspection reports the reference - or an actionable definition error; it does not silently omit it from a - supposedly complete result. -3. A prompt accepts an optional glossary. Inspection identifies its optional - status, allowing a pipeline without a glossary to remain valid. -4. Supplied variables resolve a conditional input requirement. Inspection - reflects the selected context; unresolved conditions remain visibly uncertain. -5. Selecting another prompt version or profile reports the corresponding - effective requirements and resolved identities. -6. Inspection succeeds with no generated input files, no model access, and no - generation credentials, assuming the selected configuration/resources are - otherwise valid and readable. -7. An unknown prompt, ambiguous version, or invalid profile yields a structured - failure suitable for a Narratio diagnostic naming the affected artifact. + The native inventory identifies the exact required name before either file + exists, allowing Narratio to identify the missing binding. +2. A template references an undeclared input. Recognizable references appear in + the inventory with `declared: false`; unsupported dependency analysis is + explicitly incomplete rather than silently claiming full coverage. +3. An optional glossary is reported with its native classification, allowing + Narratio to distinguish it from a required binding. +4. Variables resolve a supported conditional dependency. Unknown conditions + retain conditional findings or discovery issues according to PromptKit. +5. Selecting another qualified prompt or profile reports the corresponding + identities. Prompt selection changes the analyzed definition; profile + selection alone does not change its requirements. +6. Discovery works with no generated input files, model access, or generation + credentials when the selected configuration and resources are readable and + valid for discovery. +7. An unknown prompt or invalid profile selection returns nonzero with native + diagnostics suitable for Narratio to associate with the affected artifact. +8. Narratio treats a complete report with missing required bindings differently + from a complete, satisfied dependency check, and handles partial reports + without assuming that status zero establishes completeness. +9. Scriptorium's adapter uses consistent configuration, namespaces, and applicable + request context across discovery, validation, preview, and execution. ## Scope boundary Narratio owns its source catalog, stage ordering, source-to-input bindings, -artifact dependency graph, and session availability checks. Scriptorium only -needs to describe its prompt requirements through this public interface. -Narratio source IDs and pipeline concepts need not appear in the Scriptorium -API. A graph renderer, HTTP endpoint, batch operation, and changes to generation -behavior are outside this request. +artifact dependency graph, and session availability checks. Scriptorium exposes +PromptKit's native requirements through CommandKit. Narratio source IDs and +pipeline concepts do not become part of that interface. A graph renderer, HTTP +endpoint, batch operation, and changes to generation behavior are outside this +request. + +[requirements-cli]: https://forge.maximumdirect.net/base-layer/commandkit/src/commit/d330d7637f2e7507eca931a899397222c82bc892/docs/cli.md#promptkit-requirements +[requirements-api]: https://forge.maximumdirect.net/base-layer/promptkit/src/commit/804a4d32b23aa2e7973400bb0a322a7016574c9f/requirements.go