Align Narratio requirements with native CommandKit reports
This commit is contained in:
@@ -208,6 +208,32 @@ qualified IDs and explicit namespace flags when selecting namespaced resources,
|
||||
while retaining upstream behavior for unqualified IDs. Configurable defaults
|
||||
can be considered later if a need arises.
|
||||
|
||||
### Native inspection reports and consumer requirements
|
||||
|
||||
Decision recorded 2026-11-19: adopt native inspection reports and the native
|
||||
`promptkit requirements show` interface for Narratio's early dependency
|
||||
discovery. Its consumer needs and acceptance scenarios are recorded in the
|
||||
[aligned Narratio proposal](narratio-feature-request.md). This direction is
|
||||
accepted; Scriptorium integration remains to be implemented.
|
||||
|
||||
Consume native report fields without a Scriptorium-specific schema-version
|
||||
envelope. Use native nonzero outcomes and diagnostics for invocation failures;
|
||||
JSON report selection does not require JSON errors. Neither a separately
|
||||
versioned envelope nor programmatic invocation-failure categories blocks this
|
||||
migration. A future concrete need for either belongs in an upstream discussion.
|
||||
|
||||
PromptKit owns requirement classification, discovery completeness, and structured
|
||||
discovery issues. Completeness is distinct from satisfied bindings or execution
|
||||
readiness; Narratio owns comparison against its planned inputs. Keep early
|
||||
discovery separate from concrete request validation. Use current exact resource
|
||||
identities and native input/variable metadata, without restoring obsolete prompt
|
||||
versions or inventing input schemas or variable-default semantics.
|
||||
|
||||
Standard prompt inspection includes captured messages; prepared preview includes
|
||||
rendered content. Adopt these native reports without rebuilding Scriptorium's
|
||||
previous summary projections, and explain the changed presentation in migration
|
||||
guidance. Requirements discovery retains its distinct upstream content boundary.
|
||||
|
||||
## Upstream baseline
|
||||
|
||||
The initial review covered CommandKit **v0.7.0**, ckconfig **v0.9.0**, and
|
||||
@@ -252,33 +278,7 @@ responsibilities. Decide their mapping to the CommandKit invocation context
|
||||
and PromptKit operation timeout, separately from the standard per-attempt
|
||||
timeout flag.
|
||||
|
||||
### 2 Native reports and executable consumer needs
|
||||
|
||||
Adopt native inspection reports as the starting point. Standard prompt
|
||||
inspection includes captured prompt messages, unlike the current Scriptorium
|
||||
summary. Document that change and confirm how native inspection and preparation
|
||||
reports meet operator needs without rebuilding custom projections.
|
||||
|
||||
Reconcile the proposed [Narratio requirements request](narratio-feature-request.md)
|
||||
with native `promptkit requirements show`. Its central need remains valid:
|
||||
discover input requirements before expensive upstream work and before input
|
||||
files exist. Preserve the distinction between successful discovery and complete
|
||||
discovery; a successful partial report does not establish all requirements.
|
||||
|
||||
The proposal's legacy syntax and prompt-version assumptions need revision. Its
|
||||
illustrative versioned JSON envelope and structured-error request also need a
|
||||
decision: native reports do not add that Scriptorium envelope, and `--format
|
||||
json` does not automatically turn diagnostics into a JSON error protocol.
|
||||
Determine which consumer requirements are essential and whether any genuine
|
||||
gap belongs upstream. Avoid treating the existing proposal as a mandate for a
|
||||
new compatibility facade.
|
||||
|
||||
Keep discovery distinct from concrete request validation: validation prepares a
|
||||
real request and may read input artifacts and check relevant credentials;
|
||||
discovery must work before those artifacts exist. Neither guarantees a future
|
||||
generation will succeed.
|
||||
|
||||
### 3 Application commands and standard request flags
|
||||
### 2 Application commands and standard request flags
|
||||
|
||||
Decide the final generation command operands and application-specific flags
|
||||
around the reusable ckprompt request flag groups. Use their standard names,
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
# Feature request: inspect prompt input requirements
|
||||
# Prompt input requirements for Narratio
|
||||
|
||||
Status: proposed. This document requests an external Scriptorium API for
|
||||
Narratio and other executable consumers. It does not describe implemented
|
||||
behavior or prescribe implementation structure.
|
||||
Status: accepted direction, pending implementation through the CommandKit
|
||||
migration. This proposal records Narratio's consumer needs and their mapping
|
||||
to the native interface; it does not describe shipped Scriptorium behavior.
|
||||
The [CommandKit notepad](commandkit-notepad.md) records the migration decisions.
|
||||
|
||||
## Consumer need
|
||||
|
||||
@@ -13,153 +14,153 @@ identities: for example, the source
|
||||
`narratio.transcript.final_trimmed_markdown` might be supplied under the prompt
|
||||
input name `transcript`.
|
||||
|
||||
Narratio needs to discover the selected prompt's expected input names before
|
||||
running expensive upstream stages. It can then show operators how prompt inputs
|
||||
map to pipeline products and report missing bindings before transcription or
|
||||
generation begins. At that point, the eventual input files often do not exist.
|
||||
Narratio needs to discover expected input names before running expensive
|
||||
upstream stages and before the eventual input files exist. It can then show
|
||||
operators how prompt inputs map to pipeline products and identify missing
|
||||
bindings without reading prompt definitions or reproducing PromptKit's analysis.
|
||||
|
||||
The requested API must expose requirements without requiring callers to read
|
||||
prompt definitions or reproduce Scriptorium's interpretation of them.
|
||||
## Selected native operation
|
||||
|
||||
## Requested operation
|
||||
Expose `scriptorium promptkit requirements show PROMPT_ID` through ckprompt.
|
||||
Subprocess consumers select JSON with the standard `--format json` flag.
|
||||
Default and explicit text output use the native YAML presentation.
|
||||
The [upstream CLI contract][requirements-cli] owns syntax and report details.
|
||||
|
||||
Provide a read-only CLI operation that reports the effective input requirements
|
||||
for a selected prompt and execution context. A proposed invocation is:
|
||||
Use the same application configuration and framework source selection as
|
||||
execution, including `--config`, `--overlay`, and applicable application
|
||||
selectors. The native `--llm-profile`, `--llm-var`, `--llm-session-id`, and
|
||||
prompt/profile namespace flags supply the relevant discovery context.
|
||||
Preserve upstream selection semantics, including the prompt's default profile
|
||||
when an explicit profile is omitted. Qualified IDs identify installed resources;
|
||||
there is no separate prompt-version selector or reported prompt-version field.
|
||||
|
||||
```text
|
||||
scriptorium inspect requirements --prompt ID [--prompt-version VERSION]
|
||||
[--profile ID] [--var name=value ...]
|
||||
[--config PATH] [--config-overlay NAME]
|
||||
[--promptkit-source PATH ...] [--schema-dir DIR]
|
||||
[--format text|json]
|
||||
```
|
||||
The selected profile establishes identity, not execution readiness. Selecting
|
||||
another profile does not itself change the prompt's discovered requirements.
|
||||
Scriptorium's loading and request mapping must preserve the same effective
|
||||
selection and applicable caller context across discovery and execution.
|
||||
|
||||
Command naming is open for agreement. The required capability is distinct from
|
||||
the currently documented [prompt definition inspection](../cli.md#scriptorium-inspect-prompt):
|
||||
the result describes the caller-facing requirements of the selected execution
|
||||
context. Extending an existing public operation is acceptable if this contract
|
||||
remains explicit and machine-readable.
|
||||
Discovery requires no input bindings or output-artifact path. PromptKit performs
|
||||
no artifact loading, schema loading, credential resolution, message rendering,
|
||||
capacity reservation, or model call for this operation. Reading configuration
|
||||
and prompt resources needed to build the captured engine is expected. The
|
||||
[PromptKit discovery contract][requirements-api] owns the analysis and its
|
||||
limits.
|
||||
|
||||
Selection must follow the same public configuration, source, prompt-version,
|
||||
profile, and variable semantics as an equivalent execution request. Report the
|
||||
resolved prompt ID/version and effective profile identity, including defaults
|
||||
when the caller omits a selector. Unknown or ambiguous selections must produce
|
||||
an actionable failure consistent with execution selection.
|
||||
## Native response and consumer interpretation
|
||||
|
||||
Input paths, input contents, and an output-artifact path are not required.
|
||||
Inspection must work before upstream products are created. It must not call a
|
||||
model, require generation credentials, reserve generation capacity, or create
|
||||
execution artifacts. Reading configuration and prompt resources needed for
|
||||
inspection is expected.
|
||||
Consume the native JSON report directly, without a Scriptorium-specific wrapper
|
||||
or schema-version field. Successful report delivery writes one JSON document to
|
||||
stdout, with diagnostics on stderr under CommandKit's standard reporting rules.
|
||||
Consumers requesting a report should not enable quiet mode.
|
||||
|
||||
## Response contract
|
||||
| Field | Consumer meaning |
|
||||
| --- | --- |
|
||||
| `prompt_id`, `profile_id` | Exact selected resource identities. |
|
||||
| `inputs` | Declared and recognizable template-referenced inputs, including exact `name`, requirement `kind`, `declared`, `content_type`, `description`, and `explanation`. |
|
||||
| `variables` | Recognizable caller-variable keys with `name`, requirement `kind`, `supplied`, and `explanation`. |
|
||||
| `complete` | Whether the supported dependency analysis is complete for the supplied context. |
|
||||
| `issues` | Structured discovery limitations with a `code`, associated input or variable names where known, and a human-readable description. |
|
||||
|
||||
Offer a readable text view for operators and a versioned JSON response for
|
||||
subprocess consumers. JSON stdout must contain one response document; incidental
|
||||
diagnostics belong on stderr. Stable ordering should make repeated inspection
|
||||
of unchanged effective inputs easy to compare.
|
||||
PromptKit sorts input and variable entries by exact name, and deduplicates and
|
||||
orders issues by their code and associated names. Preserve empty inventories
|
||||
and false values. Input metadata distinguishes absent/null values from an
|
||||
explicitly empty string. Explanations and issue descriptions are for display;
|
||||
Narratio must use structured fields rather than parse their prose.
|
||||
|
||||
The response should expose:
|
||||
The native input metadata includes content type and description. Current
|
||||
PromptKit inputs have no per-input schema declaration to project. Do not infer
|
||||
constraints from names or treat the output-validation schema as an input schema.
|
||||
Variable `supplied` records exact-key presence, including an explicitly empty
|
||||
value; this interface does not introduce a separate variable-default system.
|
||||
|
||||
- Resolved prompt and profile identities.
|
||||
- Every discoverable caller-supplied input name used by the selected prompt,
|
||||
including template references rather than only explicit input declarations.
|
||||
- Whether each input is required, optional, or conditional for this request.
|
||||
- Any declared format or schema constraint that affects accepted input, using
|
||||
public identifiers. Distinguish an unspecified constraint from a known one;
|
||||
do not infer a format from an input's name.
|
||||
- A description or requirement explanation where available, sufficient for an
|
||||
operator to understand a missing-binding diagnostic.
|
||||
- Caller-supplied variable requirements separately from file input requirements,
|
||||
including whether supplied values or defaults satisfy them. Resolved variable
|
||||
values and prompt contents need not be returned.
|
||||
- Whether requirement discovery is complete for the supplied context, with
|
||||
explicit reasons for unresolved requirements.
|
||||
Requirements reports exclude artifact payloads, prompt contents, and unrelated
|
||||
supplied variable values. A supplied value used to resolve an input identifier
|
||||
can appear as that identifier. Keep this native behavior without adding a custom
|
||||
projection of the report.
|
||||
|
||||
For example, this is an illustrative proposed JSON shape, not an existing
|
||||
Scriptorium schema:
|
||||
The upstream report contract and adopted dependency versions govern the shape.
|
||||
Migration documentation must identify relevant consumer changes; this proposal
|
||||
adds no independent response-versioning mechanism or stronger compatibility
|
||||
guarantee.
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": "scriptorium.requirements.v1",
|
||||
"prompt": {"id": "session_recap", "version": "1"},
|
||||
"profile": {"id": "default"},
|
||||
"complete": true,
|
||||
"inputs": [
|
||||
{"name": "glossary", "requirement": "optional", "format": null},
|
||||
{"name": "transcript", "requirement": "required", "format": null}
|
||||
],
|
||||
"variables": [],
|
||||
"unresolved": []
|
||||
}
|
||||
```
|
||||
## Conditional and incomplete discovery
|
||||
|
||||
The final field names and schema identifier are open for agreement. Publish
|
||||
their meanings and compatibility rules so Narratio can consume the result
|
||||
without parsing human-readable text. Input names must retain the exact spelling
|
||||
used by execution.
|
||||
PromptKit classifies dependencies as required, optional, or conditional using
|
||||
the supplied context and its supported template analysis. Recognizable references
|
||||
only on known skipped paths can be optional. Explicit required declarations
|
||||
still apply. Unavailable input contents, unresolved variable values or dynamic
|
||||
names, and unsupported dependency-relevant constructs can leave discovery
|
||||
incomplete. Valid templates are not rejected merely because the analysis cannot
|
||||
fully resolve them.
|
||||
|
||||
## Conditional and incomplete requirements
|
||||
Narratio must inspect both the requirements and discovery completeness:
|
||||
|
||||
Use supplied variables and effective defaults when they determine whether an
|
||||
input is needed. An input used only by an inactive conditional branch must not
|
||||
be presented as unconditionally required.
|
||||
- `complete: true` establishes discovery coverage, not satisfied bindings,
|
||||
valid future input bytes, or execution readiness. Required inputs or missing
|
||||
required variables may still be present.
|
||||
- `complete: false` preserves useful findings and structured issues. An empty
|
||||
partial inventory does not prove that no inputs are needed.
|
||||
- A partial report can return status zero. An issue is not itself an invocation
|
||||
failure. Narratio owns its policy for proceeding with incomplete dependency
|
||||
information.
|
||||
|
||||
When a requirement depends on unavailable input contents, an unspecified
|
||||
variable, or a dynamically selected name, report that limitation explicitly.
|
||||
Return any requirements that are known, but do not present a partial list as a
|
||||
complete contract. Conditional entries should explain their condition where it
|
||||
can be expressed through the public API; otherwise report an unresolved reason.
|
||||
Once input artifacts exist, native `promptkit requirements validate` can check
|
||||
a concrete request through preparation. It is a separate operation and must not
|
||||
replace early discovery. Neither operation guarantees later generation success.
|
||||
Changes to configuration or resources between invocations can change findings.
|
||||
|
||||
An empty, complete input list means the selected context needs no caller-supplied
|
||||
file inputs. An empty, incomplete list does not establish that conclusion.
|
||||
Successfully returning an incomplete inspection result may exit zero, provided
|
||||
the response clearly identifies incompleteness. Narratio can then distinguish
|
||||
inspection success from a completed dependency check.
|
||||
## Failures
|
||||
|
||||
## Errors and externally observable guarantees
|
||||
Use CommandKit's native nonzero outcomes and diagnostic messages for selection,
|
||||
configuration, operational, and report-delivery failures. JSON report selection
|
||||
does not turn diagnostic messages into a categorized JSON error response.
|
||||
Narratio can attach the diagnostic to its own artifact or pipeline context;
|
||||
it must not depend on parsing the message to recover a stable error category.
|
||||
|
||||
Invalid selectors, unreadable required configuration/resources, and invalid
|
||||
definitions that prevent inspection must return nonzero. Provide a structured
|
||||
error with a stable category and safe human-readable message in JSON mode.
|
||||
Known incomplete analysis should use the response's incompleteness contract
|
||||
rather than masquerade as an unknown prompt or a successful complete result.
|
||||
Structured discovery issue codes belong to successful inspection reports and
|
||||
are distinct from invocation failures. Follow PromptKit's distinction between
|
||||
partial discovery and failures that return no inspection. Absence of a future
|
||||
input artifact is not a discovery failure because discovery does not load it.
|
||||
|
||||
Inspection must not fail merely because an eventual input artifact is absent.
|
||||
It must not expose credentials, input payloads, or rendered prompt content in
|
||||
its output or errors.
|
||||
|
||||
The reported requirements must agree with execution for the same effective
|
||||
configuration and selectors. This is a description of input requirements at
|
||||
inspection time, not a guarantee of generation success or validation of future
|
||||
input bytes. Changes to prompt resources or configuration after inspection may
|
||||
change the requirements of a later execution.
|
||||
The accepted consumer requirement is a nonzero return code plus failure
|
||||
information. If programmatic failure categories or separate schema versioning
|
||||
become necessary, discuss them upstream rather than introducing local wrappers.
|
||||
Neither is a prerequisite for this migration.
|
||||
|
||||
## Consumer acceptance scenarios
|
||||
|
||||
1. A prompt requires `transcript`, but Narratio binds only `session_transcript`.
|
||||
Inspection reports the exact required name, allowing Narratio to identify the
|
||||
missing binding before producing either file.
|
||||
2. A template references an undeclared input. Inspection reports the reference
|
||||
or an actionable definition error; it does not silently omit it from a
|
||||
supposedly complete result.
|
||||
3. A prompt accepts an optional glossary. Inspection identifies its optional
|
||||
status, allowing a pipeline without a glossary to remain valid.
|
||||
4. Supplied variables resolve a conditional input requirement. Inspection
|
||||
reflects the selected context; unresolved conditions remain visibly uncertain.
|
||||
5. Selecting another prompt version or profile reports the corresponding
|
||||
effective requirements and resolved identities.
|
||||
6. Inspection succeeds with no generated input files, no model access, and no
|
||||
generation credentials, assuming the selected configuration/resources are
|
||||
otherwise valid and readable.
|
||||
7. An unknown prompt, ambiguous version, or invalid profile yields a structured
|
||||
failure suitable for a Narratio diagnostic naming the affected artifact.
|
||||
The native inventory identifies the exact required name before either file
|
||||
exists, allowing Narratio to identify the missing binding.
|
||||
2. A template references an undeclared input. Recognizable references appear in
|
||||
the inventory with `declared: false`; unsupported dependency analysis is
|
||||
explicitly incomplete rather than silently claiming full coverage.
|
||||
3. An optional glossary is reported with its native classification, allowing
|
||||
Narratio to distinguish it from a required binding.
|
||||
4. Variables resolve a supported conditional dependency. Unknown conditions
|
||||
retain conditional findings or discovery issues according to PromptKit.
|
||||
5. Selecting another qualified prompt or profile reports the corresponding
|
||||
identities. Prompt selection changes the analyzed definition; profile
|
||||
selection alone does not change its requirements.
|
||||
6. Discovery works with no generated input files, model access, or generation
|
||||
credentials when the selected configuration and resources are readable and
|
||||
valid for discovery.
|
||||
7. An unknown prompt or invalid profile selection returns nonzero with native
|
||||
diagnostics suitable for Narratio to associate with the affected artifact.
|
||||
8. Narratio treats a complete report with missing required bindings differently
|
||||
from a complete, satisfied dependency check, and handles partial reports
|
||||
without assuming that status zero establishes completeness.
|
||||
9. Scriptorium's adapter uses consistent configuration, namespaces, and applicable
|
||||
request context across discovery, validation, preview, and execution.
|
||||
|
||||
## Scope boundary
|
||||
|
||||
Narratio owns its source catalog, stage ordering, source-to-input bindings,
|
||||
artifact dependency graph, and session availability checks. Scriptorium only
|
||||
needs to describe its prompt requirements through this public interface.
|
||||
Narratio source IDs and pipeline concepts need not appear in the Scriptorium
|
||||
API. A graph renderer, HTTP endpoint, batch operation, and changes to generation
|
||||
behavior are outside this request.
|
||||
artifact dependency graph, and session availability checks. Scriptorium exposes
|
||||
PromptKit's native requirements through CommandKit. Narratio source IDs and
|
||||
pipeline concepts do not become part of that interface. A graph renderer, HTTP
|
||||
endpoint, batch operation, and changes to generation behavior are outside this
|
||||
request.
|
||||
|
||||
[requirements-cli]: https://forge.maximumdirect.net/base-layer/commandkit/src/commit/d330d7637f2e7507eca931a899397222c82bc892/docs/cli.md#promptkit-requirements
|
||||
[requirements-api]: https://forge.maximumdirect.net/base-layer/promptkit/src/commit/804a4d32b23aa2e7973400bb0a322a7016574c9f/requirements.go
|
||||
|
||||
Reference in New Issue
Block a user