Allow native format selection for separate reports

This commit is contained in:
2026-11-20 03:14:07 +00:00
parent fbe7dd56e3
commit 69d2ba7038
3 changed files with 77 additions and 49 deletions

View File

@@ -63,8 +63,9 @@ dependencies remain subject to the existing
No selected-format accessor is required for this migration. The native No selected-format accessor is required for this migration. The native
invocation reporter handles the main result on stdout; the existing writer API invocation reporter handles the main result on stdout; the existing writer API
handles the separate report using fixed JSON. The upstream accessor proposal handles the separate report with automatic or explicitly selected rendering.
remains optional API work, independent of Scriptorium's adoption. The upstream accessor proposal remains optional API work, independent of
Scriptorium's adoption.
## Target End State ## Target End State
@@ -199,9 +200,9 @@ mapping, not a promise that separate invocations see an immutable snapshot.
### Generation, Reports, And Process Outcomes ### Generation, Reports, And Process Outcomes
Use a CommandKit `Report` whose value is PromptKit's native `RunResult` and whose Use a main CommandKit `Report` whose value is PromptKit's native `RunResult` and
text template emits only the resulting artifact content. The template should whose text template emits only the resulting artifact content. The template
emit artifact bytes without presentation decoration. Default/text serves raw should emit artifact bytes without presentation decoration. Default/text serves raw
artifact consumers; JSON and YAML expose the full native result. The native artifact consumers; JSON and YAML expose the full native result. The native
structured serialization, including its representation of artifact bytes, structured serialization, including its representation of artifact bytes,
remains upstream-owned. There is no Scriptorium result envelope. remains upstream-owned. There is no Scriptorium result envelope.
@@ -211,7 +212,14 @@ remains upstream-owned. There is no Scriptorium result envelope.
| Main result | Goes through the native invocation reporter to stdout using the selected format. | | Main result | Goes through the native invocation reporter to stdout using the selected format. |
| `--format text` or default | Main result contains only the artifact. | | `--format text` or default | Main result contains only the artifact. |
| `--format json` or `--format yaml` | Main result contains the native result and metadata. | | `--format json` or `--format yaml` | Main result contains the native result and metadata. |
| `--report PATH` | Additionally write the native result as JSON through CommandKit's renderer, independently of the main format and destination. | | `--report PATH` | Additionally write the native result through CommandKit's renderer, independently of the main format and destination. Without `--report-format`, use native automatic rendering. |
| `--report-format text\|yaml\|json` | Explicitly select the separate report's native rendering format. |
The separate report contains the same native `RunResult`, without the main
report's artifact-only text template. Omission of `--report-format` leaves the
format unset so CommandKit decides how to render it. Its current automatic
rendering for structured values is YAML; future upstream default changes flow
through. Scriptorium does not set a default report format.
There is no `--output` or legacy `--out` flag. Callers save the main result by There is no `--output` or legacy `--out` flag. Callers save the main result by
redirecting stdout in a shell or supplying a file as the subprocess stdout redirecting stdout in a shell or supplying a file as the subprocess stdout
@@ -234,7 +242,7 @@ invocation-error envelope is part of this migration.
Adopt CommandKit's quiet and verbosity semantics, including rendering and Adopt CommandKit's quiet and verbosity semantics, including rendering and
presentation failures. Console results use the invocation reporter; diagnostics presentation failures. Console results use the invocation reporter; diagnostics
use its diagnostic channel. An explicit `--report` write is an application use its diagnostic channel. An explicit `--report` write is an application
effect rendered as JSON through the writer API and still occurs in quiet mode. effect rendered through the writer API and still occurs in quiet mode.
Quiet does not cancel generation; it suppresses the main result even when Quiet does not cancel generation; it suppresses the main result even when
stdout is redirected, while failure explanations remain visible. Move any stdout is redirected, while failure explanations remain visible. Move any
retained generation summary into an appropriate native message category. retained generation summary into an appropriate native message category.
@@ -296,8 +304,9 @@ documentation and examples describe the shipped interface. In particular:
collections. Their upstream lifecycle differences remain visible. collections. Their upstream lifecycle differences remain visible.
- Generation defaults to the artifact alone; structured output and separate - Generation defaults to the artifact alone; structured output and separate
reports expose the native result. Main output uses stdout and native quiet reports expose the native result. Main output uses stdout and native quiet
semantics; the optional JSON report file is written independently of quiet, semantics; the optional report file uses native automatic or explicitly
with useful failure outcomes and nontransactional delivery. selected rendering, independently of quiet, with useful failure outcomes and
nontransactional delivery.
- Narratio's linked acceptance scenarios are supported, including useful partial - Narratio's linked acceptance scenarios are supported, including useful partial
discovery and discovery before generated artifacts exist. discovery and discovery before generated artifacts exist.
- Native help, aliases, completion, and explicit smoke testing are available. - Native help, aliases, completion, and explicit smoke testing are available.
@@ -309,8 +318,8 @@ documentation and examples describe the shipped interface. In particular:
reflect the cutover. The CLI migration includes positional prompt selection, reflect the cutover. The CLI migration includes positional prompt selection,
command replacements, literal versus file inputs, repeated mappings, overlay command replacements, literal versus file inputs, repeated mappings, overlay
selection, native endpoint/timeout controls, stdout redirection in place of selection, native endpoint/timeout controls, stdout redirection in place of
output-file flags, content-bearing reports, report availability, quiet output-file flags, content-bearing reports, optional report-format selection,
behavior, and changed exit statuses. report availability, quiet behavior, and changed exit statuses.
The [implementation plan](implementation.md) owns task sequencing, concrete The [implementation plan](implementation.md) owns task sequencing, concrete
adapter mappings, verification steps, and sprint status. There are no unresolved adapter mappings, verification steps, and sprint status. There are no unresolved

View File

@@ -100,18 +100,20 @@ is application work; deciding what constitutes a PromptKit failure is not.
### Native result reporting and separate report files ### Native result reporting and separate report files
Decision recorded 2026-11-19: use a CommandKit `Report` containing PromptKit's Decision revised 2026-11-19: use a main CommandKit `Report` containing PromptKit's
native `RunResult`, with a text template that emits only generated content. native `RunResult`, with a text template that emits only generated content.
Default/text output serves artifact consumers; standard JSON/YAML formats expose Default/text output serves artifact consumers; standard JSON/YAML formats expose
the native result and its metadata. This composes existing upstream APIs; the native result and its metadata. This composes existing upstream APIs;
ckprompt does not supply a general generation command or prescribe this template. ckprompt does not supply a general generation command or prescribe this template.
Include an optional `--report PATH` flag requiring an explicit file destination. Include an optional `--report PATH` flag requiring an explicit file destination.
Omitting it creates no separate report file. When requested, save the native Omitting it creates no separate report file. When requested, pass the native
result as JSON through CommandKit's renderer, using the same representation as `RunResult` to CommandKit's renderer without an artifact-only text template.
inline JSON reporting. The report file's format is JSON independently of the Optional `--report-format text|yaml|json` selects the native rendering format.
main result's `--format`; requesting a report does not change the main result's When omitted, leave the format unset and let CommandKit choose; its current
representation or destination. structured-value default is YAML, and future upstream changes flow through.
There is no Scriptorium default or filename-based selection. Report rendering
is independent of the main result's `--format`, representation, and destination.
Failure to write a requested report is a delivery failure and must produce a Failure to write a requested report is a delivery failure and must produce a
nonzero outcome and diagnostic. nonzero outcome and diagnostic.
@@ -136,7 +138,7 @@ on stderr. Do not retain the old `--out` flag.
`--format` selects the stdout representation independently of redirection: `--format` selects the stdout representation independently of redirection:
default/text emits the raw artifact, while JSON/YAML emits the native result default/text emits the raw artifact, while JSON/YAML emits the native result
with metadata. The filename does not select the format. `--report PATH` remains with metadata. The filename does not select the format. `--report PATH` remains
an additional native JSON report, so callers can save the raw artifact and its an additional native result report, so callers can save the raw artifact and its
report to separate files by redirecting stdout and supplying `--report`. report to separate files by redirecting stdout and supplying `--report`.
Adopt CommandKit's upstream verbosity and `--quiet` semantics fully, including Adopt CommandKit's upstream verbosity and `--quiet` semantics fully, including
@@ -146,8 +148,8 @@ Quiet mode does not cancel generation, and failure explanations remain visible
through the native diagnostic channel. through the native diagnostic channel.
An explicit `--report` write remains an application-owned effect in quiet mode, An explicit `--report` write remains an application-owned effect in quiet mode,
using the existing writer API with fixed JSON. Quiet suppresses the main result using the existing writer API. Quiet suppresses the main result even when stdout
even when stdout is redirected; callers wanting that result omit quiet. No is redirected; callers wanting that result omit quiet. No
selected-format accessor is needed, so the upstream proposal is no longer a selected-format accessor is needed, so the upstream proposal is no longer a
migration prerequisite. The [upstream reporting contract][upstream-cli] owns the migration prerequisite. The [upstream reporting contract][upstream-cli] owns the
shared verbosity rules. shared verbosity rules.
@@ -243,8 +245,8 @@ families. The CLI migration does not redesign the HTTP API.
Reuse ckprompt's applicable resource, input, and execution flag bindings. Reuse ckprompt's applicable resource, input, and execution flag bindings.
Preserve native parsing, explicit-value presence, repeated mapping semantics, Preserve native parsing, explicit-value presence, repeated mapping semantics,
and PromptKit's interpretation of request values. Keep application-specific and PromptKit's interpretation of request values. Keep application-specific
configuration/source selection, `--report`, server options, and the configuration/source selection, `--report`/`--report-format`, server options,
whole-operation deadline separate from framework request controls. and the whole-operation deadline separate from framework request controls.
Enable upstream convenience aliases consistently on applicable commands, Enable upstream convenience aliases consistently on applicable commands,
including `run`. Prefer their shorter spellings in user-facing documentation, including `run`. Prefer their shorter spellings in user-facing documentation,

View File

@@ -52,7 +52,8 @@ expanding this sprint into another framework upgrade.
No selected-format accessor is required. Main results go through the native No selected-format accessor is required. Main results go through the native
invocation reporter to stdout, with caller-controlled redirection. The separate invocation reporter to stdout, with caller-controlled redirection. The separate
`--report` file uses the existing `RenderReport(writer, report, FormatJSON)` API. `--report` file uses the existing `RenderReport(writer, report, format)` API,
with the format unset unless `--report-format` is supplied.
The upstream accessor proposal is optional and does not block any stage. Do not The upstream accessor proposal is optional and does not block any stage. Do not
add a second reporting-flag parser or private-API workaround. add a second reporting-flag parser or private-API workaround.
@@ -112,7 +113,7 @@ classify every `ErrInvalidConfig` wrapper as invalid user input without native
evidence. Retain safe explanations for provider failures; causes are inspectable evidence. Retain safe explanations for provider failures; causes are inspectable
Go errors, not permission to print credential-bearing provider payloads. Go errors, not permission to print credential-bearing provider payloads.
Use one `commandkit.Report` with the actual `*promptkit.RunResult` as `Value`. Use a main `commandkit.Report` with the actual `*promptkit.RunResult` as `Value`.
Its reusable text template emits `Artifact.Body` as bytes represented as a string, Its reusable text template emits `Artifact.Body` as bytes represented as a string,
with no added newline, header, escaping, or summary. Native JSON/YAML serialization with no added newline, header, escaping, or summary. Native JSON/YAML serialization
remains upstream-owned, including byte representation and native metadata. remains upstream-owned, including byte representation and native metadata.
@@ -123,9 +124,16 @@ their subprocess API; Scriptorium keeps diagnostics on stderr. Native quiet
suppresses the main result even when stdout is redirected. Consumers wanting suppresses the main result even when stdout is redirected. Consumers wanting
that result omit `--quiet`. that result omit `--quiet`.
`--report PATH` additionally renders the same native result as fixed JSON via `--report PATH` additionally renders `commandkit.Report{Value: result}` through
`commandkit.RenderReport`, regardless of the main format or stdout destination. `commandkit.RenderReport`, without the main report's artifact-only text template.
This application-owned file write still occurs in quiet mode. Omitting the flag Optional `--report-format text|yaml|json` selects its native format independently
of stdout's `--format`. When omitted, pass the zero `commandkit.Format` value
(`""`) for native automatic rendering. Do not substitute JSON, YAML, or text.
CommandKit currently renders structured values as YAML in automatic and text
modes; future upstream defaults flow through without application policy changes.
Filename extensions do not select the format.
This application-owned file write still occurs in quiet mode. Omitting `--report`
creates no separate report. Use ordinary create/truncate operations (mode 0644 creates no separate report. Use ordinary create/truncate operations (mode 0644
for newly created files, subject to umask), preserve normal OS behavior for for newly created files, subject to umask), preserve normal OS behavior for
existing files, and handle render/write/close errors. Do not create parent existing files, and handle render/write/close errors. Do not create parent
@@ -349,7 +357,7 @@ helpers in `internal/adapter/cli`, and run integration tests.
7. Build the native result report/text template and submit stdout results 7. Build the native result report/text template and submit stdout results
through the invocation reporter. Keep any retained generation summary as a through the invocation reporter. Keep any retained generation summary as a
native Info message, normally hidden at ordinary verbosity. Stage 6 adds native Info message, normally hidden at ordinary verbosity. Stage 6 adds
the separate JSON report without changing the result shape. the separate native report without changing the result shape.
**Evidence:** real-engine/controlled-provider tests exercise the assembled **Evidence:** real-engine/controlled-provider tests exercise the assembled
positional command, preferred aliases, literal/file precedence, comma-containing positional command, preferred aliases, literal/file precedence, comma-containing
@@ -359,18 +367,22 @@ serialization, completed invalid content/status 4, and resultless operational
failure/status 1. Include a small typed-error mapping table rather than parsing failure/status 1. Include a small typed-error mapping table rather than parsing
error prose. Use native config classification for representative load failures. error prose. Use native config classification for representative load failures.
## Stage 6: Add Separate Native JSON Report Delivery ## Stage 6: Add Separate Native Report Delivery
**Primary files:** CLI generation-delivery helpers and focused delivery tests. **Primary files:** CLI generation-delivery helpers and focused delivery tests.
1. Add `--report` as a presence-aware application file flag with path completion. 1. Add `--report` as a presence-aware application file flag with path completion.
Reject an explicitly empty path before executing the model. Do not create Add optional `--report-format` with typed binding and completion for the native
files during validation or add a main-output destination flag. `text`, `yaml`, and `json` formats; leave its default unset. Reject an explicitly
2. Implement the settled delivery rules using the same report as Stage 5. empty path or invalid format as usage errors before executing the model. Do
Main stdout uses `Invocation.Reporter().Report`; the separate report uses not create files during validation or add a main-output destination flag.
`RenderReport(..., FormatJSON)`. The invocation reporter owns main format 2. Main stdout uses Stage 5's report through `Invocation.Reporter().Report`.
selection, including automatic artifact-template output. No selected-format Render the separate report through `RenderReport` with
accessor is needed. Extensions never select formats. `commandkit.Report{Value: result}`, sharing the native result but no text
template. Pass the empty format when `--report-format` is omitted;
otherwise pass the explicitly selected native format. The invocation reporter
owns main format selection, including automatic artifact-template output.
No selected-format accessor is needed. Extensions never select formats.
3. Keep the report file effect independent of console quiet selection. Do not 3. Keep the report file effect independent of console quiet selection. Do not
temporarily change reporting mode, rerun `App.Execute`, capture stdout and temporarily change reporting mode, rerun `App.Execute`, capture stdout and
parse it back, or replace native structured serialization to obtain the report. parse it back, or replace native structured serialization to obtain the report.
@@ -385,12 +397,15 @@ error prose. Use native config classification for representative load failures.
native run result is eligible for the new CLI report file. native run result is eligible for the new CLI report file.
**Evidence:** cover default/text raw bytes and JSON/YAML through stdout, including **Evidence:** cover default/text raw bytes and JSON/YAML through stdout, including
a subprocess whose stdout is redirected to a file. Check separate JSON report a subprocess whose stdout is redirected to a file. Compare omitted and explicit
equivalence to inline native JSON and independence from the main format. Quiet report formats against native `RenderReport` of the same value without a template;
suppresses main output, including redirected stdout, while an explicitly requested use an empty format for the omission expectation, not hard-coded YAML. Confirm
report is still written. Test empty report paths, replacement without parent the separate report retains the full result and metadata, independently of the
creation, stdout failure followed by report attempt, report failure after main format and filename extension. Quiet suppresses main output, including
successful main delivery, and an unchanged existing report on a resultless redirected stdout, while an explicitly requested report is still written. Test
empty report paths and invalid formats before model execution, replacement
without parent creation, stdout failure followed by report attempt, report failure
after successful main delivery, and an unchanged existing report on a resultless
operational failure. Use temporary files and a small failing-writer seam for operational failure. Use temporary files and a small failing-writer seam for
stdout/render failures. Confirm status 4 survives a delivery failure while its stdout/render failures. Confirm status 4 survives a delivery failure while its
explanation identifies the failure. No exhaustive filesystem matrix. explanation identifies the failure. No exhaustive filesystem matrix.
@@ -456,7 +471,7 @@ version output. Test-local fallback deadlines prevent regressions from hanging.
| Owner | Required changes | | Owner | Required changes |
| --- | --- | | --- | --- |
| `docs/cli.md` | New command tree, applicable flags/aliases, native selection/reporting, stdout redirection and separate report-file semantics, statuses, completion, explicit smoke tests. Link upstream for standard family details. | | `docs/cli.md` | New command tree, applicable flags/aliases, native selection/reporting, stdout redirection, separate report files with optional format selection, statuses, completion, explicit smoke tests. Link upstream for standard family details. |
| `docs/config.md` | Removed render default; native overlay-name selection; application overrides and loader authority; app/framework scopes and native comparison. Preserve collection/source/default semantics. | | `docs/config.md` | Removed render default; native overlay-name selection; application overrides and loader authority; app/framework scopes and native comparison. Preserve collection/source/default semantics. |
| `docs/integrations/subprocess.md` | Artifact-first execution, native reports and result availability, content-bearing metadata reports, stream capture, partial delivery, cancellation, and caller responsibility. Link exact CLI syntax rather than duplicate it. | | `docs/integrations/subprocess.md` | Artifact-first execution, native reports and result availability, content-bearing metadata reports, stream capture, partial delivery, cancellation, and caller responsibility. Link exact CLI syntax rather than duplicate it. |
| `docs/operations.md` | Updated discovery/preview/execution workflow, explicit smoke-test effects, potentially sensitive reports, file retention, server process lifecycle, unchanged HTTP limits. | | `docs/operations.md` | Updated discovery/preview/execution workflow, explicit smoke-test effects, potentially sensitive reports, file retention, server process lifecycle, unchanged HTTP limits. |
@@ -468,9 +483,10 @@ version output. Test-local fallback deadlines prevent regressions from hanging.
CLI/consumer overview. Cover positional prompt selection; `render` and CLI/consumer overview. Cover positional prompt selection; `render` and
`inspect` replacements; file versus literal inputs; repeated mappings without `inspect` replacements; file versus literal inputs; repeated mappings without
comma splitting; `--overlay` names; endpoint and timeout spelling/units; comma splitting; `--overlay` names; endpoint and timeout spelling/units;
stdout redirection in place of output-file flags; separate JSON reports, stdout redirection in place of output-file flags; separate native reports,
native formats, and result availability; quiet behavior (omit quiet to capture optional report-format selection and upstream automatic rendering, and result
main output); status changes; and content-bearing native inspection/reports. availability; quiet behavior (omit quiet to capture main output); status
changes; and content-bearing native inspection/reports.
This prepares release migration information without selecting a version, This prepares release migration information without selecting a version,
editing published notes, tagging, pushing, or adding release gates. Eventual editing published notes, tagging, pushing, or adding release gates. Eventual
release notes must summarize the breaking change and link the guide under release notes must summarize the breaking change and link the guide under
@@ -509,9 +525,10 @@ current documentation links resolve. Migration examples match the built CLI.
provider availability. Confirm retired commands/flags are absent or rejected provider availability. Confirm retired commands/flags are absent or rejected
except where a spelling now intentionally has its native meaning (`--input`). except where a spelling now intentionally has its native meaning (`--input`).
4. Review the output/outcome matrix across stdout, caller redirection, report 4. Review the output/outcome matrix across stdout, caller redirection, report
files, quiet, completed validation failure, operational failures, presentation files with automatic/explicit formats, quiet, completed validation failure,
errors, cancellation, and partial delivery. Verify reports are native values operational failures, presentation errors, cancellation, and partial delivery.
with no Scriptorium wrapper, and HTTP's distinct contract remains unchanged. Verify reports are native values with no Scriptorium wrapper, and HTTP's
distinct contract remains unchanged.
5. Run the repository baseline on the selected toolchain: 5. Run the repository baseline on the selected toolchain:
```sh ```sh