From d97e2b8e307b113f373c6643ccb8c4f8208a3abb Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Thu, 19 Nov 2026 15:21:14 +0000 Subject: [PATCH] Remove completed PromptKit planning documents --- docs/roadmap/implementation.md | 379 --------------------- docs/roadmap/promptkit-v0.23.2-adoption.md | 260 -------------- docs/roadmap/remediation.md | 151 -------- 3 files changed, 790 deletions(-) delete mode 100644 docs/roadmap/implementation.md delete mode 100644 docs/roadmap/promptkit-v0.23.2-adoption.md delete mode 100644 docs/roadmap/remediation.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index daafe8f..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,379 +0,0 @@ -# Implement the PromptKit v0.23.2 adoption - -The [feature roadmap](promptkit-v0.23.2-adoption.md) owns the target and scope. -Start from `247b67312ef1a0552a898c558dfe4639572b0691` plus the planning -commit. Follow the [development guide](../development.md), the applicable -repository policies, and [ADR 0004](../adr/0004-definition-boundary.md) and -[ADR 0005](../adr/0005-configkit-boundary.md). Use the pinned upstream -references in the feature roadmap, not either sibling checkout's moving HEAD. - -For final verification, the selected PromptKit release is v0.23.3 at commit -`804a4d32b23aa2e7973400bb0a322a7016574c9f`. The maintainer approved this -patch update after v0.23.2 exposed an in-flight HTTP timeout attribution bug; -it supersedes the earlier PromptKit pin below. ConfigKit remains v1.2.1. - -Execute these stages in order as one sprint. Intermediate stages may leave -unresolved imports, outdated fixtures, or failing tests that a later stage -explicitly addresses. Do not add shims or split the migration into separately -buildable products. Focused checks are useful when their dependencies are -ready; the completed tree is what must pass all validation. Keep the change -to the existing product and the collection discovery example and workflow. -Do not add deferred commands, endpoints, CommandKit dependencies or -scaffolding, or conversation and coordination features. - -## Stage 1: Unify dependencies and framework source wiring - -**Status: Complete.** - -1. Update `go.mod` to require `go.maximumdirect.net/promptkit v0.23.2` and - `go.maximumdirect.net/configkit v1.2.1`. Remove the old PromptKit and - ConfigKit requirements and both old catalog requirements. Replace Go - imports throughout production code and tests with the public vanity paths. - Retain Scriptorium's module path, Go 1.27.0 minimum, and Go 1.27.4 release - toolchain. Use ordinary Go module commands with `GOWORK=off` to generate - checksums and select stable `go.yaml.in/yaml/v4 v4.0.0`; complete - `go mod tidy` once code and tests have been converted. Use published - modules, with no committed replacements, workspace, or vendor tree. -2. In `internal/adapter/cli/run.go`, convert `newEngine` to - `configkit.Input` and `configkit.Underlay`. Keep one underlay per - `promptkit_sources` selection in resolved order. A file contributes - `configkit.Input{Source: configkit.File(path), Imports: true}`. For a - directory, call `promptkit.DirectoryInputs(path)` and pass the inputs as - one strict-composition group with the helper's settings unchanged. - Keep `Sources.Main` unset; neither the application YAML nor the last - selector is implicitly a framework main. Pass through schema directory, - engine retry defaults, and injected options as before. Leave the upstream - OpenRouter bundle enabled; do not supply local copies or aliases. -3. In `application_config.go` and `framework_sources.go`, retire the - old-path diagnostic branches. Use `errors.As` for - `*configkit.LoadError` and `configkit.WriteDiagnostics` for typed errors - and both warning paths. Preserve the constructor's wrapped cause and - originating command context; use a safe fallback for other errors. Keep - the distinct application and engine load operations and their warning - timing. Stage 3 handles the inspection-specific path and record types. -4. Keep the current native application schema and overrides in - `internal/config`: ordered main discovery, explicit file selection, - imports, overlay, collection policy, working-directory-relative selector - paths, presence, and schema/domain validation. No field rename or - second schema is needed: the obsolete formats are removed, while fields - already matching the target keep their meaning. Application inspection - continues to use one ConfigKit result without opening framework sources. -5. Retarget the existing dependency guard in - `internal/adapter/dependency_test.go` and its fixtures to the vanity - public/internal boundary. Keep the existing protections against direct - catalog imports and restored framework packages. The final module audit, - rather than matching historical prose or test fixtures, checks that no old - module remains selected. - -## Stage 2: Migrate CLI execution, rendering, and terminal reports - -**Status: Complete.** - -1. In `internal/adapter/cli/run.go` remove `--prompt-version`, the stored - value, and associated usage text from `run` and `render`. Remove the - version from `buildRunRequestFromConfig` and the run summary. Forward - exact prompt and profile IDs, including namespaced IDs, without splitting, - qualifying, or otherwise rewriting them. Preserve existing unrelated - flags, explicit numeric zero, reasoning effort presence, session IDs, - transient retry presence, optional inputs, and profile defaults. -2. Keep `run` on `Engine.Run` and `render` on `Engine.Prepare`. For - `run`, keep parsing and nonnegative validation of the Go duration - `--run-timeout`. Put that duration in `RunRequest.OperationTimeout` - before invoking the runner and remove the adapter-created timeout - context. Pass the caller context (`context.Background()` in the current - CLI). The engine owns the budget for preparation, admission, retries, - generation, and validation; parsing, construction, and file writes stay - outside. Do not put the budget on `render` or change the per-attempt - `--timeout` setting. -3. For a failed CLI run use `errors.Is(err, promptkit.ErrOperationTimeout)` - to select the existing safe deadline diagnostic and exit 1. Do not use - a bare deadline cause as proof the whole operation timed out: provider - failures can carry one. Keep completed output, exit 2 for completed - invalid output, other runtime failures at exit 1, and the existing - artifact/report write ordering and failure handling. Do not override a - returned successful result based on a later context check. No local timer - is needed: the pinned upstream API attributes its own expiration and - fixes the outcome before terminal observation. -4. In `internal/format/prepared_run.go` keep native prepared JSON and - remove the version line from text. In `internal/report/report.go` - remove the version field and projection from the terminal report shared by - CLI and HTTP; do not add ancestry or content fields. Keep usage presence, - outcomes, retry/attempt counters, duration, and opt-in report behavior. - Use PromptKit's terminal event rather than reclassifying or synthesizing it. -5. Adapt the existing CLI, preparation-format, and report tests to assert - ID-only mapping, rejection of the removed flag, no version in current - output, explicit zero presence, deadline vs provider failure, and unchanged - report rules. Reuse existing coverage of output files and status handling. - Migrate inline framework fixtures in Stage 5. - -## Stage 3: Adapt existing inspection commands in place - -**Status: Complete.** - -1. In `internal/adapter/cli/inspect.go` call - `Engine.InspectPrompt(ctx, id)`. Remove the version flag, config member, - usage text, and output field. Leave the command families, format flags, - exit behavior, and full-source engine construction in place. Do not - introduce list, requirement, validation, or diagnostic commands. -2. In `internal/format/inspection.go`, keep the current narrow prompt - projection and output names other than the retired version. Map - `PromptID`, `PromptHash`, `DefaultProfileID`, declared `Inputs`, - and the new `PromptOutputInspection` fields. Map optional fields as - optional: omit absent metadata in JSON/text rather than inventing an - empty value, while showing present empty strings, zero/false values, or - an empty input list. Do not emit captured message text, description or - session fields that were not previously projected. -3. Map `ProfileInspection` directly into the existing profile formatter: - retain `profile_id` and the separate optional `api_key_required` field. - Rename the existing nested `effective_model_params` section to - `profile_model_params` in JSON and text to avoid implying completion. - Keep the existing member names in that section, mapping upstream - `Backend` to `backend_id`. Use the upstream optional pointers; omit - nil members and show present zeros, empty strings, or an empty provider - map. Do not fill missing values from the backend or perform preparation. - No new renderer, format, or extra settings projection is needed. -4. In `framework_config.go`, use `configkit.ParsePath`, - `configkit.Path`, `configkit.SourceInfo`, and the current - `Layer`, `SourceKind`, and import location fields. Keep the small - current source DTO, its field names, and captured order; pass through - native layer values without translating them to the old generation. - Keep the native effective and explanation writers and their redaction. - Validate constructs the engine; show, sources, and explain use its - captured view. For list indexes use paths such as - `/collections/support/profiles/0/model`, not paths keyed by profile ID. - Keep existing explicit-root, invalid, and unknown pointer handling. -5. Adapt only affected tests: prompt/profile optional values, lack of - message disclosure, absence of backend completion in profile inspection, - and framework captured paths, diagnostics, and existing redaction. Do - not expand the application `config` projection or add blanket redaction - for provider parameters. Leave the CommandKit transition for later. - -## Stage 4: Refactor the single-run HTTP API - -**Status: Complete.** - -1. Keep `POST /v1/runs` as the single route: the operation is still a run - and there is no parallel version to distinguish. This is a direct - contract replacement at that path, not a promise of old-client - compatibility. Update `internal/adapter/http/dto.go` and `handler.go` - rather than layering a second handler. Drop request `prompt_version`. - Retain the other current request members and their types: `prompt_id`, - `profile_id`, `session_id`, `inputs`, `vars`, `model`, - `include_raw_output`, `include_execution_report`, - `transient_retries`, and `run_timeout_seconds`. Keep input references - and model overrides in their existing nested shapes. Strictly decode a - single request, so the removed selector and raw `api_key` are unknown - fields. Leave prompt/definition validation and input necessity to - PromptKit after transport validation; do not require a profile when the - prompt defaults one or trim nonblank IDs. -2. Return the existing `artifact`, `validation`, and `metadata` - envelopes on completed runs (including failed output validation) at - 200. Keep artifact body as text, not the Go library's base64 artifact - JSON. Retain opt-in `raw_model_output` and opt-in terminal - `execution` on applicable responses. In metadata omit - `prompt_version` and eliminate redundant `model_name`, `endpoint`, - `validation_mode`, `validation_status`, and `repair_attempts_used`: - consume these values from `metadata.model_params` or `validation`. - Keep `metadata.selected_backend_id` as the backend identity and drop - the duplicate `metadata.model_params.backend_id`; upstream guarantees - equality for real results and endpoint-only profiles have no backend ID. - Retain remaining DTO keys, types, timing/usage, and optional fields - for this cutover. Keep the error envelope `{error:{code,message}}` - with the existing optional terminal report where applicable. Do not - serialize raw PromptKit requests/results, causes, or resolved secrets. -3. Map validated `run_timeout_seconds` to - `RunRequest.OperationTimeout` and invoke the shared runner with - `r.Context()`, not a second timeout context. Retain int64 range and - duration-overflow checks, absence/null/zero as no new deadline, and - request/response IO outside the budget. Preserve the current status/code - mapping in the baseline [API table](../api.md#error-responses) as the - starting contract (including 400 for malformed inputs, 404 for missing - IDs and route, 405 for wrong method, 413 for size limits, 500 for - operational validation/unclassified errors, 502 for generation failure, - and 503 for capacity). Resolve these cases explicitly: - - `ErrOperationTimeout`, or a failed run wrapping - `context.DeadlineExceeded` when the parent context itself expired: - 504 `deadline_exceeded`. - - A failed run wrapping `context.Canceled` when the parent context is - canceled: 408 `request_canceled`. Check the typed operation timeout - first; do not replace a completed outcome based on a later context - race. A provider-only timeout remains 502 while the parent is live. - - Specific selection, credential, and restricted-reader causes precede - broader PromptKit fallback categories. Use errors.Is/errors.As and - safe messages, not provider status passthrough or string matching. -4. Keep the one engine created by `serve`, request-local reports, current - body and complete encoded response limits, and compact 413 overflow - fallback. Keep file-root requirement, lexical containment with existing - symlink behavior, configured artifact limit, and inline inputs. - Verify that `artifact_reader.go` returns detached mutable body - storage and retains no aliases that can mutate a returned artifact; - fix only if needed by the ownership handoff. Do not implement a new - filesystem threat model or rebuild backend pools per request. -5. Update `handler_test.go`, `artifact_reader_test.go` as needed, and - `examples/http-run.json`. Exercise removed selector/duplicate output - fields, presence-sensitive mapping, safe errors, namespace selection, - limits, and unchanged file policy. Adapt the retry-wait and capacity-wait - deadline tests to exercise a real PromptKit engine and verify typed budget - attribution and capacity release. Use controlled offline doubles or - loopback, with no real keys or paid calls. Use representative canceled - parent and provider-only deadline cases; a fake runner alone is not - evidence that the engine honors a mapped operation budget. - -## Stage 5: Migrate maintained definitions and existing test fixtures - -**Status: Complete.** - -1. Convert selected files under `examples/prompts/`, - `examples/profiles/`, and `examples/framework/` and inline or - on-disk framework fixtures across CLI and HTTP tests. Replace keyed - backends, profiles, and version-keyed prompts with lists of `id` - records at their existing resource destinations. Keep the repository's - existing prompt IDs where there is one intended record. For tests with - distinct formerly version-selected prompts, choose distinct IDs only - when both definitions still serve a test; delete expectations about - version ambiguity. Keep role/cache, inputs, schemas, and validation - behavior as appropriate. Leave source-local message files relative to - their owning definition file. -2. Replace framework `configkit:` controls with native `config:`, - express required imports with `required: true`, and use optional - source header `version: 2` if one is needed. Replace resource - inheritance with `extends: {id: }`. Review overlays for - whole-record replacement at the same ID and replace obsolete catalog - references with an intended current bundle profile or a complete local - profile. Prefer already selected local profiles in offline fixtures; - do not infer alias mappings or a default model. Check meaningful null - endpoint/credential/map cases against new backend fallback semantics - and keep schemas self-contained. Preserve application composition and - collection selector examples (`examples/config.collections.yml` - demonstrates app source selector composition, not checkout discovery). -3. Update scripts such as `examples/render-v0.9-features.sh` and active - sample invocations to drop version selectors. Adapt remaining native - framework fixtures and assertions from the previous stages. Retain - obsolete YAML only as deliberate negative or historical material. - Add a representative adapter-boundary test for an obsolete version-keyed - definition being rejected by upstream, without matching upstream prose. - Check a valid current definition reaches preparation/execution without a - Scriptorium field filter; do not copy PromptKit's format test matrix. - -## Stage 6: Demonstrate discovery of independent collection checkouts - -**Status: Complete.** - -1. Add `examples/config.discovery.yml` selecting - `./examples/discovery/framework.yml` as a file under - `promptkit_sources`. The framework file declares two **optional** - `config.imports` globs: `collections/*/collection.yml` and - `collections/*/collection.yaml`. Omit `required: true` on each. - These paths are relative to that file; `*` selects one checkout level - in this example, not a new Scriptorium layout restriction. Leave - discovery in native ConfigKit declarations: no Python or Go scanner, - implicit path search, special per-collection registration, or auto-mount - by checkout name. -2. Under `examples/discovery/collections/`, add `support-repo` with - `collection.yml` declaring `collections.support` and a separate - `review-repo` with `collection.yaml` declaring `collections.review`. - Give support at least `base` and `fast` profiles and `answer` and - `summarize` prompts, each a separate YAML record file with an `id`. - The entry file uses explicit relative `config.imports` with - `placement: sequence-item`, destinations - `[collections, support, profiles]` or - `[collections, support, prompts]`, and `required: true` for its own - named required fragments. Use `fast` extending `base`, a synthetic - endpoint/model and `default_profile: support/fast`. Give review its - own profile and prompt (reuse a local ID to demonstrate namespace - separation if useful). Have `support/answer` read a message file - beside its own definition and use the named input `transcript` with - `examples/fixtures/transcript.md`. Include unrelated YAML that would be - unsuitable as a framework input and is not selected by the imports. -3. Run the maintained config through `inspect configuration validate` - and `render --prompt support/answer --profile support/fast` with - `--input transcript=./examples/fixtures/transcript.md`; render a review - prompt too. Add a focused real-engine CLI - test using a scratch checkout tree with the same discovery declarations: - each spelling by itself works, both work together, and placing a - second checkout after one invocation makes it selectable on a new - invocation without editing config. Assert qualified selection and - rendered content from the source-local message/inherited profile. - Confirm unrelated YAML is ignored by successfully loading and preparing - the real fixture. Test through the CLI command/engine boundary, not - through a mock glob implementation. Do not promise hot reload of a - running server. - -## Stage 7: Update canonical docs and component notices - -**Status: Complete.** - -1. Put before/after application and framework examples in - [upgrading-promptkit.md](../consumers/upgrading-promptkit.md). Explain - that unchanged app fields still work through the one native schema, - how to rewrite the selected framework files, how to replace old - version selectors and catalog IDs, whole-record replacement, null - fallback, and self-contained schemas. Do not add HTTP migration - instructions. Describe the final contract in [api.md](../api.md) - including the removal of redundant metadata, deadline/status distinctions, - and updated request example. Keep the HTTP example current. -2. In [config.md](../config.md) explain native discovery selections and - both optional filename globs with a link to the maintained example. - In [operations.md](../operations.md) describe checkout placement and - independent updates; in [cli.md](../cli.md) demonstrate the discovery - example and qualified rendering. Use the actual Stage 6 IDs and input - path. Update CLI flags, inspection field semantics and list-index - explanation examples, render/report details, and subprocess guidance - as affected. Do not describe deferred CommandKit behavior as current. -3. Refresh README where affected, `docs/development.md`, - `docs/internal/{overview,adapters,sources}.md`, - the Scriptorium-specific dependency text in - `docs/policy/architecture.md`, and affected current consumer guides. - Use the new public module links and one ConfigKit generation with - two schemas; distinguish changed implementation from existing policy. - Leave historical release notes, ADR decisions, and completed roadmaps - intact. Do not select, tag, or publish a Scriptorium release here. -4. Under the [OSS policy](../policy/oss.md), update - `docs/oss/attribution.md` for PromptKit and a single ConfigKit entry - at the exact selected commits from the feature roadmap. Remove obsolete - catalog and legacy ConfigKit entries. Inspect the final runtime - dependency list and upstream notice files; expect jsonschema v6.0.3, - x/text v0.44.0, and go-yaml v4.0.0. Use the selected versions' - copyright and license notices and commit-pinned links (the selected - PromptKit attribution is a starting point to verify). Remove stale - YAML v3 attribution if no runtime or incorporated material uses it. - Keep applicable verbatim local license texts and companion links. - Distinguish tool-only graph entries such as upstream ReleaseKit and its - tooling dependencies from Scriptorium's runtime. This is attribution - maintenance, not a project license or release procedure change. - -## Stage 8: Verify the complete cutover - -**Status: Complete.** - -1. Use the selected Go 1.27.4 toolchain. Finish `GOWORK=off go mod tidy` - after all imports and fixtures are in place and confirm - `GOWORK=off go mod tidy -diff` is empty. Inspect `go list -m all` for - the pinned modules, stable YAML, and absence of both legacy modules - and the catalogs. Use `go list -deps ./cmd/scriptorium` to distinguish - linked packages from graph-only tooling. Confirm no replacements, - workspaces, vendored code, or dependency on unpublished sources. -2. Run the [baseline Go checks](../development.md#baseline-validation) - using prepared dependencies and the deterministic offline test suite: - `go test ./...`, `go test -race ./...`, `go vet ./...`, and - `go build ./cmd/scriptorium`. Put binaries outside the repository. - Build the two existing Linux release targets (amd64 and arm64, - `CGO_ENABLED=0`, `-trimpath`, `-ldflags '-s -w'`) without running - publication. Check `gofmt` and `git diff --check`. -3. Use a freshly built binary for maintained config and render commands, - including simple, full, composed with its local overlay, inherited, - application source-list collection/overlay, and discovered collections. - Exercise app inspection without a framework load and framework validate, - show, sources, and explain with a real current path. Verify synthetic - data only, one appropriate output on stdout, and no provider calls for - render/inspection. Covered CLI/HTTP run tests should use a deterministic - fake provider or injected client. Check shell examples and affected - links, including pinned attribution targets. -4. Review the complete diff against the feature roadmap: one native load - per domain, ID-only execution, no legacy routes or translators, no - extra inspection surface, optional discovery for both spellings, - consistent disclosure and transport boundaries, and current docs. - Check for stale active references rather than rewriting history or - deleting deliberate rejection fixtures. Report verification results and - any material blockers; do not convert these sprint checks into new - permanent release gates. diff --git a/docs/roadmap/promptkit-v0.23.2-adoption.md b/docs/roadmap/promptkit-v0.23.2-adoption.md deleted file mode 100644 index d5b7dd1..0000000 --- a/docs/roadmap/promptkit-v0.23.2-adoption.md +++ /dev/null @@ -1,260 +0,0 @@ -# PromptKit v0.23.2 Adoption - -## Status And Scope - -Implemented. The original v0.23.2 target below was superseded for final -verification by PromptKit v0.23.3 at commit -`804a4d32b23aa2e7973400bb0a322a7016574c9f`, with the maintainer's approval, -to correct in-flight HTTP operation-timeout attribution. ConfigKit remains -v1.2.1. This roadmap upgrades the existing Scriptorium product in one native -cutover. Use the released PromptKit and ConfigKit APIs -for framework and configuration mechanisms that they already own. Provide -migration guidance for configuration, definitions, and CLI usage, not a -second application schema, a legacy definition reader, or compatibility -aliases. Refactor the early-beta HTTP API directly, without an HTTP migration -or backward-compatibility requirement. - -The [definition boundary](../adr/0004-definition-boundary.md) and -[configuration boundary](../adr/0005-configkit-boundary.md) remain in effect. -The existing CLI and HTTP surfaces should be able to use valid definitions for -the selected PromptKit release without copying its schema or maintaining a -narrower application-specific subset. This roadmap covers the changes to -existing surfaces needed for the migration and the collection-discovery model -below. New CLI capabilities, including -[Narratio's requirements inspection](narratio-feature-request.md), concrete -request validation, inventory, backend inspection, configuration diffs, and -smoke diagnostics, are deferred to a separate roadmap. It does not add HTTP -operations for those capabilities or introduce continuation, coordination, -prepared-handle workflows, or prepared-content capture. - -CommandKit adoption (`go.maximumdirect.net/commandkit`), including the -`ckconfig` and `ckprompt` integrations, is planned after this migration and -belongs to a separate roadmap. Its standardized operator interfaces are -expected to replace Scriptorium's current inspection interfaces. This roadmap -keeps those existing commands in maintenance-only scope and adds no -CommandKit dependencies or advance integration scaffolding. - -## Baseline And Target - -The reviewed Scriptorium baseline is -`1b1d967fc78e9c514a478b27debf1fadbc772283`. It uses -`gitea.maximumdirect.net/eric/promptkit` v0.12.0. Application configuration -already uses native ConfigKit v1.2.0, but framework inputs, inspection -records, and diagnostics still use old-path ConfigKit v0.5.0. The old -PromptKit module also selects two external catalog modules and go-yaml -v4.0.0-rc.6. Prompt versions are selectors in both executable interfaces and -appear in prepared, inspection, and execution output. The completed -[v0.12.0 roadmap](promptkit.md) records the prior migration. - -The target is `go.maximumdirect.net/promptkit` v0.23.2, commit -`382ccfb14b9630fd8f48c8364dbe2b6159637d71`, and -`go.maximumdirect.net/configkit` v1.2.1, commit -`51ef3144158db1a09fabdbd1c7ae59908c84923b`, with stable go-yaml v4.0.0. -The existing Go 1.27.0 consumer minimum and Go 1.27.4 release toolchain -already satisfy this selection. - -- [PromptKit release notes](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/382ccfb14b9630fd8f48c8364dbe2b6159637d71/docs/releases/v0.23.2.md) -- [PromptKit migration guide](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/382ccfb14b9630fd8f48c8364dbe2b6159637d71/docs/consumers/configuration-migration.md) -- [PromptKit framework formats](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/382ccfb14b9630fd8f48c8364dbe2b6159637d71/docs/formats.md) -- [PromptKit consumer API guide](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/382ccfb14b9630fd8f48c8364dbe2b6159637d71/docs/consumers/pkg-promptkit.md) -- [ConfigKit v1.2.1 release notes](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/releases/v1.2.1.md) - -## Targeted End State - -### One Configuration Mechanism, Separate Schemas - -ConfigKit resolves the application document; Scriptorium owns its schema, -defaults, domain rules, source selection, and override mapping. The app uses -one native schema and decoder. Existing fields can remain where they still -express the right application setting, without a separate legacy mode. -PromptKit loads its framework sources separately using the same released -ConfigKit types. Application-only configuration inspection still does not -construct a PromptKit engine or require framework resources. - -Scriptorium selects ordered framework files and directories and supplies -them through PromptKit's supported public APIs. The application config file -is not itself a framework source. Preserve the application selectors, -explicit override behavior, and source grouping where they already map -directly to upstream inputs. PromptKit and ConfigKit determine how those -inputs compose and how relative framework resources resolve. Scriptorium -does not infer namespaces, rewrite IDs, flatten source origins, or supply a -second merge policy. - -Adopt PromptKit's default OpenRouter bundle as its lowest-priority underlay. -Caller definitions remain able to replace bundled records by ordinary -composition. Scriptorium does not recreate the former catalogs, maintained -profile aliases, or a local implicit profile selection. Bundle availability -is not a readiness check or a promise of a fixed provider model. - -### Independently Maintained Collections - -Scriptorium provides a neutral application interface over PromptKit, including -CLI subprocess execution for consumers with limited prompting needs. One -installation may load dozens of collections, each maintained in a separate -Git repository with its own maintainer and update cadence. Operators can place -those checkouts beneath a selected directory and make their prompts available -through the same Scriptorium invocation model. - -Use ConfigKit imports in a selected framework source to discover collection -entry files. Establish `collection.yml` and `collection.yaml` as equivalent -Scriptorium conventions for the collection-root entry file, and select both -spellings through the installation's import declarations. When using a -separate glob for each spelling, leave each glob optional (omit -`required: true`) so either spelling can have zero matches. An installation -using only one spelling still discovers its collections. Each entry file -declares its own namespace under PromptKit's native `collections` mapping and -uses relative imports to assemble its profiles, prompts, and other definition -files. The checkout directory name does not determine the namespace; callers -select resources by their exact qualified IDs. - -The application selects the discovery source through its existing framework -source selection. ConfigKit owns entry-file discovery, relative imports, and -composition; PromptKit owns the resulting collection and resource semantics. -Load the repository entry files rather than treating every YAML file in each -checkout as an independent framework input. PromptKit's `DirectoryInputs` -remains useful for curated configuration trees where all YAML files are -intended contributions. Adding another checkout with a matching entry file -must make that collection available on the next CLI invocation without adding -a per-collection source registration or changing the discovery configuration. - -### Existing Commands And Execution - -Execution, rendering, and prompt inspection select a prompt by exact ID, -including a qualified collection ID where appropriate. Remove the CLI -version selector and the corresponding version members from the CLI and HTTP -request, response, inspection, and report contracts where they exist. Do not -accept a legacy version selector and then ignore it. - -Keep the existing `config` and `inspect` commands functional and accurate -against the selected upstream APIs. Limit inspection work to required import, -API, type, selector, and semantic adaptations. Do not expand capabilities, -redesign command syntax or report presentation, or build reusable inspection -abstractions for the later CommandKit adoption. Necessary output changes should -be the smallest accurate adaptation, with proportionate test and documentation -updates. - -Continue to distinguish framework configuration, resolved resources, and -request preparation. Framework configuration inspection reports the -upstream captured result, with native provenance, normalization, and -redaction where currently used. Profile inspection reports the resolved -profile, preserving unset versus explicitly supplied values, rather than -reconstructing backend-completed settings. Render shows request-completed -settings. Reuse existing native writers and make only required adjustments to -local inspection projections. Preserve existing disclosure boundaries; -captured prompt messages becoming available upstream is not reason to add them -automatically to existing prompt inspection output. Adoption of CommandKit's -report formats and disclosure behavior belongs to the later roadmap. - -PromptKit owns preparation, provider calls, retries, capacity, validation, -and repair. Preserve the configured shared server engine and established -backend admission behavior. Use PromptKit's native operation timeout for the -existing whole-run budget where it can preserve the execution boundary and -support the intended CLI and HTTP outcomes. A provider-attempt timeout is -still distinct from the whole-run deadline; caller cancellation still -applies. Keep the existing context wrapper if delegation would change those -semantics rather than introducing two independent timers. CLI process -behavior, HTTP decoding, limits, restricted artifact loading, safe -diagnostics, and application-owned execution reports remain adapter -responsibilities. - -### HTTP Contract - -Serve one ID-only HTTP execution contract. Choose the route, request and -response shapes, and status mapping that best fit the single-run model. -There is no need to preserve `POST /v1/runs` or to introduce a successor -route solely to signal incompatibility. No HTTP compatibility routes, -legacy payload adapters, or client migration procedures are needed. -Retain operational boundaries such as configured size limits, restricted -artifact access, credential handling, and shared backend admission; these -serve the application independently of wire compatibility. - -## Work Needed From The Baseline - -- **Converge the module graph and engine wiring.** Move runtime and test - imports to the vanity paths. Replace old ConfigKit input, path, provenance, - and diagnostic types at the CLI construction and inspection boundaries. - Remove the obsolete diagnostic branches and the legacy module and catalog - dependencies. Keep distinctions between application and framework error - presentation where they remain useful. Use public typed causes and sentinel - categories, including for construction, deadlines, and artifact failures. - Reconcile commit-pinned attribution and license notices against the - resulting runtime graph under the [OSS policy](../policy/oss.md), - distinguishing linked dependencies from upstream maintainer tooling. -- **Migrate maintained configuration and definitions in place.** Convert - version-keyed prompts and other keyed resource maps to explicit-ID lists; - choose one definition per ID or give separately selectable former versions - distinct IDs. Convert framework source controls to `config:`, write - `required: true` where a missing import is an error, and use version 2 only - where an optional framework source header is supplied. Use sibling - `extends: {id: ...}` and update exact references when using collections. - Review same-ID whole-record replacement, backend fallback from null - profile fields, self-contained schemas, and source-local message files. Retain - existing application discovery, overlays, and path semantics where still - appropriate; they do not need to change solely to make an old file fail. - Review maintained examples and real-engine test fixtures together. -- **Demonstrate collection discovery and composition.** Add complete, - copyable assets under `examples/` showing application selection of a - framework discovery file, ConfigKit imports matching both collection-root - filename spellings, and multiple independently namespaced collection - directories. Include a representative collection with multiple profile and - prompt definitions in separate files, relative entry-file imports, - sibling profile inheritance, and qualified prompt-to-profile references. - Supply synthetic input fixtures and offline CLI rendering examples that - exercise the discovery-to-selection flow. Keep the example collections in - the repository so validation does not require cloning external repositories. -- **Adapt the existing interfaces.** Update CLI argument mapping, - formatters, captured framework source records and list-index explanation - paths, HTTP DTOs, and terminal report projections. Inspection changes are - limited to the maintenance-only adaptations above; inspection formatter - cleanup, presentation improvements, and CommandKit command standardization - are outside this migration. Keep explicit zero overrides and optional-value - presence. Check that the restricted HTTP - artifact reader transfers ownership of returned artifacts and bytes under - the new public contract without changing Scriptorium's containment policy. - Preserve established CLI outcomes outside the intentional cutover changes. - Validate the HTTP adapter against its new contract and the application - boundaries above, not against old wire shapes. -- **Give operators a migration path in documentation.** Update the - [consumer migration guide](../consumers/upgrading-promptkit.md) with - before-and-after application and framework examples, directions for old - prompt versions and catalog references, and the changed CLI contract. - Document the collection entry-file convention and discovery configuration - in the [configuration reference](../config.md), with links to the maintained - examples. Explain the checkout layout and independently managed updates in - [operations](../operations.md), and demonstrate qualified prompt/profile - selection and offline rendering in the [CLI reference](../cli.md). - Distinguish collection-owned namespaces from checkout directory names and - explain how entry-file imports select the collection's internal resources. - Explain semantic changes that can survive a syntactic conversion, - particularly replacement, null fallback, and schema - resolution. Document the new HTTP contract directly in the - [API reference](../api.md) and update its maintained examples. Update the - canonical configuration, CLI, subprocess, and internal references where - their implementation changes. For existing inspection commands, correct - documentation affected by required API and semantic changes without expanding - their operator workflows or documenting future CommandKit behavior as current. - Refresh current contributor and architecture references to the selected - upstream release, - leaving historical release records intact. Record release impact as - an intentional incompatible change under the - [release policy](../policy/release.md#versioning-and-compatibility); this - roadmap does not select or publish a release. -- **Verify the migrated boundary, not a second upstream test suite.** Use - the repository's baseline Go checks and representative offline examples - and real-engine adapter tests. Exercise native source composition, - qualified selection, source-local resources, captured inspection, - render and execution, and representative failures for obsolete formats - and selectors. Verify collection discovery with either entry-file spelling - alone and with both spellings across multiple checkouts. Verify relative - imports and discovery of an added collection without editing the - installation's source configuration. Include unrelated YAML outside the - authored import selections to verify that entry-file discovery does not - treat an entire repository as framework configuration. - Update affected inspection tests to cover required adaptations, accurate - optional-value handling, and preserved disclosure boundaries. Reuse existing - coverage and keep new inspection checks focused on migration regressions; - do not build a broader report or command suite ahead of CommandKit adoption. - Protect behavior likely to regress in this cutover: - explicit zero overrides, shared backend admission, timeout attribution, - safe output, and unchanged HTTP artifact policy. No live inference is - needed to validate the migration. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 96f5988..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,151 +0,0 @@ -# PromptKit Adoption Review Findings - -Review baseline: Scriptorium `c5e54edaa5f6621d504cf510b4b5b5c1e4d1cfd4`, -with the approved PromptKit v0.23.3 pin and ConfigKit v1.2.1. Stage numbers -refer to the [implementation plan](implementation.md). - -## R1: Bound the HTTP timeout regression tests (Stage 4) - -**Complete.** The tests use independent parent deadlines and capture the -target invocation's error without sharing a channel. All three passed -twice under the race detector. - -In [handler_test.go](../../internal/adapter/http/handler_test.go), -`TestHandlerDeadlineDuringBuiltInGeneration` runs the handler synchronously -while the fake provider waits on a channel that is released only after the -handler returns. If the operation timeout stops reaching PromptKit, the test -can wait for the unrelated provider/transport timeout or the suite watchdog -instead of promptly failing. The retry- and capacity-wait tests likewise rely -on the behavior being tested to return control. The added -`observeRunnerFailures` helper also sends all failures to a shared, -one-slot channel before allowing each runner invocation to return. An -unexpected failure of the first request in the capacity test can fill that -slot and block the second request from returning the failure the test needs -to inspect. - -Use a test-only fallback deadline or bounded synchronization independent of -the operation budget, and capture the error of the specific observed -invocation without a blocking send shared by concurrent requests. Keep the -real engine and loopback provider, the assertion on -`ErrOperationTimeout`, and the existing status, observation, and -capacity-release checks. This is a test liveness fix, not a new production -timeout or release gate. - -## R2: Describe source-relative message paths without a colocation rule (Stage 7) - -**Complete.** The migration guide describes source-relative resolution and -links to PromptKit's path rules. A render with a message in a subdirectory -confirmed that no colocation is required. - -The [upgrade guide](../consumers/upgrading-promptkit.md) instructs users to -keep message files beside their owning definitions. PromptKit resolves -`content_file` against the field's declaring source within its rooted -filesystem; it does not require the file to be in the same directory. A -freshly built Scriptorium binary rendered a valid `content_file: -./messages/request.md` without moving the file beside its framework YAML. -The current wording can lead operators to rearrange valid layouts during -migration. - -Describe source-relative resolution and link to the pinned PromptKit format -contract for containment rules. A colocated file can remain an example, not -a requirement. No loader or path-policy change is needed. - -## R3: Link the Go Text patent grant notice (Stage 7) - -**Complete.** The attribution entry links `PATENTS` at the selected Go Text -commit. The notice exists in the pinned module, and the local BSD text and -upstream `LICENSE` link are unchanged. - -The [attribution entry](../oss/attribution.md#golangorgxtext) for -`golang.org/x/text v0.44.0` links `LICENSE` but omits `PATENTS`. Build -dependency inspection confirms Go Text is part of the executable. Its -selected module contains the separate patent grant at upstream commit -`6070ac32abcdd1670599eeb3ee4d513c8184a513`; PromptKit's attribution for -that same revision links it. The [OSS policy](../policy/oss.md#license-texts-and-upstream-links) -calls for applicable companion texts to be directly reachable. - -Add a `PATENTS` link at the already selected Go Text commit. Retain the -verified local BSD text and existing `LICENSE` link. No separate local -patent notice or change to the project license is needed under the -adopted attribution format. - -## R4: Distinguish discovery peers from ordered framework layers (Stages 6–7) - -**Complete.** The migration guide and configuration reference distinguish -peer conflicts from ordered replacement. The same two definitions were -verified to conflict as peers and replace as separate selectors. - -The [upgrade guide](../consumers/upgrading-promptkit.md) says a later source -selecting the same ID replaces the whole record. That holds for a later -framework underlay, not for two checkout entries selected by the same -discovery file. Scriptorium turns each resolved `promptkit_sources` selector -into one underlay. The entries discovered by the imports of that selector -are composed as peers; their filename, glob, or checkout order is not -override precedence. - -The built binary rejected two discovered checkout entries defining -`shared/fast` with `compose.conflict`. Selecting the very same entry files -as two ordered `promptkit_sources` items instead succeeded: the later -record's model was selected and `max_tokens`, absent from that record, did -not carry forward from the earlier one. This is the intended distinction, but -the migration wording can lead operators to put an intended override in -a discovery peer, which prevents the engine from loading. - -Qualify whole-record replacement in the upgrade guide and explain in the -[checkout discovery reference](../config.md#collection-checkout-discovery) -that same-ID conflicts within one discovery underlay fail. For an intentional -override, select a separate, later framework source using the existing -application selector list. No scanner, de-duplication, new namespace -restriction, or change to composition is needed. - -## R5: Replay framework explanation history once (Stage 3) - -**Complete.** Framework inspection calls the native explanation writer -once. The framework inspection tests passed, including path errors, -redaction, and writer failures. - -[Framework configuration inspection](../../internal/adapter/cli/framework_config.go) -calls `view.Explain(path)` to check existence, discards the complete -explanation, then calls `view.WriteExplanation`. The pinned native writer -calls `Explain` itself. A successful request therefore replays the same -captured history and builds the same redacted value snapshots twice. If -`H` is the work to construct that explanation and `W` the work to render -it, the command does roughly `2H + W` work instead of `H + W`. This is a -constant-factor cost, not a new quadratic algorithm, but it repeats a -history- and value-sized operation and its allocations. The legacy wrapper -already used two calls; the native inspection path need not retain them. - -Delegate once to `view.WriteExplanation`, as app configuration inspection -already does with its native writer. A direct probe with the selected -PromptKit and ConfigKit releases confirmed that this writes a known path, -returns an error without output for an unknown path, and preserves writer -errors. Keep JSON Pointer parsing, native redaction, and the existing exit -and stream behavior; no new formatter or replay algorithm is needed. - -## R6: Prevent artifact read-limit overflow (Stage 4) - -**Complete.** The read cap no longer overflows at `math.MaxInt64`. -The new test failed before the fix and now verifies that a small file keeps -its full body and size at that limit. The existing size-limit tests, full -Go suite, race suite, vet, and build passed. - -[The restricted HTTP artifact reader](../../internal/adapter/http/artifact_reader.go) -allows the same nonnegative `int64` limit as application configuration -and the CLI. It computes `maxBytes + 1` for `io.LimitReader` to detect -oversized files. At the valid setting `9223372036854775807` -(`math.MaxInt64`), that addition wraps to a negative number. The limited -reader then immediately returns EOF, so a nonempty file is accepted as an -empty artifact. A loopback check confirmed HTTP 200 at this setting while -the outgoing prompt silently lost the file content. The same synthetic -request included the file at zero and at the ordinary positive limit. -The defect predates the migration and remains in the reader retained -by Stage 4. - -Use overflow-safe byte-limit arithmetic: for example, retain the capped -`limit + 1` read for smaller positive limits and skip the increment at -`math.MaxInt64`, or limit the read first and probe separately for an extra -byte. Accepted files should still require linear time and buffer space in -the bytes read; normal limits should continue to reject oversized files -without a full read. Add a boundary test using a small nonempty file. -Do not reduce the accepted configuration range or change containment -or the zero-limit semantics.