base-layer/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Prepare v0.17.0 notes and retire completed roadmaps

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-03 02:56:48 +00:00
parent 32547e1e29
commit e41b8b42f6
6 changed files with 72 additions and 727 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -43,6 +43,10 @@ boundary and constraints that framework work must preserve.
## Release Guidance
The [v0.17.0 release notes](docs/releases/v0.17.0.md) cover the vanity module
path, ConfigKit v1, identified resource lists, removal of prompt-version
metadata, and the Go 1.27 minimum. This is a breaking pre-v1 migration.
Consumers using the former Go module path should follow the
[module-path migration guidance](docs/consumers/configuration-migration.md#update-go-imports).

68
docs/releases/v0.17.0.md Normal file
View File

@@ -0,0 +1,68 @@
# Promptkit v0.17.0
This release adopts the vanity module path `go.maximumdirect.net/promptkit`
and ConfigKit v1.0.0, standardizes configuration resources as identified lists,
and removes descriptive prompt-version metadata. These are intentional breaking
changes before v1. The minimum supported Go version increases from 1.26.0 to
1.27.0 for the ConfigKit v1 migration.
## Breaking Changes And Upgrade
Consumers upgrading from v0.16.0 should follow the
[configuration and API migration guide](../consumers/configuration-migration.md):
- Change Promptkit imports from `gitea.maximumdirect.net/eric/promptkit` to
`go.maximumdirect.net/promptkit`. Update packages that exchange Promptkit
types together; old-path and vanity-path types are distinct.
- Change ConfigKit imports to `go.maximumdirect.net/configkit` at v1.0.0 and
adopt its renamed public inspection types and fields where used.
- Convert root and collection `prompts`, `profiles`, and `backends` from keyed
mappings to lists whose records contain an explicit `id`. Collections remain
keyed by namespace. Sibling inheritance now uses `extends: {id: parent}`;
later same-ID records replace lower records in full.
- Change `ProfileSource` and `BackendSource` calls to pass slices of
`ProfileConfig` and `BackendConfig`, setting each record's `ID`.
- Change source-root `configkit:` controls to `config:`. Imports are optional
unless marked `required: true`. Optional Promptkit specification headers now
use `version: 2` or `version: "2"`; headerless sources remain valid.
- Remove descriptive prompt `version` values and code consuming `PromptVersion`,
`PromptIdentity.Version`, or corresponding `prompt_version` output members.
Collection `release_version` labels remain available. Prompt hashes may change
across this migration.
After adapting Go imports, configuration, and API calls, use Go 1.27.0 or newer:
```sh
go get go.maximumdirect.net/promptkit@v0.17.0
go mod tidy
```
Earlier tags retain their original module declarations. No permanent local
replacement is needed for this cutover. See the
[framework format reference](../formats.md) for current configuration semantics
and the [maintained preparation example](../../examples/go-library/prepare/main.go)
for explicit composition, source-relative imports, and scoped discovery.
## Configuration Inspection And Execution
Configuration inspection and diffs expose ConfigKit's composed values and
provenance, including list-index paths. Promptkit applies domain normalization
and execution defaults to owned interpretations without replacing the source
origins needed for relative resources. Profile inspection and preparation expose
their interpreted or effective settings. See the
[migration guide](../consumers/configuration-migration.md#select-and-compose-the-configuration-you-need)
for the distinction and inspection-path updates.
## Maintainer Tooling
Release tooling now uses the module-selected ReleaseKit v0.8.0 through `go tool`,
with page-only Gitea delivery and saved-publication-attempt recovery. The
[release procedure](../release.md) describes publication and recovery.
The three maintained offline examples now have ordinary Go workflow tests,
included in the standard suite. Their separate Python runner and custom release
hook have been removed; Python is no longer a normal release prerequisite.
The optional release-wiring diagnostic remains Python. See the
[development guide](../development.md#maintainer-validation).
[COPYRIGHT](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.17.0/COPYRIGHT) | [LICENSE](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.17.0/LICENSE) | [ATTRIBUTION](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.17.0/docs/oss/attribution.md)

314
docs/roadmap/configkit.md
View File

@@ -1,314 +0,0 @@
# ConfigKit v1 Adoption And Identified Resource Collections
Status: complete. The implemented result includes the agreed execution-owned
normalization adjustment for ConfigKit v1.0.0. Current consumer documentation
remains authoritative for supported behavior.
The completed implementation plan is preserved in Git history at `79dfd97`:
`git show 79dfd97:docs/roadmap/implementation.md`.
## Purpose
Adopt `go.maximumdirect.net/configkit` v1.0.0 and simplify authoring by using
identified lists consistently for prompts, profiles, and backends. An individual
resource file should begin with its own `id`, without a resource-name wrapper.
Collection entries can import those files beneath the appropriate resource list.
ConfigKit's sibling inheritance supplies concise variants, especially for
profiles and backends.
Prefer separately named variants when changing an existing definition. This is
authoring guidance, not a prohibition on deliberate same-ID replacement across
configuration layers. Keep ConfigKit responsible for composition and inheritance;
PromptKit interprets the final effective configuration and owns resource meaning.
## Pre-Migration Baseline
Before this migration, the repository declared Go 1.26.0 and depended on
`gitea.maximumdirect.net/eric/ConfigKit` v0.8.0. The following describes that
historical baseline; the current
[format reference](../formats.md) and [source adapter overview](../internal/sources.md)
describe the migrated behavior:
- Root `prompts`, `profiles`, and `backends` are mappings keyed by local ID.
`collections` is a namespace-keyed mapping whose entries contain the same
resource mappings and an optional `release_version` label.
- `CollectionInput` mounts a selected source at `collections/<namespace>` and
enables its imports. Namespace views qualify convenient runtime names.
- ConfigKit composes inputs and resolves mapping-sibling `extends` before
PromptKit validates, captures content, and constructs runtime indexes.
- Source-root `version: 1` or `version: "1"` is optional. The prompt `version`
field is a separate descriptive label still carried through public values.
- Public configuration and inspection APIs expose ConfigKit types. The module
identity change therefore affects downstream Go callers as well as internals.
A prior exploratory migration in a temporary copy passed the existing test
suite and maintained consumer examples against the tagged ConfigKit v1.0.0
checkout after mechanical API and control-block updates. It retained resource
mappings; it does not validate the list migration planned here.
## Targeted End State
### Dependency And Public API Migration
Use `go.maximumdirect.net/configkit` v1.0.0 throughout production code, tests,
examples, and current consumer guidance, and raise the Go consumer minimum to
1.27.0. Update build and release-toolchain guidance accordingly. PromptKit's own
module identity is outside this migration's scope.
Update PromptKit's existing uses of:
| Current use | Replacement |
| --- | --- |
| `Definition.Secret` | `Definition.Sensitive` |
| `Resolution.InputInfo` | `Resolution.EntryInfo` |
Migrate recording API names where still used; PromptKit's normalization and
defaults follow the execution-owned interpretation described below rather than
requiring ConfigKit resolution records for fields within resource lists.
Audit the remaining exposed inspection types and diagnostic consumers against
the complete [ConfigKit v1.0.0 migration notes](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.0.0/docs/releases/v1.0.0.md).
For example, version evidence fields were renamed, the main provenance layer is
now `main` rather than `base`, and the history-limit code changed. Do not require
identical historical rendering or source ordering where ConfigKit deliberately
changed those contracts or corrected attribution.
Coordinate consumers such as Scriptorium that pass `Source`, `Input`, `Underlay`,
`Path`, or inspection values across PromptKit's boundary. Old-path and new-path
ConfigKit types are different Go types; changing only a dependency requirement
does not bridge them. The released dependency must use the new module directly,
without a local replacement or an old-path compatibility adapter.
### Source Controls And Optional Headers
Migrate genuine source-root `configkit:` controls to `config:` in main files,
collection entries, fragments, and programmatic inputs. Controls are extracted
from each acquired document before placement. Nested `config` values are ordinary
application data subject to the existing application schema; placement does not
trigger another control interpretation pass. Do not globally rename arbitrary
data or the unchanged `configkit:"secret"` Go tag.
Also account for the intervening v0.9 import changes: imports are optional by
default, `optional` has been removed, and authors use `required: true` when a
missing import should fail. Preserve those upstream choices rather than imposing
a new PromptKit-wide requiredness policy. Directory imports can select filename
patterns and recurse; filesystem globs do not acquire recursive `**` semantics.
Advance the PromptKit configuration specification to version 2 for the new
resource shape. Accept the exact string `"2"` or strict decimal integer `2` when
a source declares a header; keep the existing if-present checking approach and
permit headerless files. Do not require a declaration in every resource file or
infer a missing header from an enclosing source. The header describes the
PromptKit specification, independently of ConfigKit's software version.
Remove the unused descriptive prompt `version` field and its propagation through
definitions, inventories, inspection, results, and observations. Remove the
corresponding `prompt_version` output fields and stale documentation. Retain the
collection `release_version` label and optional source specification headers;
these have distinct purposes. In a standalone prompt item file, a root `version`
is now solely a specification header; do not carry the former prompt label into
that position. Audit hashes and prepared snapshots affected by
the removed field without promising hash equality across this format migration.
### Resource Shape And Identity
Keep `collections` as a mapping keyed by namespace. Change each of `prompts`,
`profiles`, and `backends` to a list of mappings, both at the root and within a
collection. Each record supplies its local `id` explicitly. Filenames remain
organizational choices, not identities.
Preserve the existing nonblank, slash-free namespace and local-ID rules, exact
spelling, qualified `namespace/id` runtime identity, namespace views, and exact
resource references. A provider's `model` string remains provider data and can
contain `/`. Do not derive or rewrite stored backend/profile references from
the source's mount point. The same local ID can occur in different namespaces
or resource kinds.
Register `id` through `Definition.SequenceIdentities` for each root resource list
and each `collections` / any mapping key / resource-list destination. Use
`SequenceIdentityAt` and `SequenceIdentityMatching` with
`SequenceAnyMappingKey`; no source pre-scan or namespace enumeration is needed.
Authors do not need to repeat YAML identity declarations. ConfigKit owns
structural and final identity checks; PromptKit's schema and validation retain
responsibility for ID syntax, allowed value shapes, and domain semantics.
Build runtime indexes from the final validated records. Keep effective inspection
in the assembled list shape, including its index-based paths and provenance.
Do not impose a new execution-order meaning on list position or change the
existing inventory sorting solely because source resources become lists.
### Composed Configuration And Execution Interpretation
Accommodate ConfigKit v1.0.0's atomic sequence-mutation auditing. Retain the
composed typed configuration, field presence, and provenance supplied by
ConfigKit. Apply PromptKit's domain normalization and defaults to owned
interpretations of those final records during engine construction, preserving
existing execution behavior and domain validation.
Configuration inspection, explanations, and diffs describe ConfigKit's composed
values and provenance. They do not include PromptKit-side trimming, endpoint
normalization, or execution defaults as ConfigKit resolution history. Existing
prompt/profile inspection and preparation expose their respective interpreted
metadata or effective execution settings. These views have distinct purposes;
an authored whitespace difference can remain visible in a configuration diff
even when it has no effect on the interpreted setting.
Interpretation preserves exact identities, references, presence/null semantics,
and the association between records and effective index paths. Relative resources
such as `content_file` resolve through the contributing field's ConfigKit origin
and source context. Normalization does not replace that attribution with an
application-generated whole-list origin.
This remains interpretation of the final handoff, not source prevalidation or
another composition pass. It requires no upstream change or new public API.
Field-granular normalization within ConfigKit itself is a separate
[future idea](future.md#field-level-normalization-with-preserved-list-provenance)
to reevaluate after upstream support is available.
### Go Source-Builder Inputs
Add an `ID string` field to `ProfileConfig` and `BackendConfig`. Change
`ProfileSource` to accept `[]ProfileConfig` and `BackendSource` to accept
`[]BackendConfig`, replacing their keyed-map inputs. Emit each value as a record
with its explicit `id`, preserving the caller's slice order. Retain
`ProfileConfig.Extends` as a convenient parent-ID string and encode it as
`extends: {id: ...}`.
This makes Go and YAML resource definitions consistent and preserves duplicate
IDs when callers combine catalogs, allowing ConfigKit to reject them during
loading. The helpers continue encoding sources without independently validating
the final configuration. Preserving input order does not give it execution
significance. Callers that maintain keyed maps can convert them at this boundary.
Make this public API change as part of the coordinated pre-v1 migration rather
than retaining a second input representation for compatibility. Update callers,
examples, and migration guidance together.
### Inheritance And Layering
Identified siblings inherit using `extends: {id: parent-id}`. This applies to
backends and profiles as well as prompts; no additional PromptKit inheritance
engine or prompt-specific restriction is needed. Lookup stays within the same
resource list. Ordinary qualified references to other resources are separate
from this sibling relationship.
ConfigKit resolves surviving parents before children, retains the child's ID,
and applies its normal mapping and presence rules to inherited settings.
Missing parents and cycles follow upstream diagnostics. Parent records remain
ordinary definitions subject to normal validation, not automatically hidden or
abstract templates. Preserve PromptKit's existing allowance for partial profiles
that requests can complete; inheritance does not add a readiness requirement.
Within one strict tier, duplicate IDs fail. Across precedence tiers, the default
identified-list operation is upsert: a matching record is replaced in full.
Authors can select ConfigKit's append or replace modes. Under upsert or append,
`[]` contributes no records; explicit replacement with `[]` clears a list.
An ID cannot inherit its displaced lower-layer version by extending itself.
Recommend a distinct variant ID with `extends` for customization. Preserve
deliberate same-ID whole-record replacement and upstream collection operations.
Do not add a partial-record merge, ban inheritance for prompts, require new IDs
for every modification, or add constraints on source documents sharing a final
namespace. Shared prompt text can continue using `content_file`.
## Illustrative Authoring Workflow
These snippets describe the target format. They are not complete deployment
files and do not assert provider model availability.
A caller mounts an entry document under `weatherreporter` using `CollectionInput`.
The entry can declare the specification once and import headerless resources:
```yaml
version: 2
config:
imports:
- dir: profiles
pattern: ["*.yaml", "*.yml"]
recursive: true
destination: [profiles]
placement: sequence-item
- dir: prompts
pattern: ["*.yaml", "*.yml"]
recursive: true
destination: [prompts]
placement: sequence-item
```
One profile file can contain a base definition:
```yaml
id: sonnet
backend: openrouter/api
model: example-model
temperature: 0.7
```
Another profile file contains its variant:
```yaml
id: sonnet-precise
extends: {id: sonnet}
temperature: 0.2
```
A prompt file can select it by qualified ID:
```yaml
id: foo.bar
default_profile: weatherreporter/sonnet-precise
messages:
- role: user
content: Describe the weather.
output:
format: text
validation_mode: none
```
The caller separately supplies `openrouter/api`. A backend collection uses the
same item-import pattern with `destination: [backends]` and can define backend
variants using sibling `extends`. Inline resource lists are equally valid;
one-file-per-item and directory layout are conveniences, not requirements.
## Acceptance And Compatibility
The integrated target supports inline lists and imported items at the root and
in arbitrarily named collections. Equal local IDs in separate namespaces remain
independent. Registered identities govern strict composition, inheritance,
layering, and final typed values through ConfigKit. PromptKit retains its domain
validation, presence-aware settings, partial profiles, and exact runtime lookup.
Domain checks use PromptKit's interpreted values without mutating ConfigKit's
retained configuration or replaying source history.
Inherited and replaced prompt fields retain their contributing resource context.
Sensitive provider values remain protected in inspection at index-based paths.
Source builders preserve supplied record order and duplicate IDs for loading.
Namespaces, discovery, inspection, preparation, concrete-request validation,
smoke diagnostics, and execution consume the same captured resource indexes.
These behaviors have focused offline coverage without duplicating ConfigKit's
generic suite or promising incidental history formatting or list positions.
The [format reference](../formats.md), [consumer migration guide](../consumers/configuration-migration.md),
[Go consumer guide](../consumers/pkg-promptkit.md), source-adapter documentation,
GoDoc, and maintained examples describe the integrated target when it lands.
Migration guidance covers the new module/types, Go minimum, controls, import
requiredness, identified lists, explicit-ID source builders, sibling inheritance,
removed prompt labels, specification headers, and index-based inspection paths.
It also explains composed configuration inspection versus PromptKit's normalized
metadata and execution settings, including the absence of ConfigKit history for
PromptKit-side normalization/defaults.
Attribution identifies the selected ConfigKit revision and applicable notices.
Deliver this as one coordinated breaking pre-v1 minor upgrade under the existing
[release policy](../policy/release.md), so applications and collection authors
migrate once. Historical release notes remain intact. Scriptorium coordination
does not make its release or internal implementation a PromptKit dependency.
ConfigKit's own release-tooling migration does not require a change to
PromptKit's release tooling or publication workflow.
The target replaces the old resource mappings without adding dual-format
decoding, source rewriting, catalog discovery conventions, or a compatibility
module. Intermediate implementation states need not compile or be coherent;
the companion plan targets one end-to-end sprint without intermediate human
review or external publication gates. Durable behavior belongs in its canonical
owners when complete; this roadmap remains temporary coordination material.

120
docs/roadmap/example-testing.md
View File

@@ -1,120 +0,0 @@
# Go-Based Testing For Maintained Examples
Status: complete. Example assertions now run in the ordinary Go suite; the
custom release check and its Python runner have been removed.
## Purpose
Move the maintained consumer-example checks into Promptkit's ordinary Go test
suite. Developers and ReleaseKit should exercise these workflows through the
same testing entry point, without a separate Python example runner or custom
release-validation command.
Keep the examples useful as readable, copyable consumer guidance. The objective
is simpler ownership and execution of tests, not a language-only translation or
a particular reduction in line count. This is a tooling change, not a new public
library API or a new release gate.
The companion [implementation plan](implementation.md) defines the staged work
for reaching this target.
## Previous Baseline
The [preparation](../../examples/go-library/prepare/main.go),
[execution](../../examples/go-library/run/main.go), and
[observation](../../examples/go-library/observe/main.go) examples are Go programs.
Their deterministic clients and transport keep them independent of live
providers. Preparation and execution use repository-relative configuration
paths; observation embeds its assets and already has a workflow helper returning
a summary and error.
The former `scripts/check-examples.py` launched all three programs and checked
their JSON summaries. Ordinary `go test ./...` then had no example tests, and
the [ReleaseKit profile](../../releasekit.json) selected the Python script as a
custom check after shared validation.
The separate [release-wiring test](../../scripts/test-release.py) is also Python,
but is a development-only diagnostic, not a release gate. Its disposable fixture
previously assumed the example script was a configured product check.
## Target State
Ordinary Go tests alongside the examples exercise the actual example workflows.
They are discovered by `go test ./...` and can be selected with
`go test ./examples/go-library/...`. ReleaseKit's existing standard suite runs
them with its selected race and vet settings; no second example-validation pass
is needed.
Preserve the useful coverage currently supplied by the Python runner:
- Preparation: collection composition and overrides, scoped selection,
inspection/preparation, and the resulting identities and inventories.
- Execution: injected-client execution, expected output and structural validity,
model selection, and usage reporting.
- Observation: bounded transient retries, attempt metadata, incomplete usage
reporting, and application-owned interpretation of proposal/assessment results.
Assertions should remain tied to the synthetic examples and meaningful results.
Their output formatting and fixture values do not become new public API
guarantees. Preserve relevant output coverage without freezing complete JSON
documents or introducing broad golden snapshots.
## Example And Test Structure
Prefer small, unexported workflow helpers that return errors or results and,
where useful, accept an output writer. Keep `main` responsible for process-level
output and exit handling. Reuse the observation example's existing separation
where it already meets the need; uniform helper signatures are not an objective.
Tests should normally call the workflow directly, using local output buffers
and the existing deterministic collaborators. Handle resource paths explicitly
so package-directory test execution and the documented repository-root
`go run` commands both work. A small resource-base or filesystem input is
preferable to changing process-wide working directories or redirecting global
standard output merely to reach the workflow.
Use a Go subprocess test only where launching the executable protects a distinct
risk or avoids making an example materially harder to understand. Do not add
one for every example automatically. Subprocesses need their own consideration
of toolchain, offline execution, and instrumentation; parent race or coverage
settings do not automatically instrument a separately launched program.
Keep tests deterministic, offline, and independent of real credentials under the
[testing policy](../policy/testing.md). Avoid a new public package, shared
example framework, or expanded failure-case inventory solely for this migration.
Examples remain consumer programs rather than release artifacts or library
implementation dependencies, consistent with the
[architecture policy](../policy/architecture.md).
## Integration And Documentation Work
The Go tests cover the retained behavior; the former `scripts/check-examples.py`
and its custom command and environment overlay have been removed. The existing
built-in validation selections and source/hosted publication workflow remain,
without a ReleaseKit dependency upgrade.
The development-only release-wiring fixture no longer expects the removed script
or product-check marker. It retains checks for the actual launcher and profile
without manufacturing an unused custom hook. The diagnostic remains Python.
Removing Python from every repository tool is outside this scope.
The [development guide](../development.md),
[testing policy](../policy/testing.md), [release policy](../policy/release.md),
and [release procedure](../release.md) describe Go test coverage as part of the
standard suite and retain Python only for the optional release-wiring test. The
existing consumer quickstart commands remain available.
## Verification
Verify the example packages through ordinary Go tests and the selected
race-enabled suite and vet. Confirm the documented `go run` entry points still
work from the repository root, and exercise the adapted release-wiring test in
disposable repositories. These are migration checks, not new recurring release
steps. Check that a meaningful example assertion failure fails the Go suite
rather than relying only on successful compilation.
Completion means useful example behavior remains covered, normal test execution
includes that coverage, and the obsolete runner, profile hook, and documentation
references are removed. No performance thresholds, coverage percentage targets,
new platform commitments, hosted CI requirements, or changes to Promptkit's
consumer contracts are introduced by this work.

255
docs/roadmap/implementation.md
View File

@@ -1,255 +0,0 @@
# Go-Based Example Testing Implementation Plan
Implement [example-testing.md](example-testing.md) as one end-to-end sprint.
Each sequential stage below is a single prompt-sized assignment for a Luna
coding agent. The feature roadmap owns intent and scope; this document owns
the implementation steps and local design choices.
## Baseline And Working Rules
The reviewed baseline is `79dfd97afa2acc3cbf1ba6936b3a11b116702cf4`:
Go 1.27.0, module `go.maximumdirect.net/promptkit`, ConfigKit v1.0.0,
ReleaseKit v0.8.0, and three maintained programs under `examples/go-library/`.
There are no Go tests alongside those programs. `scripts/check-examples.py`
executes them and checks JSON results through the sole custom ReleaseKit hook.
`scripts/test-release.py` is a separate development-only diagnostic with a
synthetic replacement for that hook.
This plan replaces the completed ConfigKit migration plan. Its historical
instructions remain available with
`git show 79dfd97:docs/roadmap/implementation.md`; they are not assignments for
this sprint.
Read `docs/development.md` and the applicable agent, architecture, testing,
documentation, and release policies before work. Preserve unrelated changes.
Use the module-compatible local Go toolchain and prepare caches before offline
validation. No dependency, Go-minimum, consumer API, or source/hosted-release
workflow change is needed.
Intermediate stages may be incomplete or temporarily inconsistent: there is no
requirement to introduce compatibility scaffolding, compile every intermediate
state, or stop for human review. Finish all stages before claiming completion.
Run focused checks when useful; if a check depends on later work, record that
dependency rather than claiming it passed. Do not publish or contact providers.
Follow the agent policy for focused local commits.
## Implementation Choices
- Put tests in `main_test.go` with `package main` beside each example. Test
actual workflow and encoding paths directly, not reconstructed engine setups.
- For preparation and execution, extract
`run(ctx context.Context, exampleDir string, out io.Writer) error`.
`exampleDir` is that example's resource directory, not an application-wide
configuration root. Use `filepath.Join` to locate its existing files.
Each `main` passes its existing repository-relative example directory and
`os.Stdout`; tests pass `"."` and a buffer from the package directory.
- Observation already has `run() (summary, error)` and embedded assets. Keep
that workflow and add a small `execute(out io.Writer) error` which calls it
and encodes its result. Both `main` and its test use `execute`.
- Keep stderr reporting and nonzero exit handling in the thin `main` functions.
Workflow helpers return errors instead of exiting. Preserve current successful
summaries and existing example-side semantic checks.
- Use one representative happy-path workflow test per example to carry the
existing useful assertions. Decode JSON independently of production summary
tags: test-local structs or maps are sufficient. Check presence where a
missing field could masquerade as an expected false, zero, or null value.
Do not reject harmless extra fields, require exact JSON formatting, or add
whole-document golden snapshots.
- Use buffers and explicit resource directories rather than redirecting global
stdout or changing the process working directory. These are local design
choices for these examples, not new library contracts.
- No subprocess example test is needed for this baseline: the small extractions
cover the workflow and output without complicating the examples. Verify the
actual `go run` commands once during integration instead of adding redundant
executable tests to every default run.
## Stage 1: Test The Execution Example In Go
Scope: `examples/go-library/run/main.go` and a new `main_test.go` there.
1. Extract its current engine construction, injected deterministic client,
request, and summary encoding into `run(ctx, exampleDir, out)`. Resolve
`prompt.yaml` relative to `exampleDir`. Pass the supplied context to
`engine.Run` instead of creating another background context there.
2. Return construction, execution, and encoding errors. Keep the existing
indented JSON output. Make `main` call this helper with a background context,
`"examples/go-library/run"`, and stdout; retain stderr/exit handling at that
boundary. Remove obsolete error-exit helper code if no longer used.
3. Add `TestRunExample` calling the helper with `t.Context()`, `"."`, and a
`bytes.Buffer`. Decode and assert the current output:
- `output`: `Ada finished the migration review.`;
- `is_valid`: true;
- `validation_status`: `passed`;
- `model`: `offline-model`;
- `total_tokens`: 18.
4. Keep this local to the example: no new shared test framework or public API.
Validation: format changed Go files and run
`go test ./examples/go-library/run`. Confirm the test executes the workflow,
not just the summary type. These expected values describe this fixture only.
Completion: extracted the execution workflow and JSON encoding into a helper
with explicit context, resource directory, and writer. The focused Go test
executes the workflow and checks its summary; it passed on Go 1.27.0.
## Stage 2: Test Preparation With Explicit Resource Context
Scope: `examples/go-library/prepare/main.go` and its new `main_test.go`.
1. Extract the current composition, inspection, validation, preparation, and
summary encoding into the chosen `run(ctx, exampleDir, out)` helper. Replace
the three hard-coded file prefixes with paths under `exampleDir`; retain
the existing collections, source ordering, imports, and later override.
2. Preserve the inline checks for the discovered required `note` input, the
override explanation, and the effective timeout. Return their errors.
Keep those checks readable in the example instead of duplicating their
complete machinery in the test. Use the supplied context for engine calls.
3. Keep `main` callable through the existing repository-root command by passing
`"examples/go-library/prepare"` and stdout. Retain current indented encoding
and error presentation while removing exits from the workflow helper.
4. Add `TestPrepareExample` using the helper, package-local resources, and a
buffer. Assert these retained JSON results:
- `prompt_id`: `support/prepare`; `selected_profile`: `support/fast`;
- `model`: `offline-model`; `message_count` greater than zero;
- `profile_timeout_seconds`: 30;
- `namespaces`: `review`, then `support`;
- `collections`: review with an explicit null `release_version`, followed
by support with `release_version` equal to `1.4.0`;
- `scoped_prompts`: `support/prepare`; `review_prompts`: `review/check`.
Preserve the distinction between an omitted member and the explicit null
currently checked for the review collection.
Validation: format and run `go test ./examples/go-library/prepare`. Ensure
imports and relative content still resolve from their contributing sources;
do not rewrite configuration fixtures merely to bypass path handling.
Completion: the preparation workflow accepts an explicit context, example
directory, and writer while preserving its inline checks and source contexts.
Its Go test verifies the retained summary, including explicit null collection
metadata, and passed on Go 1.27.0.
## Stage 3: Test The Observation Example And Its Output
Scope: `examples/go-library/observe/main.go` and its new `main_test.go`.
1. Retain its existing `run`, embedded files, scripted HTTP transport, event
recorder, and application-owned assessment logic.
2. Extract only the workflow/encoding coordination into `execute(out)` as
specified above. It returns workflow or encoding errors. Make `main` call
it with stdout and retain the current stderr/exit behavior. Do not redesign
the transport or add a new client abstraction.
3. Add `TestObserveExample` calling `execute` with a buffer. Decode and assert:
- `proposal_attempts`: 2; `proposal_transient_retries`: 1;
- `proposal_usage_complete`: false;
- `assessment_attempts`: 1; `assessment_transient_retries`: 0;
- `assessment_outcome`: `accepted`.
Missing false/zero fields must not silently pass. Run the workflow once per
test rather than separately recomputing typed and encoded results.
Validation: format and run `go test ./examples/go-library/observe`, followed
by `go test ./examples/go-library/...`. Existing deterministic collaborators
remain the network boundary; no live provider or credential is needed.
Completion: observation workflow encoding now goes through a writer-based
helper shared by the executable and its Go test. The test checks all retained
summary fields, including false and zero presence; the observation and complete
example package tests passed on Go 1.27.0.
## Stage 4: Remove The Custom Release Hook And Adapt Its Diagnostic
Scope: `scripts/check-examples.py`, `releasekit.json`, and
`scripts/test-release.py`.
1. Delete the Python example runner after the Go tests carry its useful
coverage. Remove the profile's `checks` property and its example-specific
environment overlay. Retain every existing built-in validator, delivery
setting, and token-file protection. Do not add a replacement custom command.
2. In the release-wiring fixture, remove the generated example-check stub,
`FIXTURE_CHECK_MARKER`, `FIXTURE_CHECK_FAILURE`, related environment setup,
and assertions about invocation of that removed hook. Update its docstring
to describe launcher/profile wiring, not product-check dispatch.
3. Retain bootstrap/version/build-info checks, native usage status 2, successful
publication of the intended local fixture candidate without delivery
credentials, selected race/local-Go verification, token-path selection and
ignore checks, and module/worktree preservation.
4. Preserve a small status-1 wrapper check without manufacturing a product
hook: create only the successful version's note in the fixture, then invoke
the publication wrapper with `FAILED_VERSION`, whose note is absent.
Expect status 1 and retain the remote-tag absence check. This exercises
forwarding a native candidate failure, not a new publication algorithm.
5. Keep this diagnostic development-only and in Python. Do not add it to the
default Go suite or the release profile. The native launchers are unchanged.
Validation: run `python3 scripts/test-release.py` using the selected Go on
`PATH` and prepared caches. Its remotes must remain disposable local bare
repositories. Check the profile diff to confirm only the obsolete custom hook
was removed. Do not test by publishing a real tag or hosted page.
Completion: removed the Python example runner and its custom profile check;
the release-wiring diagnostic now tests the native missing-note failure and
preserves its other launcher, suite, publication, and token-path checks. The
diagnostic passed with Go 1.27.0 against disposable local repositories, and the
profile diff retains the existing built-in validators and delivery settings.
## Stage 5: Align Documentation And Verify The Complete Sprint
Scope: the development guide, local testing/release policies, release runbook,
affected example guidance, and integration verification.
1. Update `docs/development.md` to use normal Go testing for example validation.
Offer `go test ./examples/go-library/...` for targeted development without
making it a separate required release pass. Keep the Python release-wiring
command documented as a focused optional diagnostic.
2. Update the Promptkit-specific sections of `docs/policy/testing.md` and
`docs/policy/release.md`: example assertions now run inside the existing Go
suite, not after it as a custom check. Keep the examples' distinct coverage
ownership, all other selected checks, and the publication completion rules.
3. Remove Python from normal release prerequisites in `docs/release.md`.
Preserve Go selection, credentials, source publication, hosted delivery, and
recovery. Update any other current guidance pointing to the deleted runner.
Historical release notes remain unchanged.
4. Update the feature roadmap's completion/baseline text when recording the
implemented result. Remove its now-dead Markdown link to the deleted Python
file, using a plain historical path if still useful. Avoid leaving a broken
link in a completed coordination document. Do not retire unrelated roadmaps.
5. Format changed Go files, review the full diff, and run the integrated checks
below with the same compatible local Go toolchain:
```sh
go test -race -vet=off ./...
go vet ./...
go run ./examples/go-library/prepare
go run ./examples/go-library/run
go run ./examples/go-library/observe
python3 scripts/test-release.py
git diff --check
```
Run the three programs from the repository root. Confirm their outputs
retain the meanings checked by the new tests; JSON spacing is not a new
contract. Reuse successful focused checks for unchanged content rather than
repeating them solely to finish a stage. The separate full vet invocation
above pairs with `-vet=off` to avoid duplicate vet work.
6. As a one-time proof, temporarily make one meaningful expected value wrong in
an example test, run its package test, and confirm failure. Restore the value
and rerun the affected package before final review. Do not commit that probe
or add a permanent self-test of the testing framework.
7. Search current code/configuration/guidance for obsolete runner and hook
references, verify local documentation links, and confirm `go.mod`, `go.sum`,
public library code, and the native release launchers were not changed by
this tooling migration. Commit the focused completed work under agent policy.
Completion requires all three workflows to have real Go test coverage,
successful ordinary suite discovery, preserved useful assertions, removal of
the runner and custom hook, and accurate current documentation. No performance,
coverage-percentage, platform, or release-service gate is added. There are no
unresolved product decisions or upstream publication dependencies for this sprint.
Completion: all three examples have Go workflow and JSON checks, and the
obsolete runner and custom release hook have been removed. Updated the current
development, testing, and release guidance and marked the feature roadmap
complete. On Go 1.27.0, the full race-enabled suite, full vet, all three
repository-root example commands, and the disposable release-wiring diagnostic
passed. A deliberately incorrect example expectation failed before restoration,
then the affected package passed again. Reviewed links, the profile, formatting,
module and launcher changes, and `git diff --check`.

38
docs/roadmap/remediation.md
View File

@@ -1,38 +0,0 @@
# Review Findings
Reviewed the Go example-testing migration from `7bbdb55` through `c7d4318`,
including its integration with the existing release profile. No implementation
regressions, missing migration work, or strengthened consumer contracts were
identified. The existing release validation exposed the following preexisting
documentation issue.
## 1. Repair The Historical Profile-Inheritance Link (P2)
Status: complete.
Completion: corrected the link to `#resource-inheritance`, retaining the
historical summary. ReleaseKit's complete configured candidate check passed
in the disposable checkout with the corrected document, including Markdown
validation, the race-enabled Go suite, and full vet. No release was published.
Location: `docs/releases/v0.7.0.md:129`.
The link to `../formats.md#profile-inheritance` has no matching heading in the
current format reference. That reference now documents inheritance under
`Resource Inheritance`. ReleaseKit's configured Markdown validator rejects the
candidate at this link, preventing the release check from succeeding even
though the race-enabled package suite, including all three examples, passes.
This issue predates the example-testing sprint: both the stale link and the
replacement heading are present in `7bbdb55`. Neither file changed during the
migration. It is not caused by removing the Python example hook.
Remediation: point the historical note's link to
`../formats.md#resource-inheritance`, preserving the historical release summary.
Verify the repaired link with the existing Markdown validator and complete the
normal candidate checks when preparing the release. No new gate is needed.
Evidence: a disposable clone of `c7d4318`, with only a synthetic release note
added, failed `./scripts/releasekit.sh check v0.999.0` with
`docs/releases/v0.7.0.md:129:9: heading anchor does not exist`.
Python commands were replaced by failing shims for that check; none was invoked.