4.5 KiB
PromptKit Adoption Review Findings
Review baseline: Scriptorium c5e54edaa5f6621d504cf510b4b5b5c1e4d1cfd4,
with the approved PromptKit v0.23.3 pin and ConfigKit v1.2.1. Stage numbers
refer to the implementation plan.
R1: Bound the HTTP timeout regression tests (Stage 4)
In handler_test.go,
TestHandlerDeadlineDuringBuiltInGeneration runs the handler synchronously
while the fake provider waits on a channel that is released only after the
handler returns. If the operation timeout stops reaching PromptKit, the test
can wait for the unrelated provider/transport timeout or the suite watchdog
instead of promptly failing. The retry- and capacity-wait tests likewise rely
on the behavior being tested to return control. The added
observeRunnerFailures helper also sends all failures to a shared,
one-slot channel before allowing each runner invocation to return. An
unexpected failure of the first request in the capacity test can fill that
slot and block the second request from returning the failure the test needs
to inspect.
Use a test-only fallback deadline or bounded synchronization independent of
the operation budget, and capture the error of the specific observed
invocation without a blocking send shared by concurrent requests. Keep the
real engine and loopback provider, the assertion on
ErrOperationTimeout, and the existing status, observation, and
capacity-release checks. This is a test liveness fix, not a new production
timeout or release gate.
R2: Describe source-relative message paths without a colocation rule (Stage 7)
The upgrade guide instructs users to
keep message files beside their owning definitions. PromptKit resolves
content_file against the field's declaring source within its rooted
filesystem; it does not require the file to be in the same directory. A
freshly built Scriptorium binary rendered a valid content_file: ./messages/request.md without moving the file beside its framework YAML.
The current wording can lead operators to rearrange valid layouts during
migration.
Describe source-relative resolution and link to the pinned PromptKit format contract for containment rules. A colocated file can remain an example, not a requirement. No loader or path-policy change is needed.
R3: Link the Go Text patent grant notice (Stage 7)
The attribution entry for
golang.org/x/text v0.44.0 links LICENSE but omits PATENTS. Build
dependency inspection confirms Go Text is part of the executable. Its
selected module contains the separate patent grant at upstream commit
6070ac32abcdd1670599eeb3ee4d513c8184a513; PromptKit's attribution for
that same revision links it. The OSS policy
calls for applicable companion texts to be directly reachable.
Add a PATENTS link at the already selected Go Text commit. Retain the
verified local BSD text and existing LICENSE link. No separate local
patent notice or change to the project license is needed under the
adopted attribution format.
R4: Distinguish discovery peers from ordered framework layers (Stages 6–7)
The upgrade guide says a later source
selecting the same ID replaces the whole record. That holds for a later
framework underlay, not for two checkout entries selected by the same
discovery file. Scriptorium turns each resolved promptkit_sources selector
into one underlay. The entries discovered by the imports of that selector
are composed as peers; their filename, glob, or checkout order is not
override precedence.
The built binary rejected two discovered checkout entries defining
shared/fast with compose.conflict. Selecting the very same entry files
as two ordered promptkit_sources items instead succeeded: the later
record's model was selected and max_tokens, absent from that record, did
not carry forward from the earlier one. This is the intended distinction, but
the migration wording can lead operators to put an intended override in
a discovery peer, which prevents the engine from loading.
Qualify whole-record replacement in the upgrade guide and explain in the checkout discovery reference that same-ID conflicts within one discovery underlay fail. For an intentional override, select a separate, later framework source using the existing application selector list. No scanner, de-duplication, new namespace restriction, or change to composition is needed.