152 lines
8.2 KiB
Markdown
152 lines
8.2 KiB
Markdown
# PromptKit Adoption Review Findings
|
||
|
||
Review baseline: Scriptorium `c5e54edaa5f6621d504cf510b4b5b5c1e4d1cfd4`,
|
||
with the approved PromptKit v0.23.3 pin and ConfigKit v1.2.1. Stage numbers
|
||
refer to the [implementation plan](implementation.md).
|
||
|
||
## R1: Bound the HTTP timeout regression tests (Stage 4)
|
||
|
||
**Complete.** The tests use independent parent deadlines and capture the
|
||
target invocation's error without sharing a channel. All three passed
|
||
twice under the race detector.
|
||
|
||
In [handler_test.go](../../internal/adapter/http/handler_test.go),
|
||
`TestHandlerDeadlineDuringBuiltInGeneration` runs the handler synchronously
|
||
while the fake provider waits on a channel that is released only after the
|
||
handler returns. If the operation timeout stops reaching PromptKit, the test
|
||
can wait for the unrelated provider/transport timeout or the suite watchdog
|
||
instead of promptly failing. The retry- and capacity-wait tests likewise rely
|
||
on the behavior being tested to return control. The added
|
||
`observeRunnerFailures` helper also sends all failures to a shared,
|
||
one-slot channel before allowing each runner invocation to return. An
|
||
unexpected failure of the first request in the capacity test can fill that
|
||
slot and block the second request from returning the failure the test needs
|
||
to inspect.
|
||
|
||
Use a test-only fallback deadline or bounded synchronization independent of
|
||
the operation budget, and capture the error of the specific observed
|
||
invocation without a blocking send shared by concurrent requests. Keep the
|
||
real engine and loopback provider, the assertion on
|
||
`ErrOperationTimeout`, and the existing status, observation, and
|
||
capacity-release checks. This is a test liveness fix, not a new production
|
||
timeout or release gate.
|
||
|
||
## R2: Describe source-relative message paths without a colocation rule (Stage 7)
|
||
|
||
**Complete.** The migration guide describes source-relative resolution and
|
||
links to PromptKit's path rules. A render with a message in a subdirectory
|
||
confirmed that no colocation is required.
|
||
|
||
The [upgrade guide](../consumers/upgrading-promptkit.md) instructs users to
|
||
keep message files beside their owning definitions. PromptKit resolves
|
||
`content_file` against the field's declaring source within its rooted
|
||
filesystem; it does not require the file to be in the same directory. A
|
||
freshly built Scriptorium binary rendered a valid `content_file:
|
||
./messages/request.md` without moving the file beside its framework YAML.
|
||
The current wording can lead operators to rearrange valid layouts during
|
||
migration.
|
||
|
||
Describe source-relative resolution and link to the pinned PromptKit format
|
||
contract for containment rules. A colocated file can remain an example, not
|
||
a requirement. No loader or path-policy change is needed.
|
||
|
||
## R3: Link the Go Text patent grant notice (Stage 7)
|
||
|
||
**Complete.** The attribution entry links `PATENTS` at the selected Go Text
|
||
commit. The notice exists in the pinned module, and the local BSD text and
|
||
upstream `LICENSE` link are unchanged.
|
||
|
||
The [attribution entry](../oss/attribution.md#golangorgxtext) for
|
||
`golang.org/x/text v0.44.0` links `LICENSE` but omits `PATENTS`. Build
|
||
dependency inspection confirms Go Text is part of the executable. Its
|
||
selected module contains the separate patent grant at upstream commit
|
||
`6070ac32abcdd1670599eeb3ee4d513c8184a513`; PromptKit's attribution for
|
||
that same revision links it. The [OSS policy](../policy/oss.md#license-texts-and-upstream-links)
|
||
calls for applicable companion texts to be directly reachable.
|
||
|
||
Add a `PATENTS` link at the already selected Go Text commit. Retain the
|
||
verified local BSD text and existing `LICENSE` link. No separate local
|
||
patent notice or change to the project license is needed under the
|
||
adopted attribution format.
|
||
|
||
## R4: Distinguish discovery peers from ordered framework layers (Stages 6–7)
|
||
|
||
**Complete.** The migration guide and configuration reference distinguish
|
||
peer conflicts from ordered replacement. The same two definitions were
|
||
verified to conflict as peers and replace as separate selectors.
|
||
|
||
The [upgrade guide](../consumers/upgrading-promptkit.md) says a later source
|
||
selecting the same ID replaces the whole record. That holds for a later
|
||
framework underlay, not for two checkout entries selected by the same
|
||
discovery file. Scriptorium turns each resolved `promptkit_sources` selector
|
||
into one underlay. The entries discovered by the imports of that selector
|
||
are composed as peers; their filename, glob, or checkout order is not
|
||
override precedence.
|
||
|
||
The built binary rejected two discovered checkout entries defining
|
||
`shared/fast` with `compose.conflict`. Selecting the very same entry files
|
||
as two ordered `promptkit_sources` items instead succeeded: the later
|
||
record's model was selected and `max_tokens`, absent from that record, did
|
||
not carry forward from the earlier one. This is the intended distinction, but
|
||
the migration wording can lead operators to put an intended override in
|
||
a discovery peer, which prevents the engine from loading.
|
||
|
||
Qualify whole-record replacement in the upgrade guide and explain in the
|
||
[checkout discovery reference](../config.md#collection-checkout-discovery)
|
||
that same-ID conflicts within one discovery underlay fail. For an intentional
|
||
override, select a separate, later framework source using the existing
|
||
application selector list. No scanner, de-duplication, new namespace
|
||
restriction, or change to composition is needed.
|
||
|
||
## R5: Replay framework explanation history once (Stage 3)
|
||
|
||
**Complete.** Framework inspection calls the native explanation writer
|
||
once. The framework inspection tests passed, including path errors,
|
||
redaction, and writer failures.
|
||
|
||
[Framework configuration inspection](../../internal/adapter/cli/framework_config.go)
|
||
calls `view.Explain(path)` to check existence, discards the complete
|
||
explanation, then calls `view.WriteExplanation`. The pinned native writer
|
||
calls `Explain` itself. A successful request therefore replays the same
|
||
captured history and builds the same redacted value snapshots twice. If
|
||
`H` is the work to construct that explanation and `W` the work to render
|
||
it, the command does roughly `2H + W` work instead of `H + W`. This is a
|
||
constant-factor cost, not a new quadratic algorithm, but it repeats a
|
||
history- and value-sized operation and its allocations. The legacy wrapper
|
||
already used two calls; the native inspection path need not retain them.
|
||
|
||
Delegate once to `view.WriteExplanation`, as app configuration inspection
|
||
already does with its native writer. A direct probe with the selected
|
||
PromptKit and ConfigKit releases confirmed that this writes a known path,
|
||
returns an error without output for an unknown path, and preserves writer
|
||
errors. Keep JSON Pointer parsing, native redaction, and the existing exit
|
||
and stream behavior; no new formatter or replay algorithm is needed.
|
||
|
||
## R6: Prevent artifact read-limit overflow (Stage 4)
|
||
|
||
**Complete.** The read cap no longer overflows at `math.MaxInt64`.
|
||
The new test failed before the fix and now verifies that a small file keeps
|
||
its full body and size at that limit. The existing size-limit tests, full
|
||
Go suite, race suite, vet, and build passed.
|
||
|
||
[The restricted HTTP artifact reader](../../internal/adapter/http/artifact_reader.go)
|
||
allows the same nonnegative `int64` limit as application configuration
|
||
and the CLI. It computes `maxBytes + 1` for `io.LimitReader` to detect
|
||
oversized files. At the valid setting `9223372036854775807`
|
||
(`math.MaxInt64`), that addition wraps to a negative number. The limited
|
||
reader then immediately returns EOF, so a nonempty file is accepted as an
|
||
empty artifact. A loopback check confirmed HTTP 200 at this setting while
|
||
the outgoing prompt silently lost the file content. The same synthetic
|
||
request included the file at zero and at the ordinary positive limit.
|
||
The defect predates the migration and remains in the reader retained
|
||
by Stage 4.
|
||
|
||
Use overflow-safe byte-limit arithmetic: for example, retain the capped
|
||
`limit + 1` read for smaller positive limits and skip the increment at
|
||
`math.MaxInt64`, or limit the read first and probe separately for an extra
|
||
byte. Accepted files should still require linear time and buffer space in
|
||
the bytes read; normal limits should continue to reject oversized files
|
||
without a full read. Add a boundary test using a small nonempty file.
|
||
Do not reduce the accepted configuration range or change containment
|
||
or the zero-limit semantics.
|