base-layer/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Define the execution controls implementation plan

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-14 17:15:46 +00:00
parent f17e0620a5
commit 0e4f590f5a
3 changed files with 596 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files

441
docs/roadmap/implementation.md Normal file
View File

@@ -0,0 +1,441 @@
# Execution Controls Implementation Plan
## Objective And Sprint Scope
Implement the accepted target in [execution-controls.md](execution-controls.md)
from the v0.22.0 execution architecture. The feature roadmap owns intent and the
four accepted lifecycle decisions; this document selects the API and the work
needed to implement them. [Audita's request](audita-request.md) supplies the
consumer scenarios.
Work through Stage 1 through Stage 8 in order. Each stage is scoped for one Luna
coding-agent prompt. This is one end-to-end sprint: intermediate stages may
temporarily fail to compile or leave wiring incomplete. Do not build temporary
compatibility layers merely to make intermediate commits green. There are no
intermediate human-review or external-publication gates. Completion requires
the assembled feature and its validation, not just completion of individual
stages.
Read [development.md](../development.md), the [agent policy](../policy/agents.md),
and the applicable architecture, testing, and documentation policies before
work. Preserve unrelated changes. Follow the existing commit policy for finished
units; report intermediate validation honestly. This plan authorizes no release,
push, dependency update, configuration-format change, or new consumer workflow
beyond the execution controls described here.
## Selected API And Semantics
Add these fields to the existing public `RunRequest`, without introducing an
execution-options object, coordinator executor, or general hook framework:
```go
OperationTimeout time.Duration `json:"-" yaml:"-"`
OnPrepared func(context.Context, PreparedRun) `json:"-" yaml:"-"`
```
Add one public sentinel, alongside the current execution errors:
```go
var ErrOperationTimeout = errors.New("operation timeout exceeded")
```
The spelling of an error message is not a contract. No exported timeout error
type, new observer event, or new failure-category enum is needed.
### Entry-Point Contract
| Entry point | OperationTimeout | OnPrepared |
| --- | --- | --- |
| `Run` | A positive value starts a fresh child budget before preparation, covering preparation and execution. | Notify once if execution reaches the prepared point before first generation. |
| `Coordinate` | Each returned request receives ordinary `Run` semantics with a fresh scope. | Ordinary per-request notification; no coordinator-specific copy of the option. |
| `PrepareExecution` | Retain the value privately, without starting a timer. Preparation uses its supplied context. | Retain the selected function privately without invoking it. |
| `RunPrepared` | Start the retained positive duration after the successful claim, before execution callbacks, credential revalidation, and admission. | Notify once when the claimed execution reaches the prepared point. |
| `Prepare`, `ValidateRequest` | Do not start a timer; use the supplied context as today. | Do not invoke or retain the notification. |
| `PreparedExecution.Details`, `Discard` | Do not start a timer. | Do not invoke the notification. Discard drops its retained reference. |
- A positive duration opts in. Zero and negative durations select no additional
timeout. This follows a nonpositive-means-unselected convention, avoids a new
request rejection rule, and leaves the parent deadline effective. There is no
default operation duration, upper bound, profile setting, or engine setting.
- The provider's existing generation timeout remains independent. Do not copy
`OperationTimeout` into `ExecutionTarget`, provider JSON, or prepared/result
metadata. Neither control changes prompt hashes or configuration inspection.
- The budget is continuous across preparation, capacity waiting, notification,
generation, transient retries/backoff, validation, and structural repair. It
is not restarted between phases, attempts, or repair generations.
- Split preparation retains a duration, not a context, deadline, timer, or
cancel function. Idle time does not count. A shared caller-created deadline
remains the way to bound preparation and later execution together.
- A nil notification selects no diagnostic copying. A selected function receives
the operation context and a detached, credential-redacted `PreparedRun` value
representing the logical request. Its nested mutable values belong to the
application. Reuse the existing projection used by `Details()`.
- Notification is synchronous, has no error return, and runs after successful
preparation and admission, before the first generation. Logical admission is
still held. Its time consumes the selected operation budget. It may retain
details for later diagnostics; PromptKit does not own writes or write errors.
- Check context before notification and again before generation after it
returns. A callback may block or ignore context; do not introduce a goroutine
to interrupt it. Callback panic propagates after deferred resource cleanup,
without a new terminal-event guarantee.
- Callback selection is copied as a function value, not as a deep copy of its
closure. Shared callback state is application-synchronized; separate runs can
invoke a common callback concurrently. Later reassignment of the original
request field cannot replace a prepared handle's retained function.
- Notification is conditional on reaching that execution point. Admission or
preparation failure can prevent it. Do not render again, reorder admission,
or promise a diagnostic snapshot for every failed operation. Split callers
can still inspect a successfully prepared handle through `Details()`.
### Timeout Attribution And Completion
Use the standard library's `context.WithTimeoutCause` with a distinct cause
value for each selected scope. Record the scope's own cause identity so an
inherited timeout from an enclosing operation cannot be mistaken for this
invocation's timer. Do not classify by error text or by `DeadlineExceeded`
alone. No strict ordering of independently racing cancellations is promised.
When an execution fails through its context deadline and the child context's
recorded cause is this scope's own timer cause, return an error matching both
`ErrOperationTimeout` and `context.DeadlineExceeded`. Preserve existing operation
and collaborator error identities. An earlier parent deadline/cancellation or
a provider deadline must not acquire the local marker merely because it is a
deadline error. An unrelated failure must not acquire it merely because the
timer has also expired. Parent-provided error causes, if any, retain their
existing behavior; this feature need not reinterpret them.
Finalize attribution from the operation's returned error before canceling its
child scope and before terminal observer delivery. A nil execution error remains
a completed result, including a completed structural rejection. Do not add a
post-terminal `ctx.Err()` check that converts it into an error. Slow terminal
observers and deferred context cleanup cannot change the chosen outcome. This
does not strengthen cancellation into forced interruption or a guarantee that
every context-ignoring collaborator will return by the deadline.
The terminal event continues using the existing deadline failure category for
a timeout error. Groups continue aggregating those ordinary terminal facts.
The child scope is cleaned up before `Run`/`RunPrepared` returns to the next-turn
callback; that callback receives the coordinator parent context, not the child.
Existing parent cancellation and undelivered-result rules remain unchanged.
## Implementation Map
Use existing owners rather than adding a new public package:
- Public declarations and contract: [types.go](../../types.go),
[engine.go](../../engine.go), and [prepared_execution.go](../../prepared_execution.go).
- Public conversion and error projection: [convert.go](../../convert.go),
[errors.go](../../errors.go), and [observations.go](../../observations.go).
- Internal request values: [domain.go](../../internal/domain/domain.go).
- Timeout lifecycle and notification orchestration: `internal/usecase`, chiefly
[runner.go](../../internal/usecase/runner.go) and
[prepared_execution.go](../../internal/usecase/prepared_execution.go).
- Existing private operation-error representation:
[operation_error.go](../../internal/usecase/operation_error.go).
- Ordinary coordinator integration: [coordinate.go](../../coordinate.go) and
[internal coordinator](../../internal/usecase/coordinate.go).
The root remains a facade. Keep timer construction and cause-attribution logic
in a small internal helper; the facade may establish that scope and project its
errors while managing its existing observation lifecycle. Internal packages must
not import the root package or depend on the public `PreparedRun` representation.
Private helper names and equivalent internal organization may change; the
selected public behavior and ownership boundaries are what matter.
## Stage 1: Add The Request Contract And Conversion
Outcome: the selected API and internal request plumbing exist, ready for the
execution stages to consume.
1. Add `OperationTimeout`, `OnPrepared`, and `ErrOperationTimeout` to the public
declarations above. Write their GoDoc from the entry-point and ownership
contract in this plan, explicitly separating operation and provider timeouts.
2. Extend `domain.RunRequest` with the duration and an internal callback of shape
`func(context.Context, *domain.PreparedRun)`. Document the callback argument
as borrowed read-only internal state. It is not a public mutable snapshot.
3. Extend `toDomainRunRequest` to copy the duration and capture the selected
public callback function in a local variable. Only when it is non-nil, adapt
it by projecting the supplied prepared state through `fromDomainPreparedRun`
and passing the resulting value. This projection is the public ownership
boundary; do not deep-copy the domain snapshot first as well.
4. Keep these fields out of public prepared/result JSON and opaque handle
details. Request conversion does not start a timer, invoke a callback, or
reject an execution-only control on preparation-only entry points.
5. Add the private timeout error identity in `internal/usecase`, using the same
existing separation between internal and public sentinels. Add its public
projection to `errors.go` when introducing the attribution wrapper in Stage 2.
Validation: focused conversion tests should establish option forwarding and
conditional adaptation. Reuse existing nested-value projection tests instead of
retesting every existing `PreparedRun` field. Compilation may remain incomplete
until the execution wiring lands; do not claim the feature works at this stage.
## Stage 2: Bound Ordinary Execution And Preserve Its Error Outcome
Outcome: `Engine.Run` owns one continuous optional budget and returns the same
failure classification it publishes to observers.
1. Add a small internal operation-scope helper using `WithTimeoutCause`. For
nonpositive durations, reuse the parent context without a timer. For positive
durations, retain a fresh per-invocation cause token and a cancel function.
Use a real distinct identity, not a shared sentinel or zero-sized token whose
pointer identity cannot distinguish scopes.
2. Give the helper an error-finalization operation: for a non-nil returned error,
require both a match to the operation context's deadline error and equality
of its recorded cause with this scope's token before adding the private
timeout identity. Leave nil errors and unrelated errors unchanged. Preserve
the original error as a cause.
3. Reuse `usecase.OperationError` with the private timeout category to carry the
marker around the existing error, rather than creating a public error
hierarchy. Project that category to `ErrOperationTimeout` through the
existing `publicOperationError` machinery. Keep the original phase category
and context cause inspectable through the nested chain.
4. Establish the ordinary scope in the facade's `Run` entry, after masking
inherited execution-observation state and before start-event delivery and
request preparation. Both observed and unobserved runs use it. Pass its
context through request conversion, internal engine delegation, admission,
preparation, and all generation/validation work. Do not also start a second
timer inside `Runner.Run`.
5. Adjust the private facade `run` path as needed to finalize the execution
error before applying public error projection once. On normal return, freeze
attribution and the returned result/error, clean up the child, then classify
and deliver the existing terminal event. Preserve normal observation of
request conversion and preparation failures. Handle all early returns; install
cleanup before any user callback can panic.
6. Do not change provider timeouts, retry eligibility, repair limits, scheduling,
or existing result timing conventions to implement this feature. A timer
expiring while terminal observation runs is not another execution failure.
Validation: narrow helper tests cover disabled scopes, local expiry, parent
deadline/cancellation, inherited operation causes, unrelated errors, and nil
completion. Add public `Run` tests proving the marker survives phase wrapping
and agrees with an ordinary deadline terminal event. Provider-only deadline
errors while the operation child is active must not gain the marker.
## Stage 3: Retain Controls Across Split Preparation
Outcome: prepared execution carries the selected duration and notification,
without keeping a live operation scope while idle.
1. Extend `preparedExecutionPayload` with the duration and internal notification.
`PrepareExecution` copies them from the converted request only after its
ordinary successful preparation. Preparation itself uses its supplied
context, without starting the selected timer or calling the notification.
2. Extend payload clearing for both fields. Preserve the existing atomic claim,
owner checks, foreign/reused-handle behavior, and durable redacted details.
A discard or completed/unwound claim drops PromptKit's callback reference.
3. After a successful `Runner.RunPrepared` claim, establish the scope with the
retained duration, before the retained observer's start delivery, credential
revalidation, and admission. Defer payload and scope cleanup before callback
work. A failed claim neither starts notification nor changes existing silent
invalid-handle behavior.
4. Pass the child through execution and finalize the returned error with the
same helper before cleanup and facade terminal classification. Keep this
lifecycle in the internal prepared runner rather than borrowing duration or
callback state from an unclaimed handle in the facade.
5. `RunPrepared`'s facade projects the resulting error and emits its one existing
terminal event after resource release. Do not reapply the timer at the facade
or retain the preparation context. Namespace views keep their shared-owner
behavior and use the handle's original selections.
Validation: extend prepared-execution coverage to show that preparation and an
idle delay longer than the selected duration do not consume its execution
budget; later execution does. Reassignment of the original request does not
change the retained duration/callback. Cover earlier caller deadlines and a
shared caller deadline across preparation and execution. Existing claim-race,
discard, foreign-owner, credential, and error-identity tests remain applicable.
Notification invocation itself is completed in Stage 4.
## Stage 4: Notify From The Actual Shared Prepared Snapshot
Outcome: ordinary and prepared execution expose one detached logical request
before first generation, without repeating preparation.
1. Thread the internal notification from `Runner.Run` and the claimed payload
into the existing shared `executePreparedRun` entry. At that entry, preparation
is complete and logical admission is held on both paths.
2. When selected, check the operation context, invoke the internal adapter with
the actual retained prepared state, and check context again before entering
generation. A context failure at this boundary follows the existing
generation operation-error wrapping, preserving context identity and gaining
local-timeout identity only through the scope finalizer.
3. Invoke once per executing logical request, not per transport retry or repair.
Do not call `Prepare`, `Details`, artifact readers, renderers, or schema
preparation again to produce the notification. With nil `OnPrepared`, avoid
constructing a public details copy.
4. Keep the prepared state's API key redacted. Generation still receives its
private direct credential through the existing separate path. Preserve the
complete assembled messages, schema, settings, input hashes, session,
ancestry, and preparation metadata in the delivered public value.
5. Leave logical admission held; do not release/reacquire it around notification.
Invoke application code without holding the handle mutex or a capacity
manager lock. Preserve panic propagation while defers release the lease,
timer, and claimed payload. A panic does not make a handle reusable.
Validation: one ordinary and one split public test establish ordering before
generation and exact callback selection. Add focused context-before/after
notification tests and an internal cleanup regression if the public capacity
probe alone cannot demonstrate payload cleanup. More extensive ownership and
failure assertions belong to Stage 6, not duplicate tests here.
## Stage 5: Verify Continuous Budgets Across Execution Phases
Outcome: tests demonstrate that one scope reaches all relevant collaborators,
with correct attribution and observation on failure.
1. Add focused tests for preparation expiry through a context-aware injected
artifact reader, generation-capacity waiting on a real limited backend,
interrupted built-in retry backoff, and structural repair expiry. Verify
standard context matching, local marker, and retained phase identity where
that phase already supplies one. Cover ordinary and split paths without
mechanically duplicating every case on both.
2. Use a narrow internal runner test for validation-phase expiry, since validation
collaborators are not a public injection API. Check the same deadline reaches
preparation, validation, and generation; do not add a new public testing seam.
3. Test that notification time consumes the same budget and that expiry there
prevents generation. Cover an earlier parent deadline and parent cancellation
separately from a provider's own deadline. Preserve original injected error
identities and do not relabel a non-context error after timer expiry.
4. On both ordinary and split public execution, exercise a terminal observer
that outlasts the operation budget after successful completion. The result
stays successful and the terminal remains successful. Include the timeout
failure case to ensure public error mapping happens before terminal delivery.
5. After expiry, prove capacity remains usable and the caller parent stays
active. Respect the current capacity model: logical admission may reject
immediately, while admitted work can wait for an active generation permit.
Do not introduce a new blocking admission queue to satisfy a test label.
Use channels for ordering and `testing/synctest` for controlled elapsed-time
scenarios where its bubble rules apply. This repository's Go minimum supports
it. Prefer in-memory fixtures and controlled transports; avoid real short sleeps
or close-to-deadline assertions. Reuse the existing generation tests for retry
policy rather than building a second retry matrix. There is no production clock
framework, live provider probe, allocation threshold, or performance gate here.
Validation: run focused use-case/generation tests and the new public execution
control tests under the race detector. The new cases should protect deadline
propagation and classification, not private helper call sequences.
## Stage 6: Verify Notification Ownership And Failure Lifecycles
Outcome: applications can safely retain diagnostic details without altering
execution or acquiring new lifecycle guarantees.
1. Add a representative continuation request containing history, follow-up
messages, appended cache-control metadata, input hashes, session, settings
with nested provider values, and a structured schema. Inside `OnPrepared`,
retain and mutate the delivered mutable values; the model request, captured
exchange, and later prepared-handle `Details()` remain unchanged. Use
synthetic direct/environment credentials and verify their values are absent
from diagnostic details while the existing client credential path still works.
2. Demonstrate snapshot fidelity when mutable input/schema sources change after
preparation and before execution. Reuse existing frozen-preparation fixtures;
notification must describe the executed snapshot, not reread the changed
sources. A later provider failure or cancellation may return no result, while
the application still retains the delivered details.
3. Verify absence of notification for preparation failure and ordinary admission
rejection, without requiring extra reads to produce diagnostics. For split
execution cover invalid/foreign/discarded/reused handles and pre-notification
cancellation. `Details()`, `Prepare`, `ValidateRequest`, and preparation of an
executable handle never notify or start this execution timer.
4. Verify that retries and structural repair do not repeat notification. Reuse
an existing retry-plus-repair integration scenario with an added notification
assertion rather than duplicating its full event/usage test elsewhere.
5. Recover a notification panic only in test code, then verify ordinary capacity
is reusable and a claimed handle stays consumed. Do not require a terminal
event after panic or impose a prohibition on callback reentrancy. A capacity
one probe documents held admission rather than promising nested success.
Validation: run affected public contract and private payload-cleanup tests under
the race detector. Mutable callback state in concurrent tests must be protected
by the test; do not make PromptKit serialize application callbacks globally.
## Stage 7: Verify Coordinated Turns And Observer Composition
Outcome: coordination obtains the controls through ordinary `Run`, retaining
its existing branching and delivery semantics.
1. Check the coordinator's existing request-value copy preserves both new
controls while it replaces `ContinueFrom` and composes observers. No separate
coordinator timeout, executor, or public prepare/run sequence is needed.
2. Add a controlled-time, two-continuation success case. Each turn finishes
within its own timeout while their combined execution exceeds one turn's
duration. Callback details contain each turn's assembled history; no timer
from an earlier turn controls a later turn.
3. In the next-turn callback, verify the parent context is active and the prior
child context has been cleaned up. Run an explicitly selected assessment on
the same backend with concurrency one and queue capacity zero, using its own
timeout. This differs from `OnPrepared`, where admission is still held.
4. Exercise a locally timed-out continuation: coordination returns its ordinary
operational error with the marker, does not fabricate an outcome or completed
exchange, and leaves the parent and a sibling continuation usable. Retained
details are available if notification occurred before failure. Do not promise
them if execution stopped earlier.
5. Compose request and coordinator observers with an explicit execution group.
Check one ordinary lifecycle per selected run, normal terminal usage/attempt
totals, and no implicit membership for assessment calls. Reuse existing
final-turn and parent-cancellation/undelivered-result cases to verify that
cleanup and slow terminal observation have not changed delivery semantics.
Validation: run the coordinator, observation, group, and new execution-control
contract cases together under the race detector. Keep tests at the public
boundary for the assembled behavior; the internal coordinator loop need not
be redesigned solely because its ordinary run gained options.
## Stage 8: Document, Demonstrate, And Validate The Assembled Feature
Outcome: the implementation and its supported consumer workflow are complete,
with documentation in the canonical owners.
1. Finish GoDoc on `RunRequest`, `ErrOperationTimeout`, `Run`, `Prepare`,
`ValidateRequest`, `PrepareExecution`, `RunPrepared`, and `PreparedExecution`.
Cross-reference the owning field/error contract rather than duplicating its
entire definition. Explain conditional notification, held admission,
cooperative cancellation, split preparation, callback ownership, and outcome
finalization before terminal observation.
2. Update the [consumer guide](../consumers/pkg-promptkit.md) with small ordinary,
coordinated, and split usage sketches linked to GoDoc. Show a caller-owned
retained `PreparedRun` on failure, `errors.Is` checks for both timeout
identities, and a separate budget for callback-owned assessment. Do not add
persistence, log destinations, redaction policy, or a mandatory diagnostic
write gate.
3. Extend the maintained [observation example](../../examples/go-library/observe/main.go)
and its semantic test with a positive operation budget on continuation,
selected prepared-details retention, and a separately budgeted next-turn
assessment. Keep it offline, synthetic, and successful; Stage 5/6 tests own
failure demonstrations. Do not add another standalone example framework.
4. Update [runner documentation](../internal/runner.md),
[observation documentation](../internal/observation.md), and the
[component inventory](../internal/overview.md) only where responsibilities or
lifecycle descriptions changed. No ConfigKit, collection-format, provider
payload, or architectural policy change is required by this work.
5. Review Audita's six acceptance scenarios and the feature roadmap's validation
targets against the completed tests. Consolidate duplicate tests; do not
expand this into a full cross-product of all entry points, phases, observers,
and callback behaviors. Retain existing no-option regression coverage.
6. Select the newest installed Go toolchain per the development guide. Format
changed Go files, run `go test -race ./...` and `go vet ./...` with
`GOTOOLCHAIN=local` and `GOWORK=off`, and run `git diff --check`. The full Go
suite includes the maintained examples; no second example execution gate is
needed. Use writable caches as required by the development environment.
Validate local links and inspect the full diff for unintended files/secrets.
Do not edit published v0.22.0 release notes, create a new release/tag, or require
CI or external publication to finish the sprint. Release preparation and
retirement of these roadmaps are subsequent work.
## Completion Criteria
- Ordinary and coordinated runs apply one optional whole-operation budget;
split handles retain duration and callback selection without live idle timers.
- Deadline attribution survives public wrapping, distinguishes this scope's
recorded timer cause, and agrees with terminal observation before cleanup.
- Notification supplies a detached redacted view of the actual logical prepared
request, once before generation when reached, without repeated preparation.
- Failure, cancellation, and panic paths preserve existing resource, handle,
error, observation, grouping, and continuation behavior.
- The maintained example, canonical documentation, focused tests, full race
suite, and vet are complete. No-option use stays compatible.
No significant open questions remain. The API and nonpositive-duration choices
above complete the planning decisions left by the accepted feature roadmap.