diff --git a/README.md b/README.md index f8fdb03..ea08acf 100644 --- a/README.md +++ b/README.md @@ -45,6 +45,10 @@ boundary and constraints that framework work must preserve. ## Release Guidance +The [v0.23.0 release notes](docs/releases/v0.23.0.md) cover per-operation +timeouts and opt-in prepared-request notifications for ordinary and coordinated +execution. + The [v0.22.0 release notes](docs/releases/v0.22.0.md) cover captured conversation exchanges, application-directed turn coordination, and explicit execution groups. diff --git a/docs/releases/v0.23.0.md b/docs/releases/v0.23.0.md new file mode 100644 index 0000000..94ffed0 --- /dev/null +++ b/docs/releases/v0.23.0.md @@ -0,0 +1,62 @@ +# Promptkit v0.23.0 + +This minor release adds optional whole-operation timeouts and prepared-request +notifications to ordinary and coordinated execution. Applications can bound +each continuation independently and retain the request selected for execution +without replacing Promptkit's turn loop or preparing the same request twice. + +## Bound Individual Operations + +`RunRequest.OperationTimeout` selects a fresh cooperative budget covering +preparation and execution, including capacity waiting, retries, validation, +structural repair, and prepared-request notification. It is independent of the +provider generation timeout. Zero or negative values select no additional +timeout; existing parent cancellation and earlier deadlines remain effective. + +For split execution, `PrepareExecution` retains the duration without starting +a timer. `RunPrepared` starts that budget after claiming the handle, so prior +preparation and idle time do not consume it. Callers can still supply one shared +deadline context to bound both calls together. + +Local budget failures match both `ErrOperationTimeout` and +`context.DeadlineExceeded`. Each coordinated turn uses its selected budget; +expiry stops that coordination through the existing operational-error path +without canceling its parent or sibling work. Completed outcomes are fixed +before terminal observation and are not changed by a slow terminal observer. +Cancellation does not forcibly interrupt callbacks or collaborators that ignore +their context. + +See the [execution-controls guide](../consumers/pkg-promptkit.md#bound-execution-and-retain-prepared-details) +and the [public request contract](../../types.go) and +[timeout error contract](../../engine.go) for exact behavior. + +## Retain Prepared Details Before Generation + +`RunRequest.OnPrepared` optionally receives a detached, credential-redacted +`PreparedRun` and the operation context before first generation. It uses the +actual prepared snapshot, including continuation history, without rereading +inputs or rerendering. Applications may retain these details even if generation +later fails without returning a result. + +Notification is synchronous, at most once per logical execution, and runs while +logical admission is held. It is not repeated for retries or structural repair. +Preparation or admission failure can prevent delivery. The application owns +diagnostic storage and disclosure decisions; ordinary observers remain +metadata-only. See the [OnPrepared GoDoc](../../types.go) and updated +[offline observation example](../../examples/go-library/observe/main.go). + +## Compatibility And Upgrade + +Both controls are opt-in. Existing keyed requests and configuration files need +no migration from v0.22.0. `Prepare` and `ValidateRequest` continue using only +their supplied contexts and do not invoke `OnPrepared`. `RunRequest` gained +fields; consumers using positional struct literals must switch to keyed +literals. Prepared/result JSON representations are unchanged. + +The Go consumer minimum remains 1.27.0, and dependency versions are unchanged. + +```sh +go get go.maximumdirect.net/promptkit@v0.23.0 +``` + +[COPYRIGHT](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.0/COPYRIGHT) | [LICENSE](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.0/LICENSE) | [ATTRIBUTION](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.0/docs/oss/attribution.md) diff --git a/docs/roadmap/audita-request.md b/docs/roadmap/audita-request.md deleted file mode 100644 index fafdf0f..0000000 --- a/docs/roadmap/audita-request.md +++ /dev/null @@ -1,139 +0,0 @@ -# Audita request: coordinator execution controls - -Status: proposed follow-up to PromptKit v0.22.0. This document describes -consumer requirements; public API design and implementation remain upstream -decisions. - -## Purpose - -PromptKit v0.22.0 supplies the conversation capture, continuation, and execution -groups Audita requested. Audita can adopt those capabilities through its existing -execution adapter. To also adopt `Engine.Coordinate` and remove its local turn -loop, Audita needs two capabilities during coordinator-owned execution: - -1. A fresh, application-selected timeout for each continuation operation. -2. Access to the exact prepared request before generation, including for turns - that subsequently fail without returning a result. - -Both capabilities apply to applications that run bounded model operations and -retain optional diagnostic evidence. Neither requires PromptKit to understand -transcripts, correction proposals, or domain validation. - -## Current workflow and integration gap - -Audita's role-bound adapter currently derives a timeout context from its caller, -calls `PrepareExecution`, retains `PreparedExecution.Details()`, and executes -that same handle with `RunPrepared`. The operation deadline covers preparation, -capacity waiting, transient retries, generation, and structural repair. Each -model operation receives its own budget; application validation may make -additional, separately bounded model calls. - -When debug evidence is enabled, Audita retains the prepared messages and -effective settings alongside the available response or failure information. -It owns artifact destinations, application-specific redaction, and handling of -optional diagnostic write failures. The same adapter supplies ordinary execution -observers and classifies operation errors for the application. - -In v0.22.0, the [coordinator](../../coordinate.go) accepts a callback returning a -`RunRequest`, then directly calls `Engine.Run` with the coordinator's context. -The callback cannot supply a fresh execution context or a prepared handle. -There is no callback exposing the actual prepared request before that run. - -A timeout around the entire coordinator bounds the conversation and application -callbacks together; it does not preserve a separate timeout for each operation. -Provider generation timeouts do not cover the whole operation either. -Separately calling `Prepare` before returning a request duplicates preparation -and does not bind the inspected snapshot to the execution. Successful exchange -inspection cannot supply a failed turn's prepared request when no new exchange -is returned. Existing metadata-only observations do not contain that content. - -## Requested behavior - -### Per-operation deadlines for coordinated turns - -Allow the application to select a timeout for each requested continuation, with -these semantics: - -- Establish a fresh deadline when that continuation starts, before its - preparation. Cover preparation, admission waiting, retries, generation, and - structural repair under the same operation budget. -- Respect caller cancellation and any earlier parent deadline. Expiry of a - local operation budget must not cancel the parent context or independent - branches. -- End the operation's timeout scope before invoking the application's next-turn - callback. Assessment calls made there can retain their own operation budgets. -- Preserve useful error identity so Audita can distinguish a local deadline - from cancellation of the caller. An execution timeout can end coordination - under the existing operational-error rule; no automatic turn replay or new - recovery policy is requested. - -The existing additional-turn budget, transport retry budget, structural-repair -budget, and provider generation timeout retain their separate meanings. - -### Exact prepared-request access - -Provide an opt-in way to obtain caller-owned details of the actual prepared -continuation before its first generation attempt: - -- Include the assembled captured history, follow-up definition messages, - appended messages, and ordinary resolved execution metadata, using the - existing [prepared-details contract](../../prepared_execution.go) where useful. -- Expose the snapshot that execution will use, without a second rendering or - input-reading pass. Access to details must not mutate that execution. -- Let the application retain those details even if generation subsequently - fails or is canceled and returns no ordinary result or captured exchange. - If preparation itself fails, a complete prepared snapshot is not required. -- Preserve existing credential-redaction semantics. Keep content-bearing access - explicitly selected and separate from ordinary metadata-only observations. - -Audita needs the logical prepared request, not raw HTTP envelopes or an archive -of intermediate structural-repair requests. It can handle optional debug writes -itself without requiring PromptKit to own artifact storage or reporting policy. - -## API direction and ownership - -Prefer small execution-level controls that the coordinator honors through its -ordinary execution path. An optional operation timeout and a prepared-details -callback are one possible shape, not proposed exported names or signatures. -Their applicability to ordinary and prepared execution should follow a coherent -lifecycle rather than introducing a second coordinator-specific executor. - -An alternative is allowing a coordinator turn to delegate execution through a -caller-controlled prepared-execution path. That could reuse Audita's adapter, -but exposes more execution responsibility and requires clear ownership of -capture, observers, admission release, and prepared-handle cleanup. Evaluate -that option only if it is simpler than the narrow controls above. - -PromptKit should continue owning history assembly, turn progression, execution, -and observations. Audita retains response interpretation, validator calls, -branch policy, diagnostic storage, and application error classification. Existing -request observers, coordinator observers, and execution groups must remain -usable together. This request does not reopen the accepted conversation history, -final-turn delivery, or grouping contracts. - -## Acceptance scenarios - -1. Two successful continuation operations each finish within their own timeout, - while their combined duration exceeds one operation's budget. Both succeed - under an active parent context. -2. Preparation or admission waiting consumes the same budget as generation and - structural repair. Local expiry returns a recognizable deadline error and - stops that coordinator without canceling its parent or a sibling branch. -3. An earlier parent deadline remains effective. A validator called from the - next-turn callback runs after admission is released, including with backend - capacity one, under its own application-selected deadline. -4. The application receives the prepared continuation details before generation. - A later provider failure returns no result, but the retained details still - describe that failed operation. Mutating a details copy does not change the - messages executed. -5. Diagnostic access uses the same prepared snapshot that executes, including - when source inputs change afterward. It does not trigger another preparation - or provider call. -6. Existing observers and selected execution groups continue receiving ordinary - execution facts once per their existing contracts, including failure paths. - Applications that select neither new capability retain existing behavior. - -The consumer implementation motivating this request is Audita's -`internal/framework/llm/adapter.go`, especially `RoleClient.CompleteStructured`. -This gap affects full coordinator adoption; it does not block adopting v0.22.0 -capture, continuation, or execution groups independently. diff --git a/docs/roadmap/execution-controls.md b/docs/roadmap/execution-controls.md deleted file mode 100644 index 6e0ba3b..0000000 --- a/docs/roadmap/execution-controls.md +++ /dev/null @@ -1,291 +0,0 @@ -# Execution Controls For Coordinated Turns - -Status: all four design decisions accepted. This document defines the target -state from v0.22.0; [implementation.md](implementation.md) selects the concrete -API and staged implementation work. The feature is not yet implemented. - -## Purpose - -Enable applications using `Engine.Coordinate` to select a fresh timeout for each -continuation and retain the exact prepared request before generation. These are -execution capabilities useful for bounded operations and optional diagnostics, -without assumptions about prompt content or application workflows. - -[Audita's request](audita-request.md) supplies the motivating requirements. -Audita already uses one child context across explicit preparation and execution, -and retains prepared details for diagnostics. The goal is to make those controls -available during coordinator-owned execution without requiring a downstream -replacement for PromptKit's turn loop. - -PromptKit owns history assembly, execution, cancellation propagation, and -observations. Applications select budgets, interpret results, run validators, -schedule branches, and own diagnostic destinations, disclosure policy, and write -failure handling. This work adds no artifact store or reporting framework. - -## Current Baseline - -[PromptKit v0.22.0](../releases/v0.22.0.md) supplies captured exchanges, -continuation, sequential coordination, and explicit execution groups. - -- The [coordinator](../../coordinate.go) attaches the current exchange and composes - request/coordinator observers, then calls `Engine.Run` with the parent context. - Its callback returns a request, not an execution context or prepared handle. -- Ordinary [run execution](../../internal/usecase/runner.go) resolves the target, - obtains logical admission, completes preparation, and generates and validates - the response. Admission is released before the next-turn callback. -- Explicit [prepared execution](../../prepared_execution.go) already supplies - detached `Details()` and executes the retained snapshot. Preparation and - execution are separate calls; preparation does not reserve backend capacity. -- Ordinary [observations](../internal/observation.md) are metadata-only. Ordinary - runs observe preparation failures; split preparation itself emits no execution - events. A claimed prepared handle starts its observation lifecycle at execution. - -A timeout around the entire coordinator includes all turns and application -callbacks. Provider timeouts cover a different scope. Neither provides a fresh -whole-operation budget for each selected turn. Separately calling `Prepare` -would duplicate preparation and would not bind the inspected details to the -request actually executed. An exchange cannot supply a failed turn's prepared -request when execution returns no result. - -## Target End State - -The selected approach is optional per-request controls honored by ordinary `Run` -and therefore by `Coordinate`. The accepted decisions below specify their -lifecycle; the coordinator does not delegate execution to an application executor. - -### Optional Operation Timeout - -An application can select a duration for each requested continuation. When that -continuation starts, establish a fresh child deadline before preparation and use -it for the entire operation: preparation, admission waiting, retries, generation, -structural validation, and structural repair, including the prepared-details -notification. Do not restart the budget between phases or attempts. - -The selected timeout covers work according to the entry point: - -| Entry point | Selected timeout behavior | -| --- | --- | -| `Run`, including coordinated turns | Starts before preparation and covers preparation and execution. | -| `PrepareExecution` | Retains the duration for later execution; preparation uses its supplied context. | -| `RunPrepared` | Starts the retained duration when invoked, covering the remaining execution. | - -Idle time between split preparation and execution does not consume the retained -execution budget. Applications wanting one budget across both calls can pass the -same caller-created deadline context through them. `Prepare` and `ValidateRequest` -continue using their supplied contexts without starting the execution timeout. - -Respect parent cancellation and earlier parent deadlines. A local timeout ends -the affected operation and coordination under the existing operational-error -rule; it does not cancel the parent or siblings. Release child-context resources -before returning to the application's next-turn callback. Assessment calls from -that callback retain their own application-selected contexts and budgets. - -Determine the execution result and any local-timeout identity before publishing -the terminal observation, so the returned error and observed outcome agree. -A slow terminal observer or subsequent child-context cleanup must not -retroactively turn a completed operation into a timeout. Preserve the existing -coordinator behavior for completed responses, parent cancellation before -next-turn callback delivery, and the single undelivered response. No automatic -replay or new recovery policy is added. - -No selected timeout means existing context behavior. This adds no framework-wide -recommended duration or mandatory cap. Additional-turn, transport-retry, -structural-repair, and provider-generation budgets keep their existing meanings. -Cancellation remains cooperative; a deadline does not forcibly terminate -application callbacks or injected collaborators that ignore their context, or -guarantee that the public call returns by that deadline. - -### Opt-In Prepared Details Before Generation - -Provide explicitly selected access to caller-owned details of the logical request -selected for execution, after successful preparation and before its first -generation attempt. Reuse the existing prepared-details representation and -credential redaction where practical. Include assembled captured history, rendered -follow-up messages, caller-appended messages, and ordinary resolved execution -metadata. - -Use the execution's prepared snapshot rather than rendering or reading inputs -again. Retaining or modifying delivered details must not mutate the execution. -Once delivered, details remain available to the application after provider failure -or cancellation even when no ordinary result or exchange is returned. Diagnostic -access is available only when execution reaches the prepared-notification point, -not for every failed operation. Preparation failure requires no complete snapshot; -ordinary admission failure or timeout may occur before preparation completes. -Do not add rendering or rearrange admission solely to manufacture diagnostics for -those failures. Cancellation before notification does not promise delivery; -follow the context checks in [Decision 3](#3-what-is-the-prepared-details-callback-contract). - -This access exposes logical prepared content, not raw HTTP envelopes or internal -structural-repair requests. Keep it separate from metadata-only observers and -execution-group summaries. It is not another next-turn callback: ordinary -execution may already hold backend admission at the notification point. - -### Compatibility And Composition - -Applications selecting neither capability retain existing behavior. Preserve -conversation assembly, capture, final-turn delivery, branching, and grouping -contracts. Request observers, coordinator observers, and explicitly selected -groups continue receiving ordinary execution facts without duplicate lifecycle -events or counting attempts twice. - -Preserve ordinary admission, preparation-failure observation, and ownership -behavior. A simple replacement of ordinary `Run` with the public -`PrepareExecution`/`RunPrepared` sequence would change those behaviors. This does -not freeze the internal call structure: consolidation or refactoring is welcome -where it preserves the selected lifecycle. - -Document which controls each entry point applies or retains. Execution-only -controls need not take effect during `Prepare` or `ValidateRequest`; accepting -request-shaped input there does not itself justify rejecting such controls or -starting a timer. Follow the accepted split-preparation and callback semantics. - -## Affected Areas - -The target spans request controls, execution-context lifetime, detached prepared -details, public error projection, and ordinary coordinator integration. It is an -additive runtime feature; no collection-format or ConfigKit changes are needed. -Public GoDoc will own exact API contracts, the -[consumer guide](../consumers/pkg-promptkit.md) will explain use, and internal -lifecycle documentation will describe the implementation when it lands. - -The implementation plan owns API selections, task sequencing, concrete changes, -and validation steps. Keep that work proportional to the two requested controls. - -## Validation Targets - -Use offline public-boundary tests and existing execution collaborators, with -controlled cancellation and synchronization rather than fragile timing margins. -Cover the consumer request's [acceptance scenarios](audita-request.md#acceptance-scenarios) -and the accepted lifecycle choices: - -- Successive turns receive fresh budgets: their combined duration may exceed one - operation budget without exhausting either individual operation. -- For ordinary and coordinated runs, preparation, admission waiting, retries, - validation, and repair share the same operation deadline. Local expiry stops - coordination without canceling parent or sibling work; earlier parent - cancellation/deadlines remain effective. -- Next-turn assessment uses the parent context after release, including another - call to a backend with capacity one. Timeout identity survives public wrapping. -- A slow terminal observer or child-context cleanup does not change a completed - outcome into a local timeout; the returned error and terminal observation agree. -- Prepared details precede generation, contain the assembled continuation, and - remain available after later failure. Mutation of the delivered copy or source - changes after preparation do not change the executed snapshot. -- Diagnostic access introduces no additional preparation or inference calls, - notification per internal repair, or duplicate observer lifecycle. Ordinary - retries and structural repair remain allowed. Credential redaction remains - intact. -- Admission or preparation failure before the notification point does not require - diagnostic delivery or additional rendering to produce a snapshot. -- Preparation failure, cancellation around notification, and callback panic follow - the selected delivery rules and do not strand admission or retained resources. -- Split preparation uses its supplied context and retains the selected duration - for `RunPrepared`; idle time does not consume that execution budget. A shared - caller deadline can still bound both calls. Neither discarded nor unsuccessfully - claimed handles start execution notifications. -- The prepared notification receives detached details and the operation context, - has no error return, and runs synchronously within the operation budget. -- Local-budget expiry has a narrow inspectable identity while preserving standard - context matching; parent and provider deadlines are not mislabeled as local. -- Observers/groups retain ordinary facts on success and failure. No-option use, - completed-result delivery, and final-turn behavior remain compatible. - -## Accepted Decisions - -All four decisions below are accepted. No significant product decisions remain -open. The implementation plan selects public names, signatures, timeout-value -conventions, and the Go representation of timeout identity. - -### 1. Narrow Execution Controls Or Caller-Delegated Execution? - -**Accepted approach:** add optional per-request execution controls honored by -ordinary `Run` and thus by `Coordinate`, including a timeout and -prepared-details callback. This keeps history, capture, admission, errors, and -observers in their existing owners and makes the capability useful outside -coordination. A caller can select different controls on each returned request; -an additional coordinator-wide configuration layer is not needed for the -demonstrated use case. - -**Alternative considered, not selected:** let the coordinator delegate a turn -to a caller-provided executor or prepared-execution adapter. This would reuse -Audita's existing adapter and accommodate other execution customizations. It -also requires a larger contract for exchange assembly, capture, observer -composition, handle cleanup, and admission release. The narrow controls meet -the demonstrated need with fewer ownership questions; caller-delegated -execution is not part of this feature. - -### 2. What Does A Timeout Mean For Split Preparation And Execution? - -**Accepted approach:** bound each `Run` invocation from before preparation. For -`PrepareExecution`, retain the selected duration as an execution control and -apply it when `RunPrepared` starts; preparation itself continues using its -supplied context. Do not retain a live timer across an idle prepared handle. -Explicitly explain that this bounds the remaining execution, not preparation -already done. Callers needing one total budget across both calls can continue -passing the same caller-created deadline context, as Audita does today. - -This preserves the distinction between executing now and preparing for later, -while letting a prepared handle carry the same selected execution controls. It -requires clear documentation: the duration alone does not retroactively cover -split preparation. `Prepare` and `ValidateRequest` likewise do not become timed -executions merely because they accept request-shaped input. - -**Alternative considered, not selected:** start a budget during -`PrepareExecution` and carry its remaining deadline into `RunPrepared`. This -directly bounds preparation and execution together without caller context -plumbing. It also counts idle time, requires timer/context ownership and -discard semantics, and changes what a prepared handle retains. This is useful -if a consumer needs a whole prepared operation lifetime, but exceeds the -current coordinated-turn requirement. - -### 3. What Is The Prepared-Details Callback Contract? - -**Accepted approach:** a synchronous notification receiving the operation -context and a detached prepared value, with no error return. Invoke it once -after the snapshot is ready and before generation, only when execution proceeds -to that point. For prepared handles, retain the callback until execution and -clear it on claim/discard like the existing observer; inspection remains -available earlier through `Details()` without triggering the notification. - -Check context before notification and again before generation. Include callback -time in the operation budget and retain ordinary cooperative cancellation and -panic propagation, with resource cleanup on unwind. Do not add background delivery -or stronger terminal-event guarantees. The application can retain details and -perform optional writes under its own policy; it need not turn a diagnostic write -failure into an inference failure. - -Under the current ordinary run path this notification executes while logical -admission is held. Document that fact rather than promise the next-turn callback's -capacity-one nested-execution behavior. Do not prohibit arbitrary callback work; -applications must account for the held capacity and operation budget. Releasing -and reacquiring admission solely for notification is not necessary for this scope. - -**Alternative considered, not selected:** let the callback return an error to -veto generation. This supports mandatory audit persistence or an application -pre-generation check, with ordinary cleanup and error propagation. It also -makes the callback an execution gate and requires failure classification -semantics absent from the optional-diagnostics request. The selected -notification satisfies that request; a generation veto would require a separate -demonstrated need. - -### 4. How Precisely Should Local Timeout Errors Be Identified? - -**Accepted approach:** preserve standard context matching and provide a narrow, -inspectable operation-timeout identity when this newly selected budget causes -termination. Keep parent cancellation/deadlines and provider-local failures -under their existing identities. The marker must survive public wrapping and -must not be inferred from any arbitrary `DeadlineExceeded` returned by a -collaborator. Its Go representation is selected in the implementation plan; -no new error-report hierarchy or terminal-event category is implied. - -This lets callers distinguish the requested local budget from an earlier parent -or provider deadline without retaining the internal child context. Define cause -selection around races using the child context's recorded cause; do not promise -perfect ordering of independently simultaneous cancellations. - -**Alternative considered, not selected:** return only existing context errors -and let callers inspect their parent context. This is smaller and often -sufficient when the parent remains active. However, `DeadlineExceeded` alone -may also originate in a provider timeout, and a parent may become canceled -before the caller inspects it. That weaker attribution does not meet the -selected distinction as directly as a narrow operation-timeout identity. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 481e8fe..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,457 +0,0 @@ -# Execution Controls Implementation Plan - -## Objective And Sprint Scope - -Implement the accepted target in [execution-controls.md](execution-controls.md) -from the v0.22.0 execution architecture. The feature roadmap owns intent and the -four accepted lifecycle decisions; this document selects the API and the work -needed to implement them. [Audita's request](audita-request.md) supplies the -consumer scenarios. - -Work through Stage 1 through Stage 8 in order. Each stage is scoped for one Luna -coding-agent prompt. This is one end-to-end sprint: intermediate stages may -temporarily fail to compile or leave wiring incomplete. Do not build temporary -compatibility layers merely to make intermediate commits green. There are no -intermediate human-review or external-publication gates. Completion requires -the assembled feature and its validation, not just completion of individual -stages. - -Read [development.md](../development.md), the [agent policy](../policy/agents.md), -and the applicable architecture, testing, and documentation policies before -work. Preserve unrelated changes. Follow the existing commit policy for finished -units; report intermediate validation honestly. This plan authorizes no release, -push, dependency update, configuration-format change, or new consumer workflow -beyond the execution controls described here. - -## Selected API And Semantics - -Add these fields to the existing public `RunRequest`, without introducing an -execution-options object, coordinator executor, or general hook framework: - -```go -OperationTimeout time.Duration `json:"-" yaml:"-"` -OnPrepared func(context.Context, PreparedRun) `json:"-" yaml:"-"` -``` - -Add one public sentinel, alongside the current execution errors: - -```go -var ErrOperationTimeout = errors.New("operation timeout exceeded") -``` - -The spelling of an error message is not a contract. No exported timeout error -type, new observer event, or new failure-category enum is needed. - -### Entry-Point Contract - -| Entry point | OperationTimeout | OnPrepared | -| --- | --- | --- | -| `Run` | A positive value starts a fresh child budget before preparation, covering preparation and execution. | Notify once if execution reaches the prepared point before first generation. | -| `Coordinate` | Each returned request receives ordinary `Run` semantics with a fresh scope. | Ordinary per-request notification; no coordinator-specific copy of the option. | -| `PrepareExecution` | Retain the value privately, without starting a timer. Preparation uses its supplied context. | Retain the selected function privately without invoking it. | -| `RunPrepared` | Start the retained positive duration after the successful claim, before execution callbacks, credential revalidation, and admission. | Notify once when the claimed execution reaches the prepared point. | -| `Prepare`, `ValidateRequest` | Do not start a timer; use the supplied context as today. | Do not invoke or retain the notification. | -| `PreparedExecution.Details`, `Discard` | Do not start a timer. | Do not invoke the notification. Discard drops its retained reference. | - -- A positive duration opts in. Zero and negative durations select no additional - timeout. This follows a nonpositive-means-unselected convention, avoids a new - request rejection rule, and leaves the parent deadline effective. There is no - default operation duration, upper bound, profile setting, or engine setting. -- The provider's existing generation timeout remains independent. Do not copy - `OperationTimeout` into `ExecutionTarget`, provider JSON, or prepared/result - metadata. Neither control changes prompt hashes or configuration inspection. -- The budget is continuous across preparation, capacity waiting, notification, - generation, transient retries/backoff, validation, and structural repair. It - is not restarted between phases, attempts, or repair generations. -- Split preparation retains a duration, not a context, deadline, timer, or - cancel function. Idle time does not count. A shared caller-created deadline - remains the way to bound preparation and later execution together. -- A nil notification selects no diagnostic copying. A selected function receives - the operation context and a detached, credential-redacted `PreparedRun` value - representing the logical request. Its nested mutable values belong to the - application. Reuse the existing projection used by `Details()`. -- Notification is synchronous, has no error return, and runs after successful - preparation and admission, before the first generation. Logical admission is - still held. Its time consumes the selected operation budget. It may retain - details for later diagnostics; PromptKit does not own writes or write errors. -- Check context before notification and again before generation after it - returns. A callback may block or ignore context; do not introduce a goroutine - to interrupt it. Callback panic propagates after deferred resource cleanup, - without a new terminal-event guarantee. -- Callback selection is copied as a function value, not as a deep copy of its - closure. Shared callback state is application-synchronized; separate runs can - invoke a common callback concurrently. Later reassignment of the original - request field cannot replace a prepared handle's retained function. -- Notification is conditional on reaching that execution point. Admission or - preparation failure can prevent it. Do not render again, reorder admission, - or promise a diagnostic snapshot for every failed operation. Split callers - can still inspect a successfully prepared handle through `Details()`. - -### Timeout Attribution And Completion - -Use the standard library's `context.WithTimeoutCause` with a distinct cause -value for each selected scope. Record the scope's own cause identity so an -inherited timeout from an enclosing operation cannot be mistaken for this -invocation's timer. Do not classify by error text or by `DeadlineExceeded` -alone. No strict ordering of independently racing cancellations is promised. - -When an execution fails through its context deadline and the child context's -recorded cause is this scope's own timer cause, return an error matching both -`ErrOperationTimeout` and `context.DeadlineExceeded`. Preserve existing operation -and collaborator error identities. An earlier parent deadline/cancellation or -a provider deadline must not acquire the local marker merely because it is a -deadline error. An unrelated failure must not acquire it merely because the -timer has also expired. Parent-provided error causes, if any, retain their -existing behavior; this feature need not reinterpret them. - -Finalize attribution from the operation's returned error before canceling its -child scope and before terminal observer delivery. A nil execution error remains -a completed result, including a completed structural rejection. Do not add a -post-terminal `ctx.Err()` check that converts it into an error. Slow terminal -observers and deferred context cleanup cannot change the chosen outcome. This -does not strengthen cancellation into forced interruption or a guarantee that -every context-ignoring collaborator will return by the deadline. - -The terminal event continues using the existing deadline failure category for -a timeout error. Groups continue aggregating those ordinary terminal facts. -The child scope is cleaned up before `Run`/`RunPrepared` returns to the next-turn -callback; that callback receives the coordinator parent context, not the child. -Existing parent cancellation and undelivered-result rules remain unchanged. - -## Implementation Map - -Use existing owners rather than adding a new public package: - -- Public declarations and contract: [types.go](../../types.go), - [engine.go](../../engine.go), and [prepared_execution.go](../../prepared_execution.go). -- Public conversion and error projection: [convert.go](../../convert.go), - [errors.go](../../errors.go), and [observations.go](../../observations.go). -- Internal request values: [domain.go](../../internal/domain/domain.go). -- Timeout lifecycle and notification orchestration: `internal/usecase`, chiefly - [runner.go](../../internal/usecase/runner.go) and - [prepared_execution.go](../../internal/usecase/prepared_execution.go). -- Existing private operation-error representation: - [operation_error.go](../../internal/usecase/operation_error.go). -- Ordinary coordinator integration: [coordinate.go](../../coordinate.go) and - [internal coordinator](../../internal/usecase/coordinate.go). - -The root remains a facade. Keep timer construction and cause-attribution logic -in a small internal helper; the facade may establish that scope and project its -errors while managing its existing observation lifecycle. Internal packages must -not import the root package or depend on the public `PreparedRun` representation. -Private helper names and equivalent internal organization may change; the -selected public behavior and ownership boundaries are what matter. - -## Stage 1: Add The Request Contract And Conversion - -Status: complete. - -Outcome: the selected API and internal request plumbing exist, ready for the -execution stages to consume. - -1. Add `OperationTimeout`, `OnPrepared`, and `ErrOperationTimeout` to the public - declarations above. Write their GoDoc from the entry-point and ownership - contract in this plan, explicitly separating operation and provider timeouts. -2. Extend `domain.RunRequest` with the duration and an internal callback of shape - `func(context.Context, *domain.PreparedRun)`. Document the callback argument - as borrowed read-only internal state. It is not a public mutable snapshot. -3. Extend `toDomainRunRequest` to copy the duration and capture the selected - public callback function in a local variable. Only when it is non-nil, adapt - it by projecting the supplied prepared state through `fromDomainPreparedRun` - and passing the resulting value. This projection is the public ownership - boundary; do not deep-copy the domain snapshot first as well. -4. Keep these fields out of public prepared/result JSON and opaque handle - details. Request conversion does not start a timer, invoke a callback, or - reject an execution-only control on preparation-only entry points. -5. Add the private timeout error identity in `internal/usecase`, using the same - existing separation between internal and public sentinels. Add its public - projection to `errors.go` when introducing the attribution wrapper in Stage 2. - -Validation: focused conversion tests should establish option forwarding and -conditional adaptation. Reuse existing nested-value projection tests instead of -retesting every existing `PreparedRun` field. Compilation may remain incomplete -until the execution wiring lands; do not claim the feature works at this stage. - -## Stage 2: Bound Ordinary Execution And Preserve Its Error Outcome - -Status: complete. - -Outcome: `Engine.Run` owns one continuous optional budget and returns the same -failure classification it publishes to observers. - -1. Add a small internal operation-scope helper using `WithTimeoutCause`. For - nonpositive durations, reuse the parent context without a timer. For positive - durations, retain a fresh per-invocation cause token and a cancel function. - Use a real distinct identity, not a shared sentinel or zero-sized token whose - pointer identity cannot distinguish scopes. -2. Give the helper an error-finalization operation: for a non-nil returned error, - require both a match to the operation context's deadline error and equality - of its recorded cause with this scope's token before adding the private - timeout identity. Leave nil errors and unrelated errors unchanged. Preserve - the original error as a cause. -3. Reuse `usecase.OperationError` with the private timeout category to carry the - marker around the existing error, rather than creating a public error - hierarchy. Project that category to `ErrOperationTimeout` through the - existing `publicOperationError` machinery. Keep the original phase category - and context cause inspectable through the nested chain. -4. Establish the ordinary scope in the facade's `Run` entry, after masking - inherited execution-observation state and before start-event delivery and - request preparation. Both observed and unobserved runs use it. Pass its - context through request conversion, internal engine delegation, admission, - preparation, and all generation/validation work. Do not also start a second - timer inside `Runner.Run`. -5. Adjust the private facade `run` path as needed to finalize the execution - error before applying public error projection once. On normal return, freeze - attribution and the returned result/error, clean up the child, then classify - and deliver the existing terminal event. Preserve normal observation of - request conversion and preparation failures. Handle all early returns; install - cleanup before any user callback can panic. -6. Do not change provider timeouts, retry eligibility, repair limits, scheduling, - or existing result timing conventions to implement this feature. A timer - expiring while terminal observation runs is not another execution failure. - -Validation: narrow helper tests cover disabled scopes, local expiry, parent -deadline/cancellation, inherited operation causes, unrelated errors, and nil -completion. Add public `Run` tests proving the marker survives phase wrapping -and agrees with an ordinary deadline terminal event. Provider-only deadline -errors while the operation child is active must not gain the marker. - -## Stage 3: Retain Controls Across Split Preparation - -Status: complete. - -Outcome: prepared execution carries the selected duration and notification, -without keeping a live operation scope while idle. - -1. Extend `preparedExecutionPayload` with the duration and internal notification. - `PrepareExecution` copies them from the converted request only after its - ordinary successful preparation. Preparation itself uses its supplied - context, without starting the selected timer or calling the notification. -2. Extend payload clearing for both fields. Preserve the existing atomic claim, - owner checks, foreign/reused-handle behavior, and durable redacted details. - A discard or completed/unwound claim drops PromptKit's callback reference. -3. After a successful `Runner.RunPrepared` claim, establish the scope with the - retained duration, before the retained observer's start delivery, credential - revalidation, and admission. Defer payload and scope cleanup before callback - work. A failed claim neither starts notification nor changes existing silent - invalid-handle behavior. -4. Pass the child through execution and finalize the returned error with the - same helper before cleanup and facade terminal classification. Keep this - lifecycle in the internal prepared runner rather than borrowing duration or - callback state from an unclaimed handle in the facade. -5. `RunPrepared`'s facade projects the resulting error and emits its one existing - terminal event after resource release. Do not reapply the timer at the facade - or retain the preparation context. Namespace views keep their shared-owner - behavior and use the handle's original selections. - -Validation: extend prepared-execution coverage to show that preparation and an -idle delay longer than the selected duration do not consume its execution -budget; later execution does. Reassignment of the original request does not -change the retained duration/callback. Cover earlier caller deadlines and a -shared caller deadline across preparation and execution. Existing claim-race, -discard, foreign-owner, credential, and error-identity tests remain applicable. -Notification invocation itself is completed in Stage 4. - -## Stage 4: Notify From The Actual Shared Prepared Snapshot - -Status: complete. - -Outcome: ordinary and prepared execution expose one detached logical request -before first generation, without repeating preparation. - -1. Thread the internal notification from `Runner.Run` and the claimed payload - into the existing shared `executePreparedRun` entry. At that entry, preparation - is complete and logical admission is held on both paths. -2. When selected, check the operation context, invoke the internal adapter with - the actual retained prepared state, and check context again before entering - generation. A context failure at this boundary follows the existing - generation operation-error wrapping, preserving context identity and gaining - local-timeout identity only through the scope finalizer. -3. Invoke once per executing logical request, not per transport retry or repair. - Do not call `Prepare`, `Details`, artifact readers, renderers, or schema - preparation again to produce the notification. With nil `OnPrepared`, avoid - constructing a public details copy. -4. Keep the prepared state's API key redacted. Generation still receives its - private direct credential through the existing separate path. Preserve the - complete assembled messages, schema, settings, input hashes, session, - ancestry, and preparation metadata in the delivered public value. -5. Leave logical admission held; do not release/reacquire it around notification. - Invoke application code without holding the handle mutex or a capacity - manager lock. Preserve panic propagation while defers release the lease, - timer, and claimed payload. A panic does not make a handle reusable. - -Validation: one ordinary and one split public test establish ordering before -generation and exact callback selection. Add focused context-before/after -notification tests and an internal cleanup regression if the public capacity -probe alone cannot demonstrate payload cleanup. More extensive ownership and -failure assertions belong to Stage 6, not duplicate tests here. - -## Stage 5: Verify Continuous Budgets Across Execution Phases - -Status: complete. - -Outcome: tests demonstrate that one scope reaches all relevant collaborators, -with correct attribution and observation on failure. - -1. Add focused tests for preparation expiry through a context-aware injected - artifact reader, generation-capacity waiting on a real limited backend, - interrupted built-in retry backoff, and structural repair expiry. Verify - standard context matching, local marker, and retained phase identity where - that phase already supplies one. Cover ordinary and split paths without - mechanically duplicating every case on both. -2. Use a narrow internal runner test for validation-phase expiry, since validation - collaborators are not a public injection API. Check the same deadline reaches - preparation, validation, and generation; do not add a new public testing seam. -3. Test that notification time consumes the same budget and that expiry there - prevents generation. Cover an earlier parent deadline and parent cancellation - separately from a provider's own deadline. Preserve original injected error - identities and do not relabel a non-context error after timer expiry. -4. On both ordinary and split public execution, exercise a terminal observer - that outlasts the operation budget after successful completion. The result - stays successful and the terminal remains successful. Include the timeout - failure case to ensure public error mapping happens before terminal delivery. -5. After expiry, prove capacity remains usable and the caller parent stays - active. Respect the current capacity model: logical admission may reject - immediately, while admitted work can wait for an active generation permit. - Do not introduce a new blocking admission queue to satisfy a test label. - -Use channels for ordering and `testing/synctest` for controlled elapsed-time -scenarios where its bubble rules apply. This repository's Go minimum supports -it. Prefer in-memory fixtures and controlled transports; avoid real short sleeps -or close-to-deadline assertions. Reuse the existing generation tests for retry -policy rather than building a second retry matrix. There is no production clock -framework, live provider probe, allocation threshold, or performance gate here. - -Validation: run focused use-case/generation tests and the new public execution -control tests under the race detector. The new cases should protect deadline -propagation and classification, not private helper call sequences. - -## Stage 6: Verify Notification Ownership And Failure Lifecycles - -Status: complete. - -Outcome: applications can safely retain diagnostic details without altering -execution or acquiring new lifecycle guarantees. - -1. Add a representative continuation request containing history, follow-up - messages, appended cache-control metadata, input hashes, session, settings - with nested provider values, and a structured schema. Inside `OnPrepared`, - retain and mutate the delivered mutable values; the model request, captured - exchange, and later prepared-handle `Details()` remain unchanged. Use - synthetic direct/environment credentials and verify their values are absent - from diagnostic details while the existing client credential path still works. -2. Demonstrate snapshot fidelity when mutable input/schema sources change after - preparation and before execution. Reuse existing frozen-preparation fixtures; - notification must describe the executed snapshot, not reread the changed - sources. A later provider failure or cancellation may return no result, while - the application still retains the delivered details. -3. Verify absence of notification for preparation failure and ordinary admission - rejection, without requiring extra reads to produce diagnostics. For split - execution cover invalid/foreign/discarded/reused handles and pre-notification - cancellation. `Details()`, `Prepare`, `ValidateRequest`, and preparation of an - executable handle never notify or start this execution timer. -4. Verify that retries and structural repair do not repeat notification. Reuse - an existing retry-plus-repair integration scenario with an added notification - assertion rather than duplicating its full event/usage test elsewhere. -5. Recover a notification panic only in test code, then verify ordinary capacity - is reusable and a claimed handle stays consumed. Do not require a terminal - event after panic or impose a prohibition on callback reentrancy. A capacity - one probe documents held admission rather than promising nested success. - -Validation: run affected public contract and private payload-cleanup tests under -the race detector. Mutable callback state in concurrent tests must be protected -by the test; do not make PromptKit serialize application callbacks globally. - -## Stage 7: Verify Coordinated Turns And Observer Composition - -Status: complete. - -Outcome: coordination obtains the controls through ordinary `Run`, retaining -its existing branching and delivery semantics. - -1. Check the coordinator's existing request-value copy preserves both new - controls while it replaces `ContinueFrom` and composes observers. No separate - coordinator timeout, executor, or public prepare/run sequence is needed. -2. Add a controlled-time, two-continuation success case. Each turn finishes - within its own timeout while their combined execution exceeds one turn's - duration. Callback details contain each turn's assembled history; no timer - from an earlier turn controls a later turn. -3. In the next-turn callback, verify the parent context is active and the prior - child context has been cleaned up. Run an explicitly selected assessment on - the same backend with concurrency one and queue capacity zero, using its own - timeout. This differs from `OnPrepared`, where admission is still held. -4. Exercise a locally timed-out continuation: coordination returns its ordinary - operational error with the marker, does not fabricate an outcome or completed - exchange, and leaves the parent and a sibling continuation usable. Retained - details are available if notification occurred before failure. Do not promise - them if execution stopped earlier. -5. Compose request and coordinator observers with an explicit execution group. - Check one ordinary lifecycle per selected run, normal terminal usage/attempt - totals, and no implicit membership for assessment calls. Reuse existing - final-turn and parent-cancellation/undelivered-result cases to verify that - cleanup and slow terminal observation have not changed delivery semantics. - -Validation: run the coordinator, observation, group, and new execution-control -contract cases together under the race detector. Keep tests at the public -boundary for the assembled behavior; the internal coordinator loop need not -be redesigned solely because its ordinary run gained options. - -## Stage 8: Document, Demonstrate, And Validate The Assembled Feature - -Status: complete. - -Outcome: the implementation and its supported consumer workflow are complete, -with documentation in the canonical owners. - -1. Finish GoDoc on `RunRequest`, `ErrOperationTimeout`, `Run`, `Prepare`, - `ValidateRequest`, `PrepareExecution`, `RunPrepared`, and `PreparedExecution`. - Cross-reference the owning field/error contract rather than duplicating its - entire definition. Explain conditional notification, held admission, - cooperative cancellation, split preparation, callback ownership, and outcome - finalization before terminal observation. -2. Update the [consumer guide](../consumers/pkg-promptkit.md) with small ordinary, - coordinated, and split usage sketches linked to GoDoc. Show a caller-owned - retained `PreparedRun` on failure, `errors.Is` checks for both timeout - identities, and a separate budget for callback-owned assessment. Do not add - persistence, log destinations, redaction policy, or a mandatory diagnostic - write gate. -3. Extend the maintained [observation example](../../examples/go-library/observe/main.go) - and its semantic test with a positive operation budget on continuation, - selected prepared-details retention, and a separately budgeted next-turn - assessment. Keep it offline, synthetic, and successful; Stage 5/6 tests own - failure demonstrations. Do not add another standalone example framework. -4. Update [runner documentation](../internal/runner.md), - [observation documentation](../internal/observation.md), and the - [component inventory](../internal/overview.md) only where responsibilities or - lifecycle descriptions changed. No ConfigKit, collection-format, provider - payload, or architectural policy change is required by this work. -5. Review Audita's six acceptance scenarios and the feature roadmap's validation - targets against the completed tests. Consolidate duplicate tests; do not - expand this into a full cross-product of all entry points, phases, observers, - and callback behaviors. Retain existing no-option regression coverage. -6. Select the newest installed Go toolchain per the development guide. Format - changed Go files, run `go test -race ./...` and `go vet ./...` with - `GOTOOLCHAIN=local` and `GOWORK=off`, and run `git diff --check`. The full Go - suite includes the maintained examples; no second example execution gate is - needed. Use writable caches as required by the development environment. - Validate local links and inspect the full diff for unintended files/secrets. - -Do not edit published v0.22.0 release notes, create a new release/tag, or require -CI or external publication to finish the sprint. Release preparation and -retirement of these roadmaps are subsequent work. - -## Completion Criteria - -- Ordinary and coordinated runs apply one optional whole-operation budget; - split handles retain duration and callback selection without live idle timers. -- Deadline attribution survives public wrapping, distinguishes this scope's - recorded timer cause, and agrees with terminal observation before cleanup. -- Notification supplies a detached redacted view of the actual logical prepared - request, once before generation when reached, without repeated preparation. -- Failure, cancellation, and panic paths preserve existing resource, handle, - error, observation, grouping, and continuation behavior. -- The maintained example, canonical documentation, focused tests, full race - suite, and vet are complete. No-option use stays compatible. - -No significant open questions remain. The API and nonpositive-duration choices -above complete the planning decisions left by the accepted feature roadmap. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 5593bf9..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,72 +0,0 @@ -# Execution Controls Review - -Reviewed implementation stages 1–8 against commit `57b62d3`, the -[implementation plan](implementation.md), the accepted -[feature roadmap](execution-controls.md), and the applicable architecture, -testing, documentation, and agent policies. - -## 1. Complete The Public Prepared-Notification Contract - -Priority: P3. Status: complete. Owning stage: Stage 8. - -Location: [RunRequest.OnPrepared GoDoc](../../types.go#L97). - -The public GoDoc describes synchronous delivery, detached ownership, held -admission, and split preparation, but omits two explicitly selected lifecycle -facts: - -- Notification occurs at most once per logical execution, when execution - reaches that point before the first generation. Transient retries and - structural repair do not repeat it. -- The callback receives the operation context, including the selected operation - deadline when enabled and inherited parent cancellation. In coordination this - is distinct from the parent context delivered to the next-turn callback. - -These facts are already implemented by the -[shared execution boundary](../../internal/usecase/runner.go#L186) and covered -by the notification, retry/repair, and coordinator contract tests. They are also -expressly selected in the roadmap's -[accepted callback decision](execution-controls.md#3-what-is-the-prepared-details-callback-contract). -However, Go declarations and GoDoc own the exact public API contract; consumers -should not need a temporary roadmap or implementation inspection to discover -invocation cardinality or the scope of their callback's context. - -Required remediation: add these two facts to the field's GoDoc. Preserve -conditional delivery and cooperative cancellation; do not promise notification -for every failed request, forced callback interruption, or terminal observation -after panic. This is a documentation completion, not a runtime change. Existing -behavioral tests already cover these semantics; duplicate tests are unnecessary. - -Completion: expanded the field's GoDoc to state invocation cardinality, -non-repetition during retry/repair, and operation-context scope, including the -distinction from the coordinator's next-turn callback context. Runtime behavior -is unchanged. GoDoc rendering and the existing focused race-enabled notification, -retry/repair, and coordinated-budget tests passed. - -## Stage Review Summary - -| Stage | Result | -| --- | --- | -| 1 — API and conversion | Selected fields and sentinel are present; callback conversion captures the chosen function and detaches details only when selected. | -| 2 — Ordinary timeout | One internal scope covers ordinary execution; per-invocation cause attribution and public error projection precede terminal observation. | -| 3 — Split preparation | Handles retain duration and callback without an idle timer; claim/discard and cleanup behavior are preserved. | -| 4 — Prepared notification | Both execution paths use the actual shared snapshot, with context checks and held admission; retries and repair do not notify again. | -| 5 — Phase budgets | Tests cover preparation, capacity waiting, retry backoff, generation/repair, validation, notification time, and stable terminal outcomes. | -| 6 — Ownership and failures | Public tests cover detached messages, cache metadata, nested parameters, schema, credentials, frozen sources, retained diagnostics, and panic cleanup. | -| 7 — Coordination | Fresh turn budgets, parent-context assessment, released capacity, sibling use, observer composition, and explicit grouping are covered. | -| 8 — Documentation and example | Maintained example, consumer/internal documentation, and the completed canonical GoDoc reflect the feature; finding 1 is resolved. | - -No runtime correctness defect, additional architectural gap, or inadvertently -strengthened caller restriction or guarantee was identified in this review. - -## Validation - -- The complete package suite passed under `go test -race ./...` when combining - the initial run with successful reruns of `internal/configuration` and - `internal/llm`. Those two packages initially failed solely because the sandbox - disallowed their local `httptest` listeners; their reruns permitted listeners. -- `go vet ./...` passed using Go 1.27.3 with `GOTOOLCHAIN=local`, `GOWORK=off`, - and offline module resolution. -- Changed Go files are formatted. `git diff --check` and local document-target - checks passed. No implementation, dependency, policy, or release files were - changed during the review.