140 lines
7.6 KiB
Markdown
140 lines
7.6 KiB
Markdown
# Audita request: coordinator execution controls
|
|
|
|
Status: proposed follow-up to PromptKit v0.22.0. This document describes
|
|
consumer requirements; public API design and implementation remain upstream
|
|
decisions.
|
|
|
|
## Purpose
|
|
|
|
PromptKit v0.22.0 supplies the conversation capture, continuation, and execution
|
|
groups Audita requested. Audita can adopt those capabilities through its existing
|
|
execution adapter. To also adopt `Engine.Coordinate` and remove its local turn
|
|
loop, Audita needs two capabilities during coordinator-owned execution:
|
|
|
|
1. A fresh, application-selected timeout for each continuation operation.
|
|
2. Access to the exact prepared request before generation, including for turns
|
|
that subsequently fail without returning a result.
|
|
|
|
Both capabilities apply to applications that run bounded model operations and
|
|
retain optional diagnostic evidence. Neither requires PromptKit to understand
|
|
transcripts, correction proposals, or domain validation.
|
|
|
|
## Current workflow and integration gap
|
|
|
|
Audita's role-bound adapter currently derives a timeout context from its caller,
|
|
calls `PrepareExecution`, retains `PreparedExecution.Details()`, and executes
|
|
that same handle with `RunPrepared`. The operation deadline covers preparation,
|
|
capacity waiting, transient retries, generation, and structural repair. Each
|
|
model operation receives its own budget; application validation may make
|
|
additional, separately bounded model calls.
|
|
|
|
When debug evidence is enabled, Audita retains the prepared messages and
|
|
effective settings alongside the available response or failure information.
|
|
It owns artifact destinations, application-specific redaction, and handling of
|
|
optional diagnostic write failures. The same adapter supplies ordinary execution
|
|
observers and classifies operation errors for the application.
|
|
|
|
In v0.22.0, the [coordinator](../../coordinate.go) accepts a callback returning a
|
|
`RunRequest`, then directly calls `Engine.Run` with the coordinator's context.
|
|
The callback cannot supply a fresh execution context or a prepared handle.
|
|
There is no callback exposing the actual prepared request before that run.
|
|
|
|
A timeout around the entire coordinator bounds the conversation and application
|
|
callbacks together; it does not preserve a separate timeout for each operation.
|
|
Provider generation timeouts do not cover the whole operation either.
|
|
Separately calling `Prepare` before returning a request duplicates preparation
|
|
and does not bind the inspected snapshot to the execution. Successful exchange
|
|
inspection cannot supply a failed turn's prepared request when no new exchange
|
|
is returned. Existing metadata-only observations do not contain that content.
|
|
|
|
## Requested behavior
|
|
|
|
### Per-operation deadlines for coordinated turns
|
|
|
|
Allow the application to select a timeout for each requested continuation, with
|
|
these semantics:
|
|
|
|
- Establish a fresh deadline when that continuation starts, before its
|
|
preparation. Cover preparation, admission waiting, retries, generation, and
|
|
structural repair under the same operation budget.
|
|
- Respect caller cancellation and any earlier parent deadline. Expiry of a
|
|
local operation budget must not cancel the parent context or independent
|
|
branches.
|
|
- End the operation's timeout scope before invoking the application's next-turn
|
|
callback. Assessment calls made there can retain their own operation budgets.
|
|
- Preserve useful error identity so Audita can distinguish a local deadline
|
|
from cancellation of the caller. An execution timeout can end coordination
|
|
under the existing operational-error rule; no automatic turn replay or new
|
|
recovery policy is requested.
|
|
|
|
The existing additional-turn budget, transport retry budget, structural-repair
|
|
budget, and provider generation timeout retain their separate meanings.
|
|
|
|
### Exact prepared-request access
|
|
|
|
Provide an opt-in way to obtain caller-owned details of the actual prepared
|
|
continuation before its first generation attempt:
|
|
|
|
- Include the assembled captured history, follow-up definition messages,
|
|
appended messages, and ordinary resolved execution metadata, using the
|
|
existing [prepared-details contract](../../prepared_execution.go) where useful.
|
|
- Expose the snapshot that execution will use, without a second rendering or
|
|
input-reading pass. Access to details must not mutate that execution.
|
|
- Let the application retain those details even if generation subsequently
|
|
fails or is canceled and returns no ordinary result or captured exchange.
|
|
If preparation itself fails, a complete prepared snapshot is not required.
|
|
- Preserve existing credential-redaction semantics. Keep content-bearing access
|
|
explicitly selected and separate from ordinary metadata-only observations.
|
|
|
|
Audita needs the logical prepared request, not raw HTTP envelopes or an archive
|
|
of intermediate structural-repair requests. It can handle optional debug writes
|
|
itself without requiring PromptKit to own artifact storage or reporting policy.
|
|
|
|
## API direction and ownership
|
|
|
|
Prefer small execution-level controls that the coordinator honors through its
|
|
ordinary execution path. An optional operation timeout and a prepared-details
|
|
callback are one possible shape, not proposed exported names or signatures.
|
|
Their applicability to ordinary and prepared execution should follow a coherent
|
|
lifecycle rather than introducing a second coordinator-specific executor.
|
|
|
|
An alternative is allowing a coordinator turn to delegate execution through a
|
|
caller-controlled prepared-execution path. That could reuse Audita's adapter,
|
|
but exposes more execution responsibility and requires clear ownership of
|
|
capture, observers, admission release, and prepared-handle cleanup. Evaluate
|
|
that option only if it is simpler than the narrow controls above.
|
|
|
|
PromptKit should continue owning history assembly, turn progression, execution,
|
|
and observations. Audita retains response interpretation, validator calls,
|
|
branch policy, diagnostic storage, and application error classification. Existing
|
|
request observers, coordinator observers, and execution groups must remain
|
|
usable together. This request does not reopen the accepted conversation history,
|
|
final-turn delivery, or grouping contracts.
|
|
|
|
## Acceptance scenarios
|
|
|
|
1. Two successful continuation operations each finish within their own timeout,
|
|
while their combined duration exceeds one operation's budget. Both succeed
|
|
under an active parent context.
|
|
2. Preparation or admission waiting consumes the same budget as generation and
|
|
structural repair. Local expiry returns a recognizable deadline error and
|
|
stops that coordinator without canceling its parent or a sibling branch.
|
|
3. An earlier parent deadline remains effective. A validator called from the
|
|
next-turn callback runs after admission is released, including with backend
|
|
capacity one, under its own application-selected deadline.
|
|
4. The application receives the prepared continuation details before generation.
|
|
A later provider failure returns no result, but the retained details still
|
|
describe that failed operation. Mutating a details copy does not change the
|
|
messages executed.
|
|
5. Diagnostic access uses the same prepared snapshot that executes, including
|
|
when source inputs change afterward. It does not trigger another preparation
|
|
or provider call.
|
|
6. Existing observers and selected execution groups continue receiving ordinary
|
|
execution facts once per their existing contracts, including failure paths.
|
|
Applications that select neither new capability retain existing behavior.
|
|
|
|
The consumer implementation motivating this request is Audita's
|
|
`internal/framework/llm/adapter.go`, especially `RoleClient.CompleteStructured`.
|
|
This gap affects full coordinator adoption; it does not block adopting v0.22.0
|
|
capture, continuation, or execution groups independently.
|