base-layer/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Plan conversation continuation and execution groups

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-14 01:08:18 +00:00
parent b423de12c3
commit 5cc3fa0168
3 changed files with 812 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files

169
docs/roadmap/audita-request.md Normal file
View File

@@ -0,0 +1,169 @@
# Audita request: conversation continuation and application-directed repair
Status: proposed upstream features, requested by Audita. Public API design and
implementation remain for PromptKit to determine.
This request is based on Audita's use of PromptKit v0.21.0. It proposes two
related capabilities: captured conversation continuations, followed by bounded
repair driven by application-supplied validation feedback. The goal is to move
reusable execution machinery into PromptKit and remove its local equivalents
from Audita.
## Audita's workflow
Audita polishes transcripts using module-specific proposal, validation, and
repair prompts:
1. A proposal request supplies transcript segments and receives a structured
collection of proposed corrections.
2. Audita evaluates each correction using deterministic and LLM-backed
validators. Accepted corrections are retained for later application.
3. Each actionable rejection starts an independent repair branch containing the
original rendered request, the full final assistant response, and a follow-up
instruction rendered from that module's repair prompt definition.
4. The follow-up identifies one correction and supplies its current content,
available before-and-after transcript context, and validation findings. The
model can revise or withdraw that correction. This response uses a different
schema from the original collection of proposals.
5. A revision passes through the validators again. Further attempts retain that
branch's history and receive updated feedback. Acceptance, withdrawal,
exhaustion, or failure ends the branch; independent approvals are preserved.
Audita owns the meaning of a correction, the validation rules, repair
eligibility, and eventual application to the transcript. Conversation handling
and the bounded revision lifecycle are useful beyond transcript processing.
## Existing support and the remaining gap
PromptKit already supports [structural output repair](../consumers/pkg-promptkit.md#repair-a-structured-result),
[appended messages](../consumers/pkg-promptkit.md#append-already-rendered-messages),
prepared execution, profiles, execution observations, and backend capacity
management. These allow Audita to implement the workflow today.
Audita still captures exchanges, reconstructs the original prompt, verifies its
rendered prefix, assembles branch histories, renders follow-up instructions,
and coordinates repeated application validation and revision. The requested
APIs would make those responsibilities supported PromptKit capabilities.
## 1. Captured conversations and independent continuations
Provide a public way to capture a rendered request and its full final assistant
response, then continue that exchange without making the consumer reconstruct
the original prompt and check that its messages are unchanged.
Audita needs to:
- Preserve the original rendered message content, roles, and cache metadata
when continuing an exchange.
- Start multiple independent branches from the same exchange. A branch's later
instructions and responses must not alter the original exchange or another
branch's history.
- Render follow-up instructions using ordinary PromptKit prompt definitions and
variables, and append them to the selected branch. Audita's repair prompts
remain operator-replaceable through its existing PromptKit configuration.
- Select a follow-up profile and output contract independently of the initial
request. The first response may contain many items while a continuation
returns one revised item or a withdrawal.
- Retain the relationship between the original exchange, the follow-up prompt
definition, and each execution in inspection and observation data.
An in-process capability is sufficient for Audita. Persistence, restartable
sessions, and a conversation database are not required by this request. The
existing [prepared execution handle](../../prepared_execution.go) is single-use;
the request does not assume that copying or reusing it creates a continuation.
## 2. Bounded application-directed repair
Build a reusable revision coordinator on the continuation capability. Audita
should supply its assessment and request-building behavior while PromptKit
handles the repeated execution lifecycle.
The coordinator needs to support:
- Starting with an already-assessed candidate and its rejection findings, so
Audita need not repeat the initial batch validation simply to enter repair.
- Application-supplied assessment of each revision, including deterministic
checks and checks that make further PromptKit calls. Findings may come from
multiple validators; their meaning and explanatory content remain with the
application.
- Application-selected follow-up prompts, variables, profiles, and response
contracts. Audita supplies the candidate and context, and interprets the
response as a revision or withdrawal. PromptKit need not prescribe an Audita
correction schema or a universal feedback format.
- Reassessment before a revised candidate can be accepted, with fresh feedback
for the next attempt and accumulated history confined to that branch.
- A caller-selected additional-revision budget and caller context, with clear
terminal outcomes covering acceptance, withdrawal, exhaustion, unavailable
results, and interruption. Callers retain decisions about repair eligibility
and whether an operational failure should terminate other work.
- Attempt observations, execution relationships, and available usage accounting
sufficient to report a repair sequence without recreating its lifecycle
tracking in Audita.
The revision budget must remain distinct from transport retry and structural
repair budgets. A structurally valid response can still fail Audita's checks;
each revision generation can itself need structural repair. Accounting should
distinguish those layers and avoid double-counting shared initial work.
Application validators must be able to call the same engine and backend without
being blocked by capacity retained by their parent operation. Coordinating
ordinary completed runs is one possible approach. PromptKit should determine
the implementation in light of its [execution and admission lifecycle](../internal/runner.md#run-flow).
This is broader than exposing the current structural-repair hook: the response
schema can change between proposal and repair, validation may perform model
calls, and each rejected item has its own history and disposition.
## Ownership boundary
PromptKit would own captured exchanges, branch continuation, the bounded repair
lifecycle, and associated execution observations. Audita would retain:
- Transcript segments, proposal identities, glossary and protected-term rules,
editorial judgments, and meaning checks.
- Validator selection and ordering, actionable-rejection policy, feedback
content, and module-specific proposal, validation, and repair prompts.
- Before-and-after transcript previews, preservation of accepted sibling
proposals, application ordering, and final transcript mutation.
- Application reports, debug artifact destinations, and CLI behavior.
The separate idea of reconciling missing or duplicate items in batch responses
is outside this request. The two requested capabilities should be usable
through the supported public facade without an Audita dependency or a general
workflow framework. Exact types, callbacks, and configuration syntax are left
to PromptKit's API design.
## Suggested delivery and acceptance scenarios
Deliver conversation continuation first, allowing Audita to replace its capture
and replay machinery. Then add application-directed repair and replace the
corresponding local coordination. Existing prompt definitions and domain
validators should remain usable through the new APIs.
The following scenarios would demonstrate that the capabilities meet Audita's
needs:
1. An initial response contains three proposals. One is accepted; the other two
receive independent repair conversations containing the same original
exchange. Revising either branch leaves the accepted proposal and the other
branch unchanged.
2. A follow-up uses a separate prompt definition, profile, and single-item
response schema. Inspection identifies the follow-up and its originating
exchange.
3. A revision fails multiple application checks, receives their updated
feedback, and passes on a later attempt. An LLM-backed check can use the same
engine and backend even when backend capacity is limited to one.
4. Withdrawal, exhausted revision budget, unavailable validation, and
cancellation produce distinguishable outcomes. An unaccepted revision is
never returned as an accepted candidate, and a recoverable branch failure
leaves independent approvals available to the application.
5. A revision generation needs both a transport retry and structural repair.
Observations distinguish these from application-directed revision attempts
and expose the available usage without counting the original exchange once
per branch.
For implementation reference, Audita currently owns these mechanics in
`internal/framework/llm/adapter.go`,
`internal/framework/structuredoutput/call.go`, and
`internal/framework/runner/repair.go`. These are examples of the consumer need,
not proposed PromptKit package boundaries or public API shapes.

50
docs/roadmap/conversation-continuation.md
View File

@@ -1,8 +1,9 @@
# Conversation Continuation And Execution Groups
Status: target behavior accepted; ready for API and staged implementation planning.
No implementation is authorized by this document. Exact public types and
signatures remain to be designed. Execution groups are independently deliverable.
Status: target behavior accepted; concrete API choices and implementation steps
are recorded in the [implementation plan](implementation.md). This document
defines the target behavior and responsibility boundaries. Execution groups are
independently usable.
## Purpose And Responsibility Boundary
@@ -390,44 +391,13 @@ Applications retain their received results; the coordinator's termination value
is not an archive. These examples should remain usable through the same generic
API without manual history assembly or per-turn lifecycle bookkeeping.
## API Planning And Delivery
## API And Implementation Ownership
No product-level decisions remain open. The companion implementation plan should
settle exact public types and signatures before implementation, within the
accepted behavior above:
- Capture opt-in syntax and exchange representation. Prefer a request option;
separate capture methods remain viable. Prefer a details accessor, while a
public value is viable if it preserves ownership and generic rendering.
- Continuation request shape and placement of immediate-parent metadata, using
the message assembly order defined above.
- Callback state, next-action types, normal outcomes, and the optional
undelivered-response result.
- Collector attachment, observer composition, and summary field representation.
Start with the observer-compatible API. Explicit group fields remain a fallback
only if concrete ordinary/prepared/coordinator/callback usage sketches expose
awkwardness; either approach uses existing observation machinery.
Check the concrete API against both consumer sketches and ordinary and prepared
execution, including capture selection, exchange access, coordinator actions and
results, and observer composition. Keep these choices consistent across the
staged plan rather than leaving each implementation stage to choose its own
public contract.
Deliver in separable stages:
1. **Capture and independent continuation:** expose execution-derived exchanges,
reusable branches, fresh follow-up settings, parent metadata, and details.
This lets consumers remove reconstruction and prefix-verification machinery.
2. **Optional sequential coordinator:** build on continuation and ordinary
execution, validation, capacity, and observations. Refine both consumer
sketches against the concrete API.
3. **Optional execution groups, on an independent track:** provide collection,
composition, snapshots, and aggregation. This can ship separately and need
not wait for or delay continuation or coordination.
These stages guide a subsequent implementation plan; they do not authorize code
changes or impose an ordering dependency on execution groups.
The [implementation plan](implementation.md) selects the concrete API and steps
for reaching this target. Both consumer sketches must remain expressible without
manual history assembly or per-turn lifecycle bookkeeping. Execution groups
remain independent of capture and coordination; the plan's ordering does not
introduce a product dependency between them.
## Acceptance Scenarios

633
docs/roadmap/implementation.md Normal file
View File

@@ -0,0 +1,633 @@
# Conversation Continuation And Execution Groups Implementation Plan
## Objective And Sprint Rules
Implement the accepted [feature roadmap](conversation-continuation.md), using
[Audita's request](audita-request.md) as consumer evidence. The feature roadmap
owns product behavior and boundaries; this plan selects the API and work order.
Existing structural-output repair remains a separate, unchanged capability.
Each numbered stage is one Luna-sized implementation prompt. Execute them in
order during one end-to-end sprint. Intermediate stages need not compile or
constitute independently releasable changes. Do not build compatibility shims or
temporary implementations solely to make a stage green. Carry a concise record
of remaining wiring into the next stage; the final stage must leave the whole
repository coherent and validated. There are no intermediate human-review,
external-publication, provider-access, or dependency-release gates.
Start with [development guidance](../development.md). Follow the applicable
[agent](../policy/agents.md), [architecture](../policy/architecture.md),
[testing](../policy/testing.md), and [documentation](../policy/documentation.md)
policies. Preserve the accepted [configuration boundary](../adr/0001-configuration-boundary.md).
This sprint needs no configuration format, dependency, Go minimum, release
tooling, or licensing changes. It does not authorize tagging or publication.
Use real fast internal collaborators and replace only external boundaries in
tests. Select focused checks during stages and run the complete checks at the
end. Do not duplicate existing retry, repair, schema, or capacity test matrices.
Protect the new interactions with those mechanisms.
## Current Baseline And Integration Points
- The root facade in `engine.go` converts public requests, masks inherited
observation state, invokes the internal engine, maps errors, and finishes
observations after the runner releases admission. Preserve this boundary.
- `internal/usecase/runner.go` has shared resolution and completion preparation
phases. `completePreparationWithStructuredOutput` currently combines rendered
definition messages with appended messages and hashes the resulting prompt.
`executePreparedRun` is shared by ordinary and prepared execution and returns
the final `RawOutput`, including after exhausted structural repair.
- `internal/usecase/prepared_execution.go` retains a single-claim payload,
validation plan, direct credential, retry budget, and observer separately from
durable prepared details. Claims remain bound to the originating execution
owner. Details survive execution and discard.
- `internal/domain/message.go` owns message cloning and cache-control ownership.
Reuse it rather than inventing a second normalization contract.
- `observations.go`, `internal/domain/observation.go`, and
`internal/observation/recorder.go` own value-only events, run IDs, terminal
counters, and presence-aware usage. Terminal observations already include
completed work preceding operational failure. Do not total result usage or
attempt events on top of terminal totals.
- `internal/engine/engine.go` applies namespace views to new caller selections.
Captured history requires no original-engine lookups or namespace rewriting.
- Root conversion, `json.go`, formatting, and contract tests must account for
additions without changing existing timing, credential, or error behavior.
## Selected Public API
Implement these names and shapes in the root `promptkit` package. Code below
specifies additions and complete new types, not replacement definitions for
existing types. Keep all implementation machinery under `internal/`.
### Capture, Continuation, And Ancestry
```go
// Add to RunRequest.
CaptureExchange bool
ContinueFrom *Exchange `json:"-" yaml:"-"`
// Add to RunResult.
Exchange *Exchange `json:"-" yaml:"-"`
ParentRunID string `json:"parent_run_id,omitempty"`
// Add to PreparedRun.
ParentRunID string `json:"parent_run_id,omitempty"`
// Add to ExecutionEvent, following its existing field/tag style.
ParentRunID string
// Exchange has private state only.
type Exchange struct { /* private immutable snapshot reference */ }
type ExchangeDetails struct {
RunID string `json:"run_id"`
ParentRunID string `json:"parent_run_id,omitempty"`
Messages []RenderedMessage `json:"messages"`
}
func (e *Exchange) Details() ExchangeDetails
```
- `Run` with `CaptureExchange: true` returns `result.Exchange`. A non-nil
`ContinueFrom` supplies history and implies capture, regardless of the Boolean.
No separate continuation execution method is necessary.
- `Prepare`, `ValidateRequest`, and `PrepareExecution` accept the same request
fields and assemble the same effective messages. They create no completed
exchange. `PrepareExecution` retains capture selection and ancestry privately
for `RunPrepared`; plain preparation only returns its ordinary metadata.
- Nil `ContinueFrom` means no history. A non-nil, zero-value exchange is not a
captured exchange and fails as `ErrInvalidRequest` before source/model work.
`Details` on a nil or zero-value exchange returns a zero details value,
matching the prepared-handle accessor pattern.
- Details contain the complete logical history ending in the final assistant
content. They are detached and encodable with ordinary encoders. Give the
opaque handle constant `String` and `GoString` representations, following
`PreparedExecution`. Serializing ordinary results does not serialize the
capability or automatically disclose the additional conversation history;
callers explicitly encode `Details()` when desired. Do not add deserialization,
persistence, reconstruction, lookup, or lifecycle APIs.
- `ParentRunID` is the immediate source exchange's originating run ID, or empty
for initial runs. Expose it in prepared details, completed results, exchange
details, and ordinary observation events. Reuse the run ID already assigned to
the execution. It is metadata, not a provider request parameter, session ID,
group ID, or new source of execution settings.
- The follow-up's `PromptID`, definition hash, selected profile/backend, and input
hashes describe the new request. The rendered-prompt hash covers its complete
effective message sequence and selected session, using the existing hash
function. Do not hash capture flags or run identities into content equality.
### Optional Coordinator
```go
type ConversationTurn struct {
Exchange *Exchange
Result *RunResult
RemainingTurns int
}
type CoordinateRequest struct {
Start *Exchange
AdditionalTurns int
Next func(context.Context, ConversationTurn) (*RunRequest, error)
Observer ExecutionObserver
}
type CoordinationOutcome string
const (
CoordinationStopped CoordinationOutcome = "stopped"
CoordinationExhausted CoordinationOutcome = "exhausted"
)
type CoordinationResult struct {
Outcome CoordinationOutcome `json:"outcome,omitempty"`
Undelivered *RunResult `json:"undelivered,omitempty"`
}
func (e *Engine) Coordinate(ctx context.Context, req CoordinateRequest) (CoordinationResult, error)
```
- `Next` returning `(nil, nil)` stops. A non-nil request asks to continue; an
error stops with that error. No separate next-action enum or domain-outcome
taxonomy is needed. A zero request pointed to by a non-nil pointer is still
a continuation request, subject to ordinary validation if execution starts.
- `Start` must hold a captured exchange, `Next` must be non-nil, and
`AdditionalTurns` must be nonnegative; invalid values match `ErrInvalidRequest`.
A nil/uninitialized engine follows the existing `ErrInvalidConfig` convention.
Zero is a useful budget, not a default that expands into extra turns.
- Initial callback state has `Exchange: Start`, `Result: nil`, and the full
remaining budget. Later states contain the newly completed result and exchange
with the remaining start budget, including zero after the last permitted run.
The initial exchange is not presented as a newly delivered response.
- The coordinator copies each selected request value and sets its `ContinueFrom`
to the current exchange. It owns this one branch's progression, so a callback's
`ContinueFrom` does not redirect that branch. The rest of the request uses
ordinary rules. Callers wanting independent branches use `Run` themselves.
- Compose the request's observer first and the coordinator's observer second.
The latter selects all continuation runs, not the starting run or nested
callback calls. Applications explicitly compose additional group observers.
- On error leave `Outcome` empty. Set `Undelivered` only for a completed new
response withheld from callback invocation because cancellation was observed.
That result includes its exchange. A callback that was invoked has received
its response even if it errors or cancels. Return no result archive, latest
delivered response, failure-stage taxonomy, or coordinator accounting totals.
- Callbacks receive caller-owned public values. Retain the coordinator's current
immutable exchange separately, so replacing fields in delivered values cannot
rewrite retained history or the next continuation's ancestry.
### Observer Composition And Execution Groups
```go
func ComposeObservers(observers ...ExecutionObserver) ExecutionObserver
// Zero value ready for use; contains private synchronization/state only.
type ExecutionGroup struct { /* private collector */ }
func (g *ExecutionGroup) Observe(event ExecutionEvent)
func (g *ExecutionGroup) Snapshot() ExecutionGroupSummary
type ExecutionGroupMember struct {
RunID string `json:"run_id"`
TerminalObserved bool `json:"terminal_observed"`
}
type ExecutionGroupSummary struct {
Members []ExecutionGroupMember `json:"members"`
TerminalCoverageComplete bool `json:"terminal_coverage_complete"`
Attempts uint64 `json:"attempts"`
TransientRetries uint64 `json:"transient_retries"`
RepairCalls uint64 `json:"repair_calls"`
Usage ObservedTokenUsage `json:"usage"`
}
```
- Attach a group with `Observer: group.Observe`, or use `ComposeObservers` for
diagnostics and multiple groups. No request group field, implicit context
membership, constructor, registry, close, seal, or wait method is needed.
- Composition copies the supplied function slice, skips nil functions, and
invokes the remainder synchronously in supplied order. With no non-nil
functions return nil. Do not deduplicate functions or recover panics; a panic
prevents delivery to later observers. Closures retain their ordinary identity.
- Groups enroll a run when any ordinary event for that run is observed. The
first terminal event seen for that identity contributes counters and usage
exactly once, including when it is the first observed event. Repeated delivery
must not change totals or regress terminal coverage. Handling contradictory
caller-fabricated events is not a new public validation contract.
- Summary counters and token subtotals use terminal observations only. Pending
runs contribute no invented attempt counts or known zero usage. Members without
terminals remain identifiable. `TerminalCoverageComplete` says only that every
currently observed member has a terminal observation; it says nothing about
future members or application completion.
- For each token field, sum known terminal values; `Known` is the OR of terminal
`Known` flags and `Complete` is the AND of terminal `Complete` flags. These flags
describe received terminal subtotals; missing terminal coverage is separate.
If members exist but none has a terminal, usage is unknown/incomplete. An empty
group has complete terminal coverage and known, complete zero totals. Preserve
explicit provider zero and existing per-run failure-path usage semantics.
- Snapshots are detached and internally consistent under concurrent observation
and reads. There is no promised member ordering; tests compare by identity.
The mutex-bearing group must not be copied after first use, as with ordinary
synchronized Go values. Overlapping group totals are not generally additive.
## Internal Implementation Decisions
### Immutable History And Execution-Owned Messages
Put the internal immutable exchange value and history operations in
`internal/domain/exchange.go`. The public handle privately references this value;
internal packages never import the root facade. Retain only history and the two
run identities, not engine owners, definitions, requests, credentials, validation
plans, results, admission leases, or observer/callback closures.
Use a simple immutable parent-plus-turn representation: a parent exchange,
detached new-turn messages, the final assistant message, and a cached total
message count. Initial exchanges have no parent. New exchanges retain only their
new suffix rather than another flat copy of all ancestors. Flatten iteratively
into one correctly sized, caller/execution-owned destination when preparing a
provider request or returning details; do not recurse through arbitrary depth.
Reuse message/cache-control clone helpers at mutable boundaries. This is an
internal choice, not a public allocation or asymptotic guarantee. Avoid unrelated
history caches, ropes, eviction, limits, or pooling.
Prepare the new prompt normally, then assemble history, new definition messages,
and appended messages in the accepted order before hashing. Captured history
is not normalized or rendered again. Save capture/parent state separately in the
private prepared operation/payload. Prepared metadata contains `ParentRunID` and
the already assembled messages, not the exchange capability or capture settings.
Pass the private capture state through the shared execution function. After
successful final validation processing, capture the new-turn suffix from the
executed snapshot and the exact final `RawOutput`; failed structural validation
is still a completed result. Do not capture internal repair instructions or
intermediate candidates. On an operational error returning no result, create no
exchange. Do not change underlying execution cancellation or partial-result
rules to recover an assistant message.
### Coordinator Placement And Boundary Adapters
Put the loop in `internal/usecase/coordinate.go`, not in root business logic.
Use one small internal function generic over request/result value types, with
typed callback and execution functions and a `*domain.Exchange` current state.
Its internal turn/termination values contain only the fields projected by the
public API. This lets the facade inject ordinary `Engine.Run` and the application
callback without importing facade types internally or converting an entire run
result to domain and back just for delivery. No generic public API is added.
The execution adapter accepts `(context, currentExchange, selectedRequest)` and
returns `(completedResult, nextExchange, error)`. It creates a fresh public
exchange wrapper, copies the request, supplies the current history and composed
observer, and calls `e.Run`. Obtain the next internal exchange before delivering
the result to application code. The callback adapter projects a fresh public
wrapper and the current result into `ConversationTurn`. The shared loop does not
inspect results, interpret output, or access generation/capacity directly.
Use this exact progression:
1. With no completed response pending initially, check context before callback.
2. Invoke `Next` with the current exchange, pending result if any, and remaining
budget. Invocation marks that pending result delivered.
3. Check context again; cancellation wins over the callback's action/error.
Otherwise return a callback error unchanged, stop on nil request, or continue.
4. If a request was selected with no remaining budget, return exhaustion before
converting/preparing that request. Otherwise check context before execution,
consume one start from the budget, and call the execution adapter.
5. Return an execution error unchanged; retain no partial output. On a completed
result, update the private current exchange and repeat from step 1.
At step 1, cancellation with a pending new completed response returns that
single response in `Undelivered`; cancellation before the initial callback
returns none. Callback errors and cancellation observed after invocation return
no undelivered response. Panics propagate naturally without recovery. These
context checks do not forcibly interrupt callbacks or eliminate races.
Calling ordinary `Run` per turn ensures its logical admission and terminal
observation finish before `Next`. Do not wrap the whole coordinator in an
admission lease or an execution recorder. Nested calls remain independent.
### Group Collector Placement And Cost
Implement the collector in `internal/observation/group.go`, using domain events
and summary values. The root group is a thin adapter with private internal state;
root summary/member types are independent public declarations, not aliases.
Store a map keyed by run identity, a terminal-seen bit per member, and aggregate
terminal counters/usage under one mutex. Do not retain full event histories,
provider content, or per-attempt arrays. Event updates use map lookup rather than
rescanning all members. Snapshot copies the map's members and totals under the
lock without invoking callbacks. Observer composition is also internal logic
with thin root conversion; do not round-trip every event through both public and
domain representations simply to fan out callbacks.
## Stage 1 — Implement Immutable Exchange Storage
**Scope:** internal history value and ownership only.
1. Add the internal exchange representation and construction/flattening helpers
described above. Keep fields private and expose only the internal accessors
needed for run identities, message count, capture, and detached details.
2. Clone only new mutable suffix data when capturing; safely share immutable
parents. Preserve all role, content, empty-content, and cache metadata values.
3. Add focused internal domain tests for initial history, a second turn, two
independent branches, and mutation of source messages/cache control and
returned copies. Include a many-turn flattening case without asserting a
specific private layout or allocation count.
**Validation:** `go test ./internal/domain` when the package is buildable.
These tests own focused history behavior; the final suite verifies its assembly.
## Stage 2 — Add Public Handles And History-Aware Preparation
**Scope:** facade request conversion and the shared preparation path; generation
capture is the next stage.
1. Add `Exchange`, `ExchangeDetails`, their accessor and opaque formatting, the
request fields, and the prepared/result/observation ancestry fields selected
above. Add private domain request and prepared-operation capture state.
2. Extend `toDomainRunRequest` to translate a valid handle and capture choice.
Reject non-nil empty handles as invalid requests; do not impose engine binding
on exchanges. Preserve existing appended-message validation and copying.
3. Update completion to prepend captured messages before newly rendered messages
and caller suffixes. Compute hashes only after final assembly, leaving the
follow-up's settings, session, profile selection, and schema rules ordinary.
4. Update `PreparedRun` conversion and JSON behavior for optional `ParentRunID`.
Keep capture/private history references out of returned prepared details.
`ValidateRequest` inherits this through its existing preparation delegation.
5. Extend preparation contract coverage to assert exact order, roles, cache
metadata, hash sensitivity to prior history, and unchanged definition hashes.
Check nil/empty handle behavior and that preparation performs no model work.
**Validation:** focused root preparation/JSON tests and affected use-case tests.
Do not add a separate preparation or validation algorithm for continuation.
## Stage 3 — Capture Ordinary And Prepared Executions
**Scope:** completed-result creation and prepared payload lifetime.
1. Carry capture/parent state through `preparedOperation` and
`preparedExecutionPayload` to the shared `executePreparedRun` path. Keep it
out of the retained details object so discard can release execution-only
references. Preserve claim ownership, validation-plan freezing, direct-key
clearing, observer identity, and retry budgets.
2. Create the exchange from the request actually executed and exact final response
after the ordinary execution has a result. Initial capture is opt-in;
continuation always captures. Attach it and `ParentRunID` in result conversion.
3. Capture final failed validation/repair exhaustion as well as valid results.
On generation, validation-operation, or repair-generation errors returning no
result, create no exchange and leave the source exchange reusable.
4. Keep `RunResult.Exchange` excluded from ordinary JSON/YAML and preserve existing
result timing marshal/unmarshal behavior. Details are the explicit content
encoding path. The exchange's private state stores no execution target,
credential, result archive, callback, or lease.
5. Add focused tests for default no-capture, ordinary capture, prepared capture,
discarded/failed-claim silence, final repaired content without repair chatter,
failed validation capture, and operational-error absence of a new exchange.
Reuse existing prepared lifecycle and mutating-client fixtures.
**Validation:** targeted root capture, prepared-execution, and repair tests plus
`go test ./internal/usecase` as the touched package becomes coherent.
## Stage 4 — Propagate Ancestry Through Ordinary Observations
**Scope:** immediate-parent metadata without accounting changes.
1. Add parent metadata to the internal recorder's event construction and the root
event projection. Set it before ordinary start delivery and before a prepared
start after successful claim. Preserve it when selection identities update.
A small internal setter before `Start` is sufficient; do not add context-based
ancestry inheritance or new run IDs.
2. Propagate parent identity through start, attempt, retry-delay, and terminal
events. Initial runs and unrelated nested calls keep an empty parent. Preserve
terminal error classification and public error identities.
3. Cover observed ordinary/prepared continuation, preparation silence, operational
failure after continuation selection, and an independent nested call using
existing observation fixtures. Verify no history/content enters events.
**Validation:** focused observation and prepared-observation contract tests and
`go test ./internal/observation ./internal/generation`.
## Stage 5 — Verify Independent Continuation At The Public Boundary
**Scope:** assembled continuation behavior before adding coordination.
1. Add an external-package contract test that captures an initial exchange and
executes two branches, then continues one branch again. Assert exact outgoing
messages and immediate ancestry without changing the initial or sibling data.
2. Continue the exchange through another engine, including a scoped view. Select
different follow-up prompt/profile/output settings and session rules, and
verify that neither previous settings nor original direct credentials leak
into the new execution. Use synthetic credentials, not real environment data.
3. Remove or alter original artifact files after initial capture; continuation
still uses captured history. New follow-up inputs retain ordinary reads and
errors. Exercise prepared follow-up freezing separately from exchange reuse.
4. Mutate details and received result fields and use a mutating injected client
to prove ownership. Encode details generically and check exact final assistant
content, including empty/whitespace content when the output contract allows it.
5. Verify overlapping continuation runs can share the source exchange under the
race detector. Compare behavior, not a required scheduling order.
**Validation:** focused root continuation/capture tests under `-race`. This stage
must exercise ordinary shared preparation/execution rather than a mock history
reconstruction implemented by the test itself.
## Stage 6 — Implement The Internal Sequential Coordinator
**Scope:** the small generic loop and its deterministic state-machine tests.
1. Add the internal loop and turn/termination values described above, injecting
typed callback and run functions. Do not add a generic workflow framework or
a new generation collaborator.
2. Implement initial callback delivery, turn-start consumption, final-response
delivery, explicit stop versus exhaustion, and first-error termination.
3. Implement context checks and the single undelivered-response slot exactly as
specified. Do not prepare an exhausted request or replay delivered work.
4. Unit-test zero turns, stop on last response versus request for another turn,
execution and callback errors preserving `errors.Is`/`errors.As`, cancellation
before initial callback, cancellation withholding a completed result, and
cancellation/error after callback invocation. Use deterministic hooks, not
sleeps. Add one panic propagation check without terminal-delivery promises.
**Validation:** focused `go test ./internal/usecase -run Coordinate` (name tests
accordingly). The loop tests use cheap execution functions; provider behavior
remains covered by integration tests.
## Stage 7 — Expose Coordination Through The Facade
**Scope:** public types, argument checks, and ordinary-execution adapters.
1. Add the selected coordinator types and `Engine.Coordinate`, with GoDoc for
callback state, budgets, cancellation, errors, and normal outcomes.
2. Validate coordinator arguments without looking up old resources. Project
callback state through fresh public handle wrappers. Preserve current private
exchange state independently of any callback mutation.
3. Supply the run adapter using `Engine.Run` with the receiving engine/view.
Set current history on a value copy; the coordinator does not mutate the
callback's `RunRequest`. Never add enclosing admission or an observer recorder.
4. Wire its observer selection to composition in Stage 9; do not build a second
temporary observer mechanism to make this stage independently complete.
5. Add basic public tests for invalid arguments, zero-budget initial stop,
ordinary continuation, callback error identity, and a callback modifying the
result/handle it received without changing subsequent retained history.
**Validation:** focused coordinator tests when wiring is buildable. Carry the
explicit observer-composition dependency forward, without an intermediate gate.
## Stage 8 — Verify Coordinator Capacity And Cancellation Boundaries
**Scope:** the subtle assembled execution lifecycle.
1. Configure one backend with concurrency one and waiting capacity zero. Have the
callback make a same-engine/backend validation run after receiving a turn.
Assert it succeeds, proving the logical admission lease as well as the active
permit was released. Use ordinary admission, not test-only capacity bypasses.
2. Deliver a structurally invalid completed response normally, including one
whose structural repairs were exhausted. The application callback decides
whether to stop or select a new request; PromptKit adds no acceptance meaning.
3. Exercise operational failure stopping one coordinator while a sibling exchange
remains usable. Verify typed provider/capacity errors cross unchanged.
4. Trigger cancellation after a completed run but before callback delivery using
a terminal observer; retain its result/exchange once as undelivered. Separately
cancel inside the callback and assert no replay or undelivered duplicate.
5. Check that internal retry/repair attempts create only one delivery for that
continuation and do not consume extra turns. Reuse existing deterministic
HTTP/repair fixtures, not live providers.
**Validation:** focused root coordination tests under `-race`, with bounded test
contexts as failure guards, not timing-dependent synchronization.
## Stage 9 — Add Observer Composition
**Scope:** fan-out helper and final coordinator-observer wiring.
1. Implement internal synchronous fan-out over a copied function slice and the
root `ComposeObservers` wrapper. Keep public/domain adapters proportionate;
a small internal generic callback-composition helper can operate directly on
either event value type without repeated conversion.
2. Honor nil filtering, supplied order, no function deduplication, and ordinary
panic propagation. The helper holds no lock while invoking application code.
3. Complete coordinator observer composition, selecting each request observer
before the coordinator observer. Do not change original callback request
values. Callback-initiated calls remain explicitly selected independently.
4. Test nil/empty composition, caller mutation of the original observer slice,
ordered delivery, early panic, and simultaneous independent executions. Do
not promise stronger callback ordering across different runs.
**Validation:** focused composition and coordinator observer tests, including
the root observation-boundary tests.
## Stage 10 — Implement Terminal-Based Group Aggregation
**Scope:** internal collector, usage semantics, and concurrent snapshots.
1. Add domain summary/member values and the zero-ready internal group collector
under `internal/observation`. Implement map enrollment, terminal deduplication,
incremental terminal counters, and detached snapshots under one mutex.
2. Implement the selected known/completeness and coverage rules for all five
token fields. Do not sum `RunResult.Usage`, reconstruct usage from attempt
events, or convert missing terminals into zero evidence.
3. Cover empty groups, members without terminals, terminal-only enrollment,
duplicate terminal and repeated ordinary events, explicit zeros, wholly
unknown fields, partially known terminal usage, and terminal failures retaining
earlier completed usage. Use small event values rather than an HTTP test for
every arithmetic combination.
4. Concurrently observe independent runs and read snapshots, checking each
snapshot's counts, coverage, and member identities agree. Mutate a returned
snapshot and prove later snapshots are independent. Do not require snapshot
member ordering or introspect locks.
**Validation:** `go test -race ./internal/observation` with focused new group
tests. No background worker, callback invocation, or collector lifecycle is added.
## Stage 11 — Expose Groups And Verify Explicit Membership
**Scope:** thin public collector adapter and assembled accounting.
1. Add the public group/member/summary types and root event/summary conversions.
A private zero-ready internal group field keeps the public zero value usable.
Document synchronization, snapshot ownership, and no-copy-after-use semantics.
2. Exercise `group.Observe` alone and composed with a diagnostic observer and a
second group. Cover ordinary runs without capture, captured runs, prepared
execution, coordinator continuations, and separately selected callback calls.
3. Assert preparation/discard/failed claim creates no membership. The initial
exchange is not automatically enrolled by coordination; unselected nested
calls remain absent. Parent metadata and group membership remain independent.
4. In one assembled scenario, select initial A and branches B/C, plus a validator
into overlapping groups. Include a built-in transient retry, structural
repair, and a failure after completed usage. Check counters and known subtotals
once per identity, including replay usage reported by the provider for B/C.
5. A composed observer panic may leave an observed member without a terminal.
Verify snapshots remain usable, expose missing coverage, and do not manufacture
completion. Snapshotting inside an observer is safe because group locks are
not held while calling other observers.
**Validation:** focused root group/coordinator/observation tests under `-race`.
Reuse existing reliable-execution fixtures and avoid duplicating their complete
retry/repair matrices.
## Stage 12 — Demonstrate The Consumer Workflows
**Scope:** practical API use and one maintained offline example.
1. Extend `examples/go-library/observe` and its semantic assertions to demonstrate
captured proposal history, a coordinator follow-up, an application-owned
assessment call, and explicit group selection alongside diagnostic observation.
Keep its deterministic transport and existing retry/usage protections; no paid
calls or real credentials are needed.
2. The example's assessment remains application policy, not a library validator
hook. Use a closure to retain assessed state, and show stop versus exhausted
budget without returning an application verdict from PromptKit.
3. Add concise consumer-guide examples for direct branching and prepared capture,
plus both roadmap sketches translated to the selected callback shape. The
analysis sketch records a new result then asks another question or returns nil.
The Audita-shaped sketch uses existing findings initially, assesses each new
result, and either returns nil or an ordinary follow-up request. Domain-specific
functions remain illustrative caller code.
4. Show group use with ordinary/prepared requests and `CoordinateRequest.Observer`,
including explicit selection for a nested validator and an independent group
summary. Demonstrate checking structural validation before schema-specific
decoding; stop/exhaustion does not imply content acceptance.
**Validation:** `go test ./examples/go-library/...` and affected public example
tests. Review snippets against actual declarations; full copyable setup stays in
the maintained example rather than duplicated in prose.
## Stage 13 — Finalize Contracts, Documentation, And Sprint Validation
**Scope:** complete the target and remove sprint-only rough edges.
1. Review every new public declaration and GoDoc against this plan and the feature
roadmap. Confirm ordinary APIs still use existing error, timing, prepared-claim,
credential, namespace, usage, and cancellation behavior. Verify result/details
serialization and constant opaque formatting without introducing a restartable
exchange format or redacting caller-authored message content.
2. Update the canonical consumer guide, `doc.go` where orientation needs it, and
affected internal runner, observation, and component-overview documents.
Document capacity release for conversation callbacks separately from ordinary
per-attempt observer callback constraints. Configuration and integration docs
need only changes necessary to describe the implemented message composition;
ancestry is not an upstream wire field or configuration setting.
3. Remove temporary helpers, placeholder implementations, unused experimental
API choices, and intermediate compile/test failures. Do not remove unrelated
planned/deferred work or publish a release in this sprint. Release-note/version
selection and roadmap retirement can follow during release preparation.
4. With the newest locally available Go toolchain selected as the development
guide prescribes, format changed Go files, then run:
```sh
go test -race ./...
go vet ./...
git diff --check
```
The full suite includes the maintained examples; do not add a redundant
separate build or provider connectivity gate. Check added/changed local
documentation links, using the existing tooling where practical rather than
introducing another release validator.
5. Review the complete diff and Git status for unintended files or credentials.
Confirm no new dependency, workspace override, configuration parser, provider
parameter, or strengthened restriction/guarantee slipped into the work.
Commit completed units with plain-English messages under the agent policy.
**Completion evidence:** the implemented facade supports ordinary/prepared
capture, cross-engine independent continuations, coordinator callbacks after
admission release, cancellation's single undelivered response, and independent
observer-based groups. Public behavior tests cover those boundaries and terminal
usage completeness; the complete offline race suite and vet pass. No product or
API decisions are deferred to an implementation stage.