base-layer/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Record sequential coordinator decision and consumer sketches

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-11 21:12:13 +00:00
parent f20020d9f4
commit de2a7fcd6f
1 changed files with 144 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

186
docs/roadmap/conversation-continuation.md
View File

@@ -18,11 +18,12 @@ responses and their reasons for choosing the next action. Repair, analysis,
clarification, and branching exploration are examples of consumer workflows
that could use the same mechanisms.
Coordination and looping remain open questions. Neither is presumed necessary
or excluded. Assess their value as generally useful capabilities, rather than
as an implementation of Audita's repair loop. A coordinator, if selected,
would execute application-directed interactions without needing to understand
why the application requests another turn or stops.
Decision 3 selects an optional sequential coordinator built on independently
usable continuation primitives. Its value is generally useful execution and
conversation management, rather than implementation of Audita's repair loop.
It executes application-directed interactions without needing to understand why
the application requests another turn or stops. Applications retain orchestration
across branches; that initial scope does not prohibit future capabilities.
Apply two questions to proposed public concepts and implementation machinery:
@@ -48,12 +49,14 @@ The proposed capabilities are:
- Capture a rendered request and its final assistant content, then continue it
using ordinary configured prompts without reconstructing the original request.
- Allow independent continuations from one captured exchange.
- Consider coordination and bounded looping using application-supplied next
actions and follow-up requests.
- Provide an optional sequential coordinator for bounded looping using
application-supplied next actions and follow-up requests.
Continuation has direct value by removing consumer capture, reconstruction,
prefix comparison, and history assembly. A coordinator could also remove
repeated lifecycle work, if its general value justifies its public contract.
prefix comparison, and history assembly. The coordinator centralizes turn
progression, execution relationships, cancellation, budgets, and diagnostics.
Its value should come from removing that repeated lifecycle work, beyond merely
replacing a consumer's loop counter.
Audita would continue owning candidate selection, domain validation, feedback
meaning, branch scheduling, accepted sibling proposals, and transcript changes.
Its domain-specific repair code would not move wholesale into PromptKit.
@@ -137,41 +140,139 @@ authored roles. This does not reopen the accepted execution-setting precedence.
### 3. Should PromptKit Coordinate Turns, And Where Does Application Work Run?
**Status: open.** Evaluate whether a coordinator or loop provides generally
useful behavior beyond continuation primitives. If selected, determine its
scope and a lifecycle that permits application callbacks to call the same
engine and backend, including with capacity limited to one.
**Status: accepted; exact callback and reporting contracts remain open.** Provide
independently usable continuation primitives and an optional sequential
coordinator for one branch. The application selects each next request or decides
to stop. PromptKit manages turn progression and execution bookkeeping under the
caller's chosen additional-turn budget and context.
Applications can create several independent continuations or coordinators from
the same exchange. Scheduling branches, joining results, and deciding whether a
branch failure should stop other work remain application responsibilities in
this initial scope. Consumers can also use continuation primitives in their own
loops without adopting the coordinator.
The current [run flow](../internal/runner.md#run-flow) retains logical admission
through structural validation and repair. Releasing an active generation permit
alone does not release that admission lease.
**Recommendation:** assess the value across repair, analysis, and branching
conversation examples before choosing the abstraction. If PromptKit provides
coordination, compose completed executions: finish a turn and release its
admission lease before invoking application work. Each later turn receives
ordinary admission. A coordinator would hold no backend capacity while awaiting
application work. This reuses existing execution machinery and permits
callbacks to make their own model calls without interpreting their purpose.
**Accepted execution lifecycle:** compose completed executions. Finish a turn
and release its logical admission lease before invoking application work. Each
later turn receives ordinary admission. The coordinator holds no backend
capacity while awaiting application work. Callbacks can inspect responses,
consult application data, wait for application input, or make model calls against
the same engine and backend, including at capacity one. PromptKit need not
interpret the purpose of that work.
**Alternative:** provide continuation and observation primitives while leaving
looping to consumers. This keeps PromptKit's API smaller, but may leave repeated
execution and accounting coordination downstream. Compare the actual consumer
work under both approaches. Extending the structural-repair loop is another
possible implementation route, but would require changing lease ownership and
decoupling it from repair-specific assumptions.
**Alternatives considered, not selected:**
**Details to settle:** whether to provide coordination, looping, or both; scope
over a single branch versus multiple branches; callback cancellation boundaries;
and the sequence of request construction, execution, and application decisions.
Applications retain the meaning of those decisions and policy for stopping
other work after a failure.
- Provide only continuation and observation primitives. This would keep the
public API smaller, but leave repeated lifecycle and accounting coordination
downstream. Independent primitive use remains supported alongside the selected
coordinator.
- Provide a coordinator for an entire branching tree. This could manage more
work, but introduces scheduling, concurrency, joining, and failure-policy
questions without demonstrated shared answers across consumers. Applications
can build those workflows from the selected branch-level capabilities.
**Details to settle:** callback cancellation boundaries, exact request/stop and
outcome types, final-turn delivery at budget exhaustion, and error/report
semantics belong to Decision 4. Execution association and usage accounting belong
to Decision 5. The sketches below illustrate the selected responsibility split;
they do not settle those public contracts.
#### Illustrative Consumer Sketches
These are pseudocode, not implemented Go APIs or selected exported names.
`coordinate`, `Continue`, `Stop`, and callback state fields are illustrative.
`request` represents an ordinary PromptKit request, with the selection rules
accepted in Decision 2. Each sketch starts from an already captured exchange.
The callback can keep application state in a closure; PromptKit need not define
a universal domain-state or feedback schema.
**Analysis loop:** the application examines the latest response and selects a
question or decides that its investigation is complete.
```text
analysis = application.start_analysis(initial_exchange)
choose_next(ctx, turn):
if turn.new_result exists:
analysis.record(turn.new_result)
question = application.choose_next_question(analysis)
if question is absent:
return Stop()
return Continue(request(
prompt_id = "research/investigate",
vars = {"question": question}
))
report, error = coordinate(
ctx, start = initial_exchange,
additional_turns = 3, next = choose_next
)
application.present_analysis(analysis, report, error)
```
PromptKit preserves the history and performs each selected execution. It does
not decide whether the response answered the question, whether the evidence is
sufficient, or what to investigate next. The application callback may consult
other sources or call the engine before returning its next action.
**Audita branch:** the application starts with one already-assessed correction
and its rejection findings. It interprets and assesses each new response; the
coordinator uses the same generic controls as the analysis loop.
```text
branch = audita.branch(candidate, existing_findings)
choose_next(ctx, turn):
if turn.new_result exists:
response = audita.decode_response(turn.new_result)
if audita.is_withdrawal(response):
branch.mark_withdrawn()
return Stop()
branch.update_candidate(response)
assessment = audita.assess(ctx, engine, branch.candidate)
branch.record(assessment)
if audita.should_stop(assessment):
return Stop()
return Continue(request(
prompt_id = branch.configured_follow_up_prompt,
vars = audita.follow_up_variables(branch)
))
report, error = coordinate(
ctx, start = initial_exchange,
additional_turns = 3, next = choose_next
)
audita.record_branch_outcome(branch, report, error)
```
The first callback uses existing findings, so initial batch validation is not
repeated. `audita.assess` may make model calls after the turn's admission lease
has been released. Only Audita decides that a correction is accepted, withdrawn,
or unavailable, and retains those meanings in its own state. Further corrections
can start separate coordinators from the same initial exchange; accepted sibling
corrections and other branches remain with Audita.
Both sketches omit application-specific error handling and depict normal turn
progression. They do not prescribe how callback errors or the final result at
the budget boundary are delivered; Decision 4 must specify those paths so the
application can handle completed work without mistaking it for accepted content.
The concrete interface should let both examples use the same coordination
mechanism without manual message-history assembly or per-turn lifecycle
bookkeeping. Callback model-call association remains part of Decision 5.
### 4. How Are Application Next Actions And Execution Outcomes Represented?
**Status: open.** Subject to Decision 3, define content-neutral next actions and
distinguish application-directed completion, budget exhaustion, interruption,
and operational failure.
**Status: open.** For the coordinator selected in Decision 3, define
content-neutral next actions and distinguish application-directed completion,
budget exhaustion, interruption, and operational failure.
**Recommendation:** let the application request another turn or indicate that
the interaction should stop, separately from reporting an operational error.
@@ -194,22 +295,22 @@ small explicit next-action result without introducing domain-specific statuses.
**Details to settle:** next-action and outcome vocabulary, callback/result types,
result-plus-error behavior, zero-budget behavior, and what consumes an
additional-turn budget when preparation or generation fails. Keep such a budget
distinct from structural repair and transport retry. If coordination is left
to consumers, limit this decision to continuation execution outcomes.
distinct from structural repair and transport retry. Specify how an application
receives the last completed response when no further turns remain, including
whether and how its callback is invoked at that boundary.
### 5. How Are Related Executions And Usage Accounted For?
**Status: open.** Define accounting for the initial exchange, each continuation,
its structural repairs and transport retries, and model calls made by
application callbacks, whether or not PromptKit supplies a coordinator.
application callbacks, both for standalone continuations and coordinated turns.
The current [observation boundary](../internal/observation.md) keeps nested
invocations independent. A callback's arbitrary model calls cannot simply be
assumed to belong to the enclosing interaction's totals.
**Recommendation:** preserve independent run accounting and add explicit
relationships between exchanges, branches, and associated runs, with sequence
relationships if coordination is selected.
relationships between exchanges, branches, sequences, and associated runs.
Choose a small association mechanism, such as a scoped observation facility or
explicit execution references, so consumers need not rebuild lifecycle tracking.
Shared initial work should be referenced by branches and counted once when
@@ -256,9 +357,10 @@ an implementation plan. The recommended sequence is:
1. Deliver capture and independent continuation through the public facade,
allowing Audita to remove its reconstruction and prefix-verification work.
2. If selected after evaluating general usefulness, build coordination and/or
bounded looping on that capability, reusing ordinary execution, validation,
capacity, and observation machinery.
2. Build the optional sequential coordinator on that capability, reusing
ordinary execution, validation, capacity, and observation machinery. Refine
both consumer sketches against the selected public contracts to verify that
the same mechanism removes useful repeated work in each application.
Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios)
as consumer checks, alongside analysis and branching-conversation examples that