diff --git a/docs/roadmap/conversation-continuation.md b/docs/roadmap/conversation-continuation.md index db94583..14e9520 100644 --- a/docs/roadmap/conversation-continuation.md +++ b/docs/roadmap/conversation-continuation.md @@ -1,8 +1,9 @@ # Conversation Continuation And Application-Directed Interaction -Status: behavioral decisions accepted; ready for API and staged implementation -planning. No implementation is authorized by this document. Exact public types -and signatures remain for implementation planning. +Status: continuation and coordinator behavioral decisions accepted; optional +execution groups accepted as a separable addition with open decisions below. +No implementation is authorized by this document. Exact public types and +signatures remain for implementation planning. ## North Star: Content-Neutral, Generally Useful Capabilities @@ -51,6 +52,8 @@ The proposed capabilities are: - Allow independent continuations from one captured exchange. - Provide an optional sequential coordinator for bounded looping using application-supplied next actions and follow-up requests. +- Provide optional execution groups for explicitly selected runs and summaries + of their available execution facts, independently of conversation continuation. Continuation has direct value by removing consumer capture, reconstruction, prefix comparison, and history assembly. The coordinator centralizes turn @@ -471,9 +474,10 @@ the operational-error behavior does not reclassify it. **Status: accepted; exact placement of relationship metadata remains for implementation planning.** Reuse ordinary per-run accounting, expose a minimal -parent relationship for continuations, and leave grouping of other calls and -overall totals to applications. This applies to both standalone continuations -and coordinated turns. +parent relationship for continuations, and let applications select accounting +membership explicitly. The separable execution-group addition below supplies +optional summaries without expanding the coordinator. This applies to both +standalone continuations and coordinated turns. The current [observation boundary](../internal/observation.md) keeps nested invocations independent. A callback's arbitrary model calls cannot simply be @@ -504,10 +508,13 @@ result. **Accepted application boundary:** model calls made inside a callback remain independently observed and accounted for. PromptKit does not infer that they belong to a continuation or aggregate merely because they execute inside its -callback. Applications explicitly group those existing observations when useful, -including concurrent calls. Audita retains assembly of its overall usage report; -PromptKit supplies execution facts without owning an application-wide reporting -model or promising complete cost accounting. +callback. Applications explicitly select which runs belong together, including +concurrent calls, and may use the optional execution groups below to collect and +summarize those facts. This revises the earlier choice to leave all aggregation +to applications; membership and domain interpretation remain application-owned. +Audita retains assembly of its overall usage report and mappings to its domain +identifiers. PromptKit supplies execution facts without owning an +application-wide reporting model or promising complete cost accounting. **Alternatives considered, not selected:** @@ -518,10 +525,11 @@ model or promising complete cost accounting. consumers would have to reconstruct continuation relationships themselves. **Implementation planning:** place the minimal parent reference and preserve -ordinary per-request observation behavior for every continuation. Broader scoped -observation facilities or aggregate reporting require a separate demonstrated -need. The additional-turn budget continues to bound continuation executions, -not all provider calls performed inside those executions or application callbacks. +ordinary per-request observation behavior for every continuation. Execution +groups address the separately demonstrated need for explicit accounting across +related calls; broader reporting remains outside this scope. The additional-turn +budget continues to bound continuation executions, not all provider calls +performed inside those executions or application callbacks. ### 6. Where Should Continuation Inspection Live? @@ -561,12 +569,118 @@ but this feature does not select that registry. Explicit access through an already-held exchange satisfies the present need. Richer reporting or registry capabilities require separate consumer justification. +## Separable Addition: Optional Execution Groups + +**Status: accepted direction; the four decisions below remain open.** Audita's +follow-up request identifies a distinct need: a validator call can belong to the +same logical operation as a continuation without extending its conversation. +Conversation ancestry describes which exchange supplies history; accounting +membership describes which executions the application chooses to report together. +Neither relationship implies the other. + +An application creates a group for a logical operation and explicitly associates +selected ordinary runs or coordinated continuations with it. Calls made by a +validator or other callback join only when the application chooses. Groups use +existing run identities and observation/usage semantics to summarize available +execution facts. They are usable without capture or a coordinator, and continuation +and coordination remain usable without them. Selecting a group does not change +execution scheduling, capacity, cancellation, retry, or validation behavior. + +For example, an application may group a continuation and a separate validator +run while keeping their conversations independent. It may also include the +initial run shared by several branches. Replaying that run's captured history +does not enroll it or count its usage again; newly reported usage for sending +history in later executions still belongs to those later runs. + +The group does not infer content meaning, automatically capture arbitrary nested +calls, schedule work, determine application outcomes, or introduce a reporting +framework. Applications retain domain identifiers, presentation, and any broader +report assembly. Group summaries are obtained separately from the coordinator's +minimal termination result. They preserve unavailable versus explicit zero usage +and do not add terminal totals to the attempts they already summarize. + +### Group Decision 1: How Does An Application Select Membership? + +**Recommendation:** first evaluate a group collector compatible with the existing +observer API. Attaching it explicitly to a request would enroll the observed run; +a coordinator could receive an explicit selection for its own continuation runs. +Callback calls would still require their own selection. This reuses the existing +source of execution facts and avoids a second accounting path. + +The design must allow the application to keep its existing observer alongside +the group. Specify ordinary and prepared execution attachment and coordinator +propagation without inheriting membership into arbitrary nested calls. Preparation +alone is not an executed member; discarded handles and failed claims do not +currently emit execution observations. + +**Alternative:** explicit group options on requests and the coordinator, backed +by ordinary observation internally. This makes membership clearer and separates +it from application event delivery, but adds public API plumbing. Choose after +sketching the smallest workable interfaces for ordinary, prepared, coordinated, +and callback-initiated calls. Do not select a context-based implicit scope. + +### Group Decision 2: What Is The Group's Lifetime? + +**Recommendation:** expose summary snapshots from a caller-held group, with no +mandatory close or wait operation. Applications already own the work and decide +when to read or discard the group. Support concurrent selected executions and +summary reads without imposing serial scheduling. + +A snapshot must distinguish observed terminal runs from members whose terminal +facts are not yet available. A missing terminal event can reflect ongoing work +or interruption such as a panic; it is not evidence of zero usage or success. +Do not promise terminal observations beyond the existing observation contract. + +**Alternative:** an explicit seal/close or wait lifecycle could provide a fixed +membership boundary or completion wait. It adds lifecycle rules and overlaps +with application orchestration; defer unless a consumer demonstrates that need. + +### Group Decision 3: May Groups Overlap? + +**Recommendation:** allow a run to belong to more than one explicitly selected +group, counting each run identity once within each group. This accommodates both +an operation summary and a broader application summary without requiring a +hierarchical group model. Overlapping group totals are not necessarily additive. + +**Alternative:** require one group per run. This simplifies membership plumbing, +but restricts useful accounting views and makes applications rebuild the broader +aggregation themselves. There is no demonstrated need for that restriction. +Do not add group nesting, automatic parent membership, or group scheduling merely +to support overlap. + +### Group Decision 4: Which Facts Does A Summary Expose And Aggregate? + +**Recommendation:** start with member run identities, terminal-observation status, +and a small selection of existing execution counters and presence-aware usage +totals. Select the exact fields during API design; avoid a comprehensive new +report schema or new outcome hierarchy. Summaries describe execution facts, not +application acceptance or monetary cost. Summed run durations, if included, +must not be presented as elapsed group time when runs overlap. + +Prefer aggregating one terminal event per run. Terminal observations already +summarize retries, structural repairs, reported usage, and failure-path facts, +including executions that return no ordinary result. This is simpler than +reconstructing those totals from attempt events. Preserve each usage field's +known/completeness distinctions and expose missing terminal coverage separately, +so a total over finished members cannot imply complete accounting for all members. + +**Alternative:** accumulate attempt events for live partial totals, reconciling +terminal events without counting the same work twice. This exposes more facts +before completion or after a missing terminal event, but needs additional +reconciliation rules. Prefer terminal-based totals unless live attempt accounting +has demonstrated value. Whichever representation is selected, absent evidence +must not become a known zero, and terminal summaries must never be added to their +already-counted attempts. + ## Proposed Delivery And Validation -All six behavioral decisions are accepted. Remaining work is API and staged -implementation planning: exact types and signatures, capture opt-in syntax, -message assembly details, placement of relationship metadata, and the concrete -inspection representation. The recommended delivery sequence is: +The six continuation/coordinator behavioral decisions are accepted. Their +remaining work is API and staged implementation planning: exact types and +signatures, capture opt-in syntax, message assembly details, placement of +relationship metadata, and the concrete inspection representation. Execution +groups have four open decisions above and a separate delivery track; resolving +or implementing them is not a prerequisite for continuation. The recommended +continuation delivery sequence is: 1. Deliver capture and independent continuation through the public facade, allowing Audita to remove its reconstruction and prefix-verification work. @@ -575,6 +689,10 @@ inspection representation. The recommended delivery sequence is: both consumer sketches against the selected public contracts to verify that the same mechanism removes useful repeated work in each application. +Plan execution groups independently after resolving their open decisions, +reusing ordinary observations and preserving existing application observers. +They can ship separately without changing the coordinator termination contract. + Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios) as consumer checks, alongside analysis and branching-conversation examples that exercise the same APIs without repair-specific concepts. Additional proposed @@ -595,14 +713,23 @@ coverage should establish: - Assessment calls against the same backend at capacity one, without capacity retained by the coordinator blocking them. - Per-run usage and failure diagnostics available through ordinary observations, - including incomplete usage. Consumer examples explicitly group callback calls - and count shared initial work once, without coordinator-provided totals. + including incomplete usage. Consumer examples explicitly select accounting + membership and count shared initial work once, without coordinator-provided + totals; optional groups may supply those summaries when available. - Immediate parent relationships distinguish successive turns and independent branches using existing run identities. - Explicit exchange details expose captured logical history and minimal identity metadata through ordinary encoding. Mutating returned details does not affect retained branch state, and inspection requires no rendering or external reads. +Execution-group validation should separately cover explicit ordinary, prepared, +coordinator, and callback membership; unrelated nested calls remaining outside +the group; independent ancestry and membership; concurrent collection alongside +an existing application observer; failure-path terminal facts; unavailable and +explicit-zero usage; missing terminal observations; and no attempt/terminal +double counting. If overlapping groups are selected, verify once-per-run counting +within each group without implying that overlapping totals can be added. + These are planning targets, not new implemented guarantees. Keep additional mechanisms and constraints tied to the selected decisions and demonstrated consumer needs.