# Invocation-Scoped Execution Observation Status: accepted scope; implementation pending. ## Purpose Allow consumers to select execution observation once for an application-owned operation, rather than attach an observer to every PromptKit request. Nested, concurrent, and multi-engine calls using that operation's context should participate automatically. Include optional provider-reported cost alongside token usage, with built-in cost support initially limited to OpenRouter. The motivating consumer is CommandKit's optional `ckprompt` integration, which could use this facility to report token usage for an entire command or pipeline. PromptKit supplies application-neutral observation and aggregation; consumers choose how to present or use the results. ## Current Baseline PromptKit v0.24.0 already provides: - [ExecutionObserver and ComposeObservers](../../observations.go) for synchronous metadata callbacks and ordered composition. - [ExecutionGroup](../../execution_group.go) for concurrent collection, detached snapshots, and aggregation of the first terminal event for each observed run. - Presence-aware prompt, completion, total, cached, and cache-write token counts, including aggregate accounting across retries and repairs. - Explicit selection through `RunRequest.Observer`, retained for prepared execution, and [CoordinateRequest.Observer](../../coordinate.go) for coordinator-selected continuation runs. The missing capabilities are contextual observer selection and provider-reported cost accounting. Applications currently have to enroll each request explicitly, and PromptKit does not expose provider charges. The existing [consumer observation guide](../consumers/pkg-promptkit.md#observe-an-ordinary-run) describes that workflow. ## Target Public API Add one public helper: ```go func WithExecutionObserver(ctx context.Context, observer ExecutionObserver) context.Context ``` The returned context associates the observer with executions using that context or derived contexts that retain its values. It composes with any observer already associated with the context; it does not replace the inherited observer. A nil observer leaves the inherited selection unchanged. No engine option, process-wide default, or replacement group type is needed. Extend the existing result, event, and group surfaces with cost metadata as described below. The intended consumer workflow is: ```go var group promptkit.ExecutionGroup ctx = promptkit.WithExecutionObserver(ctx, group.Observe) err := runApplicationPipeline(ctx) // After all application-owned work has finished, including on failure: summary := group.Snapshot() ``` The consumer owns context propagation, waiting for its work, and using the snapshot and error. Calls using unrelated contexts are not automatically enrolled. Group coverage continues to describe observed members, not future work or overall application progress. ## Observer Selection And Delivery ### Composition Preserve existing explicit observers. Deliver each event to the explicit request observer first, then any coordinator observer already composed with it, then contextual observers in registration order, from inherited to newly added. Preserve the existing synchronous callback and panic behavior. Concurrent executions may invoke a shared observer concurrently; consumers remain responsible for synchronizing their callback state. `ExecutionGroup` already supports concurrent use. ### Ordinary And Nested Execution `Run` selects contextual observation from its invocation context. Each execution has its own observation identity, recorder, and accounting, including a nested execution using a context passed to an application callback or injected client. Inherited observer selection is distinct from private execution state. Existing clearing of an enclosing execution's private state at a public execution boundary must preserve the contextual observer selection. Inheriting observation does not reuse an enclosing run's recorder, totals, or lifecycle. ### Prepared Execution `RunPrepared` selects contextual observation from the context supplied when the handle is executed. A handle prepared in one scope and executed in another participates in the execution scope, not the preparation scope. The explicit `RunRequest.Observer` remains retained by a successfully prepared handle as today and composes with the execution-time contextual observers. Preparation itself emits no execution events. Discarded handles and rejected execution claims retain their existing observation behavior. ### Coordinated Execution Continuation runs receive contextual observation through their ordinary `Run` boundary. `CoordinateRequest.Observer` continues to select continuation runs only. Additional calls made by application callbacks participate when they use the scoped context, without being automatically selected by the coordinator's explicit observer. Contextual observers are incorporated once per execution, rather than being re-added by coordination or another forwarding layer. ### Duplicate Registration And Accounting Registering the same callback through multiple explicit or contextual paths may deliver the same event more than once. Do not attempt to identify or deduplicate arbitrary Go function values. Retain one execution lifecycle per run and reuse `ExecutionGroup`'s existing terminal-event deduplication. Selecting the same group through multiple paths must not inflate its totals. Terminal usage is an aggregate, not an additional contribution to be added to attempt usage. These rules apply to cost as well as token counts. ## Optional Provider-Reported Cost ### Initial Provider Support Capture OpenRouter Chat Completions' top-level `usage.cost`, which OpenRouter defines as the amount charged to the account, denominated in USD. See its [usage accounting reference](https://openrouter.ai/docs/cookbook/administration/usage-accounting) and [USD cost explanation](https://openrouter.ai/blog/insights/cost-vs-quality-tradeoff-framework-for-agent-models/). Do not add `cost_details` breakdowns or combine upstream inference cost with the top-level charge; those describe different facts and could double-count cost. For the built-in client, select this interpretation for the resolved `openrouter` backend ID. Do not infer USD semantics from an arbitrary backend's field named `cost`, or add endpoint discovery or verification. An endpoint override on that backend continues to select its OpenRouter response contract. Other built-in backend selections report unknown cost initially. Additional provider adapters or cost-capability configuration are not needed for this scope. Use the cost included in the normal completion response. No additional usage lookup, pricing request, or opt-in provider request parameter is needed. ### Monetary Representation Use `float64` amounts in USD. Make the unit explicit in public field names such as `CostUSD`, rather than introducing a generic amount with an implicit currency. At the individual model-client response boundary, use an optional amount (`*float64`): nil means unknown, and a pointer to zero means reported zero. Injected clients may supply the same optional USD value without needing an OpenRouter backend; no provider-specific interface is needed. Expose cost separately from `TokenUsage` and `TokenUsagePresence`, whose fields remain token counts. For run results, execution events, and group summaries, use a small presence-aware cost value with `Value float64`, `Known bool`, and `Complete bool`, following the existing observed-count conventions. Successful run results should expose cost even when no observer was registered; observers also provide per-attempt and terminal cost, including available facts on failure. Binary floating point is sufficient for operational telemetry and keeps decoding, addition, and the public API lightweight. Do not round to cents or another fixed scale during capture or aggregation; consumers choose display precision. Decimal strings or a decimal arithmetic dependency would be appropriate for exact financial ledgers, while fixed-scale integers would require a selected precision and rounding policy. Neither is warranted for this reporting feature. The API does not promise exact decimal arithmetic or invoice reconciliation. ### Presence, Aggregation, And Failures - An absent or null cost is unknown. Explicit numeric zero is known zero. A backend that reports no cost remains unknown even when a consumer knows it has no external charge. - On a dispatched attempt, a usable reported cost is known and complete for that attempt. Cost completeness is independent of token-field completeness. - Sum known costs across dispatched attempts, including initial generations, retries, and repairs. `Known` means at least one attempt reported cost; `Complete` means every dispatched attempt did. Unknown values contribute no amount, but make the total incomplete. With no dispatched attempts, follow the existing known, complete zero convention. - Group summaries reuse existing terminal-only aggregation and deduplication: sum known terminal values, OR their `Known` flags, and AND their `Complete` flags. Retain existing empty-group and terminal-coverage semantics. A subtotal from OpenRouter mixed with unreported local-backend cost is incomplete. - Keep valid cost observed from a decoded successful envelope when a later choice/content check fails, as with token usage. Invalid framing and non-2xx error bodies continue to supply no usage. Cancellation or failure may leave incomplete reporting even when billable upstream work occurred. - Treat an unusable optional cost value, including a nonnumeric value or one not representable as a finite `float64`, as unknown rather than making an otherwise valid generation fail. This does not relax existing response framing or content validation. An injected client likewise cannot supply a nonfinite usable cost. Completeness describes coverage of reported values, not verification of the provider's billing. Preserve reported usable amounts without pricing inference or reconciliation. ## Accounting And Responsibility Boundaries Reuse existing events, token presence/completeness, and group summaries, extending them with the cost facts above. Available usage remains observable after an error or cancellation under the existing execution-observation behavior; this does not promise that upstream providers report usage for every attempt. PromptKit owns observer selection, event generation, and aggregation. CommandKit and other consumers own invocation lifecycle, CLI flags, presentation, output delivery, and interpretation. This feature adds no CommandKit dependency or application-specific concepts to PromptKit. Smoke diagnostics retain their separate report/evidence model and are not enrolled in ordinary execution observation by this change. Reasoning-token capture, additional built-in cost providers, multiple currencies, currency conversion, pricing estimates, billing reconciliation, persistent history, and logging infrastructure are outside this scope. ## Work Needed And Completion Criteria - Add the context helper and connect observer selection to ordinary and prepared execution, keeping contextual selection separate from private run state. - Preserve existing explicit selection and coordinator behavior while applying the composition and lifetime rules above. Preserve existing observer selection when no contextual observer is selected. - Carry optional OpenRouter cost through response decoding, internal/public conversions, successful run results, attempt and terminal events, and existing group summaries. Keep result cost available without registering an observer. Preserve existing token fields and accounting. - Cover ordinary and prepared execution, coordinated continuations, nested calls using derived contexts, concurrent calls, multiple engines sharing a scope, and independent scopes on a reused engine. - Cover callback ordering, nested contextual registration, nil registration, duplicate group enrollment, prepared-handle scope selection, and existing preparation/discard/rejected-claim behavior. - Verify aggregate accounting through retries and repairs, available usage after failures or cancellation, and known zero versus unknown or incomplete counts. Use offline fixtures and injected clients; live inference is unnecessary. - Cover fractional USD values, explicit zero, omitted/null and unusable cost, non-OpenRouter built-in selection, optional injected-client cost, mixed known and unknown totals, repair/retry aggregation, and duplicate group registration. Use tolerant comparisons where floating-point addition requires them; do not turn telemetry into an exact-decimal guarantee. - Document the exact helper contract in GoDoc and add a focused invocation-level aggregation example to the consumer observation guide, including cost availability, units, and completeness. Document OpenRouter cost decoding in the [integration contract](../integrations/openai-compatible-chat.md#response-handling), and update internal observation and model-client documentation for changed flows. This roadmap defines the target behavior. A separate implementation plan can divide the work into appropriately sized stages.