Document invocation observation and cost reporting

This commit is contained in:
2026-11-22 18:38:11 +00:00
parent d4509b163e
commit 20c5ea47af
22 changed files with 231 additions and 52 deletions

View File

@@ -79,7 +79,7 @@ other backend IDs.
The coordinator does not retry injected clients; it normalizes a nil successful
response as malformed generation. The built-in HTTP client remains responsible
for provider transport behavior. The runner, rather than the
capacity layer, sums all five usage fields from the initial response and every
capacity layer, sums usage fields from the initial response and every
completed repair response into the successful run result.
An invocation-local observer receives attempt events after active permit

View File

@@ -35,6 +35,12 @@ unknown usage. Transport, timeout, malformed, and noneligible status failures
stop the sequence without retry exhaustion. Context is checked before each
wait, acquisition, and dispatch.
At every dispatched completion the coordinator resolves any reported cost
against the frozen backend zero-cost policy and adds it once to the run's
[cost accumulator](observation.md), independently of observer selection.
Injected responses may retain reported cost alongside an error. Undispatched
acquisitions and retry waits contribute no cost.
The public client adapter translates explicitly supplied public transport and
malformed-response identities into internal categories while retaining the
original error chain. This keeps injected attempt classifications consistent

View File

@@ -142,6 +142,14 @@ known. Strictly decoded usage can still be observed if later choice/content
checks fail; invalid framing or a non-success error body contributes none.
Existing successful-result accounting remains value-based.
The captured provider selects additional response details and authentication
diagnostic defaults. Optional new reasoning, cache-write, and OpenRouter cost
fields are decoded independently, without relaxing common-field or framing
validation. Cost remains an optional raw amount at this boundary; shared
accounting applies configured-zero fallback after dispatch. The integration
contract specifies the [wire mappings](../integrations/openai-compatible-chat.md#response-handling)
and [probe recognition](../integrations/openai-compatible-chat.md#explicit-smoke-diagnostics).
For a non-success response, `ProviderHTTPError` retains the HTTP status and
only normalized detail from the bounded recognized envelope. It retains
`ErrUnexpectedStatus` through unwrapping. The client owns response closure;

View File

@@ -21,7 +21,7 @@ contributor workflow and validation.
| `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) |
| `internal/domain` | Defines internal framework values for requests, immutable conversation exchanges, artifacts, prompt definitions, requirement inspection, profiles, execution targets, rendering, generation, validation, and source-neutral smoke diagnostics, and owns source-neutral invariants for shared execution settings, OpenAI-compatible base endpoints, session identifiers, output contracts, and resource-identity syntax and qualification. Source parsing, required fields, other source-specific normalization, defaulting, and boundary-specific error classification remain with their callers. | [Domain declarations](../../internal/domain/domain.go), [exchange storage](../../internal/domain/exchange.go), [requirement values](../../internal/domain/requirements.go), [endpoint invariant](../../internal/domain/endpoint.go) |
| `internal/engine` | Assembles an already captured configuration into shared capacity, rendering, validation, generation, repair, runner, and smoke-test collaborators. Immutable namespace views share those collaborators and qualify selections without rewriting captured references. Built-in engines provide the diagnostic HTTP collaborator; injected-client engines do not. It delegates resource inspection and listing to the capture. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#typed-configuration-loading), [internal runner](runner.md) |
| `internal/generation` | Coordinates frozen built-in HTTP attempts and one-shot injected calls through active backend permits, including bounded status retries, cancellable backoff, ordinary attempt observations, and invocation-local evidence for diagnostics. | [Internal generation coordination](generation.md) |
| `internal/generation` | Coordinates frozen built-in HTTP attempts and one-shot injected calls through active backend permits, including bounded retries, per-attempt cost, ordinary observations, and separate diagnostic evidence. | [Internal generation coordination](generation.md) |
| `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) |
| `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) |
| `internal/jsonvalue` | Validates and deeply copies bounded JSON-compatible provider values, and detaches previously accepted values without repeating validation. | [JSON values](../../internal/jsonvalue/jsonvalue.go) |
@@ -30,8 +30,8 @@ contributor workflow and validation.
| `internal/prompt` | Renders prompt messages from Go templates with artifact, variable, session, and cache-control data, and conservatively enumerates dependencies from captured template syntax. | [Go-template renderer](../../internal/prompt/go_renderer.go), [requirement discovery](../../internal/prompt/requirements.go) |
| `internal/artifact` | Resolves ordinary inline and unrestricted caller-selected file references into copied artifacts with metadata and hashes. | [Internal sources and validation](sources.md) |
| `internal/validate` | Creates operation-local validation plans from inline schema documents or bound filesystem resources and validates basic, JSON, and JSON Schema output. | [Framework formats](../formats.md#schemas), [internal sources and validation](sources.md) |
| `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible chat requests from resolved execution targets, including bounded response decoding, authentication, deadline handling, per-invocation HTTP evidence and built-in diagnostic probes, and ownership of the OpenAI-compatible reserved request-field policy. | [Internal model client](llm.md) |
| `internal/observation` | Owns invocation-local run IDs, presence-aware usage and phase accounting, synchronous metadata event delivery and composition, and explicit terminal-based group aggregation; smoke diagnostics keep their private evidence outside ordinary observer events. | [Internal execution observation](observation.md) |
| `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible requests, provider-specific response interpretation and authentication probes, bounded decoding, deadlines, and diagnostic evidence. | [Internal model client](llm.md) |
| `internal/observation` | Owns invocation-local run IDs, presence-aware token and cost accounting, contextual observer selection, synchronous event delivery, and terminal-based group aggregation; smoke diagnostics retain separate evidence. | [Internal execution observation](observation.md) |
| `internal/usecase` | Selects captured prompts and profiles, enumerates requirements from captured prompt syntax, applies request overrides and completion checks, coordinates ordinary and single-claim prepared execution across rendering, artifact loading, operation-local validation plans, generation, capacity, and bounded repair, advances a single conversation branch, and runs explicit smoke diagnostics with private cases, shared generation/capacity, and optional built-in HTTP probes. | [Internal runner](runner.md), [requirement inspection](../../internal/usecase/requirements_inspection.go), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) |
The internal engine assembles these internal components without exposing their

View File

@@ -216,10 +216,16 @@ ordinary `Run` reuses its run ID and carries it to the generation coordinator
through context. Prompt and profile identities are enriched only after their
respective resolutions succeed, before admission. The facade starts observation
before request conversion and emits its terminal event after public error
mapping and after the runner's deferred admission release. Preparation and
prepared execution follows the same terminal mapping after a successful claim;
mapping and after the runner's deferred admission release. Preparation emits no
events; prepared execution follows the same terminal mapping after a successful claim;
invalid or discarded handles do not acquire an observer.
Ordinary runs establish fresh private cost state before observation; a claimed
prepared execution establishes its own state after claim. Contextual observers
remain available at public boundaries, while enclosing recorder and cost state
are masked. Prepared handles retain the request observer and frozen provider
policy, not preparation-context selection or an accumulator.
The facade establishes an optional operation scope for ordinary runs before
observation and preparation. A claimed prepared execution establishes its own
scope from the retained duration; preparation and idle time do not consume it.
@@ -273,11 +279,13 @@ run identifier, and UTC timing. The same effective session reaches initial
generation and any repair attempt through the rendered prompt. The same
effective target and presence metadata, including backend identity and direct
credential during execution, reaches generation and every repair attempt.
Result usage is the field-wise sum of all five usage values from the initial
Result usage is the field-wise sum of usage values from the initial
response and every completed repair response. Final raw output, artifact, and
validation state still come from the last candidate. A repair error returns no
partial run result or partial result usage, although the invocation-local
observer retains completed-attempt usage and reports it in the terminal event.
Result cost instead snapshots the shared accumulator across all dispatched
attempts, including retries and repairs, whether or not observation is enabled.
## Failure Categories