From fed9ad082df0a9b7335015c3ba99277b605cd69b Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 22 Nov 2026 17:16:14 +0000 Subject: [PATCH] Clarify inspection and diagnostic accounting roadmap --- docs/roadmap/invocation-scoped-token-usage.md | 34 ++++++++++++++----- 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/docs/roadmap/invocation-scoped-token-usage.md b/docs/roadmap/invocation-scoped-token-usage.md index 7880781..b3f81fc 100644 --- a/docs/roadmap/invocation-scoped-token-usage.md +++ b/docs/roadmap/invocation-scoped-token-usage.md @@ -263,9 +263,12 @@ Existing explicit diagnostic endpoint overrides remain available. The adapter can dispatch without a credential; selecting vLLM does not declare authentication required or inapplicable. -Recognize a successful models-list response as acceptance of the probe request -and a 401 as rejection. Unrecognized responses leave acceptance unknown under -the existing diagnostic reporting model. Report observed credential presence and +Recognize HTTP 200 with a valid JSON object containing `object: "list"` and an +array-valued `data` as acceptance of the probe request, and a 401 as rejection. +An empty `data` array counts as acceptance; recognition does not require a +particular model or inspect model capabilities. Reuse the existing bounded +diagnostic decoding and framing rules. Unrecognized responses leave acceptance +unknown under the existing diagnostic reporting model. Report observed credential presence and response facts without claiming that authentication is enforced or that models listing establishes inference authorization. Preserve the shared override rules, including `kind: none`; explicit `vllm_models` selection also works without a @@ -480,8 +483,12 @@ Resolve the field through ordinary final ConfigKit composition and inheritance; include it in typed backend helpers, configuration capture, and backend inspection. Retain the effective policy through prepared execution without reloading configuration. It is local accounting metadata, not an outbound -request parameter. Preserve explicit/null/omitted source provenance in inspection -rather than manufacturing an explicit false value for the disabled fallback. +request parameter. `InspectBackend` shows the resolved declared setting, +preserving explicit `false`; omitted and null values can remain absent. Do not +manufacture an explicit false value for the runtime fallback or add null-tracking +machinery to resource inspection. `InspectConfiguration` retains ConfigKit's +source history and explicit/null/omitted distinctions. Source provenance belongs +to configuration inspection, not resource inspection. This supports an explicit zero declaration, not configurable input/output or cache-read/write token rates, fixed nonzero request charges, pricing tables, or @@ -622,7 +629,9 @@ outside this scope. prepared-execution plumbing. Apply the fallback outside provider-specific decoding so it works with built-in and injected clients. Verify composition and inheritance, omitted/null/false behavior, explicit overrides, provider - independence, retained inspection provenance, and absence from provider bodies. + independence, configuration-inspection provenance, and absence from provider + bodies. Verify that resource inspection preserves explicit false without + manufacturing values or source-history fields for omitted/null settings. - Consolidate provider-specific response and authentication-probe behavior in `internal/llm`; keep coordination and accounting provider-neutral. Cover provider-default probes, explicit kind and endpoint overrides, `kind: none`, @@ -644,8 +653,9 @@ outside this scope. token mappings and existing field meanings. - Add the `vllm` provider selection, reasoning and cache-detail mappings, and `vllm_models` authentication adapter. Cover supplied/missing credentials, - recognized acceptance, rejection, unrecognized responses, provider defaults, - and explicit kind/endpoint overrides, including `none` and adapter selection + recognized acceptance (including an empty models list), rejection, + unrecognized responses, provider defaults, and explicit kind/endpoint overrides, + including `none` and adapter selection without a provider declaration. Verify that ordinary generation performs no diagnostic requests and leaves monetary cost unknown unless the backend explicitly declares the zero-cost fallback. @@ -742,6 +752,14 @@ field separately so an unusable provider cost becomes unknown before applying any configured-zero fallback, without changing existing framing or content validation. This needs no general numeric-conversion layer. +Apply configured-zero fallback after decoding, outside provider-specific code, +so built-in and injected clients share the same policy. Aggregate cost sources +from known contributions even when their amounts are zero; a reported zero and +a configured zero together have a mixed source. Keep subtotal completeness +separate from terminal coverage, using the existing coverage field and the +examples above. These rules belong in shared accounting rather than separate +provider, result, and group implementations. + Use the completion criteria above to select focused offline checks for each unit. This roadmap owns target behavior; a separate implementation plan owns the detailed work sequence.