Decode provider reasoning and cache usage details
This commit is contained in:
@@ -163,7 +163,15 @@ data, a second JSON value, absent choices, missing content, `null` content,
|
||||
non-string content, and size overflow are malformed responses and return no
|
||||
partial result.
|
||||
|
||||
Each of the five token fields retains whether the provider reported a
|
||||
For OpenRouter and vLLM, the client also reads reasoning tokens from
|
||||
`usage.completion_tokens_details.reasoning_tokens`. OpenRouter cache writes
|
||||
come from `usage.prompt_tokens_details.cache_write_tokens`; vLLM uses
|
||||
`usage.prompt_tokens_details.created_cache_tokens`. The generic provider keeps
|
||||
its top-level `usage.cache_write_tokens` mapping. Missing, null, malformed,
|
||||
fractional, or unrepresentable new details remain unknown without invalidating
|
||||
otherwise usable content. Existing common count decoding remains strict.
|
||||
|
||||
Each common token field retains whether the provider reported a
|
||||
non-null integer, including zero. Omitted and `null` fields are unknown for
|
||||
observations. A strictly decoded successful envelope can yield observed usage
|
||||
even when a later missing choice or content makes the attempt malformed.
|
||||
|
||||
Reference in New Issue
Block a user