Compare commits
12 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 668d09997a | |||
| 69203dcb12 | |||
| 3b760310a6 | |||
| b39460a507 | |||
| 5177db0611 | |||
| 31f1e6a89d | |||
| f7085d76f6 | |||
| cf0bfd4d76 | |||
| 6b1323d1fe | |||
| fb87fabc2e | |||
| 5adc1c1e64 | |||
| 0d27252999 |
131
docs/roadmap/audit/01-correctness.md
Normal file
131
docs/roadmap/audit/01-correctness.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# Functional Correctness Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `d9f6aad`.
|
||||
|
||||
## Scope And Evidence
|
||||
|
||||
Reviewed the public facade and conversions; ConfigKit handoff, normalization,
|
||||
presence-aware target assembly, collections and namespace selection; artifact
|
||||
loading and template rendering/requirement discovery; ordinary and single-use
|
||||
prepared execution; output validation and repair; backend admission and active
|
||||
permits; HTTP request/response handling and transient retries; execution
|
||||
observations; and smoke-test selection and reporting. Contracts were checked
|
||||
against GoDoc, the format and integration references, internal documentation,
|
||||
and relevant existing tests. Source loading before the ConfigKit handoff remains
|
||||
ConfigKit's responsibility.
|
||||
|
||||
Validation:
|
||||
|
||||
- `go test -race -vet=off ./...` passed, including the maintained examples.
|
||||
- `go vet ./...` passed.
|
||||
- An offline, temporary public-API reproduction outside the repository confirmed
|
||||
the number conversion and captured schema payload described below. It used
|
||||
`ProfileSource`, `Prepare`, a runtime override, and `WithSchemaFS` with
|
||||
`fstest.MapFS`; no provider was contacted.
|
||||
|
||||
No production code or tests were changed. This audit does not establish provider
|
||||
availability or acceptance of arbitrary JSON Schema features.
|
||||
|
||||
## 1. Provider Source Helpers Silently Change Exact JSON Numbers
|
||||
|
||||
**Priority:** P2. Confirmed data-conversion defect.
|
||||
|
||||
**Code:** [providerValueForMapping](../../../provider_values.go:51), used by
|
||||
[ProfileSource](../../../profiles.go:33) and
|
||||
[BackendSource](../../../backends.go:100).
|
||||
|
||||
**Circumstances:** A caller supplies a valid `json.Number` in provider
|
||||
`ExtraParams` that cannot be represented exactly by the helper's chosen native
|
||||
number type. After signed and unsigned integer parsing fail, the helper converts
|
||||
the value to `float64` without checking whether that conversion loses precision.
|
||||
The same problem affects sufficiently precise fractional values.
|
||||
|
||||
**Expected:** A source helper preserves the meaning of a supported provider
|
||||
value, or reports that the value cannot be represented. It must not silently
|
||||
substitute a different number. This concerns PromptKit's own convenience-source
|
||||
conversion, not additional validation of ConfigKit's raw sources.
|
||||
|
||||
**Actual / reproduction:** With
|
||||
`ExtraParams: map[string]any{"number": json.Number("18446744073709551617")}`
|
||||
on a profile, `NewEngine` succeeds. JSON encoding of
|
||||
`Prepare(...).EffectiveModelParams.ExtraParams` produces:
|
||||
|
||||
```json
|
||||
{"number":18446744073709552000}
|
||||
```
|
||||
|
||||
Supplying the identical value through `RunRequest.Execution.ExtraParams`
|
||||
instead produces:
|
||||
|
||||
```json
|
||||
{"number":18446744073709551617}
|
||||
```
|
||||
|
||||
The built-in client serializes those effective values directly into the request,
|
||||
so configuration-source selection changes the actual provider parameter.
|
||||
`BackendSource` shares the same converter. No high-precision numeric preservation
|
||||
claim needs to be added for arbitrary source formats to correct this discrepancy.
|
||||
|
||||
**Potential remediation:** Preserve exact numbers through a numeric
|
||||
representation supported by ConfigKit, if available. If that contract cannot
|
||||
represent a particular `json.Number` exactly, surface a deferred source error
|
||||
rather than returning a rounded float or a numeric-looking string. Check numeric
|
||||
value equivalence rather than textual spelling: alternate exponent spellings
|
||||
need not be preserved. Add focused regression coverage for both source helpers
|
||||
and contrast with the already-preserving runtime path.
|
||||
|
||||
## 2. External Schema Resources Do Not Reach The Provider Constraint
|
||||
|
||||
**Priority:** P2. Confirmed schema-packaging gap; remediation requires choosing
|
||||
how provider-facing schemas accommodate the existing local-reference feature.
|
||||
|
||||
**Code:** [preparedValidation.SchemaDocument](../../../internal/validate/standard_validator.go:63),
|
||||
[StandardValidator.PrepareValidation](../../../internal/validate/standard_validator.go:74),
|
||||
[FSValidator.PrepareValidation](../../../internal/validate/standard_validator.go:126),
|
||||
and [structuredOutputFromValidationPlan](../../../internal/usecase/runner.go:419).
|
||||
|
||||
**Circumstances:** A JSON Schema output contract uses a root schema referring to
|
||||
another file within the configured schema source. The
|
||||
[schema format contract](../../formats.md#schemas) supports such references.
|
||||
Preparation compiles the transitive resources locally, but `SchemaDocument`
|
||||
exposes only the unchanged root document. The runner always uses that document
|
||||
for the native provider constraint in JSON Schema mode.
|
||||
|
||||
**Expected:** The provider-facing constraint should carry the schema resources
|
||||
needed to interpret the supported local references, or there should be an
|
||||
explicitly chosen way to retain local validation without submitting an
|
||||
unresolvable native constraint. This is not a promise that every provider accepts
|
||||
every schema dialect feature.
|
||||
|
||||
**Actual / reproduction:** With these files under `WithSchemaFS`:
|
||||
|
||||
```text
|
||||
root.json: {"type":"object","properties":{"x":{"$ref":"child.json"}},"required":["x"]}
|
||||
child.json: {"type":"integer"}
|
||||
```
|
||||
|
||||
`Prepare` succeeds, but `PreparedRun.StructuredOutput.JSONSchema.Schema` contains
|
||||
only the root object, including the literal `$ref: "child.json"`. No child
|
||||
document, bundled definition, or reference rewrite is present. The HTTP adapter
|
||||
passes that object unchanged to `response_format.json_schema.schema`. An upstream
|
||||
provider has no access to the application filesystem or private `fs.FS`, so it
|
||||
cannot resolve that reference from the request. A provider requiring resolvable
|
||||
native schemas can reject generation before local validation or repair is
|
||||
reached. No live-provider rejection was tested.
|
||||
|
||||
The internal source document accurately describes the existing root-only
|
||||
projection; the gap is between the supported local schema-resource graph and
|
||||
its mandatory projection into an outbound constraint, rather than a failure of
|
||||
local compilation or prepared-resource freezing. Self-contained schemas are
|
||||
unaffected.
|
||||
|
||||
**Potential remediation:** Produce a self-contained provider schema from the
|
||||
operation's captured resources, preserving reference semantics, identifiers,
|
||||
anchors, and recursion. Avoid naive recursive textual substitution. An
|
||||
alternative is an explicit choice to use local-only schema validation when
|
||||
native submission cannot represent the resource graph. Any alternative that
|
||||
restricts existing schema sources or silently disables native structured output
|
||||
requires a deliberate compatibility/policy decision. Protect the selected
|
||||
behavior with an offline test combining transitive schema preparation and the
|
||||
actual outgoing request, while retaining frozen local validation for
|
||||
`RunPrepared` and repair.
|
||||
147
docs/roadmap/audit/02-efficiency.md
Normal file
147
docs/roadmap/audit/02-efficiency.md
Normal file
@@ -0,0 +1,147 @@
|
||||
# Algorithmic Efficiency Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `0d27252`.
|
||||
|
||||
## Scope And Method
|
||||
|
||||
Reviewed PromptKit-owned configuration assembly, provider-value conversion and
|
||||
copying, rendering and requirement analysis, inventory selection, prepared-run
|
||||
ownership, validation orchestration, repair formatting, retry coordination,
|
||||
observations, and capacity management. Findings below follow from implementation
|
||||
and operation counts; no benchmarks or live-provider calls were needed.
|
||||
|
||||
The public facade delegates to internal components. ConfigKit supplies the final
|
||||
configuration; PromptKit constructs immutable runtime indexes and captured
|
||||
templates. Execution uses operation-local validation plans, shared backend
|
||||
capacity, and a generation coordinator. Inspection reads the captured snapshot.
|
||||
ConfigKit and ReleaseKit internals are outside this repository audit.
|
||||
|
||||
### Deliberate Choices Preserved
|
||||
|
||||
Checked current internal documentation and relevant retired plans in Git history,
|
||||
including the performance plan at `8a5c0de`, configuration adoption at `5f7b194`,
|
||||
and collections at `89af2a1`. This audit does not reopen:
|
||||
|
||||
- lexical inventory ordering, recursive provider-map overlay semantics, or
|
||||
conservative requirement-discovery semantics;
|
||||
- captured parsing, per-render artifact-text reuse, operation-local schema
|
||||
compilation, frozen retry payloads, and linked-list FIFO permits with
|
||||
constant-time cancellation removal;
|
||||
- the existing JSON depth/work bounds, output/diagnostic limits, or copies
|
||||
deliberately retained to protect public and injected-collaborator ownership;
|
||||
- synchronous filesystem/template/schema operations or the selected retry and
|
||||
repair budgets.
|
||||
|
||||
The opportunities below concern implementation work that those choices do not
|
||||
require. Complexity statements use expected constant-time Go map operations and
|
||||
treat identifier comparison/hashing cost separately. No new public performance
|
||||
guarantee, cache, caller limit, or release gate is proposed.
|
||||
|
||||
## 1. Condition Evidence Unions Can Require Quadratic Work
|
||||
|
||||
**Priority:** P2. Requirement-inspection path; most relevant to deeply nested
|
||||
conditions rather than ordinary small templates.
|
||||
|
||||
**Code:** [evaluateCommand](../../../internal/prompt/requirements.go:142) and
|
||||
[mergeConditionEvaluations](../../../internal/prompt/requirements.go:318).
|
||||
|
||||
Each comparison recursively evaluates its operands, then allocates fresh
|
||||
`missingVariables` and `contentInputs` maps and copies both operands' accumulated
|
||||
evidence. A left-heavy expression with distinct missing caller keys, such as
|
||||
successively nesting `eq (eq .v0 .v1) .v2`, carries an increasingly large missing
|
||||
key set through each ancestor. Template parsing accepts these expressions;
|
||||
inspection can encounter them even when their eventual execution would fail or
|
||||
the inspection result is partial.
|
||||
|
||||
For an expression of size n, successive unions copy approximately
|
||||
2 + 3 + ... + n entries: **Theta(n²) total set work and cumulative allocation**.
|
||||
The final unique evidence has only O(n) entries. This is separate from the
|
||||
previously fixed repeated syntax-subtree scan: visiting each syntax node once
|
||||
does not eliminate repeated copies of the evidence produced by those nodes.
|
||||
|
||||
**Opportunity:** Transfer ownership of temporary sets and merge the smaller set
|
||||
into the larger one, rather than copying both at every comparison. For this
|
||||
expression family that is O(n); small-to-large merging gives O(n log n) aggregate
|
||||
set-insertion work across general expression trees. Another option is an
|
||||
operation-local evidence accumulator that records each discovery directly,
|
||||
while keeping only condition truth/limitation state on recursive return values.
|
||||
Choose the simpler approach after checking temporary-set ownership. Preserve
|
||||
current grammar, classifications, deduplication, sorted output, cancellation,
|
||||
and partial-result behavior; interpreting more expressions is not necessary.
|
||||
|
||||
## 2. Recursive Provider Overlay Repeatedly Clones Descendant Subtrees
|
||||
|
||||
**Priority:** P3. Construction-time improvement with an existing depth bound.
|
||||
|
||||
**Code:** [overlayProviderMap](../../../internal/configuration/targets.go:131).
|
||||
|
||||
The helper deep-copies the entire lower map before considering overrides. When
|
||||
both values at a key are maps, it recursively invokes itself on a descendant
|
||||
that was already copied, and copies that whole descendant again. The first copy
|
||||
is then replaced and discarded. All descendants on an overlapping path can be
|
||||
copied once per ancestor.
|
||||
|
||||
Let N be the lower tree's node count and D its overlapping map depth. Node-copy
|
||||
work can reach **O(ND)**. For a narrow chain it is the triangular sum
|
||||
D + (D-1) + ... + 1. A wide K-entry leaf below D overlapping ancestors is cloned
|
||||
roughly D times instead of once. `jsonvalue.CopyMap` also sorts map keys, so the
|
||||
same wide leaf's sorting work is repeated at those ancestors.
|
||||
|
||||
The existing maximum container depth of 100 is important: this is **not an
|
||||
unbounded quadratic-in-N claim under today's fixed limits**. It is an avoidable
|
||||
depth-dependent multiplier in traversal and cumulative allocations within those
|
||||
limits, multiplied again when multiple profiles overlay the same backend. That
|
||||
bound and the recursive-map/value-replacement semantics remain selected choices.
|
||||
|
||||
**Opportunity:** Clone the lower tree once, then overlay into the owned clone
|
||||
without re-cloning existing descendant maps. Copy incoming replacement values
|
||||
where needed to preserve isolation from configuration inputs. Alternatively,
|
||||
construct the merged output in one traversal. Either removes the repeated
|
||||
ancestor copies: expected O(N + M) merge traversal for lower/upper trees of sizes
|
||||
N and M, apart from the retained validation/sorting costs. Preserve null and
|
||||
array replacement, empty-map behavior, input ownership, and existing JSON
|
||||
validation/bounds; do not replace ConfigKit's configuration machinery.
|
||||
|
||||
## 3. Filtered Listings Copy And Scan The Complete Inventory
|
||||
|
||||
**Priority:** P3. Scalability opportunity for engines with many namespaces.
|
||||
|
||||
**Code:** [filterIdentityList](../../../internal/engine/engine.go:306),
|
||||
[ListPrompts](../../../internal/engine/engine.go:270), and the
|
||||
[configuration inventory methods](../../../internal/configuration/listing.go:14).
|
||||
|
||||
A named-namespace request first obtains a detached copy of all N identities and
|
||||
then scans it to build its k matching results. Even an absent namespace costs
|
||||
**Theta(N) time and O(N) temporary storage**, rather than work proportional to
|
||||
the selected inventory. Listing C namespaces separately costs Theta(CN); when
|
||||
each has a bounded number of entries, that becomes quadratic in total inventory
|
||||
size despite only O(N) total returned entries.
|
||||
|
||||
The collection implementation plan at `89af2a1` expressly allowed namespace
|
||||
buckets or sorted ranges to avoid repeated full scans; it did not require the
|
||||
present whole-inventory-copy algorithm. The lexical ordering and caller-owned
|
||||
result contracts do not prevent indexed selection.
|
||||
|
||||
**Opportunity:** Capture per-kind namespace buckets or ranges alongside existing
|
||||
construction-time inventories, then copy only the selected entries. Buckets
|
||||
allow expected O(1 + k) selection/copy; binary search over sorted full IDs can
|
||||
support named-prefix ranges in O(log N + k). Keep unqualified and all-ID modes
|
||||
correct without assuming unqualified entries form one contiguous lexical range.
|
||||
Resolve filters before requesting the inventory, and retain exact namespace
|
||||
matching, full IDs, ordering, context behavior, and independent returned slices.
|
||||
An all-ID request must still spend O(N) to return N owned entries; that is not a
|
||||
defect. No public pagination API or mutable cache is needed.
|
||||
|
||||
## Review Outcome
|
||||
|
||||
These three opportunities are supported by direct operation-count analysis;
|
||||
they are not measured latency claims. The largest asymptotic issue is growing
|
||||
condition-evidence copying. The overlay finding is explicitly bounded, and the
|
||||
listing finding matters as independent namespace count grows.
|
||||
|
||||
No additional ordinary correctness defect was established in this pass. The
|
||||
findings in [the correctness audit](01-correctness.md) remain separate and were
|
||||
not duplicated here. No production code, tests, dependencies, or established
|
||||
behavior were changed, so the successful race-suite/vet evidence recorded there
|
||||
still applies to the unchanged implementation; those checks were not rerun just
|
||||
for this documentation change.
|
||||
216
docs/roadmap/audit/03-complexity.md
Normal file
216
docs/roadmap/audit/03-complexity.md
Normal file
@@ -0,0 +1,216 @@
|
||||
# Complexity And Maintainability Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `5adc1c1`.
|
||||
|
||||
## Method And Architectural Assessment
|
||||
|
||||
Used the local codebase-memory graph's cyclomatic/cognitive complexity,
|
||||
function-length, and call-relationship tools to locate candidates. Reviewed
|
||||
functions and methods, including test hotspots, rather than relying on
|
||||
function-only rankings. Inspected implementation, callers, owning interfaces,
|
||||
and current architecture/internal documentation before selecting findings.
|
||||
Metrics are analyzer-specific indicators, not correctness proofs, acceptance
|
||||
thresholds, or requirements to split every large switch.
|
||||
|
||||
PromptKit's overall boundaries remain useful: a public facade adapts values and
|
||||
errors; configuration interprets ConfigKit's final handoff and captures runtime
|
||||
indexes; the runner coordinates preparation and execution; generation owns
|
||||
attempts/retries; capacity owns admission and permits; transport and schema
|
||||
adapters own external protocols. No repository-wide architectural rewrite or new
|
||||
public abstraction is warranted by this review.
|
||||
|
||||
Representative hotspot metrics from the current graph:
|
||||
|
||||
| Function | Cyclomatic | Cognitive |
|
||||
| --- | ---: | ---: |
|
||||
| `requirementWalker.visit` | 50 | 137 |
|
||||
| `validateEffective` | 39 | 132 |
|
||||
| `requirementWalker.evaluateCommand` | 26 | 103 |
|
||||
| `providerValueForMapping` | 33 | 84 |
|
||||
| `jsonvalue.copyValue` | 29 | 66 |
|
||||
| `loaded.buildExecutionProfiles` | 27 | 58 |
|
||||
| `loaded.materializePrompt` | 30 | 52 |
|
||||
| `Coordinator.generate` | 24 | 49 |
|
||||
|
||||
The findings below are behavior-preserving refactoring opportunities, not
|
||||
confirmed bugs or automatic release blockers. They cover both package-local
|
||||
complexity and interactions between packages. No production code, tests, or
|
||||
dependencies were changed, and no empirical testing or benchmarks were needed.
|
||||
|
||||
## 1. Consolidate Schema Compilation And Express The Runner's Actual Interface
|
||||
|
||||
**Code:** [validator interfaces](../../../internal/validate/validator.go:10),
|
||||
[schema implementations](../../../internal/validate/standard_validator.go:45),
|
||||
[runner preparation](../../../internal/usecase/runner.go:398), and
|
||||
[engine assembly](../../../internal/engine/engine.go:133).
|
||||
|
||||
**Complexity:** The validator maintains direct validation and prepared
|
||||
validation, with separate filesystem and `fs.FS` compilation paths. Compiler
|
||||
construction, resource registration, error wrapping, and cancellation checks
|
||||
are repeated across direct schema-validation methods and `PrepareValidation`.
|
||||
The package's constructors return `Validator`, although the runner actually
|
||||
requires `ValidationPreparer` and discovers it through a runtime type assertion.
|
||||
An implementation satisfying the declared constructor dependency can therefore
|
||||
still be unsuitable for the execution path.
|
||||
|
||||
**Opportunity:** Make the runner's preparation dependency explicit in its
|
||||
internal field/constructor types, updating assembly and focused test
|
||||
collaborators together. Retain the direct validation interface where callers
|
||||
actually need it. Consolidate compilation behind one internal operation, with
|
||||
small source-specific loading/URL/loader adapters; direct and prepared callers
|
||||
can reuse that operation rather than separately implementing registration and
|
||||
compilation. No public validator API or generic source framework is needed.
|
||||
|
||||
**Preserve:** Direct validation currently rejects malformed JSON before trying
|
||||
to compile its schema; simply implementing direct `Validate` as eager
|
||||
`PrepareValidation` followed by plan validation would change that error/I/O
|
||||
ordering. Keep that short-circuit, operation-local compiled resources, prepared
|
||||
freezing, schema dialect and containment rules, synchronous cancellation
|
||||
precedence, and the runner's existing nil-validator behavior where used.
|
||||
Coordinate schema representation changes with correctness-audit finding 2,
|
||||
without treating this refactor alone as its fix.
|
||||
|
||||
## 2. Decompose Configuration Validation And Resource Materialization
|
||||
|
||||
**Code:** [validateEffective](../../../internal/configuration/load.go:410),
|
||||
[materializePrompt](../../../internal/configuration/prompts.go:58),
|
||||
[buildExecutionProfiles](../../../internal/configuration/targets.go:14), and
|
||||
[resolution callbacks](../../../internal/configuration/resolve_prompts.go:10).
|
||||
|
||||
**Complexity:** `validateEffective` combines namespace checks, reference indexes,
|
||||
backend settings, capacity arithmetic, smoke metadata, profile bounds, and prompt
|
||||
references in one nested traversal. Prompt materialization then combines input
|
||||
validation, presence projection, content-file I/O, message validation, template
|
||||
compilation, output-contract construction, hashing, and inspection assembly.
|
||||
Target assembly is another long presence-state decision chain. Reviewing a
|
||||
single field change requires following several distinct responsibilities inside
|
||||
large functions.
|
||||
|
||||
**Opportunity:** Keep the top-level traversal/index preparation visible, but
|
||||
delegate coherent record-level work to backend, profile, and prompt helpers
|
||||
with explicit record/path/index arguments. Isolate prompt input projection,
|
||||
message materialization/compilation, and output-contract construction into
|
||||
small functions returning the values the caller needs. Keep target presence
|
||||
interpretation in one identifiable helper rather than distributing it between
|
||||
new callers. Organize files by those responsibilities only where that helps
|
||||
navigation; moving unchanged code between files is not itself the improvement.
|
||||
|
||||
**Preserve:** Validate only final ConfigKit-provided values. Keep recorded
|
||||
normalization, field presence/null distinctions, exact references, error
|
||||
identities, source-relative resource contexts, lazy metadata capture, and
|
||||
construction-time template/hash capture. Materialization legitimately combines
|
||||
validation with resource access; this finding does not require moving every
|
||||
check into a new validation pass or aggregating formerly fail-fast errors.
|
||||
Avoid a reflection-driven field schema that obscures the domain rules.
|
||||
|
||||
## 3. Untangle Requirement Traversal, Expression Evaluation, And Evidence
|
||||
|
||||
**Code:** [requirement walker](../../../internal/prompt/requirements.go:356),
|
||||
[syntax dispatch](../../../internal/prompt/requirements.go:451),
|
||||
[condition evaluation](../../../internal/prompt/requirements.go:93), and
|
||||
[command traversal](../../../internal/prompt/requirements.go:652).
|
||||
|
||||
**Complexity:** The approximately thousand-line file contains mutually recursive
|
||||
syntax walking and partial expression evaluation, branch reachability, caller
|
||||
scope tracking, template-call handling, evidence promotion, issue construction,
|
||||
and result sorting. `if` and `with` repeat branch orchestration. `input`, `index`,
|
||||
root variables, and skipped-path behavior appear in both evaluation and
|
||||
conservative traversal. Correctness depends on the interaction of return-value
|
||||
flags and mutable walker state such as `rootKnown`, `dependencies`, and the
|
||||
unknown-effect flags, not just the local switch case.
|
||||
|
||||
**Opportunity:** Separate the existing roles into small internal units: syntax
|
||||
recognition/evaluation, branch/scope traversal, and evidence/result accumulation.
|
||||
Factor shared `if`/`with` orchestration while making their different body-dot
|
||||
rules explicit. Reuse recognition of supported command shapes instead of
|
||||
duplicating it between evaluative and inventory-only paths. Make existing
|
||||
template scope entry/exit state handling explicit. Keep a small enum/value model
|
||||
rather than adding a general expression interpreter or pluggable AST framework.
|
||||
|
||||
**Preserve:** The deliberately conservative recognition scope, skipped-path
|
||||
inventory, optional `index` behavior, named-template limitations, incomplete
|
||||
results, and cancellation semantics. Traversal and evaluation are not
|
||||
interchangeable: inactive paths must retain recognizable names without
|
||||
evaluating caller-derived names. Any shared helper must express that distinction.
|
||||
The evidence-union efficiency finding in [audit 02](02-efficiency.md) is related
|
||||
but separate; a mechanical file split would not fix it.
|
||||
|
||||
## 4. Clarify Ownership And Policies Across The JSON Value Walkers
|
||||
|
||||
**Code:** [source mapping conversion](../../../provider_values.go:31),
|
||||
[bounded JSON copying](../../../internal/jsonvalue/jsonvalue.go:67), and
|
||||
[public-value copying](../../../convert.go:406).
|
||||
|
||||
**Complexity:** Three recursive reflection-based paths independently handle
|
||||
interfaces/pointers, maps, sequences, scalars, and nil values. Each adds its own
|
||||
policy: source conversion translates into ConfigKit-compatible containers and
|
||||
defers unsupported-value errors; `jsonvalue` validates and bounds owned copies;
|
||||
facade copying detaches already-valid values while preserving their types.
|
||||
Similar-looking switches conceal different assumptions and failure behavior.
|
||||
The numeric conversion defect in correctness-audit finding 1 demonstrates a
|
||||
concrete risk at one of these boundaries, but is not a new finding here.
|
||||
|
||||
**Opportunity:** First make the three responsibilities explicit at their call
|
||||
boundaries. Isolate source-number encoding from structural traversal, and
|
||||
identify reusable scalar/indirection/container-copy primitives only where their
|
||||
semantics truly match. Prefer removing a redundant traversal path or reusing an
|
||||
existing trusted-value copy operation over maintaining another policy-rich
|
||||
reflection switch. Keep ConfigKit-specific mapping encoding in its adapter;
|
||||
do not teach the source-neutral domain about ConfigKit.
|
||||
|
||||
**Preserve:** Supported concrete types, nil behavior, cycle/depth/work handling,
|
||||
caller ownership, and deferred source errors. In particular, passing source
|
||||
builders through rejecting `CopyMap` would introduce validation before the
|
||||
ConfigKit handoff and is not an acceptable consolidation. Nor should an
|
||||
infallible facade projection acquire new rejection behavior merely to reuse a
|
||||
validator. A universal callback-configured reflection framework could increase
|
||||
complexity; retain separate small adapters where their policies differ.
|
||||
|
||||
## 5. Extract Attempt Accounting From Generation Control Flow
|
||||
|
||||
**Code:** [Coordinator.generate](../../../internal/generation/coordinator.go:98),
|
||||
[generateInjected](../../../internal/generation/coordinator.go:203),
|
||||
[execute](../../../internal/generation/coordinator.go:303), and
|
||||
[diagnostic evidence consumers](../../../internal/usecase/smoke_tester.go:246).
|
||||
|
||||
**Complexity:** The coordinator interleaves permit acquisition, dispatch,
|
||||
cancellation, retry decisions, observer events, diagnostic evidence, and
|
||||
presence-aware usage accumulation. The built-in and injected paths repeat
|
||||
generation-number setup, evidence finishing, attempt projection, and usage
|
||||
projection. The injected path also repeats its non-dispatched cancellation
|
||||
reporting. `execute` returns five parallel values, increasing the amount of
|
||||
state the caller must keep aligned while constructing two reporting forms.
|
||||
|
||||
**Opportunity:** Return one small internal attempt-result value containing
|
||||
outcome and timing/dispatch facts, and extract common projection/accounting
|
||||
operations where evidence is genuinely shared. Keep the retry loop primarily
|
||||
about eligibility, budget, waits, and terminal outcomes. Keep injected and
|
||||
built-in dispatch strategies explicit rather than manufacturing HTTP facts for
|
||||
injected clients or introducing an extensible retry framework.
|
||||
|
||||
**Preserve:** Release active permits before callbacks/backoff, one-shot injected
|
||||
dispatch, frozen built-in requests, usage-presence distinctions, private
|
||||
diagnostic evidence, ordinary observer boundaries, and retention of decoded
|
||||
facts before cancellation discards ordinary results. The HTTP adapter should
|
||||
continue owning transport facts; accounting extraction must not push provider
|
||||
protocol knowledge into the runner or observation package.
|
||||
|
||||
## Prioritization And Non-Findings
|
||||
|
||||
Start with the schema interface/compilation consolidation and configuration
|
||||
helpers: their responsibilities provide comparatively clear refactoring seams.
|
||||
The walker and generation coordinator need narrower, behavior-preserving units
|
||||
because their state transitions are more subtle. Coordinate source conversion
|
||||
cleanup with the independently required number-preservation fix.
|
||||
|
||||
High scores alone did not justify findings for explicit public error-category
|
||||
switches, presence-aware source-field encoding, regular-file/schema read loops,
|
||||
or table-driven tests. Those branches often directly express selected contracts.
|
||||
Large facade files also include substantial GoDoc and straightforward value
|
||||
projection, which is not evidence that public/internal separation has failed.
|
||||
Keep useful test scenarios; do not rewrite tests solely to reduce their metrics.
|
||||
|
||||
No additional ordinary correctness defect was established in this pass.
|
||||
[Audit 01](01-correctness.md) and [audit 02](02-efficiency.md) retain their own
|
||||
findings. No implementation changes or additional consumer guarantees are
|
||||
implied by this refactoring catalog.
|
||||
215
docs/roadmap/audit/05-resources.md
Normal file
215
docs/roadmap/audit/05-resources.md
Normal file
@@ -0,0 +1,215 @@
|
||||
# Resource Management Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `fb87fab`.
|
||||
|
||||
## Scope And Evidence
|
||||
|
||||
Reviewed file and response-body ownership, bounded reads, HTTP client/transport
|
||||
lifetimes, timeout contexts and retry timers, backend admission and permits,
|
||||
prepared-handle cleanup, retained snapshots, JSON conversion, rendering buffers,
|
||||
schema resources, and repair feedback. The architecture uses immutable
|
||||
engine-lifetime configuration, invocation-local execution/validation resources,
|
||||
and caller-owned public results. Production code launches no PromptKit-owned
|
||||
goroutines or background workers requiring a shutdown protocol.
|
||||
|
||||
Evidence consists of source/call-graph review, relevant existing tests, and a
|
||||
small temporary offline diagnostic outside the repository. The diagnostic used
|
||||
a controlled HTTP transport, counted body reads/closure, inspected the existing
|
||||
prepared-handle owner reference, and compared string backing addresses. No
|
||||
benchmark, heap-size measurement, deliberate exhaustion, live network call,
|
||||
production change, or new repository test was needed.
|
||||
|
||||
Focused existing tests passed for content-file closure, HTTP error-body limits
|
||||
and closure, independent attempt contexts/body closure, read cancellation,
|
||||
prepared discard/ownership, diagnostic formatting, synchronous schema
|
||||
cancellation, and repeated-value JSON work bounds. These tests establish the
|
||||
existing protected behavior; they do not cover all findings below.
|
||||
|
||||
## 1. Authentication Diagnostics Read An Unbounded Upstream Body
|
||||
|
||||
**Priority:** P2. Unbounded remote-response consumption; a finite limit would be
|
||||
a new policy decision, not enforcement of an existing documented byte limit.
|
||||
|
||||
**Code:** [runOpenRouterKey](../../../internal/usecase/smoke_authentication.go:66),
|
||||
[recognizedOpenRouterKeyResponse](../../../internal/usecase/smoke_authentication.go:139),
|
||||
and [diagnostic HTTP execution](../../../internal/llm/http_evidence.go:129).
|
||||
|
||||
**Circumstances:** An authentication endpoint returns HTTP 200 with a large or
|
||||
continuously growing body. Recognition calls unrestricted `io.ReadAll` and then
|
||||
unmarshals the whole document into raw-message maps. Neither declared length nor
|
||||
streamed bytes are bounded on this path. Generation's 16 MiB success-body limit
|
||||
and 64 KiB error-body limit do not apply to diagnostic GETs.
|
||||
|
||||
**Consequences / evidence:** The offline diagnostic supplied a valid small
|
||||
identifying `data` object plus 4 MiB of irrelevant padding. The probe read all
|
||||
4,194,375 bytes, closed the body, and reported credential acceptance. Increasing
|
||||
the padding has no corresponding byte-budget check. A fast misconfigured or
|
||||
unexpected endpoint can consume substantial memory before its timeout; concurrent
|
||||
authentication probes multiply consumption because HTTP-only diagnostics bypass
|
||||
generation admission. A timeout bounds elapsed time, not bytes allocated within
|
||||
that time. There is no demonstrated descriptor leak: the body is closed.
|
||||
|
||||
**Potential remediation:** Select a finite body budget appropriate to this small
|
||||
diagnostic protocol, enforce it for both declared and streamed/underreported
|
||||
lengths, and close without unbounded draining on overflow. Preserve single-JSON
|
||||
framing and the distinction between unknown acceptance and observed rejection;
|
||||
an oversized response must not be reported as credential rejection. Decide the
|
||||
budget and overflow report classification explicitly, then update the owning
|
||||
contract and tests. This need not impose new limits on application prompts,
|
||||
artifacts, or generated output.
|
||||
|
||||
## 2. Convenience Source Conversion Expands Shared Graphs Before Bounded Loading
|
||||
|
||||
**Priority:** P2. Potentially exponential allocation before upstream resource
|
||||
checks can run.
|
||||
|
||||
**Code:** [providerMapForMapping](../../../provider_values.go:19),
|
||||
[providerValueForMapping](../../../provider_values.go:31),
|
||||
[ProfileSource](../../../profiles.go:33), and
|
||||
[BackendSource](../../../backends.go:100).
|
||||
|
||||
**Circumstances:** A caller builds a small acyclic Go value graph containing
|
||||
repeated map/slice references. Each level can point twice to the same child:
|
||||
|
||||
```go
|
||||
value := map[string]any{"value": true}
|
||||
for range depth {
|
||||
value = map[string]any{"left": value, "right": value}
|
||||
}
|
||||
```
|
||||
|
||||
The source converter tracks only active ancestors to identify cycles. Once one
|
||||
branch finishes, its visit is removed, so the sibling independently converts
|
||||
the same entire child. It allocates a fresh expanded tree before calling
|
||||
`configkit.Mapping`. For d levels, O(d) distinct input maps become
|
||||
Theta(2^d) output maps. The depth guard does not constrain width or total work;
|
||||
this expansion can become impractical well below depth 100.
|
||||
|
||||
**Consequences:** Programmatic source creation can exhaust memory before
|
||||
`NewEngine` or ConfigKit's deferred diagnostics are reached. This is not a claim
|
||||
that ConfigKit lacks bounds: the selected ConfigKit v1.1.0's mapping builder
|
||||
checks node, byte, and depth budgets during its own snapshot traversal
|
||||
(`internal/engine/value.go`, `valueBuilder.build`). It is called only after
|
||||
PromptKit has already expanded the graph. PromptKit's later `jsonvalue` node
|
||||
budget is also too late to protect this conversion.
|
||||
|
||||
**Potential remediation:** Memoize completed structural conversions so repeated
|
||||
input containers reuse already-converted containers until ConfigKit snapshots
|
||||
them with its existing bounds. Keep active-ancestor tracking distinct from the
|
||||
completed memo so cycles still become deferred source errors. Account correctly
|
||||
for container identity, including distinct slice views. Alternatively, seek
|
||||
upstream support for the additional Go value forms so this conversion can be
|
||||
removed. Preserve immediate source ownership and deferred failure behavior;
|
||||
do not introduce a PromptKit raw-source validation pass or duplicate ConfigKit's
|
||||
configuration limits to fix this pre-handoff allocation problem.
|
||||
|
||||
## 3. Inert Prepared Handles Retain Their Entire Execution Owner
|
||||
|
||||
**Priority:** P3. Avoidable retained-memory dependency after use/discard.
|
||||
|
||||
**Code:** [PreparedExecution.owner](../../../internal/usecase/prepared_execution.go:24),
|
||||
[Discard](../../../internal/usecase/prepared_execution.go:96),
|
||||
[claim](../../../internal/usecase/prepared_execution.go:172), and
|
||||
[payload clearing](../../../internal/usecase/prepared_execution.go:188).
|
||||
|
||||
**Circumstances:** A consumer retains a completed or discarded handle for its
|
||||
supported `Details` access, while replacing or dropping the originating engine.
|
||||
Payload clearing releases the per-execution plan, credential, observer, and
|
||||
execution snapshot, but the handle's `owner *Runner` remains unchanged.
|
||||
|
||||
**Consequences / evidence:** The reference path is
|
||||
`handle -> Runner -> captured repositories/clients/validator/capacity manager`.
|
||||
Thus a small inert handle keeps the originating engine's complete runtime
|
||||
snapshot and collaborators reachable, not just its deliberately retained details.
|
||||
The offline diagnostic confirmed the same nonnil runner pointer after both
|
||||
discard and successful execution. Retaining handles from successive engine
|
||||
reconstructions can retain successive large snapshots. Keeping details itself
|
||||
is required behavior, and this has no additional owner-retention cost while the
|
||||
application still retains the same engine anyway.
|
||||
|
||||
**Potential remediation:** Clear the owner reference when the handle becomes
|
||||
inert, checking ownership under the same mutex to avoid racing the current
|
||||
unlocked owner read. Another option is a small standalone identity token shared
|
||||
by a runner and its handles, which establishes ownership without retaining the
|
||||
runner graph. Such a token must have reliable distinct identity; do not rely on
|
||||
distinct zero-sized allocations comparing unequal. Preserve related namespace
|
||||
views, foreign-owner rejection, single claim, discard/run races, and details
|
||||
availability. No engine `Close` method or automatic closing of caller-owned
|
||||
collaborators is implied.
|
||||
|
||||
## 4. Repair Feedback's Output Bound Does Not Bound Its Temporary Serialization
|
||||
|
||||
**Priority:** P3. Avoidable memory amplification on validation failures.
|
||||
|
||||
**Code:** [formatRepairDiagnostics](../../../internal/usecase/repairer.go:119).
|
||||
|
||||
**Circumstances:** A failed schema validation produces a large diagnostic or many
|
||||
diagnostics and repair is enabled. The formatter normalizes every string, marshals
|
||||
the complete array, then discovers that it exceeds 64 KiB. It subsequently
|
||||
marshals individual full entries again before truncating one of them.
|
||||
|
||||
**Consequences:** Although the returned feedback is bounded correctly, temporary
|
||||
storage scales with the complete diagnostic text, including text omitted from
|
||||
the request. JSON escaping can amplify its size, and normalization adds another
|
||||
copy for invalid UTF-8. Large validation messages and concurrent repairs can
|
||||
therefore create large additional buffers despite producing tiny feedback.
|
||||
The complete diagnostics already exist for the validation result; these extra
|
||||
full encodings do not help preserve them. Existing formatting tests check output
|
||||
bounds/data preservation, not scratch-space use.
|
||||
|
||||
**Potential remediation:** Encode feedback incrementally with bounded buffering
|
||||
and lookahead sufficient to distinguish a complete array from an omitted suffix.
|
||||
Stop encoding omitted text rather than allocating its full JSON representation.
|
||||
Keep the existing 64 KiB result bound, ordered entries, rune-safe truncation,
|
||||
valid UTF-8, JSON escaping, and in-bound omission notice. Leave the original
|
||||
validation diagnostics and full previous candidate unchanged. This is an
|
||||
implementation opportunity, not a claim that the existing contract promises a
|
||||
64 KiB peak heap footprint or permission to add a diagnostic-input limit.
|
||||
|
||||
## 5. Short Normalized Session IDs Can Pin Large Input Strings
|
||||
|
||||
**Priority:** P3. Avoidable backing-buffer retention in retained metadata.
|
||||
|
||||
**Code:** [NormalizeSessionID](../../../internal/domain/session.go:10),
|
||||
[preparation](../../../internal/usecase/runner.go:312), and
|
||||
[prepared metadata construction](../../../internal/usecase/runner.go:491).
|
||||
|
||||
**Circumstances:** A direct or rendered session ID contains substantial leading
|
||||
or trailing whitespace around a short valid identifier. `strings.TrimSpace`
|
||||
returns a substring; normalization validates that substring and returns it
|
||||
without detaching its backing storage. Public conversion preserves the string.
|
||||
|
||||
**Consequences / evidence:** An offline public-API probe prepared a request with
|
||||
1 MiB of spaces followed by `s`. The returned session ID had length one but its
|
||||
backing address still pointed inside the large original allocation. Retaining a
|
||||
prepared value, handle details, or run result can therefore retain that buffer
|
||||
after the original request/rendering buffer would otherwise be collectible.
|
||||
The 256-code-point semantic limit does not prevent this retention. This does not
|
||||
require changing which padded identifiers are accepted.
|
||||
|
||||
**Potential remediation:** Detach accepted normalized session metadata with a
|
||||
small owned string, for example `strings.Clone` after validation. Preserve empty
|
||||
omission, Unicode trimming/counting, hashes, and wire content. Review other
|
||||
short canonical metadata returned by trimming for the same pattern when making
|
||||
the change; do not indiscriminately clone large prompt/message payloads or add a
|
||||
new raw-session-size restriction.
|
||||
|
||||
## Cleanup Checks And Non-Findings
|
||||
|
||||
No additional descriptor, timer, connection-ownership, or goroutine leak was
|
||||
established. Artifact/schema handles use deferred closure; prompt content handles
|
||||
close after reading; generation response bodies close on success and decoding,
|
||||
status, and size failures. Diagnostic bodies close, and their timeout cancellation
|
||||
is attached to body closure. Retry timers stop on exit. Admission/active permits
|
||||
release on errors, cancellation, and panic unwinding. Prepared payloads otherwise
|
||||
clear as intended.
|
||||
|
||||
Caller-selected artifact/schema/prompt sizes, intentionally unlimited backends,
|
||||
and caller-retained ready handles are not classified as bugs merely because
|
||||
their total memory can grow. Synchronous dependency calls may delay cancellation
|
||||
by design, rather than being detached into leaking goroutines. Borrowed HTTP
|
||||
transports/filesystems/clients remain subject to their ownership contracts; this
|
||||
audit proposes no blanket shutdown or disposal API.
|
||||
|
||||
Previously recorded repeated-copying issues remain in
|
||||
[the efficiency audit](02-efficiency.md); they are not duplicated here.
|
||||
197
docs/roadmap/audit/06-error-handling.md
Normal file
197
docs/roadmap/audit/06-error-handling.md
Normal file
@@ -0,0 +1,197 @@
|
||||
# Error-Handling Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `6b1323d`.
|
||||
|
||||
## Scope And Evidence
|
||||
|
||||
Reviewed error detection, wrapping, public translation, diagnostic reporting,
|
||||
cancellation and timeout handling, retries, validation/repair failures, and
|
||||
cleanup across construction, configuration inspection, prompt/profile selection,
|
||||
artifact reading, rendering, ordinary and prepared execution, generation, and
|
||||
smoke diagnostics, plus the small release-entrypoint wrappers. Contracts were
|
||||
checked against GoDoc, integration references, the architecture policy, ADR 0001,
|
||||
and existing tests.
|
||||
|
||||
Configuration composition belongs to ConfigKit; PromptKit validates and
|
||||
interprets its final handoff and preserves upstream load errors. The runner adds
|
||||
operation context, generation coordinates attempts and status retries, the HTTP
|
||||
adapter classifies/redacts provider failures, and the public facade translates
|
||||
internal categories. Content-validation failures and concluded diagnostic
|
||||
failures are generally data rather than Go errors under their respective
|
||||
contracts. Prepared handles permit one execution claim; failure after a
|
||||
successful claim does not make the handle reusable.
|
||||
|
||||
Evidence includes source/call-graph review and a temporary public-API diagnostic
|
||||
outside the repository. It used injected collaborators and a controlled HTTP
|
||||
transport; it made no live network calls. No production code or repository
|
||||
tests were changed.
|
||||
|
||||
Existing error, failure, cancellation, timeout, panic-cleanup, and retry tests
|
||||
passed using:
|
||||
|
||||
```sh
|
||||
go test -count=1 -run 'Error|Failure|Fail|Cancel|Timeout|Panic|Release|Retry|Retries' \
|
||||
. ./internal/llm ./internal/generation ./internal/usecase \
|
||||
./internal/capacity ./internal/validate ./internal/artifact \
|
||||
./internal/configuration
|
||||
```
|
||||
|
||||
These tests establish protected behavior but do not cover the three reproductions
|
||||
below. Findings from the earlier correctness and resource-management audits are
|
||||
not repeated here.
|
||||
|
||||
## 1. Nested Collaborator Errors Can Hide The Owning Operation's Public Category
|
||||
|
||||
**Priority:** P2. Confirmed public error-classification defect.
|
||||
|
||||
**Code:** [publicErrorFor](../../../errors.go:63),
|
||||
[mapPublicError](../../../errors.go:16), and
|
||||
[wrapGenerationError](../../../internal/usecase/runner.go:271).
|
||||
|
||||
**Failure condition:** An injected `LLMClient` or `ArtifactReader` returns an
|
||||
error obtained from another PromptKit operation. Ordinary mapped errors retain
|
||||
their internal chain. `publicErrorFor` searches that complete chain with
|
||||
`errors.Is` and chooses the first matching category in a fixed priority order,
|
||||
regardless of which operation the outer runner classified.
|
||||
|
||||
**Expected:** A model-client failure matches `ErrLLMGenerate`; an artifact-reader
|
||||
failure matches `ErrArtifactLoad`. The injected cause remains available through
|
||||
`errors.Is`, as their [public GoDoc](../../../engine.go:49) specifies. Preserving
|
||||
the nested cause may also preserve its public categories; it must not replace
|
||||
the category for the operation that just failed.
|
||||
|
||||
**Actual / reproduction:** A child engine's `Run` returns a missing-prompt error.
|
||||
A parent engine selects a valid prompt, then its injected client returns that
|
||||
child error. The outer error contains the text `failed to generate output`, but:
|
||||
|
||||
```text
|
||||
errors.Is(err, ErrLLMGenerate) = false
|
||||
errors.Is(err, ErrPromptNotFound) = true
|
||||
errors.Is(err, childError) = true
|
||||
```
|
||||
|
||||
Returning the same child error from an injected artifact reader similarly
|
||||
produces `errors.Is(err, ErrArtifactLoad) == false`. This is reproducible using
|
||||
only public APIs, without importing internal sentinels. Consumers can therefore
|
||||
miss the documented failure category when composing PromptKit-backed adapters.
|
||||
|
||||
**Potential remediation:** Translate the owning operation's classification
|
||||
independently of an arbitrary collaborator's nested chain, while retaining the
|
||||
original injected cause. Explicit boundary/stage information or mapping at the
|
||||
operation boundary can do this. Merely reordering the global switch risks
|
||||
breaking intentional direct-selection behavior: a genuinely missing prompt or
|
||||
profile must retain its documented distinction from other load failures.
|
||||
Protect nested-client and nested-reader cases through public-boundary regression
|
||||
coverage; preserve existing capacity, retry-exhaustion, provider-error redaction,
|
||||
and context identities.
|
||||
|
||||
## 2. Successful-Status Body Read Failures Are Misreported As Malformed JSON
|
||||
|
||||
**Priority:** P2. Confirmed loss of failure provenance and cause.
|
||||
|
||||
**Code:** [decodeOpenAIChatResponse](../../../internal/llm/openai_compatible_client.go:373).
|
||||
|
||||
**Failure condition:** A 2xx response body returns a non-EOF I/O error other than
|
||||
`context.Canceled` or `context.DeadlineExceeded`, either during the first decode
|
||||
or while checking the required final EOF. For example, a connection can fail
|
||||
while reading the response stream after headers were received.
|
||||
|
||||
**Expected:** Distinguish an underlying read/transport failure from evidence of
|
||||
invalid JSON or trailing data. Retain the read cause through a redacting error
|
||||
so callers can inspect it with `errors.Is`/`errors.As`, consistent with the
|
||||
[transport category](../../../engine.go:65). Do not report trailing data when the
|
||||
reader failed without supplying any. This does not change the established
|
||||
classification of invalid/truncated JSON or introduce transport retries.
|
||||
|
||||
**Actual / reproduction:** The controlled transport returned HTTP 200 and a body
|
||||
that supplied either a partial JSON prefix or a complete valid chat envelope,
|
||||
then returned a distinct read-error sentinel. Both executions returned
|
||||
`ErrLLMGenerate` plus `ErrMalformedResponse`, did not match `ErrTransport`, and
|
||||
discarded the sentinel. For the complete envelope, the error said `response
|
||||
contains trailing data`, although no trailing bytes were supplied. Inference
|
||||
smoke diagnostics likewise reported `malformed_response` with nil Go error
|
||||
rather than a transport failure. Thus the issue affects both public execution
|
||||
errors and factual diagnostic classification.
|
||||
|
||||
The integration contract explicitly guarantees exact causes for
|
||||
[`http.Client.Do` failures](../../integrations/openai-compatible-chat.md#response-handling),
|
||||
not every decoder error. The finding concerns the separate loss and
|
||||
misclassification of an actual body-read failure, rather than asserting a new
|
||||
requirement to expose JSON parser errors or provider content.
|
||||
|
||||
**Potential remediation:** Track underlying reader failures separately from
|
||||
decoder/framing failures, or reliably distinguish those classes before replacing
|
||||
the decode error. Reuse the redacting transport-error path for genuine read
|
||||
failures. Keep syntax/type errors, EOF-truncated JSON, extra JSON values, and
|
||||
size overflow in the existing malformed-response category; preserve cancellation
|
||||
precedence and bounded reads/closure. Add focused coverage for interrupted
|
||||
initial decoding and interrupted EOF verification, including diagnostic failure
|
||||
classification. Default formatting must remain free of raw response content and
|
||||
private transport details.
|
||||
|
||||
## 3. Inference Diagnostics Swallow Some Invalid Request-Construction Errors
|
||||
|
||||
**Priority:** P2. Confirmed mismatch with the diagnostic error contract.
|
||||
|
||||
**Code:** [runInference](../../../internal/usecase/smoke_tester.go:207),
|
||||
[generation preparation](../../../internal/generation/coordinator.go:96), and
|
||||
[request payload validation](../../../internal/llm/openai_compatible_client.go:511).
|
||||
|
||||
**Failure condition:** An inference diagnostic's effective settings are invalid
|
||||
in a way detected by built-in request construction rather than target selection.
|
||||
A runtime `Execution.ExtraParams` override containing the reserved key `model`
|
||||
is one public example. The coordinator returns `llm.ErrInvalidRequest` before
|
||||
dispatch with no attempt evidence. `runInference` treats every generation error
|
||||
as a concluded diagnostic failure.
|
||||
|
||||
**Expected:** The [SmokeTest contract](../../../smoke_diagnostics.go:238) says invalid
|
||||
kinds, settings, and selections return an ordinary Go error before diagnostic
|
||||
work. A locally invalid request should follow that path and expose the public
|
||||
invalid-request category. This is distinct from a provider rejecting a validly
|
||||
constructed request or a diagnostic concluding with an observed failure.
|
||||
|
||||
**Actual / reproduction:** Calling text-inference `SmokeTest` with
|
||||
`Execution.ExtraParams: map[string]any{"model": "override"}` returned nil Go
|
||||
error and a report with `Completed == true`, `Failure == invalid_request`, no
|
||||
attempts, and zero transport calls. The request-construction error and its
|
||||
explanation were discarded. The caller cannot use the documented ordinary-error
|
||||
path to identify this configuration mistake.
|
||||
|
||||
**Potential remediation:** Preserve the distinction between local built-in
|
||||
request-preparation failures and concluded execution failures. Return preparation
|
||||
errors through the use-case/public invalid-request mapping rather than marking
|
||||
the diagnostic complete; preparation can remain in the generation adapter, with
|
||||
enough provenance returned to the caller to make that distinction. Do not infer
|
||||
that every zero-attempt diagnostic is an error: missing required credentials,
|
||||
capacity rejection, unsupported operations, and inapplicability have intentional
|
||||
report semantics. Preserve concluded provider failures and the separate
|
||||
caller-cancellation behavior. Add one public-boundary regression for the
|
||||
reserved-field override, retaining existing tests for those intentional reports.
|
||||
|
||||
## Reviewed Behavior Not Raised As Defects
|
||||
|
||||
- ConfigKit owns raw-source composition and errors before the accepted handoff;
|
||||
no additional PromptKit raw-source validation is proposed.
|
||||
- Non-2xx provider detail parsing is deliberately best-effort. Unreadable,
|
||||
oversized, or unrecognized error bodies retain the received status without
|
||||
exposing the raw body. That differs intentionally from decoding a 2xx body.
|
||||
- Retries are limited to selected HTTP statuses. Transport failures, malformed
|
||||
responses, and arbitrary injected-client failures are not automatically
|
||||
retried; no expansion is needed for these findings.
|
||||
- Completed output-validation failures are returned as validation results;
|
||||
operational schema failures are errors. Repair failure propagation and
|
||||
exhaustion retain that distinction.
|
||||
- Caller cancellation, configured request timeouts, cancellation during capacity
|
||||
acquisition/backoff, synchronous schema operations, and partial diagnostic
|
||||
evidence have explicit, tested handling. Callback panics propagate while
|
||||
deferred permit/admission cleanup still runs.
|
||||
- Ignored SHA-256 writer errors cannot occur with the selected implementation.
|
||||
Best-effort closure of read-only files and HTTP bodies does not imply a
|
||||
persistence guarantee or require converting a completed read into a failure.
|
||||
- Release wrappers use fail-fast shell execution and `exec` to propagate the
|
||||
selected tool's status. Publication and recovery implementation belongs to
|
||||
ReleaseKit; no release or publication operation was run for this audit.
|
||||
|
||||
No additional state-recovery or cancellation defect was substantiated in this
|
||||
pass. The findings do not call for a new recovery subsystem, automatic replay of
|
||||
possibly completed provider work, or changes to source ownership.
|
||||
205
docs/roadmap/audit/07-security.md
Normal file
205
docs/roadmap/audit/07-security.md
Normal file
@@ -0,0 +1,205 @@
|
||||
# Security Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `cf0bfd4` and the current worktree. Pending
|
||||
relocations of earlier audit documents do not change the production baseline
|
||||
and were left untouched.
|
||||
|
||||
## Scope, Architecture, And Threat Boundary
|
||||
|
||||
PromptKit is an in-process Go library with a public facade and internal
|
||||
configuration, execution, transport, validation, and observation components.
|
||||
ConfigKit owns source loading/composition; PromptKit interprets its final
|
||||
handoff and captures immutable definitions. Execution reads caller-selected
|
||||
artifacts, renders captured templates, resolves credentials, and contacts the
|
||||
selected backend. Prepared handles privately retain one execution's settings.
|
||||
Inspection and observation expose deliberately selected metadata.
|
||||
|
||||
Reviewed configuration and request validation, filesystem/resource selection,
|
||||
template and JSON handling, credential flow, public formatting and reports,
|
||||
HTTP requests/redirect handling, provider responses, schema loading, repair,
|
||||
backend capacity, prepared ownership, and release-entrypoint wiring. The review
|
||||
included interactions with the selected ConfigKit and JSON Schema dependencies
|
||||
and an advisory scan of runtime packages.
|
||||
|
||||
The [architecture policy](../../policy/architecture.md#repository-and-consumer-boundary)
|
||||
and [consumer security guidance](../../consumers/pkg-promptkit.md#protect-files-and-generated-data)
|
||||
place application authorization, untrusted path/endpoint admission, payload-size
|
||||
policy, and output disclosure with the consumer. Configuration sources,
|
||||
templates, and injected Go collaborators are application-selected capabilities.
|
||||
Upstream HTTP response bytes remain an external trust boundary. This review
|
||||
does not treat ordinary file selection, runtime endpoint overrides, namespace
|
||||
selection, or supported message roles as authorization mechanisms.
|
||||
|
||||
No production code or repository tests were changed, no secrets were collected,
|
||||
and no external provider or forge was probed. Network activity was limited to
|
||||
obtaining the public Go vulnerability checker/database and reading official
|
||||
advisories. This is a source review with focused diagnostics, not a claim that
|
||||
every possible consumer deployment is secure.
|
||||
|
||||
## Evidence
|
||||
|
||||
- Source/call-graph review and the applicable public GoDoc, framework formats,
|
||||
integration contract, policies, and ADR 0001.
|
||||
- Existing focused tests passed for credential/report formatting, sensitive
|
||||
configuration inspection/diffs, schema reference containment, provider-body
|
||||
bounds, JSON value cycles/depth, prepared ownership, and capacity boundaries.
|
||||
- A temporary offline public-API diagnostic used a controlled HTTP transport
|
||||
and a synthetic credential. It verified the authentication-body behavior in
|
||||
finding 2 and confirmed that credential was absent from the serialized report.
|
||||
- `govulncheck` v1.9.0 scanned `./...` using Go 1.27.0 and the Go vulnerability
|
||||
database last modified `2026-11-02T22:01:43Z`. The runtime scan covered the
|
||||
module, its examples, ConfigKit v1.1.0, jsonschema/v6 v6.0.2, YAML v4.0.0-rc.6,
|
||||
and x/text v0.14.0. It is not a complete audit of ReleaseKit's implementation
|
||||
or every dependency used only by the release tool.
|
||||
|
||||
The text-mode scan reported ten symbol-level standard-library advisories, plus
|
||||
package/module-only matches. Those are candidate matches, not ten established
|
||||
PromptKit vulnerabilities. Applicability was checked against the actual
|
||||
operations described below rather than inferred from interface-call traces.
|
||||
|
||||
## 1. The Audited Go Toolchain Predates Applicable Networking Security Fixes
|
||||
|
||||
**Priority:** P2. Build/deployment security finding, not a newly discovered
|
||||
PromptKit algorithm defect.
|
||||
|
||||
**Relevant code:** [declared Go minimum](../../../go.mod:3),
|
||||
[default HTTP-client construction](../../../internal/llm/openai_compatible_client.go:58),
|
||||
[generation dispatch](../../../internal/llm/openai_compatible_client.go:211), and
|
||||
[diagnostic dispatch](../../../internal/llm/http_evidence.go:129).
|
||||
|
||||
**Condition and impact:** The local toolchain used for this audit is Go 1.27.0.
|
||||
Applications built with that patch can exercise affected standard-library
|
||||
networking through PromptKit's normal outbound client:
|
||||
|
||||
- [GO-2026-6611](https://pkg.go.dev/vuln/GO-2026-6611) describes excessive CPU
|
||||
consumption caused by a malicious HTTP/2 peer repeatedly changing initial
|
||||
stream windows. An application using HTTP/2 against a hostile or compromised
|
||||
backend, with enough active streams for amplification, can suffer availability
|
||||
loss. The Go 1.27 fix is in 1.27.2. Body-size limits do not address protocol
|
||||
processing below response decoding.
|
||||
- [GO-2026-6809](https://pkg.go.dev/vuln/GO-2026-6809) adds explicit rejection of
|
||||
compromised intermediate certificates. The affected conditions depend on the
|
||||
platform and trust-store state. When the old trust path remains accepted, an
|
||||
attacker able to present a matching certificate chain and intercept the
|
||||
connection could compromise confidentiality/integrity of prompts, responses,
|
||||
or bearer credentials. The Go 1.27 fix is in 1.27.3. This is not a claim that
|
||||
every default system trust store remains affected.
|
||||
|
||||
**Evidence and limits:** The advisories establish affected versions and attack
|
||||
conditions; source review establishes use of the standard outbound HTTP/TLS
|
||||
path. No certificate forgery, HTTP/2 attack, or running consumer binary was
|
||||
tested. The scanner's surprising interface-based paths from unrelated methods
|
||||
are not relied upon as proof of exploitation. PromptKit is source-only: a
|
||||
consumer's actual compiled toolchain and transport configuration determine its
|
||||
exposure. The `go 1.27.0` directive is a compatibility minimum, not a record of
|
||||
which patch built an application.
|
||||
|
||||
**Potential remediation:** Use an appropriate current patched Go toolchain,
|
||||
at least 1.27.3 within this series for these two fixes, for release validation
|
||||
and ReleaseKit execution; advise consumers to rebuild affected applications
|
||||
with patched Go. Recheck advisories against that selected toolchain. This fits
|
||||
the existing distinction between consumer minimum and maintainer toolchain in
|
||||
the [release procedure](../../release.md#prepare-the-candidate).
|
||||
Whether to raise the consumer minimum for the identified fixes is a separate
|
||||
compatibility decision under the existing policy, not an automatic consequence
|
||||
of this audit. These fixes do not require custom TLS verification, a blanket
|
||||
HTTP/2 ban, or a new recurring release gate.
|
||||
|
||||
## 2. Authentication Probes Permit Unbounded Remote-Body Memory Consumption
|
||||
|
||||
**Priority:** P2. Existing availability finding, confirmed at the external
|
||||
response boundary. This is the same mechanism as resource-management audit
|
||||
finding 1, not a second independent remediation item.
|
||||
|
||||
**Relevant code:** [authentication recognition](../../../internal/usecase/smoke_authentication.go:139),
|
||||
[authentication probe execution](../../../internal/usecase/smoke_authentication.go:66),
|
||||
and [diagnostic HTTP execution](../../../internal/llm/http_evidence.go:129).
|
||||
|
||||
**Condition:** A consumer explicitly invokes an `openrouter_key` authentication
|
||||
diagnostic. The selected endpoint, a compromised provider, or an intermediary
|
||||
on an operator-selected plaintext connection returns HTTP 200 with a large or
|
||||
continuously growing body. Recognition uses unrestricted `io.ReadAll` followed
|
||||
by whole-document JSON unmarshalling. Unlike generation response decoding,
|
||||
this path has no byte budget. HTTP-only probes also do not use generation
|
||||
capacity admission.
|
||||
|
||||
**Impact:** The remote peer controls memory consumption before the diagnostic
|
||||
returns. A fast response can cause large allocations or process termination
|
||||
before a deadline, and concurrent probes multiply the exposure. A configured
|
||||
timeout bounds time, not bytes allocated during that interval. This requires a
|
||||
diagnostic call; it is not an unauthenticated inbound attack on a PromptKit
|
||||
server, and ordinary `Run` does not execute the probe.
|
||||
|
||||
**Evidence:** The offline diagnostic returned a minimal recognized account
|
||||
object and 4 MiB of irrelevant padding. PromptKit read all 4,194,374 bytes,
|
||||
closed the body, and concluded with credential acceptance. The body used an
|
||||
unknown content length. No deliberate memory exhaustion was attempted. The
|
||||
synthetic bearer credential was supplied to the controlled transport and was
|
||||
absent from the serialized diagnostic report.
|
||||
|
||||
**Potential remediation:** Select a finite body budget for this small diagnostic
|
||||
protocol and enforce it for declared and streamed lengths, closing on overflow
|
||||
without unbounded draining. Preserve factual reporting: overflow establishes
|
||||
neither credential rejection nor acceptance. Agree the limit and overflow
|
||||
classification explicitly; no existing authentication-body byte guarantee is
|
||||
being enforced. The existing resource-audit remediation can own this fix;
|
||||
there is no need for separate global limits on prompts or artifacts.
|
||||
|
||||
## Other Candidates And Boundary Checks
|
||||
|
||||
- **Scanner applicability:** Server-only advisories GO-2026-6603, 6612, 6613,
|
||||
and 6617 do not establish an attack on PromptKit's outbound client. ECH
|
||||
server parsing (6607), multipart-form parsing (6608), and Range-serving
|
||||
behavior (6609) likewise lack a matching operation here. The CONNECT-with-body
|
||||
condition in 6605 and raw HTTP/2-to-HTTP/1 header forwarding in 6610 are absent
|
||||
from built-in POST/GET generation and factual diagnostic reporting. Windows
|
||||
junction-directory creation (6604) is not established by read-only resource
|
||||
opens. These exclusions concern PromptKit, not all possible code in a host
|
||||
application.
|
||||
- **Module-only matches:** x/text advisories GO-2026-5970 and 6629 were not
|
||||
reported as called vulnerable symbols; no matching use of `norm.Iter` or
|
||||
`precis.Nickname` was established. html/template advisories 6599 and 6600 do
|
||||
not apply to the text/template renderer. No separate runtime third-party
|
||||
vulnerability was substantiated by this scan; that is not a general safety
|
||||
certification of those modules.
|
||||
- **Files and schemas:** Ordinary artifacts intentionally allow unrestricted
|
||||
caller-selected regular files. Host-backed `content_file` contexts use
|
||||
ConfigKit's rooted filesystem, while the standard schema loader checks
|
||||
canonical containment and rejects remote reference schemes. Generic `fs.FS`
|
||||
inputs remain caller-supplied capabilities; logical path checks do not turn an
|
||||
arbitrary filesystem implementation into a host sandbox. No additional
|
||||
confinement guarantee is proposed.
|
||||
- **Injection:** Variables and artifact text are rendered as data rather than
|
||||
reparsed as Go templates. Captured templates have the input helper and normal
|
||||
text/template functionality, not a shell or process-execution helper.
|
||||
Provider payloads use JSON encoding and reserved-field checks. Model output
|
||||
is data for validation/repair, not executable code or an authorization
|
||||
decision. Prompt injection and domain-aware interpretation remain consumer
|
||||
concerns.
|
||||
- **Credentials and disclosure:** Direct keys are request-scoped, excluded from
|
||||
supported JSON and public request summaries, and removed from diagnostic
|
||||
settings and prepared details. Provider error defaults redact details;
|
||||
explicit provider accessors and arbitrary application content retain the
|
||||
documented consumer disclosure responsibility. Inspection/diff tests protect
|
||||
configured sensitive provider paths. No blanket redaction of arbitrary
|
||||
prompt, schema, variable, error, or provider content is promised.
|
||||
- **Ownership and availability:** Engine snapshots and copied public values
|
||||
prevent ordinary cross-call mutation. Prepared handles enforce owner binding
|
||||
and single claim. Capacity and retry cleanup are tested. Intentionally
|
||||
unlimited backends, synchronous template/schema dependency calls, and
|
||||
application-selected large files are not newly classified as vulnerabilities.
|
||||
The other resource-audit findings remain maintenance items rather than
|
||||
independently demonstrated remote attacks.
|
||||
- **Release tooling:** Local launchers pass quoted arguments through `exec`;
|
||||
there is no PromptKit runtime command-execution surface. The delivery token
|
||||
path is ignored and selected for committed-file protection. Publication,
|
||||
token-file permission enforcement, and recovery implementation belong to
|
||||
ReleaseKit; no publication command was run during this audit.
|
||||
|
||||
## Recommended Follow-Up
|
||||
|
||||
Use a patched Go toolchain before release preparation and address the existing
|
||||
authentication-body availability finding. Neither requires a new application
|
||||
authorization subsystem or stronger filesystem, endpoint, or semantic-output
|
||||
contracts. No additional PromptKit-specific confidentiality, integrity, or
|
||||
authorization defect was substantiated within the documented boundary.
|
||||
284
docs/roadmap/audit/08-testing.md
Normal file
284
docs/roadmap/audit/08-testing.md
Normal file
@@ -0,0 +1,284 @@
|
||||
# Test Coverage And Effectiveness Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `f7085d7` and the current worktree. Existing
|
||||
audit-document moves were preserved. No production code or tests were changed.
|
||||
|
||||
## Assessment And Method
|
||||
|
||||
The suite broadly follows the [testing policy](../../policy/testing.md): real
|
||||
ConfigKit composition, filesystems, rendering, validation, and engine assembly
|
||||
are exercised alongside controlled model clients and HTTP transports. Internal
|
||||
tests own focused rules and coordination, external-package root tests protect
|
||||
the public facade, and the three maintained examples have Go workflow tests.
|
||||
Capacity, prepared-handle state transitions, ownership, cancellation, retries,
|
||||
repair, observations, and configuration inspection have substantial behavioral
|
||||
coverage. This is a targeted improvement list, not a recommendation to replace
|
||||
the test architecture or pursue a percentage threshold.
|
||||
|
||||
Both required analyses were performed: review of missing behavioral protection,
|
||||
and review of assertion/fixture cost and coupling. Evidence includes source and
|
||||
call-graph review, existing GoDoc and integration contracts, full-suite coverage,
|
||||
repeated race-enabled execution, and reuse of the earlier error audit's temporary
|
||||
offline diagnostic. No live provider, paid API, new repository test, production
|
||||
mutation, or external mutation-testing service was needed.
|
||||
|
||||
Validation used the local Go 1.27.0 toolchain:
|
||||
|
||||
```sh
|
||||
go test -count=1 -covermode=atomic -coverpkg=./... -coverprofile=/tmp/promptkit-testing-audit-LHAyNm/coverage.out ./...
|
||||
go tool cover -func=/tmp/promptkit-testing-audit-LHAyNm/coverage.out
|
||||
go test -race -vet=off -shuffle=42 -count=2 ./...
|
||||
```
|
||||
|
||||
Both suite runs passed, including the examples. The race run exercised two
|
||||
shuffled passes; this is evidence against readily reproducible order/race
|
||||
problems, not proof that no flakes exist. Coverage output remains outside the
|
||||
repository. The commands are audit evidence, not additional release gates.
|
||||
|
||||
Combined coverage is **86.4%**, or 4,782 of 5,536 instrumented statements.
|
||||
Selected package totals below combine hits across all test binaries. The
|
||||
per-binary percentages printed by `-coverpkg=./...` are not individual-package
|
||||
coverage percentages; repeated profile spans were combined before calculating
|
||||
these totals.
|
||||
|
||||
| Component | Combined statement coverage |
|
||||
| --- | ---: |
|
||||
| Public facade | 81.0% |
|
||||
| Configuration | 91.0% |
|
||||
| Domain | 98.0% |
|
||||
| Capacity | 98.0% |
|
||||
| Generation coordination | 91.2% |
|
||||
| HTTP model client | 94.2% |
|
||||
| Execution use cases | 90.5% |
|
||||
| Rendering and requirement inspection | 78.4% |
|
||||
| Output/schema validation | 75.8% |
|
||||
|
||||
Coverage located candidates; the findings below concern specific risks and
|
||||
assertions, not the percentages themselves. Testing recommendations associated
|
||||
with already recorded implementation defects should accompany those fixes,
|
||||
rather than becoming duplicate remediation projects.
|
||||
|
||||
## 1. RunRequest's Privacy Formatting Contract Is Not Exercised
|
||||
|
||||
**Priority:** P2. Missing protection for a documented confidentiality boundary.
|
||||
|
||||
**Relevant code:** [RunRequest formatting](../../../formatting.go:5) and the
|
||||
analogous existing [SmokeTestRequest privacy test](../../../smoke_test_contract_test.go:135).
|
||||
|
||||
**Deficiency:** `RunRequest.String`, `GoString`, and their shared formatter all
|
||||
have 0% combined statement coverage. The suite protects formatting of generated
|
||||
client requests, smoke-test requests, and opaque prepared handles, but does not
|
||||
exercise the ordinary run request's own formatting contract.
|
||||
|
||||
**Unverified behavior:** A change that prints the direct key, input bodies, or
|
||||
variable values from `RunRequest` can pass the complete suite. This is an
|
||||
assertion gap, not evidence that current formatting actually leaks those values.
|
||||
|
||||
**Potential improvement:** Add one small public-boundary test with distinct
|
||||
synthetic private markers and exercise the supported String/GoString formatting
|
||||
paths on values and pointers. Assert absence of the data covered by the GoDoc;
|
||||
do not snapshot the complete summary or freeze incidental field order/count
|
||||
formatting. If adding a JSON check, assert only the documented direct-key
|
||||
exclusion, not a stable JSON shape or omission of ordinary request data.
|
||||
Reflection-based disclosure is explicitly outside this formatter guarantee.
|
||||
|
||||
## 2. The Public Single-Schema-File Option Has No Execution Coverage
|
||||
|
||||
**Priority:** P2. An exported source-selection path is entirely unprotected.
|
||||
|
||||
**Relevant code:** [WithSchemaFile](../../../engine.go:212),
|
||||
[SetSchemaFile](../../../internal/engine/engine.go:75), and
|
||||
[fileSource](../../../internal/engine/engine.go:211).
|
||||
|
||||
**Deficiency:** All three functions have 0% combined coverage. The existing
|
||||
[single-file validator test](../../../internal/validate/standard_validator_test.go:596)
|
||||
constructs `FSValidator` directly with `fstest.MapFS`; it does not test the public
|
||||
option, host-file selection, or its engine wiring.
|
||||
|
||||
**Unverified behavior:** The public option could ignore its argument, fail to
|
||||
replace `SchemaDir`, select the wrong root/base name, or mishandle invalid files
|
||||
without this validator test noticing. Direct validator success does not establish
|
||||
that a consumer can reach that behavior through the supported API.
|
||||
|
||||
**Potential improvement:** Use `t.TempDir` and a real schema file in a small
|
||||
public integration test. Exercise construction plus preparation/validation
|
||||
through that file's base name, and representative invalid selections such as a
|
||||
missing file or directory. Include source-option replacement where it protects
|
||||
the documented last-valid-option behavior. Assert public error identity and
|
||||
observable schema selection, not private helper calls. This does not require a
|
||||
helper-by-helper test matrix or model-provider access.
|
||||
|
||||
## 3. Programmatic Provider-Value Conversion Lacks Representative Value Cases
|
||||
|
||||
**Priority:** P2. Conversion and ownership coverage misses known data-integrity
|
||||
and deferred-error risks.
|
||||
|
||||
**Relevant tests:** [programmatic source ownership/order](../../../sources_contract_test.go:14)
|
||||
and [programmatic construction](../../../construction_contract_test.go:226).
|
||||
**Implementation:** [providerValueForMapping](../../../provider_values.go:31).
|
||||
|
||||
**Deficiency:** The converter has only 11.9% combined coverage. The principal
|
||||
source-ownership test mutates a flat string-valued map. It does not establish
|
||||
preservation and ownership for representative nested/typed containers or the
|
||||
`json.Number` branch, nor deferred failure for unsupported provider values.
|
||||
The more extensive `internal/jsonvalue` tests protect a different traversal and
|
||||
do not execute this pre-ConfigKit convenience conversion.
|
||||
|
||||
**Unverified behavior:** The suite passes with the exact-number rounding defect
|
||||
already recorded in [the correctness audit](01-correctness.md#1-provider-source-helpers-silently-change-exact-json-numbers).
|
||||
It also would not detect many regressions in nested source capture or the
|
||||
documented source-creation-versus-loading failure boundary.
|
||||
|
||||
**Potential improvement:** Exercise both `ProfileSource` and `BackendSource`
|
||||
through `NewEngine` with a compact set of representative supported nested
|
||||
values, mutations after source creation, and unsupported/cyclic values that
|
||||
should fail during loading. Add the numeric regression when addressing the
|
||||
existing precision finding. Compare meaningful final values or encoding, not Go
|
||||
container identity, numeric spelling, or a required float representation.
|
||||
Keep tests at the final handoff/public source-helper boundary; do not duplicate
|
||||
ConfigKit's raw-source validation or its entire value-limit suite. Work-bound
|
||||
coverage for the separately identified shared-graph issue belongs with that fix.
|
||||
|
||||
## 4. Error Tests Miss Composed Causes And Pre-Dispatch Diagnostic Failures
|
||||
|
||||
**Priority:** P2. Covered code paths lack important boundary-case assertions.
|
||||
|
||||
**Relevant tests:** [public generation failures](../../../generation_error_contract_test.go:111),
|
||||
[direct mapper cases](../../../errors_internal_test.go:13),
|
||||
[body cancellation](../../../internal/llm/prepared_generation_test.go:161), and
|
||||
[inference diagnostic cases](../../../internal/usecase/smoke_inference_test.go:14).
|
||||
|
||||
**Deficiency and demonstrated blind spots:**
|
||||
|
||||
- Injected failures are mostly plain marker errors or a single explicit
|
||||
category. They do not check a real nested PromptKit error returned through a
|
||||
different operation. The suite passes although an injected client/reader can
|
||||
lose the outer public generation/artifact category.
|
||||
- Tests cover `http.Client.Do` failures and cancellation during successful-body
|
||||
reading, but not an ordinary non-EOF read error during initial decoding or
|
||||
final EOF verification. Such errors currently lose their cause and become
|
||||
malformed-response failures.
|
||||
- Diagnostic tests cover invalid selection/settings and concluded generation
|
||||
failures separately, but miss a runtime setting rejected by built-in request
|
||||
construction before dispatch. A reserved `ExtraParams["model"]` override
|
||||
currently returns a completed report with nil Go error.
|
||||
|
||||
These are the three implementation findings in the error-handling audit, not
|
||||
new duplicate defects. Re-running its temporary offline public-API diagnostic
|
||||
against the current tree confirmed all three while the complete suite passed.
|
||||
|
||||
**Potential improvement:** Add narrowly owned regressions with those fixes:
|
||||
real nested public causes through the injection boundary; a controlled body
|
||||
reader failing at each decode/framing phase; and a public `SmokeTest` call with
|
||||
a reserved runtime field and counted transport. Assert relevant error identities,
|
||||
cause preservation, factual failure classification, and whether dispatch occurred.
|
||||
Avoid reproducing the internal error topology by hand or matching complete
|
||||
messages. Preserve intentional nil-error diagnostic findings, cancellation
|
||||
precedence, best-effort non-2xx detail parsing, and status-only retry policy.
|
||||
|
||||
## 5. Multi-Resource Schema Tests Do Not Establish A Usable Provider Constraint
|
||||
|
||||
**Priority:** P2. Missing assembled validation-to-wire assertion.
|
||||
|
||||
**Relevant tests:** [prepared execution source freezing](../../../prepared_execution_contract_test.go:19),
|
||||
its [referenced schema fixture](../../../prepared_execution_contract_test.go:809),
|
||||
and [local reference validation](../../../internal/validate/standard_validator_test.go:627).
|
||||
|
||||
**Deficiency:** Local tests establish that referenced schemas compile and
|
||||
validate. The prepared-execution test checks that the injected client's schema
|
||||
matches frozen details. Its opaque model client accepts that schema without
|
||||
interpreting references. Equality between those two values does not establish
|
||||
that the provider-facing constraint contains the resources it needs.
|
||||
|
||||
**Unverified behavior:** Both sides can contain the same unresolved `value.json`
|
||||
reference while the test passes. This is the packaging gap already identified
|
||||
in [the correctness audit](01-correctness.md#2-external-schema-resources-do-not-reach-the-provider-constraint).
|
||||
The freezing test's fake is appropriate for its stated ownership purpose; the
|
||||
missing protection is a separate built-in-path integration case.
|
||||
|
||||
**Potential improvement:** Once the schema-packaging policy is selected, add one
|
||||
offline assembled test with a root and child schema, capture the actual built-in
|
||||
HTTP payload, and assess the chosen provider-facing representation independently
|
||||
of `Details`. For a bundled native schema, compile/evaluate the captured
|
||||
constraint without granting access to PromptKit's original local files; if the
|
||||
chosen policy supports a local-only fallback, assert that behavior instead.
|
||||
Keep the existing freezing test. Do not require live provider acceptance or
|
||||
promise support for arbitrary provider schema dialect features.
|
||||
|
||||
## 6. Artifact Tests And Benchmarks Pin An Explicitly Opaque Hash Algorithm
|
||||
|
||||
**Priority:** P3. Remove or rewrite change-detector assertions, retaining the
|
||||
substantive file-reading tests.
|
||||
|
||||
**Relevant assertions:** [size-hint test](../../../internal/artifact/reader_test.go:287)
|
||||
and [benchmark correctness check](../../../internal/artifact/reader_benchmark_test.go:156).
|
||||
|
||||
**Deficiency:** Both independently calculate a hex SHA-256 digest and require
|
||||
that exact value. The [public Artifact.Hash contract](../../../types.go:290)
|
||||
expressly leaves its format and algorithm unspecified. A legitimate algorithm
|
||||
or encoding change would fail tests unrelated to the requested behavior.
|
||||
|
||||
**Potential improvement:** Replace those algorithm-specific checks with the
|
||||
opaque equality relationships needed for each test. For the under/overreported
|
||||
size cases, comparing with the inline reader for the same complete body still
|
||||
detects hashing only a reported-size prefix. The existing
|
||||
[source-parity test](../../../internal/artifact/reader_test.go:27) already owns
|
||||
repeatability, same-content equality, and changed-content inequality. Retain
|
||||
body, actual-size, metadata, and read-boundary assertions. The benchmark may
|
||||
reuse an opaque expected hash prepared outside measurement or rely on the
|
||||
owning behavioral test; it need not maintain its own SHA-256 contract.
|
||||
|
||||
## 7. Bundled Catalog Expectations Have Multiple Test Owners
|
||||
|
||||
**Priority:** P3. Consolidation/removal candidate for duplicated assertions.
|
||||
|
||||
**Relevant tests:** [internal complete catalog table](../../../internal/configuration/bundled_test.go:12),
|
||||
[public option/inventory table](../../../bundled_openrouter_contract_test.go:54),
|
||||
[HTTP payload matrix](../../../bundled_execution_contract_test.go:51), and
|
||||
[bundled smoke defaults](../../../bundled_smoke_contract_test.go:16).
|
||||
|
||||
**Deficiency:** The current count of 17, complete ID inventory, selected
|
||||
ID-to-model mappings, tier-plugin data, and diagnostic default model recur
|
||||
across configuration, facade, wire, and diagnostic tests. A catalog update can
|
||||
require expectation edits in several layers even when loading, option handling,
|
||||
routing, and serialization are unchanged. The internal plugin assertion also
|
||||
searches `fmt.Sprint` output instead of comparing the plugin's structure.
|
||||
|
||||
**Potential improvement:** Choose one owner for the complete maintained asset
|
||||
inventory and expected model/plugin/default data. Remove duplicate full-catalog
|
||||
counts/lists from option permutations and keep those tests focused on enable,
|
||||
disable, last-option selection, isolation, and override behavior. Retain a small
|
||||
assembled wire/diagnostic set for distinct integration risks; controlled
|
||||
overrides can isolate routing assertions from a changing catalog default.
|
||||
Use semantic structural assertions for plugin data. Keep independent expected
|
||||
asset values in the selected owner rather than deriving every expectation
|
||||
from the production loader, which could conceal an omitted or incorrect asset.
|
||||
This is consolidation, not a proposal to stop checking bundled configuration.
|
||||
|
||||
## Findings Not Supported And Work To Preserve
|
||||
|
||||
- No general test-flakiness, default-suite live-provider dependency, or race
|
||||
failure was reproduced. Bounded timeout guards and channel synchronization
|
||||
are not classified as flaky merely because a test involves concurrency.
|
||||
- The public API/type-boundary checks and their checker fixtures protect an
|
||||
explicit architectural rule; they are not automatically obsolete because
|
||||
they inspect structure. Stable JSON shape tests protect selected contracts.
|
||||
- Fakes at model/HTTP boundaries, counted paid-effect calls, and no-work-after-
|
||||
rejection checks are appropriate. A fake is not deficient merely because it
|
||||
does not model unrelated provider behavior.
|
||||
- Exhaustive domain-rule tables plus representative boundary wiring tests are
|
||||
generally sound. No wholesale deletion of error, cancellation, capacity,
|
||||
ownership, or prepared-lifecycle coverage is recommended.
|
||||
- Low coverage of thin wrappers, example `main` functions, defensive impossible
|
||||
branches, and unexercised reflection variants is not independently a finding.
|
||||
Interface-only packages do not need artificial tests to obtain a package
|
||||
percentage. Conservative requirement inspection is not obliged to fully
|
||||
evaluate unsupported template expressions just to cover every AST branch.
|
||||
- The standalone release-wiring fixture remains scoped to local disposable
|
||||
repositories under the documented testing model. This audit does not propose
|
||||
moving it into every normal test run or retesting ReleaseKit's algorithms.
|
||||
|
||||
Prioritize the missing public privacy/source-option protection and regressions
|
||||
for the already identified defects. Make the two assertion-cleanup changes as
|
||||
focused rewrites, retaining their useful behavioral coverage. None of these
|
||||
recommendations requires new public guarantees or a coverage-percentage gate.
|
||||
199
docs/roadmap/audit/09-interfaces.md
Normal file
199
docs/roadmap/audit/09-interfaces.md
Normal file
@@ -0,0 +1,199 @@
|
||||
# API And Interface Contract Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `31f1e6a` and the current worktree. Existing
|
||||
audit-document moves were preserved. No production code or repository tests
|
||||
were changed.
|
||||
|
||||
## Scope And Assessment
|
||||
|
||||
The public root package remains a facade over configuration, execution,
|
||||
generation, capacity, rendering, validation, and observation components.
|
||||
ConfigKit owns composition before the final configuration handoff; PromptKit
|
||||
validates that handoff and captures its runtime indexes and templates. Public
|
||||
adapters translate owned request/result values and errors. Namespace views
|
||||
share execution ownership and capacity, while separately constructed engines
|
||||
remain independent.
|
||||
|
||||
The review compared exported GoDoc, configuration and integration contracts,
|
||||
internal collaborator interfaces, call paths, and existing behavioral tests.
|
||||
It covered preparation/preflight/execution parity, prepared-handle lifecycle,
|
||||
optional values and numeric presence, configuration inspection and listing,
|
||||
injected collaborator ownership, output repair, diagnostics, observations,
|
||||
serialization, cancellation, and public error categories. It did not treat
|
||||
different intentionally documented workflows as defects or infer additional
|
||||
admission, filesystem, provider, or semantic-validation guarantees.
|
||||
|
||||
Four findings warrant remediation. The configuration behaviors in findings 2
|
||||
and 3 are consistent with the current format reference and implementation;
|
||||
their older contradictory descriptions should be corrected rather than used
|
||||
to change those behaviors.
|
||||
|
||||
## 1. ArtifactReader Permits Buffer Reuse Before PromptKit Can Copy It
|
||||
|
||||
**Priority:** P2. A documented ownership permission permits data races and
|
||||
inconsistent input snapshots.
|
||||
|
||||
**Relevant interfaces/code:** [ArtifactReader GoDoc](../../../types.go:294),
|
||||
[public reader adapter](../../../artifact_reader.go:19), and
|
||||
[copyBytes](../../../convert.go:490).
|
||||
|
||||
**Contract defect:** The interface says that `Read` may be called concurrently
|
||||
and that the reader may reuse or mutate the returned artifact and body after
|
||||
`Read` returns. The adapter can only copy those values after that return.
|
||||
There is no synchronization or acknowledgement between return and completion
|
||||
of the copy. “Immediately copies” does not make concurrent reuse safe.
|
||||
|
||||
**Circumstances and consequence:** A reader serializes its own `Read` calls
|
||||
with a mutex and returns a reusable buffer. Another invocation can overwrite
|
||||
that buffer as soon as the first call returns, while the first adapter is still
|
||||
copying it. Both calls follow the published reuse permission, yet preparation
|
||||
can race, mix contents, or associate metadata with the wrong bytes. This is
|
||||
not a demonstrated defect in the default file reader or in readers that
|
||||
return independent snapshots.
|
||||
|
||||
**Evidence:** A temporary, offline public-API diagnostic used two concurrent
|
||||
`Prepare` calls and a mutex-protected reader returning a shared 1 MiB buffer.
|
||||
`go run -race` reported a write in the next `Read` racing with `copyBytes`
|
||||
through `publicArtifactReaderAdapter.Read`. The adapter's existing copy test
|
||||
establishes detachment after copying, not safety during that copy.
|
||||
|
||||
**Potential remediation:** Resolve the return-value ownership contract before
|
||||
v1.0. The straightforward approach is for a successful `Read` to return a
|
||||
per-invocation snapshot that remains stable while PromptKit consumes it, with
|
||||
reuse permitted after the enclosing preparation/execution method returns.
|
||||
Returning fresh owned data is the simplest implementation for concurrent
|
||||
readers. If reuse immediately after `Read` itself returns must remain
|
||||
supported, it requires an explicit borrowing/release arrangement; another
|
||||
unsynchronized copy cannot provide that permission. Add focused coverage for
|
||||
the selected lifetime, without globally serializing unrelated engine work.
|
||||
|
||||
## 2. Profile Inspection Misstates Required-Credential Semantics
|
||||
|
||||
**Priority:** P2. The public inspection contract describes valid results as
|
||||
impossible and incorrectly limits the sources that can satisfy a requirement.
|
||||
|
||||
**Relevant interfaces/code:** [Engine.InspectProfile](../../../engine.go:313),
|
||||
[ProfileInspection.APIKeyRequired](../../../types.go:372),
|
||||
[profile-inspection consumer example](../../consumers/pkg-promptkit.md#inspect-a-profile-before-prompt-work),
|
||||
[effective-target construction](../../../internal/configuration/targets.go:99),
|
||||
and [credential availability check](../../../internal/usecase/runner.go:611).
|
||||
|
||||
**Contract defect:** Both method and field GoDoc claim that `APIKeyRequired`
|
||||
is mutually exclusive with a nonblank `EffectiveModelParams.APIKeyEnv`. They
|
||||
also describe the flag as requiring a direct credential or a request-level
|
||||
environment override. In fact, `api_key_required: true` suppresses a
|
||||
backend-derived environment source only when no explicit profile source was
|
||||
provided. An explicit profile environment source remains usable, as the
|
||||
[credential format reference](../../formats.md#credentials) already states.
|
||||
Inspection can therefore legitimately return both the flag and a nonblank
|
||||
environment name. The consumer example also calls every reported environment
|
||||
source optional, regardless of the flag.
|
||||
|
||||
**Circumstances and consequence:** A profile sets `api_key_required: true` and
|
||||
`api_key_env: PROMPTKIT_INTERFACE_AUDIT_KEY`. Inspection returns `true` and
|
||||
that name; preparation succeeds using its populated environment variable
|
||||
without a direct key or request override. Consumers following the GoDoc may
|
||||
reject this valid combination or report credential readiness incorrectly.
|
||||
|
||||
**Evidence:** A temporary public-API diagnostic demonstrated that combination
|
||||
and successful preparation. The existing
|
||||
[effective-target test](../../../internal/configuration/targets_test.go:88)
|
||||
also includes a required profile with an explicit environment source.
|
||||
|
||||
**Potential remediation:** Describe the flag as a local credential-availability
|
||||
requirement, distinguish source selection from required availability, and
|
||||
remove the mutual-exclusion/direct-only claims. Align the consumer explanation
|
||||
with the format reference and retain inspection's intentional lack of an
|
||||
environment-value lookup. Do not reject the valid configuration or change
|
||||
the public target's stable JSON shape to make the old wording true.
|
||||
|
||||
## 3. The Wire Contract Incorrectly Says Configured Provider Maps Replace
|
||||
|
||||
**Priority:** P2. Conflicting published contracts predict different outbound
|
||||
provider parameters.
|
||||
|
||||
**Relevant interfaces/code:** [integration request-body contract](../../integrations/openai-compatible-chat.md#request-body),
|
||||
[profile defaulting reference](../../formats.md#defaults-and-overrides),
|
||||
[overlayProviderMap](../../../internal/configuration/targets.go:133), and
|
||||
[runtime override merge](../../../internal/usecase/runner.go:562).
|
||||
|
||||
**Contract defect:** The integration document says backend, profile, and request
|
||||
extra-parameter maps all use whole-map replacement. Actual configured profile
|
||||
maps recursively overlay backend defaults, including nested maps. A nonempty
|
||||
runtime `ExecutionTargetOverride.ExtraParams` instead replaces the entire
|
||||
configured map. The format reference and runtime-override GoDoc describe
|
||||
these distinct behaviors correctly.
|
||||
|
||||
**Circumstances and consequence:** A backend supplies `backend_only: true` and
|
||||
`nested: {backend: true}`; its profile supplies `profile_only: true` and
|
||||
`nested: {profile: true}`. All entries survive into the effective target and
|
||||
built-in HTTP body. A reader of the integration contract would expect the
|
||||
backend entries to disappear, potentially leading to surprising provider
|
||||
options or ineffective attempts to clear defaults. A runtime map containing
|
||||
only `request_only: true` does remove the configured entries.
|
||||
|
||||
**Evidence:** A temporary public-API diagnostic observed both forms through an
|
||||
injected client and confirmed the configured merged fields through a controlled
|
||||
built-in HTTP transport. No external request was made. The existing
|
||||
[effective-target test](../../../internal/configuration/targets_test.go:70)
|
||||
also exercises recursive configured overlay.
|
||||
|
||||
**Potential remediation:** Replace the stale configuration-precedence claim
|
||||
with a short link to the format reference. Keep the integration contract
|
||||
focused on flattening the resulting effective map into the HTTP body and on
|
||||
reserved request fields. Preserve the distinct runtime replacement behavior;
|
||||
do not add another merge policy or duplicate its full configuration reference.
|
||||
|
||||
## 4. Identity GoDoc Still Describes Removed Normalization And Version Fields
|
||||
|
||||
**Priority:** P3. Small but concrete public-contract inconsistencies.
|
||||
|
||||
**Relevant interfaces/code:** [ProfileInspection.ProfileID](../../../types.go:364),
|
||||
[ListPrompts GoDoc](../../../listing.go:90), and
|
||||
[literal profile identity test](../../../profile_inspection_contract_test.go:13).
|
||||
|
||||
**Contract defect:** `ProfileInspection.ProfileID` is described as trimmed,
|
||||
but configured IDs and lookups preserve meaningful whitespace exactly.
|
||||
`ListPrompts` promises ownership of each `Version` pointer even though
|
||||
`PromptIdentity` contains only `ID`; collection-level `ReleaseVersion` belongs
|
||||
to the separate collection inventory.
|
||||
|
||||
**Circumstances and consequence:** Inspecting the valid profile `" padded "`
|
||||
returns that exact ID, while looking up `"padded"` fails. Callers expecting the
|
||||
documented trimmed identity can fail later lookups. The nonexistent version
|
||||
field creates an erroneous expectation about available prompt metadata.
|
||||
|
||||
**Evidence:** The existing literal-identity test explicitly protects preserved
|
||||
whitespace and failure of the trimmed lookup; a temporary public-API diagnostic
|
||||
confirmed it. The current `PromptIdentity` declaration and facade conversion
|
||||
have no version field.
|
||||
|
||||
**Potential remediation:** State that the profile ID is the exact selected
|
||||
identity after any namespace qualification, and remove the nonexistent version
|
||||
pointer language. Keep literal identity behavior and collection metadata
|
||||
unchanged; no new normalization, version field, or compatibility guarantee is
|
||||
needed.
|
||||
|
||||
## Existing Findings And Validation
|
||||
|
||||
The earlier error-handling audit already records loss of owning public error
|
||||
categories for nested injected causes, loss of successful-body I/O causes, and
|
||||
pre-dispatch smoke failures incorrectly becoming concluded reports. Those are
|
||||
also interface-contract concerns, not new duplicate findings here. The hidden
|
||||
`ValidationPreparer` requirement behind the runner's `Validator` parameter is
|
||||
already recorded in the [complexity audit](03-complexity.md). No additional
|
||||
independent defect was established in the reviewed prepared-handle lifecycle,
|
||||
optional-value handling, inspection ownership, or observation/result mapping.
|
||||
|
||||
Focused existing tests passed with the local Go 1.27.0 toolchain:
|
||||
|
||||
```sh
|
||||
go test -race -count=1 -run 'Test(InspectProfile|ListIdentities|ScopedIdentityListings|PublicArtifactReaderAdapter|PublicLLMClientAdapter|ConfigurationInspection|LoadBuildsEffectiveTargetsAfterInheritance)' . ./internal/configuration
|
||||
```
|
||||
|
||||
The separate temporary reader-reuse diagnostic intentionally produced a race
|
||||
report and exited unsuccessfully, substantiating finding 1 despite the green
|
||||
existing tests. Temporary programs and outputs remain outside the repository.
|
||||
All diagnostics were offline with synthetic configuration and credentials;
|
||||
no paid provider calls, new repository tests, or production changes were used.
|
||||
These checks are review evidence, not additional release gates.
|
||||
249
docs/roadmap/audit/10-equivalence.md
Normal file
249
docs/roadmap/audit/10-equivalence.md
Normal file
@@ -0,0 +1,249 @@
|
||||
# Behavioral Equivalence And Shared Abstraction Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `5177db0` and the current worktree. Existing
|
||||
audit-document moves were preserved. No production code or repository tests
|
||||
were changed.
|
||||
|
||||
## Assessment And Method
|
||||
|
||||
The public package adapts consumer values into internal configuration and
|
||||
execution components. ConfigKit owns source composition; PromptKit interprets
|
||||
its final handoff and captures indexes and templates. The runner shares
|
||||
preparation and execution machinery across public workflows, generation owns
|
||||
attempts and retries, and capacity owns admission and permits. Domain rules,
|
||||
validation, JSON-value ownership, and observation have existing shared owners.
|
||||
The architecture does not need another public abstraction or a general-purpose
|
||||
utility package to address the opportunities below.
|
||||
|
||||
This review started from tests: direct cross-path comparisons, common test
|
||||
matrices, and separately asserted matching behavior. It then examined the
|
||||
corresponding implementation and callers to distinguish a shared responsibility
|
||||
from similar-looking code. Matching syntax, similar coverage, or duplicated
|
||||
test setup alone did not establish an extraction opportunity.
|
||||
|
||||
Three bounded refactoring opportunities warrant consideration. They provide
|
||||
test-backed scope for findings 1, 4, and 3 respectively in the
|
||||
[complexity audit](03-complexity.md), rather than three additional independent
|
||||
bugs or duplicate remediation projects. None is a newly demonstrated behavioral
|
||||
failure or an automatic release blocker. Existing behavior, not a stronger
|
||||
equivalence promise, is the preservation target.
|
||||
|
||||
## 1. Share The Schema Compilation Lifecycle Across Source And Execution Paths
|
||||
|
||||
### Evidence
|
||||
|
||||
- [TestJSONSchemaNumericConstraintsRetainJSONPrecision](../../../internal/validate/standard_validator_test.go:721)
|
||||
runs the same exact-number constraints through operating-system and `fs.FS`
|
||||
validators using `jsonSchemaValidatorSources`. Both must distinguish adjacent
|
||||
large integers, preserve precise decimal boundaries, and accept the same
|
||||
valid instances.
|
||||
- [TestJSONSchemaDecodingRequiresOneCompleteDocument](../../../internal/validate/standard_validator_test.go:805)
|
||||
uses that same source matrix to require equivalent schema/instance framing
|
||||
and the distinction between operational schema errors and rejected output.
|
||||
- [TestSchemaReferencesPreserveEscapedFilenames](../../../internal/validate/standard_validator_test.go:492)
|
||||
applies the same reference cases to both source implementations, with an
|
||||
explicit platform exception for filenames illegal on Windows.
|
||||
- [TestStandardValidatorPreparedSchemaSurvivesSourceRemoval](../../../internal/validate/standard_validator_test.go:228)
|
||||
and [TestFSValidatorPreparedSchemaSurvivesSourceMutation](../../../internal/validate/standard_validator_test.go:416)
|
||||
separately require the same captured schema to accept an integer-valued
|
||||
object and reject a string-valued one after its source changes. Their source
|
||||
operations differ, but the prepared validation responsibility matches.
|
||||
|
||||
These tests support common schema semantics and compilation ownership; they do
|
||||
not require identical filesystem representations or diagnostic strings.
|
||||
|
||||
### Implementation
|
||||
|
||||
[StandardValidator and FSValidator](../../../internal/validate/standard_validator.go:74)
|
||||
each implement `PrepareValidation` and direct `validateJSONSchema`. The four
|
||||
paths repeat compiler setup, root-resource handling, cancellation boundaries,
|
||||
compilation/error wrapping, and construction or use of executable schema
|
||||
state. The direct host-file path lets its loader fetch the root, whereas the
|
||||
other paths explicitly load/register it. Decoding, compiler construction,
|
||||
schema execution, and content-result classification already have shared helpers;
|
||||
the remaining duplication is the lifecycle around them.
|
||||
|
||||
### Opportunity And Potential Remediation
|
||||
|
||||
Keep source-specific resolution/loading adapters, but introduce one private
|
||||
compilation operation in `internal/validate` that produces the root document
|
||||
and compiled executor for the caller's selected source. Reuse it from prepared
|
||||
validation and from the direct schema-validation callback. Retain the already
|
||||
shared artifact-validation/result logic. This makes future dialect, framing,
|
||||
number-preservation, and cancellation changes less likely to land in only some
|
||||
of the four paths.
|
||||
|
||||
The existing package is the appropriate owner; neither a public compiler API
|
||||
nor a generic filesystem framework is needed. The related runner dependency
|
||||
clarification remains part of complexity-audit finding 1.
|
||||
|
||||
### Tradeoffs And Test Independence
|
||||
|
||||
Direct validation currently rejects malformed output JSON before accessing its
|
||||
schema. Eagerly implementing it as `PrepareValidation` followed by plan
|
||||
validation would change I/O and error ordering. Share compilation when that
|
||||
work is reached, not necessarily the entire entry point. Preserve contained
|
||||
host paths and symlink handling, logical `fs.FS` paths, escaped resource URLs,
|
||||
single-file sources, synchronous cancellation precedence, and operation-local
|
||||
prepared freezing. These source differences belong in their adapters.
|
||||
|
||||
Keep the source matrix and independently specified valid/invalid instances.
|
||||
Do not compute expected results using the new compilation helper or delete
|
||||
source-specific containment/cancellation tests merely because compilation is
|
||||
shared. Provider-facing packaging of external schema references is a separate
|
||||
correctness finding, not solved by this extraction.
|
||||
|
||||
## 2. Centralize Structural Copying Of Already-Normalized JSON Values
|
||||
|
||||
### Evidence
|
||||
|
||||
- [TestPublicLLMClientAdapterGivesClientOwnedNestedValues](../../../llm_adapter_internal_test.go:14)
|
||||
requires nested provider maps, schema values, messages, and cache metadata to
|
||||
remain independent when an injected client mutates and retains its request.
|
||||
- [TestRunnerPreparedExecutionDetachesTransferredSnapshotSources](../../../internal/usecase/prepared_execution_test.go:634)
|
||||
separately requires nested provider maps and schema values to survive source
|
||||
mutation, repeated `Details` calls, and execution without aliasing.
|
||||
- [TestCopyHandlesIndirectionAndIsolatesNestedMutations](../../../internal/jsonvalue/jsonvalue_test.go:156)
|
||||
exercises the underlying JSON-value ownership responsibility with typed
|
||||
collections and repeated acyclic children. The adjacent scalar/collection
|
||||
tests protect compatible concrete types and exact `json.Number` values.
|
||||
- [TestCopyAndCopyMapApplyDistinctEmptyKeyRules](../../../internal/jsonvalue/jsonvalue_test.go:204)
|
||||
explicitly demonstrates a necessary policy distinction: general schema
|
||||
objects can have empty keys, while provider maps cannot.
|
||||
|
||||
The common requirement is an independently owned copy of accepted nested
|
||||
values. These tests do not establish universal equivalence of validation,
|
||||
normalization, and copying on arbitrary Go values.
|
||||
|
||||
### Implementation
|
||||
|
||||
The facade's [copyAny/copyReflectValue](../../../convert.go:406) implements a
|
||||
separate recursive reflection copier for values projected into client requests,
|
||||
prepared details, results, and profile inspections. Internal prepared-state
|
||||
cloning uses [clonePreparedRun](../../../internal/usecase/prepared_execution.go:226)
|
||||
and [jsonvalue.Copy/CopyMap](../../../internal/jsonvalue/jsonvalue.go:30).
|
||||
The latter package already owns compatible JSON-tree copying and validation.
|
||||
|
||||
Ordinary runtime provider values reach facade projection after JSON-value
|
||||
validation/copying. The runner also copies its schema document through
|
||||
[structuredOutputFromValidationPlan](../../../internal/usecase/runner.go:419)
|
||||
before projecting it. Thus these normal runtime paths have a common accepted
|
||||
value domain, even though their entry-point policies are different.
|
||||
|
||||
### Opportunity And Potential Remediation
|
||||
|
||||
Move ownership of trusted JSON-tree cloning into `internal/jsonvalue`, reusing
|
||||
its compatible structural-copy machinery where semantics match, so the facade
|
||||
does not maintain another general reflection traversal. Keep public/domain DTO
|
||||
field projection in the facade. A small internal trusted-value clone operation
|
||||
is a possible approach; reuse of existing fallible operations is another where
|
||||
the caller can preserve the existing error boundary. Select the simpler option
|
||||
after inventorying the concrete callers and values, rather than introducing a
|
||||
configurable reflection framework.
|
||||
|
||||
The meaningful payoff is removing an independent traversal implementation,
|
||||
not wrapping the existing facade copier under another name. This does not
|
||||
require consolidating every pointer or slice helper in the repository.
|
||||
|
||||
### Tradeoffs And Test Independence
|
||||
|
||||
`Copy` validates and bounds values and normalizes null/indirection; the facade
|
||||
copier assumes prior acceptance and can preserve typed nils and pointers.
|
||||
Neither is a drop-in replacement for the other on all possible inputs.
|
||||
Preserve observed types, nil behavior, exact numbers, schema empty keys, and
|
||||
caller ownership at the relevant boundary. An infallible projection must not
|
||||
silently turn a new copy error into a zero value, a panic, or an additional
|
||||
consumer restriction.
|
||||
|
||||
The [configuration-boundary ADR](../../adr/0001-configuration-boundary.md)
|
||||
also remains authoritative. `providerValueForMapping` serves a different,
|
||||
pre-handoff source-encoding purpose and defers unsupported-value errors to
|
||||
ConfigKit. It must not be routed through a rejecting runtime copier as part of
|
||||
this work. Source number conversion is separately recorded in the correctness
|
||||
audit; visual similarity between its reflection switch and a copier is not
|
||||
enough to equate their contracts.
|
||||
|
||||
Keep literal expected values and mutation-based assertions in both facade and
|
||||
internal tests. A shared production copier must not also manufacture their
|
||||
expected snapshots. Retain focused tests for policy differences rather than
|
||||
replacing them with one “all copy functions are equivalent” test.
|
||||
|
||||
## 3. Share If/With Requirement-Branch Orchestration, Not Template Execution
|
||||
|
||||
### Evidence
|
||||
|
||||
- [TestAnalyzeRequirementsClassifiesKnownConditions](../../../internal/prompt/requirements_test.go:268)
|
||||
verifies required/optional classification for selected and skipped `if`
|
||||
branches, and directly compares the recognized branch choice with the real
|
||||
Go renderer. This anchors recognized condition semantics in actual rendering.
|
||||
- [TestAnalyzeRequirementsHandlesWithDotAndNestedReachability](../../../internal/prompt/requirements_test.go:332)
|
||||
requires the same selected/skipped classification for `with` and nested
|
||||
`if`, while separately protecting `with`'s changed dot and preserved root
|
||||
caller-map access.
|
||||
- [TestAnalyzeRequirementsReportsUnresolvedDecisionAndContentConditions](../../../internal/prompt/requirements_test.go:369)
|
||||
protects conditional dependencies and meaningful incompleteness for unknown
|
||||
decisions. It also establishes that an unresolved condition around plain
|
||||
text alone does not make discovery incomplete.
|
||||
- [TestAnalyzeRequirementsDoesNotHideCallerMapScopes](../../../internal/prompt/requirements_test.go:14)
|
||||
compares several scope forms against actual template execution while allowing
|
||||
incomplete analysis for constructs outside recognition scope.
|
||||
|
||||
The tests support one reachability/evidence policy for recognizable branches,
|
||||
with explicit scope differences. They do not require the inspector to execute
|
||||
or completely interpret arbitrary templates.
|
||||
|
||||
### Implementation
|
||||
|
||||
The `IfNode` and `WithNode` cases in
|
||||
[requirementWalker.visit](../../../internal/prompt/requirements.go:451)
|
||||
independently perform inactive-path traversal, condition evaluation, then/else
|
||||
reachability selection, dependency-counter comparison, and limitation recording.
|
||||
`with` adds a different body-dot calculation. `branchReach`, condition
|
||||
evaluation, and `template.IsTrue` already provide shared lower-level behavior,
|
||||
but their surrounding branch protocol is repeated.
|
||||
|
||||
### Opportunity And Potential Remediation
|
||||
|
||||
Extract a small package-private branch traversal operation that owns the common
|
||||
reachability and evidence sequence, with explicit body/else scope handling.
|
||||
Leave node recognition and `withDotIdentity` visible at the node-specific
|
||||
boundary. Keeping this in `internal/prompt` would allow a change to skipped-path
|
||||
handling or incompleteness reporting to be reviewed and implemented once.
|
||||
|
||||
This is the narrowly supported part of complexity-audit finding 3. It does not
|
||||
justify sharing the requirement walker with the real template executor or
|
||||
building an interpreter abstraction.
|
||||
|
||||
### Tradeoffs And Test Independence
|
||||
|
||||
An abstraction that treats `if` and `with` identically would lose caller-map
|
||||
scope information. Preserve `with`'s body-dot behavior, else scope, inactive
|
||||
computed-name handling, and the placement of dependency bookkeeping used to
|
||||
decide whether an unresolved condition matters. A helper with numerous mode
|
||||
flags or callbacks would obscure those differences; prefer a small explicit
|
||||
operation over a generalized AST visitor framework.
|
||||
|
||||
Retain separate node-specific assertions and the independent standard-library
|
||||
rendering comparisons. Extracting production traversal is not a reason to
|
||||
calculate expected inventories using the production branch helper or to
|
||||
strengthen `Complete` into an execution-readiness guarantee.
|
||||
|
||||
## Recommendation And Validation
|
||||
|
||||
Start with schema compilation, which has the clearest shared lifecycle and
|
||||
cross-source test matrix. The branch helper is a smaller local follow-up.
|
||||
Treat JSON-copy consolidation as a more careful ownership refactor: remove a
|
||||
traversal only after its caller/policy differences are accounted for. No new
|
||||
package boundary is necessary solely to carry out these changes, and retaining
|
||||
small source/boundary-specific adapters is appropriate.
|
||||
|
||||
Existing offline tests passed with the local Go 1.27.0 toolchain:
|
||||
|
||||
```sh
|
||||
go test -race -count=1 ./internal/validate ./internal/prompt ./internal/jsonvalue
|
||||
go test -race -count=1 -run 'Test(PublicLLMClientAdapter|PreparedExecutionFreezesSourcesAndReturnsIndependentDetails|RunnerPreparedExecutionDetachesTransferredSnapshotSources|RunnerRunAndPrepareResolveSameProfileAndEffectiveSettings|RunnerPreparationNormalizesOutputContractConsistently|ValidateRequestMatchesPrepareContract)' . ./internal/usecase
|
||||
```
|
||||
|
||||
Passing tests substantiate the reviewed current behavior, not equivalence of a
|
||||
future refactor. No new tests, production changes, live provider requests,
|
||||
benchmarks, or additional release gates were introduced by this audit.
|
||||
229
docs/roadmap/audit/11-boundaries.md
Normal file
229
docs/roadmap/audit/11-boundaries.md
Normal file
@@ -0,0 +1,229 @@
|
||||
# Package Cohesion And Architectural Boundary Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `b39460a` and the current worktree. Existing
|
||||
audit-document moves were preserved. No production code, tests, dependencies,
|
||||
or functionality were changed.
|
||||
|
||||
## Scope, Method, And Architectural Assessment
|
||||
|
||||
Reviewed the 17 repository-owned internal Go packages, their production import
|
||||
relationships, engine assembly, relevant callers, and tests. The public facade
|
||||
was checked where it establishes the boundary around those packages. Third-party
|
||||
package implementations were outside the audit's scope.
|
||||
|
||||
The [current component inventory](../../internal/overview.md) broadly matches
|
||||
the implementation. Configuration owns the final ConfigKit handoff and captured
|
||||
resource indexes; the engine assembles collaborators; use cases coordinate
|
||||
selection, preparation, execution, and diagnostics. Generation owns dispatch
|
||||
and retries, capacity owns admission and permits, and outbound transport,
|
||||
rendering, artifacts, schema validation, observations, and shared values have
|
||||
identifiable owners. No repository-wide package reorganization is warranted.
|
||||
|
||||
The public facade does not leak internal types or receive reverse production
|
||||
imports from internal packages in the checks run below. Deliberate boundaries
|
||||
remain useful: configuration completion differs from request overrides;
|
||||
diagnostic orchestration differs from ordinary execution; domain invariants
|
||||
differ from source-specific presence and error handling. A broad package, a
|
||||
small interface-only package, or an import of another internal component is not
|
||||
itself a cohesion defect.
|
||||
|
||||
Three focused ownership improvements warrant consideration. They are
|
||||
maintainability opportunities, not newly demonstrated runtime bugs or automatic
|
||||
release blockers.
|
||||
|
||||
## 1. Retire The Test-Only Capacity Client As A Production Scheduling Surface
|
||||
|
||||
**Relevant code:** [capacity.NewClient and client.Generate](../../../internal/capacity/client.go:17),
|
||||
[generation constructors and dispatch](../../../internal/generation/coordinator.go:37),
|
||||
[engine assembly](../../../internal/engine/engine.go:157), and
|
||||
[runner scheduling test](../../../internal/usecase/runner_test.go:2391).
|
||||
|
||||
**Basis:** The engine selects `generation.NewBuiltIn` or `NewInjected` for every
|
||||
production engine. The coordinator owns active-permit acquisition around each
|
||||
attempt, retry waits, observation, and diagnostic evidence. The earlier
|
||||
`capacity.NewClient` still provides a separate model-client wrapper that
|
||||
acquires/releases a permit around an opaque call. Both use the same manager;
|
||||
there are not two independent FIFO algorithms or pools.
|
||||
|
||||
Caller analysis found only tests invoking the older constructor: capacity tests
|
||||
and a runner test that assembles initial/repair scheduling with this wrapper.
|
||||
The [capacity document](../../internal/capacity.md#fifo-generation-permits)
|
||||
acknowledges its retention for focused callers/tests, but no production caller
|
||||
was found. Its presence leaves capacity owning a model-client adapter after
|
||||
production dispatch responsibility moved to generation, and makes one runner
|
||||
assembly test exercise a different scheduling composition than the engine.
|
||||
|
||||
**Consequences:** The extra adapter and its documented role remain maintenance
|
||||
surface without a production need. Future scheduling changes can require
|
||||
reasoning about a legacy assembly alongside the real one. Green tests of the
|
||||
wrapper alone do not demonstrate that coordinator dispatch applies the same
|
||||
permit lifetime. Existing coordinator and facade tests do protect that actual
|
||||
path, so this is not a claim that production scheduling is untested or broken.
|
||||
|
||||
**Potential remediation:** Keep the manager and its admission/FIFO primitives
|
||||
in `internal/capacity`, and generation dispatch integration in
|
||||
`internal/generation`. Replace the runner test's wrapper with the production
|
||||
injected coordinator. For focused capacity scenarios, test `Manager.Acquire`
|
||||
directly or retain the small one-shot adapter as test-owned scaffolding rather
|
||||
than a production constructor. Once callers are adjusted, the production
|
||||
wrapper can be retired and its documentation updated.
|
||||
|
||||
Preserve the useful FIFO, cancellation-race, independent-pool, and panic-release
|
||||
assertions when adjusting test ownership. This finding does not call for moving
|
||||
queue internals into generation, deleting those scenarios, or wrapping the
|
||||
coordinator in another capacity client, which could acquire the same limit
|
||||
twice. Retaining the wrapper is viable if a real internal production need is
|
||||
identified; current test reachability alone does not establish one.
|
||||
|
||||
## 2. Put OpenRouter Authentication Protocol Details Behind The Outbound Adapter
|
||||
|
||||
**Relevant code:** [authenticationProbe.runOpenRouterKey](../../../internal/usecase/smoke_authentication.go:66),
|
||||
[recognizedOpenRouterKeyResponse](../../../internal/usecase/smoke_authentication.go:139),
|
||||
[diagnostic endpoint derivation](../../../internal/usecase/smoke_selection.go:166),
|
||||
and [HTTP authentication transport](../../../internal/llm/http_evidence.go:111).
|
||||
|
||||
**Basis:** `internal/usecase` currently interprets OpenRouter-specific HTTP
|
||||
statuses and the account-response keys `data`, `label`, `is_free_tier`, and
|
||||
`usage`. It owns JSON envelope parsing, identifying-field type checks, and body
|
||||
consumption. The same package derives the OpenRouter-relative `/key` endpoint.
|
||||
Meanwhile `internal/llm` owns authenticated HTTP execution, credential
|
||||
resolution, evidence collection, and response ownership mechanics. One provider
|
||||
integration is therefore divided between the application coordinator and its
|
||||
outbound adapter.
|
||||
|
||||
This division is different from a use case consuming factual HTTP evidence:
|
||||
the precise external account document and provider endpoint are protocol
|
||||
knowledge. The [architecture policy](../../policy/architecture.md#architectural-approach)
|
||||
calls for encapsulating external integration details in internal adapters.
|
||||
The parser is already functionally isolated, making the ownership seam fairly
|
||||
small. The authentication tests in
|
||||
[smoke_authentication_test.go](../../../internal/usecase/smoke_authentication_test.go:64)
|
||||
also show that this is provider protocol behavior rather than application
|
||||
selection policy.
|
||||
|
||||
**Consequences:** A change to OpenRouter's account endpoint or identifying
|
||||
response fields requires changing application-layer code as well as the
|
||||
transport integration. Adding another authentication adapter would extend
|
||||
provider-specific branching inside the use-case package. This complicates the
|
||||
otherwise clear distinction between choosing a diagnostic and implementing
|
||||
the upstream protocol.
|
||||
|
||||
**Potential remediation:** Move the OpenRouter request/response adapter,
|
||||
including its default destination and response recognition, into
|
||||
`internal/llm` alongside the existing diagnostic transport. Have the use case
|
||||
consume a small factual outcome/evidence value. Keep target selection,
|
||||
configured applicability, unsupported-operation reporting, caller cancellation,
|
||||
overall completion, and report assembly in the use case. The existing
|
||||
consumer-owned diagnostic interface is a useful seam; engine assembly can
|
||||
supply the concrete outbound implementation without requiring protocol parsing
|
||||
inside the coordinator.
|
||||
|
||||
An alternative is a focused internal diagnostic-adapter package if separation
|
||||
from inference transport becomes useful. Another package or a provider plugin
|
||||
framework is not necessary for the current single protocol adapter. Do not
|
||||
introduce a new dependency from the outbound adapter back into `usecase`, or
|
||||
move application report/completion policy into transport merely to relocate
|
||||
the parser. The diagnostic feature legitimately deals with HTTP facts; a
|
||||
blanket prohibition on HTTP types or status observations in use cases is not
|
||||
implied by this finding.
|
||||
|
||||
Preserve anonymous authentication probing, explicit endpoint overrides,
|
||||
neutral tri-state facts, private body/credential handling, and the absence of
|
||||
ordinary observer events. The previously recorded unbounded authentication-body
|
||||
read is a separate resource/security finding; relocation alone does not fix it.
|
||||
|
||||
## 3. Narrow Smoke-Value Copy Helpers To Their Actual Ownership Boundaries
|
||||
|
||||
**Relevant code:** [domain.CloneSmokeTestRequest](../../../internal/domain/smoke_diagnostics.go:157),
|
||||
[domain.CloneSmokeTestReport](../../../internal/domain/smoke_diagnostics.go:179),
|
||||
[smokeReportTarget](../../../internal/usecase/smoke_tester.go:432), and
|
||||
[public smoke-value conversion](../../../smoke_diagnostics.go:279).
|
||||
|
||||
**Basis:** `CloneSmokeTestRequest` is called only by its own domain test;
|
||||
actual request ownership is established by public-to-domain conversion.
|
||||
`CloneSmokeTestReport` has one production caller: `smokeReportTarget` creates
|
||||
an otherwise empty report containing a target, clones the full report, and
|
||||
extracts just that target. Whole-report facts, timing, and attempt cloning have
|
||||
no runtime use through that caller. Public report conversion independently
|
||||
copies the actual facts, timings, attempts, and provider map into public types.
|
||||
|
||||
Thus the domain package maintains a broad request/report-copy API, but the
|
||||
real internal consumer needs only a detached target/provider map. The
|
||||
whole-report ownership functionality is exercised by its domain test rather
|
||||
than by actual whole-report production use. Public DTO conversion is a
|
||||
legitimate separate responsibility, not something that should disappear into
|
||||
domain types or public aliases.
|
||||
|
||||
**Consequences:** Smoke-value changes have additional copy lists and tests to
|
||||
maintain even where those copies are not needed in production. A temporary
|
||||
report container obscures the narrow target-detachment operation. Keeping a
|
||||
general cloner solely because its own test calls it reverses the relationship
|
||||
between production responsibility and supporting tests.
|
||||
|
||||
**Potential remediation:** Express the target-copy operation directly at its
|
||||
use-case boundary using existing JSON-value ownership, or a small target clone
|
||||
helper if actual callers justify one. Retain public request/report projection
|
||||
in the facade and shared JSON-tree copying in its existing owner. Retire the
|
||||
unused request cloner and the unnecessary whole-report clone surface after
|
||||
confirming no additional callers need them. Remove only clone-specific helper
|
||||
code that is then unused. The backend-cloning helpers have real registry
|
||||
consumers and are outside the proposed retirement; retain that metadata
|
||||
ownership behavior.
|
||||
|
||||
Preserve provider-map error propagation, caller-owned result values, and
|
||||
credential/endpoint redaction. Keep the meaningful
|
||||
[public conversion ownership test](../../../smoke_conversion_internal_test.go:13)
|
||||
and relevant use-case report tests. If genuine internal whole-report snapshots
|
||||
are needed, retaining an explicitly used domain cloner is a reasonable
|
||||
alternative. This finding does not require sharing public/internal DTO types
|
||||
or routing public conversion through a full internal clone followed by another
|
||||
copy. Coordinate any JSON traversal consolidation with
|
||||
[audit 10](10-equivalence.md#2-centralize-structural-copying-of-already-normalized-json-values).
|
||||
|
||||
## Related Existing Findings
|
||||
|
||||
The runner's declared `Validator` dependency and actual `ValidationPreparer`
|
||||
requirement remain the interface-boundary clarification in
|
||||
[complexity-audit finding 1](03-complexity.md#1-consolidate-schema-compilation-and-express-the-runners-actual-interface).
|
||||
The facade's generic JSON reflection copier has an existing internal owner
|
||||
identified in audit 10. These are relevant ownership concerns, not additional
|
||||
independent findings to count again. Large orchestration functions and repeated
|
||||
schema compilation within an already coherent package likewise retain their
|
||||
existing complexity/equivalence remediation scope.
|
||||
|
||||
## Reachability And Validation Evidence
|
||||
|
||||
Production imports were enumerated with `go list` and compared with the graph
|
||||
and source callers. Rapid Type Analysis used the locally available pinned
|
||||
`golang.org/x/tools` v0.52.0 command, with reporting restricted to
|
||||
repository-owned internal packages:
|
||||
|
||||
```sh
|
||||
go list -f '{{.ImportPath}}|{{join .Imports " "}}' ./internal/...
|
||||
go run golang.org/x/tools/cmd/deadcode@v0.52.0 -test -filter='^go.maximumdirect.net/promptkit/internal/' ./...
|
||||
go run golang.org/x/tools/cmd/deadcode@v0.52.0 -filter='^go.maximumdirect.net/promptkit/internal/' ./...
|
||||
```
|
||||
|
||||
With tests included, the tool reported only the unused test helper
|
||||
`internal/usecase/runner_test.go:stringPtr`. That small test cleanup does not
|
||||
justify a separate architectural finding. Test-only production helpers remain
|
||||
reachable to this analysis when their tests call them.
|
||||
|
||||
Without tests, the roots are the repository's example executables, not every
|
||||
possible downstream library consumer. The output includes valid public
|
||||
workflows that those examples do not exercise, so it is not a deletion list.
|
||||
The capacity-wrapper and request-cloner observations above were additionally
|
||||
checked against actual source references and assembly; graph in-degree or
|
||||
example reachability alone was not treated as proof of obsolescence. The
|
||||
analysis also covers only the selected local build configuration.
|
||||
|
||||
The following existing offline checks passed with local Go 1.27.0:
|
||||
|
||||
```sh
|
||||
go test -race -count=1 -run 'Test(PublicFacadeDoesNotExposeInternalTypes|InternalProductionPackagesDoNotImportFacade|RepositoryDoesNotImportFormerModule|SmokeTestConcludedInferenceMismatchHasNilGoErrorAndDetachedValues)' .
|
||||
go test -race -count=1 ./internal/capacity ./internal/generation ./internal/domain ./internal/usecase
|
||||
```
|
||||
|
||||
No new release gate, consumer contract, provider call, production edit, or test
|
||||
removal is proposed as part of this audit pass. The tools left module metadata
|
||||
unchanged.
|
||||
233
docs/roadmap/audit/12-deadcode.md
Normal file
233
docs/roadmap/audit/12-deadcode.md
Normal file
@@ -0,0 +1,233 @@
|
||||
# Dead Code And Obsolete Functionality Audit
|
||||
|
||||
Reviewed 2026-11-04 against commit `3b76031` and the current worktree. Existing
|
||||
audit-document moves were preserved. No production code, tests, dependencies,
|
||||
or functionality were changed.
|
||||
|
||||
## Scope And Assessment
|
||||
|
||||
Reviewed repository-owned Go packages, public entry points, configuration and
|
||||
runtime values, collaborator interfaces, default/optional paths, embedded
|
||||
resources, build constraints, and the release launchers and their wiring test.
|
||||
The architecture remains a public facade over captured configuration, use-case
|
||||
orchestration, generation/capacity, and focused internal adapters. The former
|
||||
standalone configuration repositories have already been retired; this review
|
||||
found smaller residues rather than another abandoned subsystem.
|
||||
|
||||
The review combined Rapid Type Analysis, Staticcheck unused/data-flow checks,
|
||||
graph caller discovery, source inspection, and the format, internal, policy,
|
||||
and roadmap documents. Tests count as real callers, but a helper's self-test
|
||||
does not by itself establish a production need. Conversely, absence from the
|
||||
examples or default execution path is not evidence that an optional public
|
||||
feature is obsolete.
|
||||
|
||||
Four cleanup areas warrant consideration. These are low-priority reductions in
|
||||
state and maintenance surface, not newly demonstrated runtime bugs. None calls
|
||||
for deleting a supported public feature or imposing a new release gate.
|
||||
|
||||
## 1. Remove Backend Presence Bookkeeping With No Runtime Consumer
|
||||
|
||||
**Relevant code:** [loaded.backends](../../../internal/configuration/load.go:122),
|
||||
[backendFields and presence capture](../../../internal/configuration/load.go:215),
|
||||
[effective backend construction](../../../internal/configuration/targets.go:14),
|
||||
and [Backend.QueueCapacitySet](../../../internal/domain/domain.go:187).
|
||||
|
||||
**Evidence:** Construction allocates a per-backend field-state map and fills it
|
||||
through `captureFields(..., state.backends)`. No production code reads the
|
||||
recorded backend states. The only lookup of the captured queue state is in
|
||||
[TestLoadDefaultsAbsentQueueCapacity](../../../internal/configuration/load_test.go:755).
|
||||
Unlike profile presence, backend presence is not consumed while constructing
|
||||
effective execution targets.
|
||||
|
||||
`QueueCapacitySet` is also assigned during target construction and asserted in
|
||||
tests, but never read to select a production behavior. The current assignment
|
||||
is simply whether concurrency is positive; it no longer distinguishes an
|
||||
explicit queue setting from a derived default. The registry's capacity-policy
|
||||
projection uses `ConcurrencyLimit` and `QueueCapacity`, not this flag.
|
||||
|
||||
**Why obsolete:** Queue defaulting and its provenance are already handled by
|
||||
[resolveBackends](../../../internal/configuration/resolve_targets.go:11) and
|
||||
ConfigKit's derived-default record. Typed final values drive runtime capacity;
|
||||
the retained side map and Boolean do not contribute to that behavior or to the
|
||||
public inspection view. No documented compatibility or planned-feature need for
|
||||
these extra records was found.
|
||||
|
||||
**Potential remediation:** Remove the backend-only presence capture, its field
|
||||
list and retained map, and the unused runtime Boolean/assignment. Adjust tests
|
||||
to assert effective queue behavior and, where relevant, the existing ConfigKit
|
||||
inspection/provenance result. Keep `captureFields`, profile field-state
|
||||
tracking, and prompt-input metadata presence: those have actual consumers and
|
||||
preserve required absent/null/explicit-value distinctions. Retain explicit
|
||||
zero queue capacity, derived defaulting, and operator history unchanged.
|
||||
|
||||
This removes unnecessary construction work and retained per-backend maps; it
|
||||
does not justify dropping configuration provenance or changing its ownership
|
||||
under the [configuration-boundary ADR](../../adr/0001-configuration-boundary.md).
|
||||
|
||||
## 2. Retire Legacy Fields In The Captured Prompt Representation
|
||||
|
||||
**Relevant code:** [PromptDefinition.OutputFormat](../../../internal/domain/domain.go:148),
|
||||
[PromptMessageTemplate.ContentFile](../../../internal/domain/domain.go:173),
|
||||
[prompt materialization](../../../internal/configuration/prompts.go:58),
|
||||
[resolveOutputContract](../../../internal/usecase/runner.go:639), and
|
||||
[definition hashing](../../../internal/promptdef/hash.go:12).
|
||||
|
||||
**Evidence:** Supported engine construction requires and validates a complete
|
||||
output contract. Materialization sets `definition.Validation` to that contract
|
||||
and duplicates its format into `definition.OutputFormat`. The runner reads
|
||||
the duplicate only when `Validation.Format` is empty, a condition that the
|
||||
captured definitions supplied by engine construction cannot have. The standard
|
||||
runner test-definition helper fills both fields with the same value; no
|
||||
supported public repository-injection path supplies an alternative definition.
|
||||
|
||||
For messages, materialization resolves the real source `content_file` into
|
||||
captured `Content` and does not populate the domain `ContentFile` field. The
|
||||
renderer uses `Content`/captured templates, not that path field. Its one
|
||||
nonempty assignment is a renderer fixture demonstrating that already-loaded
|
||||
content is rendered without opening a file. The source-format
|
||||
`promptMessageConfig.ContentFile` remains actively used and is a different
|
||||
field.
|
||||
|
||||
**Why obsolete:** The runtime representation still carries source-era message
|
||||
path data and a second output-format source after configuration loading became
|
||||
the authoritative materialization boundary. The empty-format fallback is
|
||||
disconnected from supported engine definitions. No current documentation or
|
||||
roadmap records a compatibility requirement for these internal duplicate fields.
|
||||
|
||||
**Potential remediation:** Use `PromptDefinition.Validation.Format` as the one
|
||||
captured format and remove the legacy duplicate/fallback. Remove only the
|
||||
runtime message path field, adjusting its fixture to exercise captured text.
|
||||
Preserve the actual YAML `content_file` feature, source-relative resource
|
||||
resolution, ConfigKit origin inspection, and construction-time capture. Also
|
||||
preserve the separate, supported request-override rule that an empty replacement
|
||||
`OutputContract.Format` defaults to text.
|
||||
|
||||
**Serialization caveat:** These fields are included indirectly by
|
||||
`HashDefinition`'s `json.Marshal`, even when a message path is empty. They are
|
||||
therefore not completely inert bytes: removing them changes current prompt
|
||||
hash values. Account for that effect explicitly and protect the documented
|
||||
opaque-equality relationships between inspection and preparation. Do not claim
|
||||
bit-identical hashes after changing the hash input, or invent a permanent hash
|
||||
encoding guarantee merely to perform this cleanup. The fields are internal
|
||||
representation, not part of the public stable DTO shapes.
|
||||
|
||||
## 3. Remove Unused Diagnostic State And A Discarded Helper Result
|
||||
|
||||
**Relevant code:** [reachabilityProbeOutcome.initialEndpoint](../../../internal/usecase/smoke_reachability.go:24),
|
||||
[HTTP report assembly](../../../internal/usecase/smoke_tester.go:376),
|
||||
[fileSource](../../../internal/engine/engine.go:211), and
|
||||
[SetSchemaFile](../../../internal/engine/engine.go:75).
|
||||
|
||||
**Evidence:** The reachability probe assigns its private outcome's
|
||||
`initialEndpoint`, but the smoke tester never reads that member. It records
|
||||
the real public attempt's `InitialEndpoint` from the resolved report target.
|
||||
Only `TestReachabilityProbeRemovesURLCredentials` inspects the private echo.
|
||||
Actual sanitized request/evidence and report behavior are established through
|
||||
other values.
|
||||
|
||||
Separately, `fileSource` returns a filesystem, file name, source-directory
|
||||
string, and error. Its only caller, `SetSchemaFile`, always discards the
|
||||
directory result. The directory is still needed internally to construct
|
||||
`os.DirFS`; exposing it as another return value serves no caller.
|
||||
|
||||
**Potential remediation:** Remove the unused probe-outcome member and its
|
||||
assignment; retain credential-removal assertions against the actual request,
|
||||
evidence, or public diagnostic report. Reduce `fileSource`'s private signature
|
||||
to the consumed results. Keep its directory calculation, file checks, selected
|
||||
filesystem/root, and the public `WithSchemaFile` option intact.
|
||||
|
||||
Neither cleanup removes `SmokeTestAttempt.InitialEndpoint`, destination
|
||||
evidence, or single-schema-file support. Those public features are live even
|
||||
though an extra private field/result is unnecessary.
|
||||
|
||||
## 4. Remove Orphan Helpers Whose Only Callers Are Their Own Tests
|
||||
|
||||
**Relevant code and evidence:**
|
||||
|
||||
- [filecatalog.RelativePath](../../../internal/filecatalog/catalog.go:71) is
|
||||
called only by `TestRelativePathNested`. Current source discovery and
|
||||
source-relative resource resolution use other helpers; no production or
|
||||
public-facade caller uses this wrapper around `filepath.Rel`.
|
||||
- [resourceNamespaceFromID](../../../internal/domain/resource_identity.go:34)
|
||||
is called only by `TestResourceNamespaceFromID`. Actual namespace inventory
|
||||
and filtering paths do not call it. The exported identity-validation and
|
||||
qualification operations remain active and are not candidates for removal.
|
||||
- [runner_test.go:stringPtr](../../../internal/usecase/runner_test.go:2592)
|
||||
has no callers, including in tests. Both reachability analysis and
|
||||
Staticcheck `U1000` report it.
|
||||
|
||||
Caller searches found no reflection, registration, build-constrained caller,
|
||||
or supported external API exposing these helpers. The first two belong to
|
||||
internal packages and have only their direct self-tests; no documented future
|
||||
or compatibility purpose was found. The third is already test-only and unused.
|
||||
|
||||
**Potential remediation:** Remove the orphan helpers and their now-irrelevant
|
||||
self-tests, keeping behavioral coverage of real discovery, namespace
|
||||
inventory/filtering, and identity qualification. Remove the unused pointer
|
||||
helper without replacing it. There is no reason to introduce a new abstraction
|
||||
or move the self-tests to another layer solely to preserve these unused APIs.
|
||||
|
||||
## Intentional Retention And Related Findings
|
||||
|
||||
The following distinctions prevent removal based on misleading reachability:
|
||||
|
||||
- `WithSchemaFile` and `InlineWithURI` are supported public helpers despite
|
||||
being unreachable from the current test executables. The former's missing
|
||||
public wiring coverage is already in [audit 08](08-testing.md); it is not
|
||||
evidence that the option should be deleted.
|
||||
- The older capacity client wrapper and no-repair runner constructor are
|
||||
expressly retained for focused internal callers/tests in the current
|
||||
[capacity](../../internal/capacity.md#fifo-generation-permits) and
|
||||
[runner](../../internal/runner.md#collaborators) documentation. This audit
|
||||
does not classify them as dead functionality or recommend deleting them.
|
||||
[Audit 11](11-boundaries.md) discusses the wrapper's architectural placement;
|
||||
altering that deliberate retention is a separate decision.
|
||||
- Repository capability fallbacks, standalone rendering/validation, injected
|
||||
clients' nil usage-presence compatibility, nil-validator/admitter behavior,
|
||||
and optional provider settings have meaningful callers or documented
|
||||
purposes. Normal engine assembly taking another path does not obsolete them.
|
||||
- ConfigKit YAML fields and stable JSON DTO fields can be used by reflection;
|
||||
formatter methods and interfaces can be invoked indirectly. Embedded
|
||||
OpenRouter and smoke-schema assets, the Linux FIFO test, and release scripts
|
||||
likewise have consumers outside a simple direct-call list. They were not
|
||||
treated as dead. No unused public configuration option or abandoned release
|
||||
launcher was established.
|
||||
|
||||
The domain smoke-request/full-report cloners have a narrow-or-unused consumer
|
||||
surface already described in [audit 11, finding 3](11-boundaries.md#3-narrow-smoke-value-copy-helpers-to-their-actual-ownership-boundaries).
|
||||
That ownership/remediation scope is not counted again here. The schema and
|
||||
JSON-copy consolidations in prior audits likewise remain their existing work,
|
||||
not an excuse to delete intentional alternative implementations wholesale.
|
||||
|
||||
## Tooling And Validation
|
||||
|
||||
The checks used local Go 1.27.0 and pinned audit-tool versions:
|
||||
|
||||
```sh
|
||||
go run golang.org/x/tools/cmd/deadcode@v0.52.0 -test -filter='^go.maximumdirect.net/promptkit(/|$)' ./...
|
||||
go run honnef.co/go/tools/cmd/staticcheck@v0.8.2 -checks 'U1000,SA4*' ./...
|
||||
go list -f '{{.ImportPath}} {{join .GoFiles " "}} {{join .IgnoredGoFiles " "}}' ./...
|
||||
```
|
||||
|
||||
`deadcode -test` reported the two public helpers and unused test pointer helper
|
||||
listed above. Staticcheck reported only `stringPtr` (`U1000`) and exited with
|
||||
status 1 because of that finding; it reported no additional selected `SA4*`
|
||||
diagnostics. These are audit observations, not successful lint sign-offs.
|
||||
Neither tool establishes that all tested or serialized state is necessary,
|
||||
which is why the write-only and redundant fields required manual review.
|
||||
|
||||
Rapid Type Analysis starts from executable roots; tests add roots, and example
|
||||
programs do not exercise every library consumer path. The analysis is local-build
|
||||
evidence, not proof about all platforms or downstream programs. In particular,
|
||||
platform-specific schema handling and optional library operations were retained.
|
||||
|
||||
Existing offline race-enabled tests passed:
|
||||
|
||||
```sh
|
||||
go test -race -count=1 ./internal/configuration ./internal/domain ./internal/filecatalog ./internal/prompt ./internal/engine ./internal/usecase
|
||||
go test -race -count=1 -run 'Test(ConfigurationInspection|PreparedExecution|ValidateRequestMatchesPrepareContract|AppendedMessagesComposeFrozenEffectivePrompt)' .
|
||||
```
|
||||
|
||||
Tool execution left `go.mod` and `go.sum` unchanged. No production changes,
|
||||
test removals, provider calls, or new validation gates were made during this
|
||||
review.
|
||||
570
docs/roadmap/audit/catalog.md
Normal file
570
docs/roadmap/audit/catalog.md
Normal file
@@ -0,0 +1,570 @@
|
||||
# Consolidated Pre-1.0 Audit Catalog
|
||||
|
||||
Consolidated on 2026-11-04 from the 11 audit documents in this directory.
|
||||
Their 41 numbered findings are represented by 34 entries below. Entries that
|
||||
describe the same underlying issue share one remediation scope; missing
|
||||
regression coverage for an identified defect is included with that defect.
|
||||
Some broad testing findings consequently support more than one entry.
|
||||
|
||||
The order reflects consequence, exposure, and the value of clarifying public
|
||||
contracts before v1.0. Confirmed defects and conditional security exposure come
|
||||
first, followed by contract/test gaps, resource efficiency, and maintainability.
|
||||
This ordering does not upgrade the original severities or turn every cleanup
|
||||
into a release gate. The source audits retain their evidence, qualifications,
|
||||
and validation results. None of these entries is marked remediated by this
|
||||
documentation-only consolidation.
|
||||
|
||||
## 1. Update The Audited Maintainer Toolchain For Applicable Security Fixes
|
||||
|
||||
**Conditional security exposure.** The audited Go 1.27.0 toolchain predates
|
||||
networking fixes applicable to HTTP clients, including HTTP/2 CPU exhaustion
|
||||
and conditional certificate-trust exposure. Use a patched maintainer toolchain
|
||||
and repeat the focused security assessment against it. Actual downstream
|
||||
exposure depends on the toolchain that builds the consumer and its transport
|
||||
and trust configuration; raising PromptKit's minimum Go requirement is a
|
||||
separate compatibility decision, not an implied remedy.
|
||||
|
||||
Sources: [Security 1](07-security.md#1-the-audited-go-toolchain-predates-applicable-networking-security-fixes).
|
||||
|
||||
## 2. Bound Authentication-Probe Response Consumption
|
||||
|
||||
**Confirmed resource/security defect.** Authentication diagnostics read a
|
||||
successful upstream body with unbounded `io.ReadAll`. A deadline does not bound
|
||||
the bytes a fast server can supply. Introduce finite response consumption and
|
||||
clean closure, with an explicit choice of byte budget and overflow reporting.
|
||||
Overflow should not be interpreted as proof of rejected authentication.
|
||||
Relocating the provider adapter in entry 26 would not itself fix this defect.
|
||||
|
||||
Sources: [Resources 1](05-resources.md#1-authentication-diagnostics-read-an-unbounded-upstream-body),
|
||||
[Security 2](07-security.md#2-authentication-probes-permit-unbounded-remote-body-memory-consumption).
|
||||
|
||||
## 3. Avoid Exponential Expansion During Convenience Source Conversion
|
||||
|
||||
**Confirmed resource defect.** `ProfileSource` and `BackendSource` conversion
|
||||
expand repeated references before ConfigKit performs its bounded snapshot.
|
||||
A small shared acyclic graph can therefore cause exponential work and
|
||||
allocation. Preserve shared references during conversion, with suitable cycle
|
||||
and slice-view handling, rather than duplicating ConfigKit's validation or
|
||||
introducing PromptKit-side raw-source limits. Add focused shared-graph and
|
||||
deferred-failure regressions for both convenience helpers.
|
||||
|
||||
Sources: [Resources 2](05-resources.md#2-convenience-source-conversion-expands-shared-graphs-before-bounded-loading),
|
||||
[Testing 3](08-testing.md#3-programmatic-provider-value-conversion-lacks-representative-value-cases).
|
||||
|
||||
## 4. Resolve The ArtifactReader Buffer-Ownership Contract
|
||||
|
||||
**Confirmed race under the documented collaborator contract.** The public
|
||||
reader contract permits buffer reuse after `Read` returns, before PromptKit
|
||||
can finish copying it. Concurrent preparation with a compliant custom reader
|
||||
can race. Clarify ownership/lifetime per invocation and align adapters, GoDoc,
|
||||
and concurrency tests. This needs a deliberate public-contract decision before
|
||||
v1.0: an immediate copy alone cannot guarantee safety if the reader is allowed
|
||||
to mutate the buffer immediately. No default-reader defect was established.
|
||||
|
||||
Sources: [Interfaces 1](09-interfaces.md#1-artifactreader-permits-buffer-reuse-before-promptkit-can-copy-it).
|
||||
|
||||
## 5. Preserve Exact Numbers In Programmatic Provider Sources
|
||||
|
||||
**Confirmed data-integrity defect.** Convenience source conversion falls back
|
||||
from integer parsing to `float64` without checking precision, silently changing
|
||||
some accepted `json.Number` values. Preserve an exact ConfigKit-compatible
|
||||
representation, or defer an unsupported-value error to ConfigKit instead of
|
||||
rounding or converting the number into a string. Regression coverage should
|
||||
include exact-number boundaries, nested typed collections, mutation isolation,
|
||||
and deferred unsupported/cyclic-value errors for both source helpers.
|
||||
|
||||
Sources: [Correctness 1](01-correctness.md#1-provider-source-helpers-silently-change-exact-json-numbers),
|
||||
[Testing 3](08-testing.md#3-programmatic-provider-value-conversion-lacks-representative-value-cases).
|
||||
|
||||
## 6. Make Provider-Native Schema Constraints Self-Contained
|
||||
|
||||
**Confirmed implementation gap; packaging policy needs selection.** Local
|
||||
validation freezes referenced schema resources, but the provider receives only
|
||||
the root document and cannot resolve its local external references. Select
|
||||
correct resource packaging, or an explicit local-only fallback policy, without
|
||||
naively inlining recursive schemas or silently dropping native constraints.
|
||||
Add a wire-level regression demonstrating a usable provider constraint, not
|
||||
merely successful local compilation. Entry 21 is a separate compilation
|
||||
refactor and does not solve this packaging problem.
|
||||
|
||||
Sources: [Correctness 2](01-correctness.md#2-external-schema-resources-do-not-reach-the-provider-constraint),
|
||||
[Testing 5](08-testing.md#5-multi-resource-schema-tests-do-not-establish-a-usable-provider-constraint).
|
||||
|
||||
## 7. Preserve The Owning Operation's Public Error Category
|
||||
|
||||
**Confirmed classification defect.** Public error conversion searches nested
|
||||
causes in a global priority order, allowing a collaborator's sentinel to hide
|
||||
the enclosing operation's category. Map the owning boundary independently
|
||||
while retaining the original cause for `errors.Is`/`errors.As`. Add composed
|
||||
cause regressions; simply reordering global checks can break legitimate direct
|
||||
not-found versus load-failure distinctions.
|
||||
|
||||
Sources: [Error handling 1](06-error-handling.md#1-nested-collaborator-errors-can-hide-the-owning-operations-public-category),
|
||||
[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures).
|
||||
|
||||
## 8. Distinguish Response Stream Failures From Malformed JSON
|
||||
|
||||
**Confirmed error-handling defect.** Non-context I/O failures while decoding a
|
||||
successful HTTP response or checking its end are reported as malformed JSON,
|
||||
losing the transport cause; complete JSON followed by a read failure can also
|
||||
be misreported as trailing data. Retain and classify reader failures separately
|
||||
using the existing redacted transport boundary. Preserve cancellation
|
||||
precedence and genuine malformed/truncated/oversized-content handling. Cover
|
||||
both read locations without adding transport retries.
|
||||
|
||||
Sources: [Error handling 2](06-error-handling.md#2-successful-status-body-read-failures-are-misreported-as-malformed-json),
|
||||
[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures).
|
||||
|
||||
## 9. Return Invalid Diagnostic Request Construction As A Go Error
|
||||
|
||||
**Confirmed diagnostic-contract defect.** Some invalid inference settings fail
|
||||
before dispatch yet produce a completed report with a nil Go error. Propagate
|
||||
local request-construction failures through `ErrInvalidRequest` and add a
|
||||
no-dispatch regression. Preserve intentionally completed diagnostic outcomes
|
||||
for missing credentials, capacity outcomes, and unsupported or inapplicable
|
||||
operations; not every no-dispatch outcome is invalid configuration.
|
||||
|
||||
Sources: [Error handling 3](06-error-handling.md#3-inference-diagnostics-swallow-some-invalid-request-construction-errors),
|
||||
[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures).
|
||||
|
||||
## 10. Remove Quadratic Requirement-Evidence Union Work
|
||||
|
||||
**Confirmed algorithmic inefficiency.** A left-heavy recognized condition can
|
||||
repeatedly copy growing missing-key evidence sets, producing quadratic work.
|
||||
Use operation-local accumulation or ownership-aware merging while preserving
|
||||
classification and conservative inspection coverage. Do not address this by
|
||||
adding expression limits or expanding the recognized template language. Entry
|
||||
23's traversal cleanup is related but is not a substitute for fixing this
|
||||
specific accumulation algorithm.
|
||||
|
||||
Sources: [Efficiency 1](02-efficiency.md#1-condition-evidence-unions-can-require-quadratic-work).
|
||||
|
||||
## 11. Exercise RunRequest's Privacy Formatting Boundary
|
||||
|
||||
**Missing protection, not a demonstrated leak.** Request formatting methods
|
||||
have no execution coverage despite owning a privacy-sensitive public boundary.
|
||||
Add focused tests with distinct synthetic private markers for the relevant
|
||||
value/pointer formatting paths and JSON omission behavior. Assert privacy and
|
||||
the documented boundary, not an incidental complete diagnostic or JSON shape.
|
||||
|
||||
Sources: [Testing 1](08-testing.md#1-runrequests-privacy-formatting-contract-is-not-exercised).
|
||||
|
||||
## 12. Correct Profile Inspection Credential Documentation
|
||||
|
||||
**Public-contract documentation defect.** Inspection GoDoc incorrectly makes
|
||||
required credentials and an environment variable mutually exclusive. Actual
|
||||
behavior permits an explicit profile environment variable while suppressing
|
||||
only the backend-derived default when required credentials are selected.
|
||||
Correct the field/method GoDoc and consumer example, keeping the established
|
||||
configuration behavior and DTO shape.
|
||||
|
||||
Sources: [Interfaces 2](09-interfaces.md#2-profile-inspection-misstates-required-credential-semantics).
|
||||
|
||||
## 13. Correct The Configured Provider-Map Wire Documentation
|
||||
|
||||
**Integration-contract documentation defect.** The wire document says backend
|
||||
and profile provider maps replace one another, but configuration recursively
|
||||
overlays them; runtime replacement is a separate rule. Align the integration
|
||||
summary with the canonical format reference rather than maintaining a second
|
||||
conflicting precedence definition. Entry 19 concerns the efficiency of the
|
||||
existing overlay, not a change to its semantics.
|
||||
|
||||
Sources: [Interfaces 3](09-interfaces.md#3-the-wire-contract-incorrectly-says-configured-provider-maps-replace).
|
||||
|
||||
## 14. Remove Stale Identity And Version GoDoc
|
||||
|
||||
**Public-contract documentation defect.** Inspection comments still describe
|
||||
trimmed profile IDs, and listing comments describe a removed prompt-version
|
||||
field. Update them to the implemented literal identity and current collection
|
||||
version model. Do not normalize meaningful whitespace or reintroduce a field
|
||||
to match obsolete comments.
|
||||
|
||||
Sources: [Interfaces 4](09-interfaces.md#4-identity-godoc-still-describes-removed-normalization-and-version-fields).
|
||||
|
||||
## 15. Cover The Public Single-Schema-File Option
|
||||
|
||||
**Public wiring coverage gap.** Direct filesystem-validator tests do not
|
||||
exercise `WithSchemaFile` through engine construction. Add offline temporary
|
||||
file cases through public preparation/validation, including option precedence
|
||||
and relevant file errors. This is a supported feature, not dead code; entry
|
||||
32 removes only an unused private helper result.
|
||||
|
||||
Sources: [Testing 2](08-testing.md#2-the-public-single-schema-file-option-has-no-execution-coverage).
|
||||
|
||||
## 16. Detach Inert Prepared Handles From The Execution Owner
|
||||
|
||||
**Avoidable retention.** Used or discarded prepared handles retain their
|
||||
runner, and therefore the engine's captured configuration and collaborators.
|
||||
Use a narrower ownership identity or clear the unnecessary owner reference
|
||||
safely. Preserve foreign-owner rejection, one-time claim, namespace-view
|
||||
behavior, and available details. This is unnecessary retention when a handle
|
||||
outlives its engine, not evidence of an ever-growing global leak.
|
||||
|
||||
Sources: [Resources 3](05-resources.md#3-inert-prepared-handles-retain-their-entire-execution-owner).
|
||||
|
||||
## 17. Bound Repair Feedback's Temporary Serialization
|
||||
|
||||
**Avoidable peak allocation.** Feedback formatting serializes potentially
|
||||
large diagnostic text before truncating the final message, so its existing
|
||||
output bound does not bound scratch memory. Produce bounded encoded feedback
|
||||
incrementally, preserving the existing output semantics and complete stored
|
||||
validation diagnostics and previous candidate. A new raw-diagnostic input
|
||||
limit is not implied.
|
||||
|
||||
Sources: [Resources 4](05-resources.md#4-repair-feedbacks-output-bound-does-not-bound-its-temporary-serialization).
|
||||
|
||||
## 18. Detach Normalized Session IDs From Large Input Strings
|
||||
|
||||
**Avoidable retention.** A short trimmed session ID can retain the allocation
|
||||
of a much larger padded caller string. Clone the accepted normalized value at
|
||||
its ownership boundary. Preserve trimming, accepted content, code-point
|
||||
validation, and resulting wire/hash content rather than adding a raw-input
|
||||
size restriction.
|
||||
|
||||
Sources: [Resources 5](05-resources.md#5-short-normalized-session-ids-can-pin-large-input-strings).
|
||||
|
||||
## 19. Avoid Repeated Descendant Cloning During Provider Overlay
|
||||
|
||||
**Construction-time inefficiency.** Recursive overlays repeatedly clone and
|
||||
sort overlapping descendant maps, giving work proportional to tree size times
|
||||
depth. Existing depth bounds mean this is not unbounded quadratic growth.
|
||||
Clone ownership once or use a single traversal while retaining recursive-map,
|
||||
null, and array semantics. Coordinate with entry 24 without assuming all JSON
|
||||
walkers have interchangeable policies.
|
||||
|
||||
Sources: [Efficiency 2](02-efficiency.md#2-recursive-provider-overlay-repeatedly-clones-descendant-subtrees).
|
||||
|
||||
## 20. Index Filtered Listings By Namespace
|
||||
|
||||
**Repeated inspection overhead.** Filtered listing currently copies and scans
|
||||
the complete inventory even when few or no entries match. Capture namespace
|
||||
buckets or equivalent ranges so queries operate on the relevant subset.
|
||||
Preserve ordering, exact identity matching, cancellation, and caller-owned
|
||||
results. A pagination API or runtime cache is not needed for this improvement.
|
||||
|
||||
Sources: [Efficiency 3](02-efficiency.md#3-filtered-listings-copy-and-scan-the-complete-inventory).
|
||||
|
||||
## 21. Share Schema Compilation And Declare The Runner's Real Dependency
|
||||
|
||||
**Maintainability and interface clarity.** Four host-file/`fs.FS`,
|
||||
direct/prepared paths repeat the schema compilation lifecycle. The runner also
|
||||
declares a validator interface while requiring preparation through a runtime
|
||||
assertion. Share the lifecycle behind source-specific adapters and make that
|
||||
dependency explicit. Preserve malformed-output short-circuiting before schema
|
||||
I/O, source containment, exact numbers, cancellation, and frozen plans. Retain
|
||||
independent cross-source tests; do not force whole entry points to be identical.
|
||||
|
||||
Sources: [Complexity 1](03-complexity.md#1-consolidate-schema-compilation-and-express-the-runners-actual-interface),
|
||||
[Equivalence 1](10-equivalence.md#1-share-the-schema-compilation-lifecycle-across-source-and-execution-paths).
|
||||
|
||||
## 22. Decompose Configuration Validation And Materialization
|
||||
|
||||
**Maintainability.** Large configuration operations mix record-specific
|
||||
validation, resource capture, and effective-profile construction. Extract
|
||||
focused helpers with visible inputs and ownership, rather than another generic
|
||||
validation framework. Preserve final-handoff validation, presence/null rules,
|
||||
origins, and meaningful fail-fast/error-precedence behavior. Do not turn
|
||||
incidental ordering of independent validation failures into a new guarantee. The configuration-boundary ADR remains
|
||||
authoritative; this is not a proposal to inspect raw ConfigKit sources.
|
||||
|
||||
Sources: [Complexity 2](03-complexity.md#2-decompose-configuration-validation-and-resource-materialization).
|
||||
|
||||
## 23. Clarify Requirement Traversal And Share Branch Orchestration
|
||||
|
||||
**Maintainability.** Separate recognition/evaluation from reachability and
|
||||
evidence bookkeeping, particularly the repeated `if`/`with` branch protocol.
|
||||
Keep their scope differences explicit and retain independent comparisons with
|
||||
the actual Go renderer. Preserve conservative partial coverage and inactive
|
||||
computed-name behavior. This does not justify a general template interpreter
|
||||
or stronger execution-readiness promises from requirements inspection.
|
||||
|
||||
Sources: [Complexity 3](03-complexity.md#3-untangle-requirement-traversal-expression-evaluation-and-evidence),
|
||||
[Equivalence 3](10-equivalence.md#3-share-ifwith-requirement-branch-orchestration-not-template-execution).
|
||||
|
||||
## 24. Centralize Compatible JSON Copying Without Conflating Policies
|
||||
|
||||
**Maintainability and ownership clarity.** The facade maintains a separate
|
||||
reflection traversal for already-normalized values despite an existing
|
||||
`internal/jsonvalue` owner. Consolidate genuinely compatible structural
|
||||
copying while retaining DTO projection at the facade and distinct source
|
||||
encoding/runtime validation policies. Preserve concrete types, nils, exact
|
||||
numbers, schema keys, and ownership. Infallible projection must not acquire
|
||||
new errors, panics, or silent zero-value fallbacks. Entries 3 and 5 remain
|
||||
separate source-conversion defects, not reasons to reject pre-handoff sources.
|
||||
|
||||
Sources: [Complexity 4](03-complexity.md#4-clarify-ownership-and-policies-across-the-json-value-walkers),
|
||||
[Equivalence 2](10-equivalence.md#2-centralize-structural-copying-of-already-normalized-json-values).
|
||||
|
||||
## 25. Separate Attempt Accounting From Generation Control Flow
|
||||
|
||||
**Maintainability.** Generation coordination interleaves dispatch/retry policy
|
||||
with parallel return values and repeated evidence/observation projection.
|
||||
Introduce a small attempt result and focused accounting helpers. Preserve
|
||||
built-in versus injected-client differences, permit release before callbacks
|
||||
or backoff, frozen requests, existing retry classification, private evidence,
|
||||
and known/unknown usage distinctions. Do not collapse legitimately different
|
||||
dispatch strategies merely to reduce lines.
|
||||
|
||||
Sources: [Complexity 5](03-complexity.md#5-extract-attempt-accounting-from-generation-control-flow).
|
||||
|
||||
## 26. Move OpenRouter Authentication Protocol Details Into Its Adapter
|
||||
|
||||
**Architectural ownership.** Use-case code currently interprets OpenRouter's
|
||||
account-response fields and provider-relative endpoint while the outbound
|
||||
adapter owns HTTP execution. Place protocol construction/recognition with the
|
||||
outbound integration; keep diagnostic selection, applicability, cancellation,
|
||||
completion, and neutral reporting in the use case. Preserve credential/body
|
||||
privacy and anonymous probing. No plugin framework, reverse dependency, or
|
||||
blanket prohibition on HTTP facts is needed. Coordinate the body-bound fix
|
||||
in entry 2, which is a distinct behavioral change.
|
||||
|
||||
Sources: [Boundaries 2](11-boundaries.md#2-put-openrouter-authentication-protocol-details-behind-the-outbound-adapter).
|
||||
|
||||
## 27. Narrow Smoke-Value Copy Helpers To Real Consumers
|
||||
|
||||
**Ownership cleanup.** A request cloner has only its self-test as a caller;
|
||||
the full-report cloner is used to detach only a target. Express that actual
|
||||
copy operation directly and retire unused broader cloning after checking
|
||||
callers. Keep facade request/report projection, active backend cloning,
|
||||
provider-map errors, ownership tests, and redaction. Do not introduce shared
|
||||
public/internal DTOs or an additional full-report copy.
|
||||
|
||||
Sources: [Boundaries 3](11-boundaries.md#3-narrow-smoke-value-copy-helpers-to-their-actual-ownership-boundaries).
|
||||
|
||||
## 28. Give Bundled Catalog Expectations One Test Owner
|
||||
|
||||
**Test duplication.** Complete bundled profile/model/provider expectations are
|
||||
repeated across several test layers. Keep one independently specified asset
|
||||
oracle, with smaller tests for distinct loading, wire, and diagnostic behavior.
|
||||
Use controlled fixtures where bundled churn is irrelevant and structural
|
||||
assertions rather than formatted-map strings. Retain meaningful bundled
|
||||
coverage instead of deriving every expected value from the loader under test.
|
||||
|
||||
Sources: [Testing 7](08-testing.md#7-bundled-catalog-expectations-have-multiple-test-owners).
|
||||
|
||||
## 29. Stop Pinning The Opaque Artifact Hash Algorithm In Tests
|
||||
|
||||
**Over-specified tests.** Artifact tests and a benchmark assert a literal hash
|
||||
algorithm despite the public value being opaque. Assert the documented
|
||||
equality/ownership relationships and actual byte/size behavior instead; keep
|
||||
benchmark setup outside its timed work. Coordinate with entry 31, but do not
|
||||
turn today's hash representation into a new compatibility guarantee.
|
||||
|
||||
Sources: [Testing 6](08-testing.md#6-artifact-tests-and-benchmarks-pin-an-explicitly-opaque-hash-algorithm).
|
||||
|
||||
## 30. Remove Unconsumed Backend Presence Bookkeeping
|
||||
|
||||
**Low-priority state cleanup.** Backend field-state maps and
|
||||
`QueueCapacitySet` have no runtime reader; effective values and ConfigKit
|
||||
already own queue defaulting and provenance. Remove only these unused records
|
||||
and adjust tests to assert effective behavior or operator inspection. Keep
|
||||
profile/input presence tracking, explicit zero capacity, and derived defaults.
|
||||
|
||||
Sources: [Dead code 1](12-deadcode.md#1-remove-backend-presence-bookkeeping-with-no-runtime-consumer).
|
||||
|
||||
## 31. Retire Duplicate Fields In Captured Prompt Definitions
|
||||
|
||||
**Low-priority representation cleanup.** Captured definitions duplicate the
|
||||
output format and carry an unused runtime message-file field after content
|
||||
materialization. Use the captured output contract and text as their respective
|
||||
sources of truth. Preserve real YAML `content_file` loading, origins, and the
|
||||
request-override default rule. Removing serialized internal fields changes
|
||||
current definition hashes: retain documented opaque equality relationships,
|
||||
not an invented bit-identical encoding promise.
|
||||
|
||||
Sources: [Dead code 2](12-deadcode.md#2-retire-legacy-fields-in-the-captured-prompt-representation).
|
||||
|
||||
## 32. Remove Unused Private Diagnostic State And Helper Results
|
||||
|
||||
**Low-priority cleanup.** A private reachability outcome echoes an endpoint
|
||||
that report assembly never reads; the schema-file helper returns a directory
|
||||
that its only caller discards. Remove the unused state/result while preserving
|
||||
actual sanitized request/report evidence, directory calculation, file checks,
|
||||
and public single-file schema support. Keep privacy tests against the values
|
||||
that real execution consumes.
|
||||
|
||||
Sources: [Dead code 3](12-deadcode.md#3-remove-unused-diagnostic-state-and-a-discarded-helper-result).
|
||||
|
||||
## 33. Remove Orphan Internal Helpers And Their Self-Tests
|
||||
|
||||
**Low-priority cleanup.** `filecatalog.RelativePath` and
|
||||
`resourceNamespaceFromID` have only their own tests as callers, and the runner
|
||||
test's `stringPtr` helper is entirely unused. Remove those narrow surfaces
|
||||
without replacement, preserving real discovery, identity qualification, and
|
||||
namespace inventory/filtering coverage. Reachability from examples alone is
|
||||
not a reason to remove optional public APIs.
|
||||
|
||||
Sources: [Dead code 4](12-deadcode.md#4-remove-orphan-helpers-whose-only-callers-are-their-own-tests).
|
||||
|
||||
## 34. Decide Whether To Retain The Legacy Capacity Client Wrapper
|
||||
|
||||
**Decision-dependent architectural cleanup.** The old capacity client wrapper
|
||||
is used by tests, whereas production dispatch uses the generation coordinator.
|
||||
It is also explicitly retained in current internal documentation; the dead-code
|
||||
audit does not classify it as obsolete functionality. If that retention decision
|
||||
is revised, exercise the production coordinator in runner tests and move any
|
||||
needed one-shot capacity scaffolding into tests before retiring the wrapper.
|
||||
Preserve FIFO, cancellation, independent-pool, and panic-release coverage.
|
||||
This is not a second scheduling algorithm or a confirmed concurrency defect.
|
||||
|
||||
Sources: [Boundaries 1](11-boundaries.md#1-retire-the-test-only-capacity-client-as-a-production-scheduling-surface),
|
||||
[Dead-code retention qualification](12-deadcode.md#intentional-retention-and-related-findings).
|
||||
|
||||
## Open Decisions
|
||||
|
||||
Work through the decisions below in this order. These are unresolved choices,
|
||||
not additional findings or approved changes to public contracts. A decision
|
||||
blocks its dependent remediation, not unrelated fixes. In particular, updating
|
||||
the maintainer toolchain need not wait for the consumer-minimum decision.
|
||||
Implementation details and optional cleanup do not become release gates merely
|
||||
by appearing in this catalog.
|
||||
|
||||
### Decision 1: ArtifactReader Result Ownership And Lifetime
|
||||
|
||||
**Related finding:** [4](#4-resolve-the-artifactreader-buffer-ownership-contract).
|
||||
|
||||
**Question:** What ownership does a successful `Read` transfer, and when may
|
||||
an injected reader mutate or reuse the returned storage?
|
||||
|
||||
**Recommendation:** Transfer an independent result to PromptKit on successful
|
||||
return. The reader must stop mutating the returned artifact and its body,
|
||||
including through retained aliases. Fresh per-invocation storage is the simplest
|
||||
implementation; immutable shared data can remain shared if no collaborator
|
||||
mutates it. This removes the impossible immediate-reuse permission without
|
||||
adding synchronization or a release protocol to the interface. Explain the
|
||||
consumer impact as a deliberate pre-1.0 contract correction.
|
||||
|
||||
**Alternative:** Permit borrowing until the enclosing preparation/execution
|
||||
operation returns. This can support application-managed reuse, but a reader
|
||||
cannot generally observe that lifetime and concurrent calls need independent
|
||||
storage or caller coordination. If reuse immediately after `Read` returns is
|
||||
required, an explicit acknowledgement/release API is another viable design;
|
||||
it enables pooling at the cost of a larger interface and lifecycle change.
|
||||
|
||||
**Resolve before implementation:** Select the ownership/lifetime rule and update
|
||||
GoDoc, adapter assumptions, and the regression scenario together. Another
|
||||
unsynchronized copy is not a remedy for the current permission.
|
||||
|
||||
### Decision 2: Provider Submission Of Multi-Resource Schemas
|
||||
|
||||
**Related findings:** [6](#6-make-provider-native-schema-constraints-self-contained)
|
||||
and [21](#21-share-schema-compilation-and-declare-the-runners-real-dependency).
|
||||
|
||||
**Question:** How should native structured output represent a schema whose
|
||||
local validation depends on external resources?
|
||||
|
||||
**Recommendation:** First investigate packaging the operation's captured
|
||||
resource graph into a self-contained provider constraint. Check identifiers,
|
||||
relative references, anchors, recursion, and supported dialects before choosing
|
||||
an encoding. Prefer correct packaging when it fits a proportionate scope,
|
||||
while keeping resource completeness distinct from any provider's supported
|
||||
schema subset. A standalone offline validation of the actual emitted constraint
|
||||
should establish the selected packaging behavior.
|
||||
|
||||
**Alternative:** Offer an explicit local-only validation choice when native
|
||||
submission cannot represent the resource graph. This preserves useful local
|
||||
validation and avoids building a general schema transformation system, but
|
||||
forgoes native generation constraints for that selected mode and needs a clear
|
||||
consumer-facing selection mechanism. Silently dropping native constraints is
|
||||
not equivalent to an explicit choice.
|
||||
|
||||
**Resolve before implementation:** Establish packaging feasibility and select
|
||||
the submission/fallback policy, including behavior for unsupported cases.
|
||||
Coordinate the compilation refactor with that result; it does not itself fix
|
||||
provider packaging or establish universal provider acceptance.
|
||||
|
||||
### Decision 3: Authentication Response Budget And Overflow Outcome
|
||||
|
||||
**Related findings:** [2](#2-bound-authentication-probe-response-consumption)
|
||||
and [26](#26-move-openrouter-authentication-protocol-details-into-its-adapter).
|
||||
|
||||
**Question:** What finite response budget should the authentication adapter
|
||||
apply, and how should a probe report exceeding it?
|
||||
|
||||
**Recommendation:** Select a conservative fixed budget for this small protocol,
|
||||
check actual streamed bytes regardless of declared length, and close on overflow
|
||||
without unbounded draining. Report the diagnostic failure with acceptance
|
||||
unknown, retaining any independently observed HTTP facts. Agree the exact byte
|
||||
count and its representation in the existing diagnostic report before coding;
|
||||
the catalog does not select either implicitly.
|
||||
|
||||
**Alternative:** Expose a caller-configurable budget with a finite default.
|
||||
This accommodates unusual legitimate responses but expands configuration and
|
||||
validation surface. It is worthwhile if a concrete consumer requirement cannot
|
||||
be served by an appropriately chosen fixed budget; it is not necessary merely
|
||||
to address the demonstrated unbounded read.
|
||||
|
||||
**Resolve before implementation:** Choose the size, overflow classification,
|
||||
and whether configurability is warranted. Keep this fix independent of adapter
|
||||
relocation so architectural cleanup does not delay the resource remedy.
|
||||
|
||||
### Decision 4: Exact Numbers And The Source-Conversion Boundary
|
||||
|
||||
**Related findings:** [3](#3-avoid-exponential-expansion-during-convenience-source-conversion),
|
||||
[5](#5-preserve-exact-numbers-in-programmatic-provider-sources), and
|
||||
[24](#24-centralize-compatible-json-copying-without-conflating-policies).
|
||||
|
||||
**Question:** Which exact numeric values can the convenience helpers preserve
|
||||
through the current ConfigKit mapping boundary, and what happens to others?
|
||||
|
||||
**Recommendation:** Preserve supported values through the complete conversion,
|
||||
load, and outbound-encoding path; defer an unsupported-value error when exact
|
||||
preservation is unavailable. Compare numeric meaning rather than spelling or
|
||||
successful parsing alone. Address shared-graph expansion in the same adapter
|
||||
with completed-conversion memoization, distinct cycle tracking, and correct
|
||||
slice-view identity, preserving source capture and deferred failures.
|
||||
|
||||
**Alternative:** Request upstream support for additional exact numeric/value
|
||||
forms, then simplify or remove the relevant PromptKit conversion. This may
|
||||
provide a cleaner long-term boundary and broader preservation, but introduces
|
||||
an upstream dependency and should not postpone fixing silent rounding or
|
||||
exponential expansion. Runtime JSON copying remains a separate policy.
|
||||
|
||||
**Resolve before implementation:** Establish the supported numeric round-trip
|
||||
cases and whether broader upstream support is needed now. Do not substitute
|
||||
numeric-looking strings or introduce a second raw-source validation system.
|
||||
|
||||
### Decision 5: Consumer Go Minimum After The Toolchain Update
|
||||
|
||||
**Related finding:** [1](#1-update-the-audited-maintainer-toolchain-for-applicable-security-fixes).
|
||||
|
||||
**Question:** Should the security fixes also change PromptKit's declared minimum
|
||||
consumer Go version?
|
||||
|
||||
**Recommendation:** Update the maintainer toolchain and repeat the focused
|
||||
assessment promptly; initially keep the consumer minimum decision separate.
|
||||
Advise affected consumers to rebuild with a patched toolchain. This recognizes
|
||||
that PromptKit publishes source and that downstream build/transport choices
|
||||
control the deployed exposure.
|
||||
|
||||
**Alternative:** Raise the declared minimum to the selected patched release.
|
||||
This makes the supported floor clearer and steers module consumers toward the
|
||||
fixed toolchain, but changes compatibility and does not repair already-built
|
||||
applications. Assess it under the existing release policy rather than treating
|
||||
it as an automatic consequence of the scanner output.
|
||||
|
||||
**Resolve before changing the minimum:** Confirm current advisory applicability,
|
||||
select the patched version, and decide whether changing the supported consumer
|
||||
floor is justified. No new recurring release gate is implied.
|
||||
|
||||
### Decision 6: Retention Of The Legacy Capacity Client Wrapper
|
||||
|
||||
**Related finding:** [34](#34-decide-whether-to-retain-the-legacy-capacity-client-wrapper).
|
||||
|
||||
**Question:** Does the wrapper still justify a production surface for its
|
||||
currently documented test-focused purpose?
|
||||
|
||||
**Recommendation:** Retire the production wrapper after moving the runner's
|
||||
integration test to the production generation coordinator. Keep direct manager
|
||||
coverage and any useful one-shot scaffolding in tests. This reduces maintenance
|
||||
and makes assembled scheduling tests reflect production while preserving the
|
||||
actual FIFO, cancellation, independent-pool, and panic-release scenarios.
|
||||
|
||||
**Alternative:** Retain it if a concrete internal use justifies the small
|
||||
adapter. This avoids test migration and preserves that convenient composition,
|
||||
but keeps a second assembly surface to understand and maintain. Existing
|
||||
self-tests alone do not establish a future production need.
|
||||
|
||||
**Resolve before removal:** Confirm the intended retention decision and test
|
||||
ownership. This cleanup can be deferred without blocking the behavioral fixes;
|
||||
it is not evidence of a second scheduler or a concurrency defect.
|
||||
Reference in New Issue
Block a user