diff --git a/docs/roadmap/correction-repair.md b/docs/roadmap/correction-repair.md index b21fda6..8a38220 100644 --- a/docs/roadmap/correction-repair.md +++ b/docs/roadmap/correction-repair.md @@ -109,10 +109,9 @@ for separately. Do not manufacture hidden provider messages or reasoning. Keep required conversation state in memory when repair is enabled, independently of debug artifact saving. Its lifetime need only cover the associated work; this -does not introduce durable conversation storage or restart/resume support. Count -the complete replay, feedback, and applicable output contract when assessing -request size. Apply the accepted -[context-limit behavior](#full-history-that-does-not-fit). +does not introduce durable conversation storage or restart/resume support. Send +the full replay and feedback under the revision output contract, subject to the +selected provider's [request limits](#provider-request-limits). ### Module Repair Prompts and Response Contract @@ -200,9 +199,10 @@ Use the accepted [reporting scope](#ordinary-reports-and-attempt-history). `framework/responseschema`. Provide revision/withdrawal decoding and correspondence checks without changing ordinary span-deletion semantics. 4. **Adapt feedback collection.** Update validator findings where their current - messages are insufficient for repair. Change runner filtering to collect the - agreed applicable findings, distinguishing failures, unavailable decisions, - and checks that could not run. Keep complete validation mandatory for approval. + messages are insufficient for repair. When repair is enabled, collect the + agreed applicable findings and distinguish failures, unavailable decisions, + and checks that could not run. With an effective budget of zero, use the + existing validator filtering. Keep complete validation mandatory for approval. 5. **Implement the bounded branch loop.** Integrate revision calls and full revalidation into the current stage/section flow. Preserve accepted proposals, branch isolation, deterministic application, cancellation, and existing @@ -241,8 +241,8 @@ boundaries. Cover the following behavior with proportionate tests: containing only its own branch's history. Exercise debug enabled and disabled. - Repair success, explicit withdrawal, budget exhaustion, the default of one semantic attempt, additional configured attempts, and global or stage-level - zero disabling repair. Distinguish withdrawal from an empty-string deletion - proposal. + zero disabling both repair and extra feedback collection. Distinguish + withdrawal from an empty-string deletion proposal. - Validator-owned findings collected under the accepted continuation policy; inapplicable or unavailable checks do not masquerade as editorial rejection. Cover known rejection alongside an unavailable assessment, unavailable @@ -255,18 +255,17 @@ boundaries. Cover the following behavior with proportionate tests: - A revision can correct its source span or segment reference within the original request's section, preserving proposal lineage and passing ordinary target mapping and full validation. -- Context-size handling, provider failure, structural-repair exhaustion, and - cancellation follow the accepted policies while preserving existing - partial-success and fatal-error boundaries. Account for semantic attempts, - structural repairs, and transport retries without conflating them. - An oversized conversation ends only the affected branch with a reported size - failure, preserving unrelated accepted work. +- A provider error on a repair request, including a context-size rejection, + fails only that proposal after any applicable PromptKit retries. Other approved + work remains eligible for application. Cover structural-repair exhaustion and + cancellation under the existing failure boundaries. Account for semantic + attempts, structural repairs, and transport retries separately. - Embedded module repair prompts and operator replacements work through the shared engine. Root and stage settings follow the accepted scope and fallback rules, including inheritance of the effective proposal profile after stage - settings are combined and explicit repair-profile overrides. Reports identify - attempts and final outcomes without presenting every attempt as a separate - applied correction or exposing raw history by default. + settings are combined and explicit repair-profile overrides. Verify that + ordinary reporting records one compact outcome per repair branch and that + optional redacted debug output carries the detailed attempt history. Run focused checks during development and assembled application validation at completion under the [testing policy](../policy/testing.md). Representative @@ -281,23 +280,24 @@ evidence that a revised correction is acceptable. ### Semantic Repair Budget and Configuration Scope Allow one additional semantic repair attempt per rejected proposal by default, -configurable at the root and per stage; zero disables the loop. A revision that -fails validation can continue when the operator configures -more attempts. Use a distinct setting from structural `llm.repair_attempts`. +configurable at the root and per stage. Zero disables both the repair loop and +extra feedback collection. A revision that fails validation can continue when +the operator configures more attempts. Use a distinct setting from structural +`llm.repair_attempts`. Retain existing per-call timeouts and parent cancellation, and document that each semantic attempt may include transport retries and structural repairs. This provides useful feedback-driven revision with a simple finite bound. The default and scope are accepted; exact configuration names remain an implementation-planning choice. -### Full History That Does Not Fit +### Provider Request Limits -Account for the complete conversation and feedback using the supported -request-sizing facilities; if a repair cannot fit the selected -limit, end that branch with a reported size failure and keep unrelated accepted -work. Preserve existing context-estimation limits rather than promising exact -provider token accounting. This honors the agreed full-history design without -silently truncating context or changing the execution profile. +Submit the full repair conversation and let the selected provider enforce its +request limits. If a provider error prevents revision after any applicable +PromptKit retries, including when the conversation exceeds the provider's context +window, record the repair as unavailable and end that proposal's branch. +Continue processing other proposals. Audita's existing fatal application error +and cancellation behavior still applies. ### Repair Execution Profile @@ -309,9 +309,11 @@ difficult revision. Backend capacity remains shared through PromptKit. ### Feedback Eligibility and Unavailable Validation -Collect all meaningful deterministic failures, then run the applicable semantic -validators when the candidate has the usable target and -before/after data they need, even if another independent check rejected it. +When repair is enabled, collect all meaningful deterministic failures, then run +the applicable semantic validators when the candidate has the usable target and +before/after data they need, even if another independent check rejected it. With +an effective repair budget of zero, later validators receive only the candidates +that advance under the current validation flow. An actionable rejection may enter repair; no-effect proposals can finish without repair because they contribute no edit. Build a small applicability map from the actual checks rather than running validators on unusable inputs or encoding @@ -327,16 +329,16 @@ as evidence against the proposed wording. ### Ordinary Reports and Attempt History -Retain compact proposal lineage, attempt counts, and final dispositions in -ordinary results, with meaningful errors and recoveries in the -existing report framework. Distinguish approval, actual application, withdrawal, -and exhausted or unavailable repair. Keep full conversations and detailed -per-attempt evidence in optional redacted debug artifacts. A rejected proposal -or voluntary withdrawal is not automatically an operational error. This makes -outcomes explainable without turning ordinary reports into conversation dumps. -Choose concrete fields and any output-schema version change during implementation -planning, preserving the existing failure and exit-status policy unless an -explicitly identified requirement calls for a change. +Use one compact repair-outcomes section in ordinary reporting. Give each repair +branch one entry with the original proposal reference, its attempt count, and +final disposition. Use the existing work-unit context for stage and chunk +identity. Make it clear whether an approved revision was applied. Record +operational errors and recoveries through the existing report framework and keep +the current exit-status policy. +Editorial rejection and voluntary withdrawal are correction outcomes. Keep +conversation text and per-attempt details in optional redacted debug artifacts. +Choose the report location, fields, and any schema version change during +implementation planning. ### Revising the Span or Target Reference