Resolve native correction prompts and validator guidance ownership

This commit is contained in:
2026-11-20 14:42:26 +00:00
parent 2d69c26813
commit b904021138

View File

@@ -145,8 +145,9 @@ CommandKit reporters. Pure domain packages retain artifact models, codecs,
reference resolution, and deterministic rules. `internal/preflight` retains its
separate known-dependency checks. Exact argument grouping and helper signatures
can be refined against representative extraction and normalization during
implementation; ownership and placement are settled. Role-specific prompts,
budgets, and fallback policy remain U4–U6. Implementation is pending.
implementation; ownership and placement are settled. U4 selects prompt roles and
guidance ownership below; budgets and fallback policy remain U5/U6. Implementation
is pending.
Retain the fixed `input -> chunk -> extract -> merge -> normalize -> output`
shape, ordered steps and generated references, typed artifact-family boundaries,
@@ -335,10 +336,11 @@ consumers disappear; no replacement filesystem abstraction is needed.
Declare response format, validation mode, schema path, and structural-repair
budget in native prompt definitions by default. Modules select prompt IDs, supply
inputs, and interpret responses. Initial and correction prompts may reference
the same schema file. Execution, inspection, and prepared-request preview then
inputs, and interpret responses. U4 selects the same schema file for each
initial/correction pair. Execution, inspection, and prepared-request preview then
use the same declared contract without repeating schema selection in Go.
Role-specific prompt contents remain U4, and retry defaults remain U5.
U4 selects prompt roles and contracts; exact prose is implementation work, while
retry defaults remain U5.
Use a request-level native `OutputContract` only for a demonstrated runtime-
dependent contract. It replaces the whole configured contract, so supply the
@@ -388,6 +390,84 @@ complete replacement model responses. Normalization may continue to return a
reconciliation plan that deterministic code applies to produce the artifact.
Per-entity branching is not part of this migration.
#### Prompt Roles And Correction Guidance (U4 Resolved)
Give each correctable producer an initial native prompt and a domain-specific
correction prompt. Retain existing initial IDs and identify correction IDs
explicitly in the module, for example `notarius/dnd.spells.repair`. Preflight uses
those same declarations to check known prompt dependencies. Shared correction
wording can live in ordinary asset files; distinct domain/operation templates
supply the task and response expectations without a universal application-aware
correction protocol.
The initial request supplies source material, references, and the task. Enduring
domain principles belong in its system instructions. The correction template
supplies a new user assignment with the current candidate's actionable findings;
PromptKit carries the preceding exchange. Do not rerender the original source or
manually replay prior responses. Review initial wording so later correction
assignments remain valid. Include additional application-derived context
only when needed to interpret the findings. Keep source/reference meanings and
normalization candidate handles fixed throughout the conversation.
Use the same declared response schema for each initial/correction pair:
| Role | Response contract |
| --- | --- |
| LLM chunk planning | Complete replacement chunk plan. |
| Extraction | Complete replacement extracted artifact candidate. |
| Normalization | Complete replacement reconciliation plan, applied afresh to the fixed original candidates. |
| LLM validation | The validator's assessment response; no separate semantic correction conversation by default. |
No correction envelope, patch language, or special repair-result type is needed.
PromptKit retains structural validation and repair inside each native execution.
Notarius assesses the response returned by that public boundary, independently
of how many internal attempts produced it.
Use the same native default profile for an initial/correction pair. Preserve
explicit application profile selections and run-wide execution overrides across
both requests. An operator may select a different default profile in a correction
prompt, subject to explicit request overrides. Validators retain their own
profile selection. Pass native selections and overrides without copying resolved
settings or adding a Notarius-wide repair-model configuration layer.
The component that detects a defect owns its actionable correction guidance:
- Each rejecting validator supplies guidance describing the defect and the
constraints a corrected answer must satisfy. A deterministic validator authors
it alongside the check; an LLM validator requests an actionable explanation in
its prompt and returns it through the application validation contract.
- Preserve the explicit `CorrectionGuidance` boundary separately from operator
diagnostics. Guidance need not prescribe an exact replacement, and validators
do not construct the complete correction prompt.
- Module interpretation and normalization safety checks supply their own guidance
when defects arise before an artifact candidate can be materialized.
- The pipeline collects guidance in configured validator order. The module's
domain-specific template renders it into the correction assignment. Neither
the pipeline helper nor the template duplicates validators' defect-detection
logic. Reason codes and operator messages are not automatically instructions.
Assess the same candidate through every applicable configured validator before
requesting correction. A rejection does not short-circuit later validators;
collect all actionable findings they return. Reassess every revised candidate
through the complete applicable chain, including earlier approvals and the final
permitted response. The existing validator-owned guidance and ordered-chain
semantics remain useful; replace the hard-coded `CorrectionRequest` message
builder with native template rendering and retain useful ordered aggregation.
Applicability requires usable input: if interpretation cannot construct a typed
candidate, its module supplies guidance and artifact validators cannot yet run.
Cancellation or fatal execution failures can interrupt assessment. An unavailable
validator assessment is a separate outcome, not a detected defect or an instruction
to change the artifact. Do not convert provider errors into correction guidance;
detailed failure disposition remains U5/U6.
Ownership, response contracts, and default profile behavior are resolved. Refine
exact template prose, correction IDs, and necessary operation-specific inputs
while implementing representative workflows. Implementation is pending; semantic
budgets and terminal/fallback policies remain U5/U6.
#### Execution And Assessment Flow
The application flow is:
1. Prepare source context, references, and candidate-handle mappings once. Build
@@ -397,8 +477,9 @@ The application flow is:
or debug inspection needs one.
2. Consume the native usable response or terminal failure, decode, and apply
module interpretation and safety checks. A module can supply actionable
findings before it has a materialized candidate. Run configured artifact
validation on each candidate.
findings before it has a materialized candidate. Assess each candidate through
all applicable configured validators and collect their actionable guidance
before requesting correction, as selected in U4.
3. For actionable correction, call `Engine.Coordinate` with the captured exchange
and `AdditionalTurns`. Its callback selects a repair prompt and supplies
findings; PromptKit retains the initial exchange and subsequent turns.
@@ -410,12 +491,8 @@ The application flow is:
failure, or supported safe-fallback policy after exhaustion. Native `stopped`
and `exhausted` outcomes do not themselves mean application acceptance.
Repair prompts supply a new user task; enduring principles belong in the original
system instructions. Keep source/reference and candidate-handle meanings stable
throughout the conversation. Avoid replaying the original task and constraints
that permit only an initial response. Accumulated history increases context use;
select the semantic turn budget accordingly. Separate repair profiles or schemas
are supported but not yet selected.
Accumulated history increases context use; select the semantic turn budget
accordingly. Exact budgets remain U5.
Remove `single_response_v1`, raw-response `ModelCandidate`/`SemanticCorrection`
transport, manual assistant/user replay, and correction-protocol capability
@@ -769,7 +846,6 @@ reopen the architecture. IDs retain continuity with the earlier roadmap.
| ID | Remaining choice or dependency | Resolve for |
| --- | --- | --- |
| U1 | Review the message-content hash API/release documented upstream in v0.24.0, including its ConfigKit dependency; verify the final compatible dependency set against the selected baseline. | Track in Phase 1; required to finish cache integration in Phase 5. |
| U4 | Role-specific initial/repair definitions, response contracts, and profile selection within complete replacement and fixed operation inputs. | Phases 3–4. |
| U5 | Producer/validator retry-setting migration to native controls and semantic budgets: field names/defaults, non-LLM failure handling, and any justified recovery beyond native exhaustion. | Phases 3–4 and configuration migration. |
| U6 | Detailed terminal/fallback policy, especially whether an earlier safe candidate survives a later failed turn. No implicit best-so-far acceptance. | Phases 3–4. |
| U7 | Independent worker defaults; application-supplied native backend/queue limits, if any; retired `total_llm` migration and admission-failure handling by role. | Phase 4. |
@@ -779,14 +855,22 @@ reopen the architecture. IDs retain continuity with the earlier roadmap.
**U2 resolved:** Ordinary embedded assets, prompt-declared output contracts,
native schema-source selection, and removal of duplicate schema/filesystem
machinery are selected in [the schema design](#prompt-assets-and-response-schemas-u2-resolved).
Implementation remains in Phase 2; role-specific prompts and retry budgets remain
U4/U5.
Implementation remains in Phase 2; U4 selects prompt roles and contracts, while
retry budgets remain U5.
**U3 resolved:** Typed stage boundaries, composition-root engine construction,
and the shared pipeline producer helper with module functions and a pipeline
assessment callback are selected in [the execution design](#module-execution-and-assessment-u3-resolved).
Implement in Phase 3 and refine signatures during Phase 4 without reopening
ownership. U4–U6 still own role-specific behavior.
ownership. U4 selects prompt/guidance behavior; U5/U6 retain budget and terminal
policy choices.
**U4 resolved:** Domain-specific initial/correction prompt pairs, shared response
schemas, default profile behavior, validator-owned guidance, and complete
applicable-chain assessment are selected in
[the prompt and guidance design](#prompt-roles-and-correction-guidance-u4-resolved).
Implement templates and operation-specific inputs in Phases 3–4; exact prose does
not reopen ownership. Budgets and terminal/fallback policy remain U5/U6.
**U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight,
output, debug policy, and coordinated cutover sequencing are selected above.
@@ -808,8 +892,9 @@ major boundary changes in successor ADRs without rewriting accepted decisions.
Keep the cache dependency explicit without blocking independent work.
- Map affected ADRs, current callers, contracts, and removal inventory. Carry
forward Audita evidence without adopting its application wrappers.
- Preserve U4–U9 with their dependent phases; do not reopen resolved module
execution, assets/schemas, CommandKit, debug, cache, or output ownership.
- Preserve U5–U9 with their dependent phases; do not reopen resolved module
execution, prompt/guidance ownership, assets/schemas, CommandKit, debug, cache,
or output ownership.
**Completion evidence:** reviewed dependency baseline, upstream dependency, scope,
and outstanding choices are explicit. Upstream availability alone does not
@@ -834,9 +919,11 @@ without credentials or synthetic requests. `config diff` handles two selections.
### Phase 3: Prove A Representative Native Pipeline
- Implement the resolved U3 collaboration and settle relevant U4–U6 details.
- Implement the resolved U3/U4 collaboration and settle relevant U5/U6 details.
Connect module request/interpretation functions to the pipeline assessment
callback through the shared producer helper; refine concrete signatures here.
Implement domain-specific initial/correction prompts using the same schema and
default profile, supplied overrides, and validator-owned correction guidance.
- Exercise an LLM validator on the same engine and normalization with fixed
handles and fresh plan application. Select actual observer/group enrollment.
- Run through the new command surface and alias, using the small completion
@@ -844,15 +931,18 @@ without credentials or synthetic requests. `config diff` handles two selections.
Begin U8/U9 integration without a provisional local message hash.
**Completion evidence:** fake-provider workflows exercise accumulated history,
collection of independent validator findings without rejection short-circuiting,
full candidate revalidation through the same assessor, final-turn acceptance
without duplicate final validation, selected exhaustion/fallback, and useful
failures. The executable publishes artifacts and reports them through native
presentation without the private completion/correction API.
failures. Unavailable assessments remain distinct from actionable defects. The
executable publishes artifacts and reports them through native presentation
without the private completion/correction API.
### Phase 4: Migrate All Roles And Remove Superseded Machinery
- Migrate every LLM-backed chunker, extractor, normalizer, and validator; preserve
deterministic paths and ordered validation. Complete U4–U7 across roles.
deterministic paths and ordered validation. Apply U4's prompt/guidance design
across roles and complete U5–U7.
- Remove the adapter, copied completion contracts, manual replay/capability checks,
outer LLM retries, global scheduler/ceiling, and coupled worker settings.
- Finish native command/inspection migration and remove custom dispatch, flags,
@@ -905,7 +995,7 @@ Release preparation/publication is governed separately by release policy.
| `AssetRegistry`, `PromptAssetManifest`, `ModulePromptFS`, synthetic mounts, flattened schema namespaces, `readonlyfs`, manual fingerprint enumeration | Ordinary embedded content, native source assembly, and module meaning. |
| Private response-schema registry/wrappers, byte copies, digest and reporting-only identity metadata, duplicate structural validation | Native prompt output contracts and schema loading; application interpretation and durable artifact contracts. |
| `single_response_v1`, raw `ModelCandidate`/`SemanticCorrection`, manual replay, correction capability checks | Typed candidates, actionable findings, native exchanges, assessment. |
| LLM retry loops in `runProducerAttempts`/`executeValidationChain`, module rerun directives | Ordered validators, domain acceptance/rejection/fallback. |
| LLM retry loops in `runProducerAttempts`/`executeValidationChain`, module rerun directives, hard-coded `CorrectionRequest` message builder | Ordered validators, validator/module-owned guidance, native domain correction templates, and domain acceptance/rejection/fallback. |
| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, worker coupling | Bounded pipeline work and native backend admission. |
| Checkpoint stage/dependency/configuration fingerprints, invocation-wide barriers, manifests, acceptance/provenance envelopes | Small accepted-artifact cache using the upstream message hash, ordinary load checks, explicit controls. |
| Inspection wrappers, copied effective parameters, usage summaries/conversions/accumulators, debug completion wrapper, synthetic IDs, profile recorder | Native facts with application scope for diagnostics. |
@@ -928,6 +1018,7 @@ retry tests or freeze complete help text and native debug snapshots.
| Commands/loaders | Required operands, native alias, supplied-value distinctions, references, and run-wide overrides reach intended handlers/roles; help/version/completion need no config; diff selections remain distinct. |
| Preflight/inspection | Same definitions across execution and inspection; missing statically selected resources fail with context; partial profiles alone pass; no provider calls, transcript, or credential requirement for offline preflight. |
| Domain workflow | Representative extraction and normalization preserve source/reference identity, fixed handles, safe plan application, ordered validators, artifact schemas, and stable output ordering; one assessor checks initial/corrected candidates without an extra final pass; deterministic stages and LLM validators need no producer conversation loop. |
| Prompt/guidance contracts | Domain-specific initial/correction pairs use the same response schema and default profile; explicit profile/execution overrides reach both; operator correction-profile defaults work when not overridden; all applicable validators assess the same candidate despite earlier rejection; their actionable findings reach the correction template; module interpretation defects and unavailable assessments remain distinct. |
| Continuation/failures | Accumulated history, full revalidation and final-turn assessment; no blanket retries beyond native exhaustion; validator failure differs from rejection; selected fallback, incomplete-validation, and cancellation policy hold. |
| Admission/workers | Shared backend pools across roles, independent backends, bounded pipeline work, nested validators, queue exhaustion, and cancellation without leaked capacity or a global ceiling. |
| Accepted-artifact cache | Downstream recovery; initial-message keys; corrected final artifact stored late under the initial key; changed messages miss; settings/ID-only changes can hit; no model/revalidation on hits; unusable entries miss; failed/rejected/incomplete work and ineligible derivatives excluded; no required history. |