Set semantic turn defaults and delegate structural repair upstream

This commit is contained in:
2026-11-20 15:34:05 +00:00
parent b904021138
commit e737167730

View File

@@ -146,8 +146,8 @@ reference resolution, and deterministic rules. `internal/preflight` retains its
separate known-dependency checks. Exact argument grouping and helper signatures
can be refined against representative extraction and normalization during
implementation; ownership and placement are settled. U4 selects prompt roles and
guidance ownership below; budgets and fallback policy remain U5/U6. Implementation
is pending.
guidance ownership below; U5 selects retry controls and the correction budget.
Remaining failure and fallback policies stay in U5/U6. Implementation is pending.
Retain the fixed `input -> chunk -> extract -> merge -> normalize -> output`
shape, ordered steps and generated references, typed artifact-family boundaries,
@@ -194,10 +194,11 @@ a singleton flag and use the last successfully parsed value.
| Capture destination | Application `--debug-dir PATH`, with the semantics below. |
Keep worker counts and semantic budgets in configuration, adjustable through
application `--set`; their fields/defaults remain open. Use ckprompt's reusable
profile, session, and execution flags, including model, temperature, timeout,
reasoning effort, and transient retries, for run-wide overrides across producers,
validators, and continuations. Omission preserves normal selection; explicitly
application `--set`. U5 selects `max_additional_turns`; worker defaults remain U7.
Use ckprompt's reusable profile, session, and execution flags, including model,
temperature, timeout, reasoning effort, and transient retries, for run-wide
overrides across producers, validators, and continuations. Omission preserves
normal selection; explicitly
empty reasoning effort clears inheritance. Use native `--llm-profile` and
`--llm-session-id`, retiring the separate reasoning-clearing control. Native
transient retries remain distinct from semantic additional-turn budgets.
@@ -275,7 +276,8 @@ ConfigKit continues to own loading and composition under
Retire `promptkit.profile_file`, `profile_dir`, and `local_backend` in favor of
native companion sources; do not introduce a separate `promptkit.config_file`.
Retire `concurrency.total_llm` and revise retry/worker settings as described below.
Their replacement defaults and detailed migration treatment remain open.
U5 selects `max_additional_turns` and removes application structural-repair
controls; worker defaults and retired global-limit migration remain U7.
Use one source assembly for runtime, preflight, and inspection:
@@ -334,21 +336,23 @@ Remove synthetic mounts, file copying, flattened schema namespaces, and module
asset manifests. Retire `readonlyfs` when its prompt-mounting and asset-flattening
consumers disappear; no replacement filesystem abstraction is needed.
Declare response format, validation mode, schema path, and structural-repair
budget in native prompt definitions by default. Modules select prompt IDs, supply
inputs, and interpret responses. U4 selects the same schema file for each
initial/correction pair. Execution, inspection, and prepared-request preview then
use the same declared contract without repeating schema selection in Go.
U4 selects prompt roles and contracts; exact prose is implementation work, while
retry defaults remain U5.
Declare response format, validation mode, and schema path in native prompt
definitions by default. Omit structural-repair attempt settings in bundled
prompts to inherit PromptKit's upstream default under U5. Modules select prompt
IDs, supply inputs, and interpret responses. U4 selects the same schema file for
each initial/correction pair. Execution, inspection, and prepared-request preview
then use the same declared contract without repeating schema selection in Go.
U4 selects prompt roles and contracts; exact prose is implementation work.
U5 selects budget ownership and defaults.
Use a request-level native `OutputContract` only for a demonstrated runtime-
dependent contract. It replaces the whole configured contract, so supply the
intended format, validation mode, schema, and repair policy together. Do not
introduce a wrapper or per-call registry solely to reproduce Audita's approach.
Operator prompt replacements must still produce responses the selected module
can interpret; that module boundary does not require duplicating each prompt's
schema selection in Go.
intended format, validation mode, and schema together while leaving structural
repair at its native default. Do not inject a repair count merely to construct
the override, or introduce a wrapper or per-call registry solely to reproduce
Audita's approach. Operator prompt replacements must still produce responses the
selected module can interpret; that module boundary does not require duplicating
each prompt's schema selection in Go.
Default to native `WithSchemaFS(assets.FS(), ".")`. Schema paths are relative to
that selected root, for example `dnd/spells/schemas/dnd_spells_llm.v1.json`.
@@ -459,12 +463,12 @@ candidate, its module supplies guidance and artifact validators cannot yet run.
Cancellation or fatal execution failures can interrupt assessment. An unavailable
validator assessment is a separate outcome, not a detected defect or an instruction
to change the artifact. Do not convert provider errors into correction guidance;
detailed failure disposition remains U5/U6.
terminal/fallback disposition remains U6.
Ownership, response contracts, and default profile behavior are resolved. Refine
exact template prose, correction IDs, and necessary operation-specific inputs
while implementing representative workflows. Implementation is pending; semantic
budgets and terminal/fallback policies remain U5/U6.
while implementing representative workflows. Implementation is pending; U5 selects
semantic budgets and U6 retains terminal/fallback policy choices.
#### Execution And Assessment Flow
@@ -481,8 +485,9 @@ The application flow is:
all applicable configured validators and collect their actionable guidance
before requesting correction, as selected in U4.
3. For actionable correction, call `Engine.Coordinate` with the captured exchange
and `AdditionalTurns`. Its callback selects a repair prompt and supplies
findings; PromptKit retains the initial exchange and subsequent turns.
and the resolved `max_additional_turns` as native `AdditionalTurns`. Its callback
selects a repair prompt and supplies findings; PromptKit retains the initial
exchange and subsequent turns.
4. Interpret every delivered revised response and rebuild from the fixed original
inputs. Run the complete configured validation chain again for changed
candidates, including validators that previously approved. Assess the final
@@ -491,8 +496,8 @@ The application flow is:
failure, or supported safe-fallback policy after exhaustion. Native `stopped`
and `exhausted` outcomes do not themselves mean application acceptance.
Accumulated history increases context use; select the semantic turn budget
accordingly. Exact budgets remain U5.
Accumulated history increases context use; U5's allowance bounds additional
semantic correction while stopping as soon as the candidate is accepted.
Remove `single_response_v1`, raw-response `ModelCandidate`/`SemanticCorrection`
transport, manual assistant/user replay, and correction-protocol capability
@@ -502,16 +507,71 @@ findings and explicit fallback information. Retain useful ordered collection and
deduplication of actionable findings, rendered through a native repair prompt.
Reason codes and operator messages are not automatically model instructions.
Transport retries, structural repair, and semantic additional turns remain
separate controls. Preserve the ability to override native transport retries,
using the upstream default when omitted. Remove blanket outer LLM retries after
native transport/structural budgets are exhausted, including repeated LLM
validator invocations in `executeValidationChain`. Keep the ordered validator
chain and aggregation. A validator execution failure is distinct from rejection
of a producer candidate. Existing producer/validator `retries` settings cannot be
mechanically renamed; decide their migration and non-LLM failure handling
explicitly. Use native `OperationTimeout` when that operation scope is needed,
without adding an arbitrary default or replacing broader application deadlines.
#### Retry Controls And Correction Budget (U5 Controls Resolved)
Delegate transport retries and structural repair to PromptKit. Notarius selects
only the allowance for application-directed semantic correction:
| Concern | Control and selected default |
| --- | --- |
| Temporary provider failures | Native PromptKit transient retries; inherit the upstream default when omitted and preserve native execution overrides. |
| Structurally invalid model output | Native output-contract policy; bundled prompts and application requests inherit PromptKit's upstream structural-repair default. |
| Actionable domain defects | Application `max_additional_turns`, passed directly to native `AdditionalTurns`; application default `2`. |
`max_additional_turns` is a nonnegative maximum, excluding the initial producer
response. Resolve producer-binding override, then pipeline override, then the
application default of `2`. Omission inherits; explicit `0` disables semantic
correction. It applies to LLM-backed producers. Validators do not receive a
separate semantic continuation budget, and deterministic stages do not acquire
conversation machinery. Keep the field in configuration, adjustable through the
existing application `--set` mechanism, without another dedicated CLI flag.
For example, a producer binding can override its pipeline allowance:
```yaml
extract:
module: dnd/spells
max_additional_turns: 2
```
Two additional turns are an allowance, not mandatory executions. An accepted
initial candidate needs none; an accepted first correction needs only one. A
second turn is available when actionable defects remain and continuation is
appropriate. Assess every delivered response, including the final permitted turn,
and stop on acceptance or a terminal condition. PromptKit enforces the turn
limit; do not add an application counter or retry loop. Internal transport
attempts and structural repairs do not consume this semantic-turn allowance.
The default is an application policy choice, not a measured claim about how
frequently corrections will be needed.
For structural repair, omit `output.repair_attempts` from all bundled initial,
correction, and validator definitions. Remove Notarius's pipeline/binding/
validator `structured_output_repair_attempts` fields and resolution plumbing.
Do not replace them with a hard-coded default, including zero, in request
construction or a native output-contract override. Explicit operator
customization remains available through native PromptKit prompt configuration.
Notarius deliberately inherits future upstream default changes. An anticipated
PromptKit default change is not a migration prerequisite, and its timing or
chosen count is not a Notarius contract. Notarius still declares response format,
validation mode, and schema; PromptKit owns the mechanics of obtaining a valid
response. At the public boundary, handle a usable result or terminal failure,
including a returned result reporting unsuccessful structural validation.
Do not restart the operation to extend PromptKit's exhausted repair budget.
Remove blanket outer LLM retries, including repeated LLM validator invocations in
`executeValidationChain`. Keep ordered candidate assessment and distinguish a
validator execution failure from a domain rejection. Use native `OperationTimeout`
when that operation scope is needed, without adding an arbitrary default or
replacing broader application deadlines. Cancellation stops work.
Retire the broad producer/validator `retries` settings for LLM execution rather
than automatically renaming them. Migration guidance maps intent: a producer
allowance intended for semantic correction becomes `max_additional_turns`; native
transport controls and prompt-owned structural customization cover their own
concerns. Do not promise equivalent total provider-attempt counts. Generic retry
handling for non-LLM operations remains the outstanding U5 choice; detailed
terminal and fallback dispositions remain U6. Implementation is pending.
Keep source grounding, reference resolution, candidate isolation, domain
invariants, and reconciliation safety checks. Unknown or overlapping handles
@@ -846,7 +906,7 @@ reopen the architecture. IDs retain continuity with the earlier roadmap.
| ID | Remaining choice or dependency | Resolve for |
| --- | --- | --- |
| U1 | Review the message-content hash API/release documented upstream in v0.24.0, including its ConfigKit dependency; verify the final compatible dependency set against the selected baseline. | Track in Phase 1; required to finish cache integration in Phase 5. |
| U5 | Producer/validator retry-setting migration to native controls and semantic budgets: field names/defaults, non-LLM failure handling, and any justified recovery beyond native exhaustion. | Phases 3–4 and configuration migration. |
| U5 | Decide generic retry handling for non-LLM operations. LLM controls, `max_additional_turns` naming/default/precedence, structural-default delegation, and intent-based LLM setting migration are resolved below. | Phases 3–4 and configuration migration. |
| U6 | Detailed terminal/fallback policy, especially whether an earlier safe candidate survives a later failed turn. No implicit best-so-far acceptance. | Phases 3–4. |
| U7 | Independent worker defaults; application-supplied native backend/queue limits, if any; retired `total_llm` migration and admission-failure handling by role. | Phase 4. |
| U8 | Cache directory configuration and reuse/bypass/refresh controls (names, defaults, scope); old-checkpoint treatment and unusable-entry diagnostics. Hash inputs, lookup timing, accepted value, and ownership are settled. | Design with Phase 3; finish in Phase 5. |
@@ -855,22 +915,29 @@ reopen the architecture. IDs retain continuity with the earlier roadmap.
**U2 resolved:** Ordinary embedded assets, prompt-declared output contracts,
native schema-source selection, and removal of duplicate schema/filesystem
machinery are selected in [the schema design](#prompt-assets-and-response-schemas-u2-resolved).
Implementation remains in Phase 2; U4 selects prompt roles and contracts, while
retry budgets remain U5.
Implementation remains in Phase 2; U4 selects prompt roles and contracts, and U5
selects native repair delegation and the application correction allowance.
**U3 resolved:** Typed stage boundaries, composition-root engine construction,
and the shared pipeline producer helper with module functions and a pipeline
assessment callback are selected in [the execution design](#module-execution-and-assessment-u3-resolved).
Implement in Phase 3 and refine signatures during Phase 4 without reopening
ownership. U4 selects prompt/guidance behavior; U5/U6 retain budget and terminal
policy choices.
ownership. U4/U5 select prompt/guidance and budget behavior; remaining failure
and terminal-policy choices stay in U5/U6.
**U4 resolved:** Domain-specific initial/correction prompt pairs, shared response
schemas, default profile behavior, validator-owned guidance, and complete
applicable-chain assessment are selected in
[the prompt and guidance design](#prompt-roles-and-correction-guidance-u4-resolved).
Implement templates and operation-specific inputs in Phases 3–4; exact prose does
not reopen ownership. Budgets and terminal/fallback policy remain U5/U6.
not reopen ownership. U5 selects budgets; terminal/fallback policy remains U6.
**U5 controls resolved:** `max_additional_turns` defaults to `2`, with binding over
pipeline precedence and explicit zero disabling correction. Native transient and
structural-repair defaults remain upstream-owned; bundled prompts and application
requests do not pin structural repair. The [retry-control design](#retry-controls-and-correction-budget-u5-controls-resolved)
also selects intent-based LLM setting migration and removal of outer LLM retries.
Non-LLM retry handling remains open; U6 owns terminal/fallback decisions.
**U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight,
output, debug policy, and coordinated cutover sequencing are selected above.
@@ -893,8 +960,8 @@ major boundary changes in successor ADRs without rewriting accepted decisions.
- Map affected ADRs, current callers, contracts, and removal inventory. Carry
forward Audita evidence without adopting its application wrappers.
- Preserve U5–U9 with their dependent phases; do not reopen resolved module
execution, prompt/guidance ownership, assets/schemas, CommandKit, debug, cache,
or output ownership.
execution, prompt/guidance ownership, retry controls/defaults, assets/schemas,
CommandKit, debug, cache, or output ownership.
**Completion evidence:** reviewed dependency baseline, upstream dependency, scope,
and outstanding choices are explicit. Upstream availability alone does not
@@ -905,7 +972,8 @@ of the migration.
- Implement the resolved U2 design, update available dependencies, introduce
configuration version 6, convert native embedded/companion resources, and use
native schema loading with embedded/default or operator-selected roots.
native schema loading with embedded/default or operator-selected roots. Omit
bundled structural-repair settings and remove their application override fields.
- Build the small CommandKit application, shared ConfigKit/engine loaders,
standard suites, pipeline declarations/native alias, and `internal/preflight`.
- Add maintained embedded-only and operator-override examples. Remove duplicated
@@ -919,11 +987,14 @@ without credentials or synthetic requests. `config diff` handles two selections.
### Phase 3: Prove A Representative Native Pipeline
- Implement the resolved U3/U4 collaboration and settle relevant U5/U6 details.
- Implement the resolved U3/U4 collaboration and U5 controls; settle remaining
U5/U6 failure-policy details.
Connect module request/interpretation functions to the pipeline assessment
callback through the shared producer helper; refine concrete signatures here.
Implement domain-specific initial/correction prompts using the same schema and
default profile, supplied overrides, and validator-owned correction guidance.
Resolve `max_additional_turns` with omission/zero distinctions and pass it to
native coordination; inherit native transport and structural-repair defaults.
- Exercise an LLM validator on the same engine and normalization with fixed
handles and fresh plan application. Select actual observer/group enrollment.
- Run through the new command surface and alias, using the small completion
@@ -942,7 +1013,7 @@ without the private completion/correction API.
- Migrate every LLM-backed chunker, extractor, normalizer, and validator; preserve
deterministic paths and ordered validation. Apply U4's prompt/guidance design
across roles and complete U5–U7.
and U5's selected controls across roles; settle remaining U5–U7 choices.
- Remove the adapter, copied completion contracts, manual replay/capability checks,
outer LLM retries, global scheduler/ceiling, and coupled worker settings.
- Finish native command/inspection migration and remove custom dispatch, flags,
@@ -996,6 +1067,7 @@ Release preparation/publication is governed separately by release policy.
| Private response-schema registry/wrappers, byte copies, digest and reporting-only identity metadata, duplicate structural validation | Native prompt output contracts and schema loading; application interpretation and durable artifact contracts. |
| `single_response_v1`, raw `ModelCandidate`/`SemanticCorrection`, manual replay, correction capability checks | Typed candidates, actionable findings, native exchanges, assessment. |
| LLM retry loops in `runProducerAttempts`/`executeValidationChain`, module rerun directives, hard-coded `CorrectionRequest` message builder | Ordered validators, validator/module-owned guidance, native domain correction templates, and domain acceptance/rejection/fallback. |
| Application `structured_output_repair_attempts` fields/resolution and explicit bundled `output.repair_attempts` | Native upstream structural-repair default, with operator customization through native prompt configuration. |
| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, worker coupling | Bounded pipeline work and native backend admission. |
| Checkpoint stage/dependency/configuration fingerprints, invocation-wide barriers, manifests, acceptance/provenance envelopes | Small accepted-artifact cache using the upstream message hash, ordinary load checks, explicit controls. |
| Inspection wrappers, copied effective parameters, usage summaries/conversions/accumulators, debug completion wrapper, synthetic IDs, profile recorder | Native facts with application scope for diagnostics. |
@@ -1019,6 +1091,7 @@ retry tests or freeze complete help text and native debug snapshots.
| Preflight/inspection | Same definitions across execution and inspection; missing statically selected resources fail with context; partial profiles alone pass; no provider calls, transcript, or credential requirement for offline preflight. |
| Domain workflow | Representative extraction and normalization preserve source/reference identity, fixed handles, safe plan application, ordered validators, artifact schemas, and stable output ordering; one assessor checks initial/corrected candidates without an extra final pass; deterministic stages and LLM validators need no producer conversation loop. |
| Prompt/guidance contracts | Domain-specific initial/correction pairs use the same response schema and default profile; explicit profile/execution overrides reach both; operator correction-profile defaults work when not overridden; all applicable validators assess the same candidate despite earlier rejection; their actionable findings reach the correction template; module interpretation defects and unavailable assessments remain distinct. |
| Retry controls | `max_additional_turns` resolves binding, pipeline, then application default `2`; explicit zero disables correction; acceptance stops early and the final allowed turn is assessed. Bundled declarations/application overrides leave structural repair unspecified, native operator customization remains effective, and tests do not freeze upstream default attempt counts. |
| Continuation/failures | Accumulated history, full revalidation and final-turn assessment; no blanket retries beyond native exhaustion; validator failure differs from rejection; selected fallback, incomplete-validation, and cancellation policy hold. |
| Admission/workers | Shared backend pools across roles, independent backends, bounded pipeline work, nested validators, queue exhaustion, and cancellation without leaked capacity or a global ceiling. |
| Accepted-artifact cache | Downstream recovery; initial-message keys; corrected final artifact stored late under the initial key; changed messages miss; settings/ID-only changes can hit; no model/revalidation on hits; unusable entries miss; failed/rejected/incomplete work and ineligible derivatives excluded; no required history. |
@@ -1053,8 +1126,11 @@ Provide practical release guidance for operators and subprocess callers:
- Version 6 and native resources, embedded-only versus companion examples in
appropriate directories, imports, qualified IDs, replacement/inheritance,
native local backends, removed catalogs/versions, and separate source versions.
- Native admission and worker defaults, removed global limit, retry-budget
migration, accumulated-history costs, and selected role failure policy.
- Native admission and worker defaults, removed global limit, `max_additional_turns`
default/precedence/zero behavior, and intent-based migration from LLM `retries`.
Explain upstream-owned structural defaults, removal of application repair
controls and bundled counts, native operator customization, accumulated-history
costs, and selected role failure policy.
- Accepted-artifact caching, message-based misses, settings-only reuse, refresh/
bypass for changed interpretation or validation, and old-checkpoint handling.
- Artifact delivery without mandatory manifests/receipts; preserve intrinsic