diff --git a/docs/roadmap/promptkit.md b/docs/roadmap/promptkit.md index 21b485a6..2f4686c6 100644 --- a/docs/roadmap/promptkit.md +++ b/docs/roadmap/promptkit.md @@ -196,8 +196,9 @@ a singleton flag and use the last successfully parsed value. | Presentation | Native `--format`, `--quiet`, `--verbose`, `--debug`, and `--progress`. | | Capture destination | Application `--debug-dir PATH`, with the semantics below. | -Keep worker counts and semantic budgets in configuration, adjustable through -application `--set`. U5 selects `max_additional_turns`; worker defaults remain U7. +Keep the worker limit and semantic budgets in configuration, adjustable through +application `--set`. U5 selects `max_additional_turns`; U7 selects +`concurrency.workers` with a default of `64`. Use ckprompt's reusable profile, session, and execution flags, including model, temperature, timeout, reasoning effort, and transient retries, for run-wide overrides across producers, validators, and continuations. Omission preserves @@ -280,7 +281,7 @@ Retire `promptkit.profile_file`, `profile_dir`, and `local_backend` in favor of native companion sources; do not introduce a separate `promptkit.config_file`. Retire `concurrency.total_llm` and revise retry/worker settings as described below. U5 selects `max_additional_turns` and removes application structural-repair -controls; worker defaults and retired global-limit migration remain U7. +controls; U7 selects the shared worker pool and retired global-limit migration. Use one source assembly for runtime, preflight, and inspection: @@ -321,8 +322,8 @@ and its reference to the bundled unqualified `openrouter` backend. Inherited application profiles use sibling local IDs in the same collection. The bundled inventory differs from the old external catalogs and selects no model for an application request; document intentional replacements for removed catalog IDs. -The bundled backend supplies no concurrency limit. Application/operator limits -need explicit native configuration; application-supplied defaults remain U7. +The bundled backend supplies no concurrency limit. Under U7, Notarius injects no +backend concurrency or queue defaults; operators use native backend configuration. Preserve prompt prose, message ordering, roles, and cache boundaries where their meaning carries forward. Review and deliberately revise initial/repair prompts @@ -667,12 +668,39 @@ conversation state in process, outside ordinary artifact/cache payloads; explicitly requested detailed debug is separate. U6 is resolved; implementation and module-specific construction details remain in Phases 3–4. -### Admission And Pipeline Workers +### Admission And Pipeline Workers (U7 Resolved) Remove `Scheduler`, `scheduledClient`, and custom global queue/permit plumbing. Retire the separate cross-backend ceiling and the default/maximum extraction -worker coupling to `concurrency.total_llm`. Keep bounded extraction workers and -bounded lane continuations with independent defaults to be selected. +worker coupling to `concurrency.total_llm`. Use one shared pool for ready pipeline +jobs, configured by the positive integer `concurrency.workers`, default `64`. +Omission uses the default; explicit `1` provides serial job execution. Keep this +in application configuration, adjustable through `--set`, without a dedicated +CLI flag or separate extraction/lane worker settings. + +The pool serves both kinds of work: + +| Job | Work covered by one worker slot | +| --- | --- | +| Extraction | One chunk-and-lane extraction, including its validation and semantic correction. | +| Lane completion | A ready lane's merge and normalization, including their validation and correction. | + +The limit counts total active jobs across both kinds, with no separate reserved +capacity. The existing pipeline coordinator tracks dependencies and makes lane +completion ready after that lane's extraction finishes. A worker never holds a +slot while waiting for prerequisite jobs. Validators and correction turns execute +within the current job instead of acquiring another worker slot. Preserve stable +public result ordering regardless of completion order. Input preparation, chunking, +and final publication retain their sequential placement around lane execution. +This remains execution of the fixed pipeline, not a general workflow scheduler. + +Count application jobs, not OS threads or all process goroutines. Do not derive +the default from CPU count or `GOMAXPROCS`; model-backed jobs often wait for I/O +or native admission. The pool bounds active application state and preparation as +well as model-backed work. `64` supplies practical headroom for backend utilization +while remaining well below the reviewed native default queue capacity; it is not +a benchmark-derived optimum or a guarantee of backend saturation. Operators may +tune it for workload and host resources independently of backend settings. Use one shared native engine across all roles. Native capacity belongs to the engine and backend ID: profiles selecting the same ID share its pool; different @@ -680,6 +708,15 @@ backend IDs or separate engines do not automatically share limits, even for the same endpoint. Endpoint-only profiles and backends without a positive limit remain unrestricted by native backend admission. +Inject no Notarius-owned backend concurrency or queue defaults. Preserve native +operator configuration and inherit upstream defaults rather than deriving backend +settings from the worker limit. In the reviewed PromptKit baseline, omitted/zero +backend concurrency is unlimited and has no queue; with positive concurrency, +omitted queue capacity defaults to `1024`. Explicit queue capacity, including zero, +uses native semantics. These are upstream defaults, not values pinned by Notarius. +With an unlimited backend, active jobs can generate concurrent calls up to the +application's workload; the worker default does not imply a separate LLM ceiling. + For limited backends, the reviewed baseline bounds active model attempts by `concurrency_limit` and admitted operations by that limit plus `queue_capacity`. Ordinary execution admission spans preparation, retries, and repairs; split @@ -687,12 +724,36 @@ Ordinary execution admission spans preparation, retries, and repairs; split `RunPrepared`. Active permits are released during retry waits. A completed continuation turn releases admission before assessment, allowing a nested validator to use the same backend. Configure native limits at the intended -capacity boundary and handle `ErrCapacityExceeded` through application policy. +capacity boundary. Worker counts need not be less than backend concurrency; +native admission can queue waiting operations. + +Handle `ErrCapacityExceeded` through U6's existing strict/permissive policy: + +- A producer admission failure fails in strict mode or uses the module's + supported fallback in permissive mode. +- A validator admission failure means assessment is unavailable; apply the + producing module's resolved mode. Do not turn it into a semantic rejection + or a correction instruction. +- Cancellation stops work and does not initiate fallback. + +Preserve the error and backend identity in diagnostics. Add no application retry, +backoff, or second admission queue. A full native queue rejects admission without +calling the model; the large default queue makes this unusual with modest worker +counts, but explicit operator limits remain authoritative. Use ordinary fallback +reporting and cache exclusions when permissive handling continues. Aggregate concurrency across independent backends may exceed the former global ceiling. That change is intentional. Bounded pipeline workers are application -work controls, not a substitute cross-backend LLM ceiling. Defaults, retired-field -migration, and role-specific admission-failure dispositions remain open. +work controls, not a substitute cross-backend LLM ceiling. + +Retire `concurrency.stage_workers.extract` along with `concurrency.total_llm`. +Migration guidance selects `concurrency.workers` independently and configures +each backend natively. Existing explicit extraction-worker values can inform the +new shared limit, but its scope now includes lane completion. Do not automatically +translate a cross-backend ceiling into per-backend limits or preserve worker +inheritance/caps from that field. Provide a native limited-backend example with +usable queue capacity, without requiring operators to override the upstream +queue default. U7 is resolved; implementation remains in Phase 4. ### Accepted-Artifact Cache @@ -991,7 +1052,6 @@ reopen the architecture. IDs retain continuity with the earlier roadmap. | ID | Remaining choice or dependency | Resolve for | | --- | --- | --- | | U1 | Review the message-content hash API/release documented upstream in v0.24.0, including its ConfigKit dependency; verify the final compatible dependency set against the selected baseline. | Track in Phase 1; required to finish cache integration in Phase 5. | -| U7 | Independent worker defaults; application-supplied native backend/queue limits, if any; retired `total_llm` migration and admission-failure handling by role. | Phase 4. | | U8 | Cache directory configuration and reuse/bypass/refresh controls (names, defaults, scope); old-checkpoint treatment and unusable-entry diagnostics. Hash inputs, lookup timing, accepted value, and ownership are settled. | Design with Phase 3; finish in Phase 5. | | U9 | Observer/group enrollment and application scope; concrete conditional report fields and debug filenames/serialization within the agreed contracts; publication-layout/consumer migration details preserving intrinsic artifact schemas. | Implementation details in Phases 3–5; no new receipt/manifest inventory or capture-policy decision required. | @@ -1034,6 +1094,14 @@ Chunking falls back to one whole-transcript chunk; spell extraction uses its emp artifact, and normalization uses safe final-response groups or merged input. Implement module-specific construction details in Phases 3–4. +**U7 resolved:** [One pipeline worker pool and native admission](#admission-and-pipeline-workers-u7-resolved) +replace the separate worker pools and application LLM scheduler. +`concurrency.workers` defaults to `64` and bounds ready extraction/lane completion +jobs together, independently of CPU count and backend settings. PromptKit owns +backend capacity with no injected application defaults; admission failures use +U6's mode without extra retries or queues. Retire `total_llm` and the old extraction +worker field with explicit migration guidance. Implement in Phase 4. + **U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight, output, debug policy, and coordinated cutover sequencing are selected above. The former later CommandKit phase is retired. CLI compatibility is a non-goal; @@ -1054,9 +1122,9 @@ major boundary changes in successor ADRs without rewriting accepted decisions. Keep the cache dependency explicit without blocking independent work. - Map affected ADRs, current callers, contracts, and removal inventory. Carry forward Audita evidence without adopting its application wrappers. -- Preserve U7–U9 with their dependent phases; do not reopen resolved module +- Preserve U8–U9 with their dependent phases; do not reopen resolved module execution, prompt/guidance ownership, retry controls/defaults, assets/schemas, - CommandKit, debug, cache, or output ownership. + worker/admission policy, CommandKit, debug, cache, or output ownership. **Completion evidence:** reviewed dependency baseline, upstream dependency, scope, and outstanding choices are explicit. Upstream availability alone does not @@ -1108,7 +1176,8 @@ without the private completion/correction API. - Migrate every LLM-backed chunker, extractor, normalizer, and validator; preserve deterministic paths and ordered validation. Apply U4's prompt/guidance design - and U5's retry policy across roles; implement U6's module fallbacks and settle U7. + and U5's retry policy across roles; implement U6's module fallbacks and U7's + shared worker pool and native admission handling. - Remove the adapter, copied completion contracts, manual replay/capability checks, generic outer attempt loop, all producer/validator `retries` fields, global scheduler/ceiling, and coupled worker settings. @@ -1120,7 +1189,9 @@ without the private completion/correction API. composition root and producer collaboration to the pipeline. **Completion evidence:** every role uses the shared native engine and selected -admission policy. No production path depends on the private completion API, +admission policy. One worker limit bounds extraction and lane completion together, +including serial operation without dependency or nested-validator deadlock. +No production path depends on the private completion API, latest-response-only correction, generic stage retries, global ceiling, or legacy CLI infrastructure. Obsolete tests retire with the machinery; surviving domain requirements remain. @@ -1170,7 +1241,7 @@ Release preparation/publication is governed separately by release policy. | Generic `runProducerAttempts` outer loop, repeated validator executions in `executeValidationChain`, all producer/validator `retries` fields, module rerun directives, hard-coded `CorrectionRequest` message builder | Ordinary deterministic execution, native LLM execution/continuation, ordered validators, validator/module-owned guidance, native domain correction templates, and domain acceptance/rejection/fallback. | | Application `structured_output_repair_attempts` fields/resolution and explicit bundled `output.repair_attempts` | Native upstream structural-repair default, with operator customization through native prompt configuration. | | Failure-category `validation_policy` fields and separate `fail_run`/`reject_output`/`warn_continue` actions | Strict/permissive selection, module-owned fallback construction, and explicit fallback outcomes. | -| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, worker coupling | Bounded pipeline work and native backend admission. | +| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, `concurrency.stage_workers.extract`, separate extraction/lane worker pools | One ready-job pool controlled by `concurrency.workers`, default `64`, and native backend admission. | | Checkpoint stage/dependency/configuration fingerprints, invocation-wide barriers, manifests, acceptance/provenance envelopes | Small accepted-artifact cache using the upstream message hash, ordinary load checks, explicit controls. | | Inspection wrappers, copied effective parameters, usage summaries/conversions/accumulators, debug completion wrapper, synthetic IDs, profile recorder | Native facts with application scope for diagnostics. | | Custom dispatch/help/version/parsing, command-specific loading, copied inspection, legacy aliases/flags, parallel encoders and verbosity handling | Native declarations/loaders, pipeline handlers, domain input checks. | @@ -1197,7 +1268,7 @@ retry tests or freeze complete help text and native debug snapshots. | Non-LLM failure handling | Deterministic errors propagate without generic retries; rejected deterministic candidates reach terminal policy without regeneration; cancellation stops work. Producer/validator `retries` fields and the generic outer attempt loop are retired. | | Continuation/failures | Accumulated history, full revalidation and final-turn assessment; no blanket retries beyond native exhaustion; validator failure differs from rejection; no earlier-candidate recovery or cross-turn proposal union; cancellation stops without fallback work. | | Strict/permissive outcomes | `mode` resolves module binding, pipeline, then strict; omission inherits and either explicit value overrides. Strict is the default for every role, including normalization, without module-specific defaults. Strict failure when no acceptable result is obtained; permissive handling produces module fallback with required artifact/downstream invariants, without rerunning the ordinary full validation chain or claiming normal acceptance. Chunking fallback is one whole-transcript chunk preserving source order/references. Fallback construction failure fails execution; successful fallback use is disclosed in human/machine completion reports. | -| Admission/workers | Shared backend pools across roles, independent backends, bounded pipeline work, nested validators, queue exhaustion, and cancellation without leaked capacity or a global ceiling. | +| Admission/workers | One shared extraction/lane completion pool obeys `concurrency.workers`, default `64`; explicit `1` runs serially without dependency or nested-validator deadlock. Only ready jobs occupy slots; result ordering remains stable. Backend limits remain independent with no application-injected defaults. Producer/validator admission failures use U6's mode, preserve useful diagnostics, and add no retries; cancellation releases capacity. No separate global LLM ceiling. | | Accepted-artifact cache | Downstream recovery; initial-message keys; corrected final artifact stored late under the initial key; changed messages miss; settings/ID-only changes can hit; no model/revalidation on hits; unusable entries miss. Failed/rejected/incomplete work is not stored; fallbacks and affected downstream work skip lookup/storage while unaffected accepted predecessors remain reusable; no required history. | | Cache migration | Selected reuse/bypass/refresh and old-record treatment work; invocation configuration no longer blocks lookup; canonical chunk caching retains its distinct policy. | | Output/outcomes | Artifact files independent of report format; native text/JSON/YAML summary and qualified outcomes; no mandatory metadata bundle; quiet suppression, verbose stderr, debug path; native nonzero failures and useful explanations. Publication survives later capture/report failure without model reruns. | @@ -1230,8 +1301,12 @@ Provide practical release guidance for operators and subprocess callers: - Version 6 and native resources, embedded-only versus companion examples in appropriate directories, imports, qualified IDs, replacement/inheritance, native local backends, removed catalogs/versions, and separate source versions. -- Native admission and worker defaults, removed global limit, `max_additional_turns` - default/precedence/zero behavior, and intent-based migration from LLM `retries`. +- Shared `concurrency.workers` default `64`, ready-job scope, explicit serial + execution, native backend concurrency/queue settings, and removal of the global + limit and old extraction-worker field without automatic backend translation. + Explain strict/permissive handling of producer and validator admission failures. +- `max_additional_turns` default/precedence/zero behavior, and intent-based + migration from LLM `retries`. Explain retirement of all producer/validator `retries` fields and generic non-LLM retries, upstream-owned structural defaults, removal of application repair controls and bundled counts, native operator customization, accumulated-history