diff --git a/docs/adr/0004-package-modules-by-domain.md b/docs/adr/0004-package-modules-by-domain.md index d9298b8a..7a525970 100644 --- a/docs/adr/0004-package-modules-by-domain.md +++ b/docs/adr/0004-package-modules-by-domain.md @@ -1,6 +1,10 @@ # ADR-0004: Package modules by domain, not by stage -**Status:** Accepted — its asset-co-location rule is superseded by [ADR-0011](0011-centralize-llm-assets.md); its domain-first module packaging decision remains accepted. +**Status:** Accepted in part. Asset co-location is superseded by +[ADR-0011](0011-centralize-llm-assets.md), and prompt/schema registration by +[ADR-0021](0021-compose-native-kits-and-prompt-declared-resources.md). +Domain-first packaging, typed domain ownership, and registrar dependency direction +remain accepted. **Date:** 2026-07-13 ## Context diff --git a/docs/adr/0005-cache-canonical-chunk-plans-by-source.md b/docs/adr/0005-cache-canonical-chunk-plans-by-source.md index bf98ec7d..bd82a89f 100644 --- a/docs/adr/0005-cache-canonical-chunk-plans-by-source.md +++ b/docs/adr/0005-cache-canonical-chunk-plans-by-source.md @@ -1,6 +1,9 @@ # ADR-0005: Cache one canonical chunk plan per source -**Status:** Accepted +**Status:** Superseded by +[ADR-0023](0023-unify-accepted-artifact-and-plan-caching.md). Source-addressed plan +semantics and deterministic materialization remain; source-only canonical reuse +and its separate store do not. **Date:** 2026-07-17 ## Context diff --git a/docs/adr/0006-separate-output-cache-and-debug-state.md b/docs/adr/0006-separate-output-cache-and-debug-state.md index a3a8c40e..5360729b 100644 --- a/docs/adr/0006-separate-output-cache-and-debug-state.md +++ b/docs/adr/0006-separate-output-cache-and-debug-state.md @@ -1,6 +1,12 @@ # ADR-0006: Separate output, cache, and debug state -**Status:** Superseded by [ADR-0007](0007-separate-checkpoint-recording-from-reuse.md) +**Status:** Accepted in part. The checkpoint invocation policy was superseded by +[ADR-0007](0007-separate-checkpoint-recording-from-reuse.md); the two-cache model +is now superseded by [ADR-0023](0023-unify-accepted-artifact-and-plan-caching.md), +and required output records and the debug summary/trace split by +[ADR-0024](0024-publish-artifacts-with-native-reports-and-debug-evidence.md). +Independent output/cache/debug surfaces, explicit debug selection, and +operator-owned retention remain accepted. **Date:** 2026-07-17 ## Context diff --git a/docs/adr/0007-separate-checkpoint-recording-from-reuse.md b/docs/adr/0007-separate-checkpoint-recording-from-reuse.md index 3c7ea2b7..a4d00b55 100644 --- a/docs/adr/0007-separate-checkpoint-recording-from-reuse.md +++ b/docs/adr/0007-separate-checkpoint-recording-from-reuse.md @@ -1,6 +1,8 @@ # ADR-0007: Separate checkpoint recording from reuse -**Status:** Accepted +**Status:** Superseded by +[ADR-0023](0023-unify-accepted-artifact-and-plan-caching.md), which replaces +checkpoint recording/resume with unified accepted-result reuse. **Date:** 2026-07-19 ## Context diff --git a/docs/adr/0010-workload-oriented-llm-profile-defaults.md b/docs/adr/0010-workload-oriented-llm-profile-defaults.md index 5bdf7555..cec6bac6 100644 --- a/docs/adr/0010-workload-oriented-llm-profile-defaults.md +++ b/docs/adr/0010-workload-oriented-llm-profile-defaults.md @@ -1,7 +1,13 @@ # ADR-0010: Use workload-oriented LLM profile defaults -**Status:** Accepted; profile source replacement behavior superseded by -[ADR-0019](0019-adopt-native-promptkit-configuration.md) +**Status:** Accepted in part. Profile source replacement was superseded by +[ADR-0019](0019-adopt-native-promptkit-configuration.md), whose native source +boundary is now superseded by +[ADR-0021](0021-compose-native-kits-and-prompt-declared-resources.md). +Checkpoint identity is superseded by +[ADR-0023](0023-unify-accepted-artifact-and-plan-caching.md). +Workload-oriented defaults, D&D-owned profile policy, binding exceptions, and +run-wide profile precedence remain accepted. **Date:** 2026-08-03 ## Context diff --git a/docs/adr/0011-centralize-llm-assets.md b/docs/adr/0011-centralize-llm-assets.md index b65f5d3f..04729ef0 100644 --- a/docs/adr/0011-centralize-llm-assets.md +++ b/docs/adr/0011-centralize-llm-assets.md @@ -1,6 +1,11 @@ # ADR-0011: Centralize LLM-facing assets in a content-only package -**Status:** Accepted +**Status:** Accepted in part. Asset manifests, module schema registration, and +scoped source assembly are superseded by +[ADR-0021](0021-compose-native-kits-and-prompt-declared-resources.md); +checkpoint fingerprints are superseded by +[ADR-0023](0023-unify-accepted-artifact-and-plan-caching.md). +The content-only root assets leaf and domain semantic ownership remain accepted. **Date:** 2026-08-05 ## Context diff --git a/docs/adr/0013-use-request-local-candidate-handles-for-semantic-reconciliation.md b/docs/adr/0013-use-request-local-candidate-handles-for-semantic-reconciliation.md index a1568333..12156506 100644 --- a/docs/adr/0013-use-request-local-candidate-handles-for-semantic-reconciliation.md +++ b/docs/adr/0013-use-request-local-candidate-handles-for-semantic-reconciliation.md @@ -1,6 +1,9 @@ # ADR-0013: Use request-local candidate handles for semantic reconciliation -**Status:** Accepted +**Status:** Accepted; only the checkpoint-fingerprint requirement is superseded by +[ADR-0023](0023-unify-accepted-artifact-and-plan-caching.md). +Request-local handles, fixed mapping, domain ownership, and safe typed application +remain accepted. **Date:** 2026-08-09 ## Context diff --git a/docs/adr/0014-feedback-aware-validation-retries.md b/docs/adr/0014-feedback-aware-validation-retries.md index c78a96f4..76109c57 100644 --- a/docs/adr/0014-feedback-aware-validation-retries.md +++ b/docs/adr/0014-feedback-aware-validation-retries.md @@ -1,6 +1,10 @@ # ADR-0014: Use feedback-aware validation retries -**Status:** Accepted +**Status:** Superseded by +[ADR-0022](0022-use-native-continuation-and-application-assessment.md) for execution, +assessment, and disposition, and +[ADR-0024](0024-publish-artifacts-with-native-reports-and-debug-evidence.md) for +diagnostic persistence. Guidance remains separate from operator diagnostics. **Date:** 2026-08-26 ## Context diff --git a/docs/adr/0015-separate-process-warnings-from-quality-diagnostics.md b/docs/adr/0015-separate-process-warnings-from-quality-diagnostics.md index 80340636..727da211 100644 --- a/docs/adr/0015-separate-process-warnings-from-quality-diagnostics.md +++ b/docs/adr/0015-separate-process-warnings-from-quality-diagnostics.md @@ -1,6 +1,10 @@ # ADR-0015: Separate process warnings from quality diagnostics -**Status:** Accepted +**Status:** Accepted in part. Durable warning/diagnostic files and receipt +presentation are superseded by +[ADR-0024](0024-publish-artifacts-with-native-reports-and-debug-evidence.md). +Process-warning classification, module-owned meaning, and bounded origin-aware +aggregation remain accepted. **Date:** 2026-08-27 ## Context diff --git a/docs/adr/0016-feedback-aware-module-requested-retries.md b/docs/adr/0016-feedback-aware-module-requested-retries.md index d29c3edd..d13a73ac 100644 --- a/docs/adr/0016-feedback-aware-module-requested-retries.md +++ b/docs/adr/0016-feedback-aware-module-requested-retries.md @@ -1,6 +1,10 @@ # ADR-0016: Use feedback-aware module-requested retries -**Status:** Accepted +**Status:** Superseded by +[ADR-0022](0022-use-native-continuation-and-application-assessment.md) for execution, +assessment, and fallback, and +[ADR-0024](0024-publish-artifacts-with-native-reports-and-debug-evidence.md) for +diagnostic persistence. Module-authored guidance remains distinct from diagnostics. **Date:** 2026-08-28 ## Context diff --git a/docs/adr/0018-adopt-configkit.md b/docs/adr/0018-adopt-configkit.md index 6538fde5..a8060a34 100644 --- a/docs/adr/0018-adopt-configkit.md +++ b/docs/adr/0018-adopt-configkit.md @@ -1,6 +1,10 @@ # ADR-0018: Adopt ConfigKit for configuration loading -**Status:** Accepted +**Status:** Accepted in part. The version 5 and dependency API selection is +superseded by [ADR-0021](0021-compose-native-kits-and-prompt-declared-resources.md). +ConfigKit adoption, application semantic ownership, and discovery-only environment +selection remain, as clarified by +[ADR-0020](0020-delegate-configuration-mechanics-to-configkit.md). **Date:** 2026-09-13 ## Context diff --git a/docs/adr/0019-adopt-native-promptkit-configuration.md b/docs/adr/0019-adopt-native-promptkit-configuration.md index 7d07d399..8010674d 100644 --- a/docs/adr/0019-adopt-native-promptkit-configuration.md +++ b/docs/adr/0019-adopt-native-promptkit-configuration.md @@ -1,6 +1,10 @@ # ADR-0019: Adopt native PromptKit configuration sources -**Status:** Accepted +**Status:** Superseded by +[ADR-0021](0021-compose-native-kits-and-prompt-declared-resources.md) for native +composition, resource ownership, and execution types, and +[ADR-0023](0023-unify-accepted-artifact-and-plan-caching.md) for cache identity. +ConfigKit-owned mechanics and captured source contexts remain in force. **Date:** 2026-10-04 ## Context diff --git a/docs/adr/0021-compose-native-kits-and-prompt-declared-resources.md b/docs/adr/0021-compose-native-kits-and-prompt-declared-resources.md new file mode 100644 index 00000000..727adbda --- /dev/null +++ b/docs/adr/0021-compose-native-kits-and-prompt-declared-resources.md @@ -0,0 +1,75 @@ +# ADR-0021: Compose native kits and prompt-declared resources + +**Status:** Accepted +**Date:** 2026-11-22 + +## Context + +Notarius duplicated configuration assembly, resource registration, completion +types, inspection, CLI parsing, and provider scheduling that the adopted kits +now supply. Keeping those wrappers would preserve competing ownership rather +than protect the application's typed artifact boundary. + +## Decision + +Use ConfigKit, PromptKit, CommandKit, ckconfig, and ckprompt directly at their +respective application boundaries. The composition root owns one application +load per selected configuration and constructs one shared native engine for +all selected model roles from that original result. Runtime and whole-app or +focused inspection share captured sources and native provenance, not a settings +reserialization or a second loader. ConfigKit retains ownership of composition +mechanics under [ADR-0020](0020-delegate-configuration-mechanics-to-configkit.md). +Version 6 replaces version 5 without a compatibility parser. + +Assemble bundled provider resources, embedded application resources, and optional +operator companions beside the selected main source through native contexts and +precedence. Native declarations own qualified resources, inheritance, default +output contracts, and schema selections. PromptKit prepares schemas and owns +structural validation and repair. Application modules retain prompt meaning, +interpretation, and static resource-ID dependencies, not manifests, response-schema +registries, asset fingerprints, virtual mounts, or fallback-profile registration. +The root assets package remains an ordinary content-only embedded leaf. + +Model-executing code uses native requests, results, exchanges, errors, and +observations. Pure domain models, codecs, source resolution, and deterministic +helpers remain independent. CommandKit and its integrations own standard command +declarations, parsing, inspection, and presentation; Notarius supplies loaders, +pipeline commands, request inputs, and report content. Shared loaders and offline +preflight do not allocate run storage or perform implicit provider probes. + +One ready-job worker pool bounds extraction and merge/normalize continuations +together. Prerequisite waits occupy no worker; nested validation and correction +stay inside the current job. Native engine/backend admission is independent. +Inject no backend concurrency/queue defaults, cross-backend ceiling, extra queue, +or application capacity retries. Preserve stable scope ordering and cancellation +draining. A completed turn releases admission before nested assessment. + +This supersedes [ADR-0019](0019-adopt-native-promptkit-configuration.md)'s old +source adapter and private-type boundary; its fingerprint policy is replaced by +[ADR-0023](0023-unify-accepted-artifact-and-plan-caching.md). It replaces the asset +registration portions of [ADR-0004](0004-package-modules-by-domain.md) and +[ADR-0011](0011-centralize-llm-assets.md), retaining domain-first packaging, +semantic ownership, and the content-only leaf. It replaces ADR-0018's version 5 +selection, not [ConfigKit adoption](0018-adopt-configkit.md) or ADR-0020's ownership. +[ADR-0010](0010-workload-oriented-llm-profile-defaults.md)'s workload-oriented +profile intent remains; native qualified resources replace its older source form. + +## Alternatives considered + +- Keep private completion, loader, and inspection adapters. This duplicates kit + behavior and obscures the actual execution contract. +- Adopt only PromptKit and defer CommandKit. Temporary command/report wrappers + would need another cutover and could select different configuration. +- Retain a global LLM scheduler above native admission. This adds a second queue + and couples independent backends without a selected product requirement. + +## Consequences + +The kits are deliberate dependencies at execution and composition boundaries; +typed application assessment and durable domain artifacts remain distinct. +Upstream changes require review of the public APIs actually consumed, rather +than preservation of old wrappers. Workers do not promise a global provider-call +ceiling or backend saturation. CLI and configuration compatibility is intentionally +broken; artifact schemas are not redesigned. Exact public syntax and dependency +integration details belong in the canonical configuration, CLI, and integration +references, not copied library manuals here. diff --git a/docs/adr/0022-use-native-continuation-and-application-assessment.md b/docs/adr/0022-use-native-continuation-and-application-assessment.md new file mode 100644 index 00000000..3f69d237 --- /dev/null +++ b/docs/adr/0022-use-native-continuation-and-application-assessment.md @@ -0,0 +1,82 @@ +# ADR-0022: Use native continuation and application assessment + +**Status:** Accepted +**Date:** 2026-11-22 + +## Context + +Private retry machinery conflated transient failures, structural repair, +semantic defects, and validator availability. Latest-response replacement also +discarded conversation history while requiring an application replay protocol. +Native execution can own those mechanics without owning domain acceptance. + +## Decision + +One small typed producer helper connects native execution to module functions +for fixed input preparation, interpretation, correction requests, and supported +fallback construction. The pipeline supplies the operation's typed assessor. +PromptKit owns transient retries, structural repair, growing conversation history, +and semantic turn iteration. Notarius selects the semantic-turn allowance but +does not pin omitted native repair/retry counts or extend exhausted native work +with an outer retry loop. Deterministic operations have no generic retry budget. + +Each correctable producer has initial and correction prompts using the same +response schema and bundled default profile. Preserve explicit request selections +and run overrides. Enduring principles and source/reference material are supplied +initially; the correction template adds current findings and a new assignment. +Native coordination carries the exchange, without manual assistant replay or +rerendering original inputs. Each response is a complete replacement, not a patch. +Normalization applies each proposal afresh to fixed originals and candidate +handles; never union groups across turns. + +Check native errors and structural validation status before interpretation, +including failed validation returned with nil Go error. A usable typed candidate +receives the complete applicable validator chain once, in configured order. +Earlier rejection does not suppress later guidance. Revisions, unchanged +candidates, and the final permitted response receive the whole chain; there is +no duplicate assessment after the helper returns. + +The detecting validator or module authors actionable guidance separately from +reason codes and operator messages. Unavailable assessment is not a semantic +defect; provider failures do not become instructions. LLM validators execute +once per assessment, without their own semantic continuation loop. Native +success, stopped, and exhausted statuses do not imply application acceptance. + +Strict is the application default. Explicit permissive mode may construct a +module-supported fallback after terminal nonacceptance. A fallback satisfies +intrinsic artifact and downstream invariants without ordinary semantic approval +or rerunning the normal validator chain. Failure to construct it fails normally. +Caller cancellation and undelivered responses never initiate fallback. A +result-less failed continuation cannot recover an earlier candidate. Reuse +exclusions and disclosure follow ADRs 0023 and 0024. Exact configuration precedence +and defaults belong in [Configuration](../config.md). + +This supersedes [ADR-0014](0014-feedback-aware-validation-retries.md) and +[ADR-0016](0016-feedback-aware-module-requested-retries.md)'s budgets, private +candidate transport, latest-only replay, and terminal policies. It retains their +useful separation of actionable guidance from diagnostic prose and of execution +failure from detected defects. Diagnostic persistence is governed by +[ADR-0024](0024-publish-artifacts-with-native-reports-and-debug-evidence.md). +Contextual identity and request-local reconciliation safety under ADRs +[0012](0012-resolve-opaque-entity-identifiers-deterministically.md) and +[0013](0013-use-request-local-candidate-handles-for-semantic-reconciliation.md) +remain in force. + +## Alternatives considered + +- Retain blanket producer/validator retries. These can extend native budgets + and repeat deterministic work without information capable of fixing it. +- Keep latest-only manual replay. This duplicates conversation mechanics and + discards earlier application-visible turns. +- Let validators decide terminal disposition or revalidate every fallback. + That distributes acceptance policy and confuses qualified recovery with + ordinary validation. + +## Consequences + +Native execution and application acceptance are separate facts. Accumulated +history uses more context, bounded by the selected allowance. Not every failure +has a supported fallback: an empty scene description, for example, cannot meet +its required artifact shape. Permissive output is explicitly qualified and +reuse-ineligible rather than silently accepted. Paid-model quality and exact +provider-attempt parity with the retired loops are not promised. diff --git a/docs/adr/0023-unify-accepted-artifact-and-plan-caching.md b/docs/adr/0023-unify-accepted-artifact-and-plan-caching.md new file mode 100644 index 00000000..933a100a --- /dev/null +++ b/docs/adr/0023-unify-accepted-artifact-and-plan-caching.md @@ -0,0 +1,82 @@ +# ADR-0023: Unify accepted artifact and plan caching + +**Status:** Accepted +**Date:** 2026-11-22 + +## Context + +Source-keyed plans and fingerprinted checkpoints imposed two recovery models +and substantial identity/provenance machinery. The selected need is narrower: +reuse successful model-backed predecessors after downstream failure without +repeating their complete production and assessment. + +## Decision + +Use one best-effort filesystem cache for final accepted typed artifacts and +LLM-generated chunk plans. Prepare the initial native request once and use its +opaque MessageContentHash unchanged, with no additional identity dimensions. +Execute that same prepared handle on a miss and discard it on a hit. Corrections +do not change the key. Store the final accepted codec artifact, including applied +normalization, rather than a native response, exchange, or reconciliation proposal. + +Message roles, exact text, boundaries, and order select reuse. Model/settings, +session, schema metadata, policy, library versions, and implementation changes +add no key dimensions. Text rendered into the messages participates as content. +Changed policy may therefore reuse old acceptance; operators deliberately choose +bypass or refresh when they want re-execution. + +Hits receive ordinary codec and current-source usability checks, not renewed +semantic assessment, replayed exchanges, fabricated execution, or historical +usage. Cached plans are rematerialized against the current source. Unusable +entries miss. Deterministic operations remain uncached without synthetic requests. + +Store only final ordinary acceptance. Failed, rejected, incomplete, canceled, +and fallback results cannot be stored. Fallback and actually affected dependent +work skip lookup as well as storage; unrelated accepted predecessors remain +reusable. Eligibility is a small handoff fact, not a dependency fingerprint graph. + +Auto reads and stores acceptance; bypass does neither; refresh skips lookup and +replaces only after new acceptance. Failed refresh preserves earlier accepted +bytes. Cache I/O failures produce diagnostics without failing otherwise successful +work or repeating model calls. Records are plain accepted JSON, not envelopes, +manifests, acceptance ledgers, or copied source/execution histories. Exact root, +layout, and control precedence belong in configuration and operations references. + +Ignore legacy checkpoint and source-plan records without importing, rewriting, +or deleting them. Remove explicit resume, recomputation barriers, and the separate +canonical-partition guarantee. Matching initial messages may reuse a plan; changed +prompt/reference/input content may select another partition. Retain plan range, +annotation, source-order, and deterministic materialization correctness. + +This supersedes [ADR-0005](0005-cache-canonical-chunk-plans-by-source.md) and +[ADR-0007](0007-separate-checkpoint-recording-from-reuse.md), and the two-cache +model in [ADR-0006](0006-separate-output-cache-and-debug-state.md). It replaces +checkpoint/fingerprint clauses in ADRs +[0010](0010-workload-oriented-llm-profile-defaults.md), +[0011](0011-centralize-llm-assets.md), +[0013](0013-use-request-local-candidate-handles-for-semantic-reconciliation.md), and +[0019](0019-adopt-native-promptkit-configuration.md), not their retained domain, +content, or identity boundaries. + +Apply [ADR-0017](0017-defer-cache-access-policy-to-deployment.md)'s permission +ownership to this unified cache: deployment umask, setgid, and default ACLs govern +effective access. Retain confined paths, symlink rejection, validation, atomic +publication, and recovery without permission convergence or automatic migration. +Output and debug retain independent roots and lifecycles. + +## Alternatives considered + +- Keep one canonical partition per source. This ignores changed initial task + content and retains a second lookup/publication mechanism. +- Use comprehensive execution or policy fingerprints. That changes the selected + reuse semantics and restores the removed compatibility bookkeeping. +- Cache native responses or restore conversations. Neither is the accepted typed + application result, and each requires replay or renewed interpretation. + +## Consequences + +Ordinary reruns recover reusable accepted predecessors without a resume request. +Acceptance under changed model or validation policy is not implied by a hit. +Cache is reconstructible and best effort, not an execution history or a storage +durability guarantee. Legacy data stays available for operator-managed cleanup; +no automatic conversion, expiry service, or garbage collector is introduced. diff --git a/docs/adr/0024-publish-artifacts-with-native-reports-and-debug-evidence.md b/docs/adr/0024-publish-artifacts-with-native-reports-and-debug-evidence.md new file mode 100644 index 00000000..bbb5b8c8 --- /dev/null +++ b/docs/adr/0024-publish-artifacts-with-native-reports-and-debug-evidence.md @@ -0,0 +1,90 @@ +# ADR-0024: Publish artifacts with native reports and debug evidence + +**Status:** Accepted +**Date:** 2026-11-22 + +## Context + +Required manifests, receipts, indexes, reporting sidecars, and copied execution +ledgers made consumers depend on accounting rather than the domain artifacts. +Split sanitized-summary/detailed-trace machinery also duplicated native evidence +and conflated useful inspection with a stable reporting protocol. + +## Decision + +Domain artifacts are the primary deliverables, including supported qualified +fallbacks. Preserve intrinsic schemas and existing run-specific logical paths, +including selected chunk-map/evidence auxiliaries. Consumers need no index, +manifest, persisted receipt, warnings/diagnostics sidecars, or execution ledger. +Publish artifacts, finalize requested capture, then submit one small native +completion value and human template through CommandKit. Native formats render +that same value; no separate encoder or universal envelope is introduced. + +The completion report identifies the pipeline, deliverable count, publication +directory, available debug directory, and positive fallback count. Count each +operation that emits fallback once, not its affected artifacts or attempts. +Disclose fallback independently of diagnostic verbosity. Standard commands keep +their native values. [CLI Reference](../cli.md) owns exact report fields, +stdout/stderr behavior, and outcome codes. + +Preserve native operation causes separately from rendering and delivery failures. +Post-publication capture/report failure returns nonzero with available locations +while retaining valid artifacts and evidence. Failed stdout can be partial; +consumers check status before decoding a successful report. Do not rerun model +work, rewrite history, or remove published files to compensate for delivery failure. + +Use one native invocation ExecutionGroup, enrolling actual initial, continuation, +and nested validator executions once. Preserve native IDs, terminal coverage, +and unknown usage without parallel totals, synthetic IDs, or a profile ledger. +Cache hits contribute no execution or historical usage. + +Explicit debug selection and a nonempty effective destination enable one run +recorder. A directory alone does not enable it; explicit empty disables capture. +Allocate after configuration/run identity but before engine construction/preflight. +Update the relative latest link at start, never completion. Persist convenient +run/operation evidence incrementally and finalize once before successful reporting. +Requested allocation, recording, or finalization failures fail the invocation, +retaining primary causes and whatever evidence/publication is available. + +Capture fixed inputs, configuration/composition, prepared contracts, native events, +available results and full application-visible exchanges, typed candidates, +findings/guidance, correction/normalization material, fallbacks, deterministic +results, and cache decisions. Reuse prepared evidence without another preparation +or provider call. Label undelivered responses unassessed and result-less failures +with available evidence; do not fabricate missing provider bodies. Exact filenames +and serialization remain private diagnostic choices, not a public format. + +Use ConfigKit secret-aware inspection and native credential omissions. Do not +directly serialize marked secrets or collect unrelated environment/filesystem +content. Do not heuristically scrub or rewrite selected source, prompts, responses, +or arbitrary strings. A debug bundle may retain sensitive application data; it +is not promised sanitized. Operators own access and retention. Do not automatically +delete output or requested bundles. Output, cache, and debug remain independent. + +This replaces [ADR-0006](0006-separate-output-cache-and-debug-state.md)'s required +output records and summary/trace split while retaining its three surfaces and +explicit debug selection. It supersedes durable reporting in +[ADR-0015](0015-separate-process-warnings-from-quality-diagnostics.md), retaining +module-owned diagnostic classification and bounded framework aggregation. It +also replaces the private attempt-record restrictions of ADRs +[0014](0014-feedback-aware-validation-retries.md) and +[0016](0016-feedback-aware-module-requested-retries.md) without moving conversation +state into ordinary artifacts or cache entries. + +## Alternatives considered + +- Rebuild manifests and receipts with native execution data. This preserves an + unnecessary consumer dependency and another reporting model. +- Retain sanitized summaries beside sensitive traces. Heuristic content rewriting + can alter evidence without establishing a selected sanitized-bundle guarantee. +- Treat capture failures as harmless warnings or retry model work. Requested + capture is required work, but its failure cannot repair model output. + +## Consequences + +Consumers use configured lane/auxiliary paths and optional native reports rather +than receipt traversal. Diagnostic files can change independently of artifacts. +Debug may duplicate evidence for convenient inspection without a complete provider +transcript guarantee. Quiet presentation does not erase fallback qualification +from the report value or turn delivery failure into success. No new access policy, +automatic retention service, release gate, or release authorization is created. diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index daf0b456..b2a5b5d1 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -2,30 +2,33 @@ **Base Policy Version:** 1.3 -**Last Updated**: 2026-11-17 +**Last Updated**: 2026-11-22 **Upstream Source**: https://gitea.maximumdirect.net/eric/policy/src/commit/593c9b52273c5c879785ae9e995d21026df467b4/docs/policy/architecture.md +The source above identifies the adopted baseline. Local native-kit architecture +decisions below are repository adaptations, not a new upstream policy adoption. + ## Purpose And Ownership The adopted policy defines the application's purpose and the architectural constraints contributors must follow. It owns system boundaries, responsibility and dependency rules, established invariants, and explicit non-goals. -Describe the concrete package inventory and implementation mechanics in internal -documentation. Link to canonical public contracts for their exact behavior and -to ADRs for decision rationale. +Describe concrete package inventory and implementation mechanics in internal +documentation. Link to canonical public contracts for exact behavior and to +ADRs for decision rationale. ## Architectural Approach -Hexagonal architecture is strongly recommended, including encapsulating external -integrations and APIs within internal adapters. Keep domain and application -logic independent of external integration details, and make dependency direction -clear in the adopted policy. +Hexagonal architecture is strongly recommended. Keep pure domain logic independent +of external integration details and make dependency direction clear. Native kit +types are an intentional contract for model execution, configuration composition, +and CLI presentation; do not hide them behind a replacement completion API. -Implement that separation with proportionate, idiomatic code. Choose interfaces -and package layout to serve the application's actual boundaries. Use adapters -for collaborators with external integration responsibilities. +Implement separation with proportionate, idiomatic code. Choose interfaces and +package layout to serve demonstrated boundaries. Use adapters where they serve +an integration need, not merely to rename an adopted library's public types. ## Adopted Constraints And Implemented Structure @@ -38,16 +41,19 @@ Constraints must reflect deliberately selected requirements and decisions. Do not promote incidental implementation choices into architectural invariants. Notarius's fixed pipeline, artifact-family boundaries, and deployment-owned -cache permissions are established product decisions. The linked ADRs retain -their rationale; the [internal overview](../internal/overview.md) inventories -their implementation. +cache permissions remain established product decisions. Successor ADRs +[0021](../adr/0021-compose-native-kits-and-prompt-declared-resources.md), +[0022](../adr/0022-use-native-continuation-and-application-assessment.md), +[0023](../adr/0023-unify-accepted-artifact-and-plan-caching.md), and +[0024](../adr/0024-publish-artifacts-with-native-reports-and-debug-evidence.md) +record the native cutover. Earlier accepted bodies remain historical decisions, +with status links identifying superseded portions. ## System Shape -Notarius is a small, dependency-light Go application for extracting structured -artifacts from source material. It is a general extraction platform whose -source formats, extraction domains, validation policies, LLM providers, and -output formats are isolated behind explicit boundaries. +Notarius is a Go application for configured structured extraction workflows. +Source formats, artifact domains, validation, and output encoding remain explicit +boundaries. The application has one fixed pipeline shape: @@ -55,337 +61,308 @@ The application has one fixed pipeline shape: input -> chunk -> extract -> merge -> normalize -> output ``` -Pipelines are configured compositions of this shape. They are not arbitrary -DAGs or a general workflow language. Every stage remains explicit; general -chunking, merging, or normalization behavior must not be hidden inside an -extractor. +Pipelines are configured compositions of this shape, not arbitrary DAGs or a +general workflow language. Every stage remains explicit; chunking, merging, or +normalization must not be hidden inside an extractor. -A stage module is one configured implementation of one pipeline stage. An -artifact family is the cohesive domain feature that owns an artifact across -the explicit stages and supporting codecs, validators, prompts, identity -rules, and reference projections. Artifact-family ownership does not combine -stages or alter the fixed pipeline. +A stage module implements one pipeline stage. An artifact family owns a cohesive +domain feature across its stages, codecs, validators, prompts, identity rules, +and reference projections. Family ownership does not combine stages. Input and chunking are pipeline-wide. Each selected artifact lane owns its -extract, merge, and normalize stages, and the output stage aggregates the run's -lane outcomes. +extract, merge, and normalize stages. Output aggregates the run's domain results. +Ordered steps support generated references without changing the fixed shape. -Notarius is contract-first without being abstraction-heavy. Interfaces and -extension points should protect demonstrated boundaries. New abstraction is not -itself an architectural goal. - -Implementation discipline is owned by [Agent Policy](agents.md). +Notarius is contract-first without being abstraction-heavy. Extension points +protect demonstrated boundaries; new abstraction is not itself an architectural +goal. [Agent Policy](agents.md) owns implementation discipline. ## Layers And Dependency Direction -The application boundary is the composition root and may depend on concrete -implementations. Domain-neutral model and framework layers provide reusable -policy, contracts, and orchestration. Concrete input, pipeline, output, and -validation extensions depend inward on those generic layers. +The composition root may depend on concrete implementations. Domain-neutral +models and framework layers provide reusable contracts and orchestration. +Concrete input, pipeline, output, and validation extensions depend inward on +those generic layers. Generic layers must not depend on production extensions; +extensions must not compose the application or own process behavior. -Semantic reconciliation is one such domain-neutral framework mechanism. It -prepares bounded source context, invokes a shared model-judgment protocol, -validates proposals, and applies safe plans through typed policies supplied by -the consuming artifact family. It does not own domain identity, durable IDs, -warning semantics, or artifact construction rules. +Semantic reconciliation is a domain-neutral mechanism for fixed candidate +preparation, native request construction, proposal decoding, and safe typed +application. It does not own domain identity, durable IDs, warning semantics, +or artifact construction rules. Native execution uses the shared producer path. -Generic layers must not depend on production extensions. Concrete extensions -must not compose the application or take ownership of process behavior. The -current packages implementing these layers are inventoried in -[Internal Overview](../internal/overview.md). - -The root `assets` package is a content-only dependency leaf. It may expose a -read-only embedded filesystem, but it must contain no business logic and must -not depend on `internal` packages or PromptKit. Consumers scope that filesystem -to the content they own; the root package is not a behavioral registry or a -public extension contract. The rationale and compatibility consequence are -recorded in [ADR-0011](../adr/0011-centralize-llm-assets.md). +The root `assets` package remains a content-only dependency leaf. It may expose +a read-only embedded filesystem but contains no business logic or dependencies +on internal packages or PromptKit. Native source assembly retains its filesystem +context; the package is not a behavioral registry or public extension contract. +This retains [ADR-0011](../adr/0011-centralize-llm-assets.md)'s content boundary. The following dependency boundaries are mandatory: - extractors and validators do not depend on concrete input adapters; -- provider-specific types do not cross the LLM runtime boundary; -- external dependency types do not leak across internal package boundaries - unless that dependency is the package's explicit contract. +- provider-specific transport and authentication remain inside PromptKit; +- model-executing framework code, modules, and validators use native PromptKit + requests, results, exchanges, errors, and observations directly; +- pure domain types, codecs, reference resolution, and deterministic helpers do + not acquire PromptKit dependencies merely because surrounding producers use it; +- other external dependency types cross internal boundaries only where that + dependency is an explicit contract. -Shared helpers may support demonstrated common needs, but must not move -source-format or extraction-domain knowledge into generic framework packages. -External dependencies require a clear correctness, security, interoperability, -or complexity benefit. +Shared helpers support demonstrated common needs without moving source-format +or domain knowledge into generic packages. Dependencies require a clear +correctness, security, interoperability, or complexity benefit. +[Internal Overview](../internal/overview.md) owns the concrete package inventory. ## Source And Domain Boundaries -Input modules translate external source formats into the generic source model. -Format-specific schemas, fields, and validation remain with the input module -and its integration contract. +Input modules translate external formats into generic source documents, units, +and references. Format-specific fields, schemas, and validation remain with the +input module and its integration contract. -Framework stages operate on source documents, source units, and source -references rather than format-specific structures. A source reference identifies -an ordered range of generic source units. Framework code preserves those ranges -and does not merge or rewrite them unless a stage module explicitly owns that -behavior. Every source unit carries a validated self-reference to its containing -document and its own unit ID. +A source reference identifies an ordered range of generic units. Framework code +preserves ranges unless a module explicitly owns their transformation. Every +source unit carries a validated self-reference to its document and unit ID. +Auxiliary references provide context or disambiguation, not source evidence; +they must not be converted into source references. -Extract modules own artifact semantics, prompt use, response schemas, and -domain interpretation. Domain-specific concepts remain in the relevant module, -validator, shared domain helper, and artifact contract. +Modules own domain semantics, prompt selection, interpretation, deterministic +identity, and supported fallback construction. Physical asset centralization does +not transfer semantic ownership. Native prompt declarations own default output +contracts and schema selections; PromptKit prepares schemas and performs +structural validation and repair. Modules declare small static resource +dependencies for preflight, not manifests, schema registries, or fingerprints. +Intrinsic durable artifact schemas remain application contracts. -Physical centralization of LLM-facing content does not transfer semantic -ownership from those modules. Modules retain their manifests, response-schema -identities, prompt ordering, and registration, while reading only their scoped -content subtree. Generic framework code remains domain-neutral when it reads -its own scoped generic assets from the shared content container. +Typed artifact registrations declare one stable kind and exact Go type through +extraction, merge, normalization, and semantic validation. Resolution requires a +compatible codec and matching kind-specific variants. Private framework erasure +reports incompatibility as an error rather than a panic. -Typed artifact registrations declare one stable artifact kind and exact Go -type from extraction through merge, normalization, and semantic validation. -Pipeline resolution requires a compatible codec and matching kind-specific -variants before a typed lane can be accepted. Framework-owned erasure remains -private and must report type incompatibility as an error rather than a panic. +A kind may provide a typed evidence projection copying direct generic source +references. Preparation proves its exact codec-type compatibility. Evidence is +reconstructed from serialized normalized artifacts, including supported fallbacks, +without generic code inspecting domain JSON or importing domain types. The output +boundary owns publication. -An artifact kind may additionally provide a typed evidence projection that -copies its direct generic source references. Preparation proves that projection -matches the artifact codec's exact Go type before retaining it for an output -policy. The runner reconstructs evidence only from accepted serialized -normalized artifacts, and the output boundary owns any resulting publication. -Generic framework code never infers evidence by inspecting domain JSON or -depends on domain artifact types. - -Auxiliary references provide context or disambiguation. They are not source -evidence and must not be converted into source references. +When a model selects an entity, modules supply contextual selections and attach +opaque application IDs deterministically after exact resolution. Models do not +receive or reproduce opaque application IDs. Reconciliation may expose +request-local candidate handles, resolved before typed application and never +used as durable identity. Retain +[ADR-0012](../adr/0012-resolve-opaque-entity-identifiers-deterministically.md) and +[ADR-0013](../adr/0013-use-request-local-candidate-handles-for-semantic-reconciliation.md)'s +identity and safety decisions. ## Pipeline Composition And Ownership -Module selection is configuration- and registry-driven. The framework resolves -named pipeline definitions, applies explicit defaults and runtime overrides, -and verifies module availability and capabilities before execution. Structural -pipeline choices must not be scattered through conditionals or hidden behind -ad hoc command flags. +Configuration and registries select modules, explicit defaults, references, and +runtime overrides. Validate selected module and validator options before +execution. Preparation constructs the complete implementation set; the runner +accepts only that prepared set. Construction failures precede source parsing. -Resolution validates every selected module and validator option set. A separate -preparation boundary then constructs the complete input, chunk, lane, -validation, and output implementation set in pipeline order. The runner accepts -only that prepared set, so construction and dependency failures occur before -source parsing or any other module operation. +Stage ownership remains explicit: -Stage ownership is explicit: +- input translates external material into the generic source model; +- chunk produces source-addressed plans; +- extract produces artifacts from chunks; +- merge combines extraction outputs; +- normalize reconciles merged output; +- output encodes domain results and enabled auxiliaries into logical files. -- input modules convert external material into the generic source model; -- chunk modules partition source material for extraction; -- extract modules produce domain artifacts from chunks; -- merge modules combine accepted extraction outputs; -- normalize modules reconcile merged output; -- output modules encode accepted results and run outcomes into logical files. - -Chunk modules produce source-addressed chunk plans rather than materialized -chunks. The framework validates and materializes those plans into the generic -chunk representation before chunk validation and lane execution. Plan reuse is -therefore independent of the configured pipeline, module options, references, -lanes, validators, and LLM profile: the canonical source digest selects the -plan, while the current run still applies its configured chunk validators to -the materialized chunks. +The framework materializes plans into chunks before candidate assessment and +lane execution. LLM plans use the accepted-result cache; hits rematerialize +against the current source without new semantic assessment. Source identity alone +does not select a canonical partition. Deterministic chunkers execute normally. The framework owns orchestration, origin enrichment, aggregation, and handoff -provenance. Modules return logical results and classified diagnostics; they do -not own CLI reporting, physical output, cache, or -debug roots, durable file placement, or checkpoint and debug lifecycle. +provenance. Modules return logical results and classified diagnostics, not CLI +reports, physical roots, durable placement, or capture lifecycle. -After pipeline-wide chunking, extraction uses bounded framework concurrency. -One run-wide worker pool receives chunk-scoped lane jobs in deterministic -chunk-first, lane-second order. A lane may begin its merge and normalize -continuation only after all of its extract jobs are terminal; that continuation -remains serial within the lane, while bounded continuations for different lanes -may overlap. The framework must not create unbounded goroutines per lane or -chunk. +After pipeline-wide chunking, one pool bounds ready extraction and ready +merge/normalize lane-completion jobs together. Extraction dispatch uses stable +chunk-first, lane-second order. A continuation becomes ready after its lane's +extraction finishes; a worker does not wait for prerequisites while holding a +slot. Validation and correction run inside the current job without another +worker acquisition. Ordered steps retain their barriers. Do not create unbounded +goroutines per lane or chunk. -Completion timing does not choose public ordering or errors. The coordinator -orders accepted artifacts, grouped diagnostics, rejections, checkpoint events, and -framework errors by stable pipeline scope. Rejections do not cancel unrelated -work. A framework error cancels derived work, prevents undispatched work from -starting, waits for started work, and prevents output encoding. +Completion timing does not choose public ordering or terminal errors. The +coordinator orders artifacts, grouped diagnostics, and errors by stable scope. +Cancellation stops dispatch, drains started work, and releases workers and native +admission before returning. -Warnings are process-only signals: configuration degradation, approved fallback, -or incomplete configured validation. Quality uncertainty and grounding findings -are advisories; successful canonicalization and cleanup are observations. -Modules choose that semantic classification, while the framework attaches -origin, aggregates groups, enforces bounds, and presents final collections. -An ordinary successful run therefore has zero warnings. See -[ADR-0015](../adr/0015-separate-process-warnings-from-quality-diagnostics.md) -for the decision rationale. +Warnings are process signals: configuration degradation, qualified fallback, +or incomplete configured work. Quality and grounding findings are advisories; +accepted cleanup is observation. Modules own classification, while the framework +adds origin and aggregates bounded groups. A non-degraded successful run has no +actionable warnings. Retain +[ADR-0015](../adr/0015-separate-process-warnings-from-quality-diagnostics.md)'s +classification decision, not its retired durable reporting contracts. -## Validation +## Validation And Disposition -Validation is a framework-managed boundary around outputs from chunk, extract, -merge, and normalize stages. Validators receive immutable stage output and -make an explicit whole-output decision: approve, reject, fail, or skip when a -runtime prerequisite is unavailable. +Validation is a framework-managed boundary around chunk, extract, merge, and +normalize candidates. Validators receive immutable views and return approve, +reject, fail, or unavailable findings. Typed validators receive domain values; +chunk validators receive generic chunks; serialized validators receive bytes +and schema metadata. Incompatible targets or kinds cannot substitute for one +another. -Typed artifact validators receive the domain value directly. Chunk validators -receive source-zone chunks, while serialized validators receive immutable -representation bytes and declared schema metadata. A validator registered for -one target or artifact kind cannot satisfy an incompatible selection. +Assess each usable initial or revised candidate once through every applicable +validator in configured order. Rejections do not short-circuit later validators. +Earlier approvals and unchanged revisions do not excuse reassessment; the final +allowed response also receives the whole chain. Cancellation or fatal framework +failures can interrupt assessment. There is no duplicate final assessment. -The framework runs every applicable validator sequentially in configured order. -It aggregates rejections, exhausted validator failures, and skips before the -producer policy chooses a disposition. A completed rejection never advances. -With no rejection, an exhausted validator failure may fail the run or, under -the configured `warn_continue` policy, advance a structurally valid candidate -with explicit incomplete-validation provenance. Validators report findings; -they do not choose candidate disposition. +Actionable correction guidance is distinct from reason codes, operator messages, +and failures. The detecting validator or module owns it. The pipeline aggregates +it in order; a native module template supplies the correction assignment. Missing +required guidance is a contract failure, not an invitation to invent instructions. +Unavailable assessment is not a semantic defect or ordinary acceptance. -A completed rejection supplies a stable reason code for internal provenance -and bounded actionable correction guidance for the candidate producer. Reason -codes, validator keys, and operator-facing messages remain diagnostic data; -they are not model instructions. The framework constructs model-facing retry -text only from the semantic guidance and fails the contract rather than -inventing or truncating missing guidance. +The producer's resolved mode owns terminal disposition. Strict requires +acceptance; explicitly selected permissive mode may construct a supported +fallback. A fallback must satisfy intrinsic shape and downstream invariants but +does not rerun the ordinary validator chain or claim semantic approval. Unsupported +or invalid fallback construction fails normally. Caller cancellation never +initiates fallback. Exact mode and turn controls belong in [Configuration](../config.md). -An LLM-backed module may also request a feedback-aware retry when its own -deterministic translation or typed safety policy rejects a structurally valid -model response. It must supply model-facing guidance separately from its -reason code, operator message, and fallback diagnostics, together with the -exact `single_response_v1` candidate. A feedback-free module retry remains -valid when no exact candidate exists. +Default validator chains remain central production composition policy. +Configuration can replace a stage-local default, including with an explicit empty +chain. Configured order is authoritative. Intrinsic codec/source correctness +still applies with an empty chain. -Default validator chains are production composition policy and are registered -centrally by stage and module. Configuration may replace a stage-local default, -including with an explicitly empty chain. Configured validator order is -authoritative; the framework must not silently reorder it. +## Native Execution Boundary -## LLM Boundary +The composition root constructs one shared native PromptKit engine from the +application's ConfigKit result. All model roles use that engine. PromptKit owns +transport, authentication, transient retries, structural validation and repair, +conversation history, turn iteration, and backend admission. Notarius does not +pin omitted native budgets or add outer retries to extend them. Deterministic +operations have no generic retry budget or synthetic model request. -Modules and validators use transport-neutral structured completion contracts. -Provider request and response types, authentication, transport behavior, and -provider error adaptation remain inside the LLM runtime. +One small typed producer helper connects module-owned fixed inputs, native +request builders, interpretation, and supported fallback functions with the +pipeline assessor. Initial/correction pairs use complete replacement responses +and the same declared schema. Enduring principles belong in initial system +context; correction supplies current findings and a new assignment. Native +coordination carries the growing exchange without manual response replay or +regeneration of original references. Normalization applies each proposal afresh +to fixed originals, without cross-turn unions. -The caller of the LLM owns prompt selection, prompt inputs, response schema, -and interpretation of structured output. Provider adapters do not own source- -or domain-specific prompt logic. +Check native errors and validation status before interpretation, including failed +validation with nil Go error. Native success, stopped, or exhausted status alone +does not establish domain acceptance. Do not recover an earlier candidate after +a result-less continuation failure or assess an undelivered response. LLM +validators execute once per assessment, without a separate semantic correction +loop. Calls respect cancellation and applicable native timeouts. -PromptKit owns bounded structural correction within one structured completion. -Notarius owns outer stage attempts, semantic validation, and acceptance policy; -the two budgets must remain separate. - -An LLM-backed producer can participate in semantic correction only when it -declares `single_response_v1` and returns the exact one response that directly -controlled its candidate. On an actionable validator rejection or -feedback-capable module retry, the framework rebuilds the ordinary request and -appends only the latest defective response as an `assistant` message plus one -aggregated `user` correction message. This is a fresh replacement request, not -a growing conversation. The retry budgets, terminal policy, and sensitive-data -rationale are recorded in -[ADR-0014](../adr/0014-feedback-aware-validation-retries.md) and -[ADR-0016](../adr/0016-feedback-aware-module-requested-retries.md). - -When a model selects an application entity, callers must supply a contextual -selection and deterministically attach the opaque application identity whenever -the selection resolves exactly. Models do not receive or reproduce opaque -application identifiers. Semantic reconciliation may instead expose -contiguous, one-based candidate handles that exist only for one request; -deterministic code resolves them before typed application, and they never -become durable identity. This is the approved request-local-label application -of [ADR-0012](../adr/0012-resolve-opaque-entity-identifiers-deterministically.md) -recorded by -[ADR-0013](../adr/0013-use-request-local-candidate-handles-for-semantic-reconciliation.md). - -LLM calls and other external operations accept cancellation and respect -timeouts. Concurrency control belongs in shared runtime plumbing rather than in -individual modules. - -The application-wide LLM scheduler bounds actual provider calls independently -of framework worker limits. Every LLM-backed module, retry, and validator uses -the single injected scheduled client, including work performed by overlapping -lanes. Provider runtime adapters may enforce a narrower backend-specific limit -beneath this mandatory application-wide scheduler. +Worker limits and native backend admission are independent. Profiles selecting +one backend ID share its engine capacity; different IDs do not imply an endpoint +limit. Notarius injects no backend concurrency/queue defaults and has no +cross-backend scheduler, second admission queue, or capacity retry loop. Completed +turns release admission before assessment so nested validators can use the same +limited backend. Native failures retain their causes and follow the producer's +mode; they do not become correction guidance. ## Configuration And Provenance -Configuration loading, precedence, defaults, redaction, and validation are -centralized through the ConfigKit-backed boundary. `NOTARIUS_CONFIG` is limited -to discovery; field-level environment overlays are not configuration input. -Named pipeline definitions make structural composition explicit and discoverable. -Explicit command overrides are permitted -when they do not obscure the configured pipeline structure. +ConfigKit owns loading, imports, composition, overlays, inheritance, source +contexts, provenance, and secret-aware inspection. Notarius owns application +meaning through supported definition rules and domain resolution, retaining +[ADR-0020](../adr/0020-delegate-configuration-mechanics-to-configkit.md). +`NOTARIUS_CONFIG` is discovery only; field-level environment overlays are not +application configuration. Named pipelines keep structural composition explicit. -Run preparation fails before stage execution when statically discoverable -modules, capabilities, reference bindings, or explicitly selected profiles are -invalid or incompatible. +One application-owned result supplies engine construction and whole-app/focused +inspection. Preserve source contexts and captured selection evidence rather than +reparsing settings or maintaining duplicate loaders. Request overrides retain +native supplied-value semantics. CommandKit, ckconfig, and ckprompt own standard +commands, parsing, presentation, and delivery outcomes. Notarius supplies domain +commands and report content, not a universal result envelope. -Run manifests record enough resolved pipeline, module, source, reference, and -LLM provenance to make a run auditable after configuration changes. Manifests -record identities and bounded validation summaries rather than secret, raw -model, correction, or large payload content. +Offline preflight shares domain resolution and checks statically known module, +reference, initial/correction/validator prompt, and applicable profile dependencies +before execution. It requires no transcript, reference bytes, credentials, +provider call, or run storage. Dynamic selections remain runtime checks. +Shared loaders allocate no run storage; capture belongs to the run path. ## Trust Boundaries And Threat Scope -Where useful, state the trust boundaries and threat scenarios relevant to the -application's adopted requirements, along with scenarios it explicitly excludes. -For example, distinguish handling untrusted application input from defending -against a privileged actor concurrently modifying the host filesystem. +Source material and model responses cross application validation boundaries. +Host access policy belongs to deployments. Hypothetical privileged actors +concurrently modifying the filesystem do not justify new application defenses +without an explicit decision. This does not remove adopted correctness boundaries +or claim excluded scenarios cannot occur. Raise new threats or guarantees under +[Agent Policy](agents.md). -Threat exclusions define the scope of the adopted requirements. Base defenses -and guarantees on those requirements and raise unsettled scope for a deliberate -decision under the [agent policy](agents.md). - -Keep this discussion proportionate to the application. Configuration, -operations, and internal documents retain ownership of credential supply, -runtime handling, and implementation mechanisms under the documentation policy. +Configuration, operations, and internal documents own credential supply, +runtime handling, and mechanisms under [Documentation Policy](documentation.md). ## State, Output, And Safety -Source material and model responses cross application validation boundaries. -Host access policy belongs to the deployment; hypothetical privileged actors -concurrently changing the filesystem do not justify new application defenses -without an explicit decision. This scope statement does not remove the adopted -correctness boundaries below or claim excluded scenarios cannot occur. Follow -[Agent Policy](agents.md) when a new threat or guarantee needs a decision. +Output, cache, and debug retain independent roots and lifecycles: -Notarius exposes three filesystem surfaces with independent roots and -lifecycle: +- output is durable domain data; modules encode logical files and the CLI places them; +- cache is reconstructible accepted-artifact and LLM-plan state; +- debug is explicitly requested run/operation evidence, not a consumer API. -- output is durable user data; output modules define logical files and the CLI - owns their placement; -- cache is reconstructible state, with separate chunk-plan and checkpoint - families; and -- debug is explicitly requested inspection data, combining a redacted summary - with a detailed trace. +Cache identity is the opaque native initial prepared message-content hash. +Add no configuration, schema, model, session, implementation, or dependency +dimensions. Store only final accepted codec artifacts or materialized/assessed +plans, not responses, exchanges, proposals, provenance envelopes, or historical +usage. Hits perform ordinary decoding/source-usability checks without new +semantic assessment. Policy/settings changes may still hit; explicit +bypass/refresh controls re-execution. Deterministic work is not cached through +manufactured requests. -Chunk plans are keyed only by canonical source digest. Configured checkpoint -recording is independent of checkpoint reuse; checkpoints are loaded only for -an invocation that explicitly requests resume. Debug is never a cache input and -is never created without an explicit request. Pipeline modules receive -collaborator interfaces and never physical roots. +Fallback and affected dependent work skip lookup and storage; unaffected accepted +predecessors remain reusable. Refresh replaces an entry only after new acceptance. +Failure preserves an old entry. Cache I/O is best effort and cannot fail otherwise +successful work. Legacy caches remain unread and unmodified, with no checkpoint +resume, canonical source-only plan store, or automatic migration. -Deployments own cache access policy. Cache writers request conventional maximum -creation modes and let the process umask, setgid inheritance, and default ACLs -determine effective access. Notarius owns the separate safety boundary: -confined paths, symlink rejection, atomic publication, validation, and -recovery. It does not chmod, chown, chgrp, or migrate cache entries to impose -an application policy. This rule applies to chunk-plan and checkpoint cache -only; output and debug retain their separate contracts. [Operations](../operations.md) -owns deployment provisioning and retention guidance. +Deployments own cache access policy. Writers request conventional maximum modes +and let umask, setgid inheritance, and default ACLs select effective access. +Retain confined paths, symlink rejection, atomic publication, validation, and +recovery without chmod/chown/chgrp convergence or cache migration. +[ADR-0017](../adr/0017-defer-cache-access-policy-to-deployment.md)'s boundary now +applies to the unified cache; output and debug retain separate contracts. -Only accepted, completely validated producer output is reusable checkpoint or -chunk-plan state. Rejected, structurally invalid, and validation-incomplete -results cannot become cache or checkpoint inputs, even when a -`warn_continue` result is allowed to advance in the current run. This -ineligibility follows derived merge and normalize results and generated -references for the remainder of the run: current-run handoff remains allowed, -but no dependent cache or checkpoint may be loaded or published. +Preserve intrinsic domain payload schemas and logical paths, including supported +fallbacks. Consumers need no index, manifest, persisted receipt, or reporting +sidecar. Publish artifacts and finish capture before a native completion report. +Disclose fallback use independently of diagnostic verbosity. Delivery failure may +leave partial stdout and valid published artifacts; retain those artifacts, +useful locations, and distinct operation/delivery causes without model retries. +Exact report fields and outcomes belong in [CLI Reference](../cli.md). -Writes are atomic where practical. Paths for writes, moves, overwrites, and -deletion must be narrow and explicit. Notarius never automatically deletes -output or requested debug bundles; cache cleanup is explicit and recoverable. +One native execution group records actual initial, continuation, and nested +validator executions once. Preserve identity, terminal coverage, and +unknown-versus-zero usage without synthetic IDs, copied totals, or historical +execution claims on cache hits. -Secrets must not appear in errors, logs, output, cache, debug summaries, -traces, manifests, documentation, examples, or redacted configuration. Raw -assistant responses and complete correction messages are attempt-local and are -excluded from ordinary durable records and summaries; the requested detailed -debug trace is the sole diagnostic surface allowed to retain them. Debug -collection is allowlisted to application-owned payloads and must not capture -unrelated process environment values or filesystem content. Trace data may -contain application data and therefore inherits its sensitivity; operators own -access controls and retention. Physical layout and operation are defined in -[Operations](../operations.md). +Debug is never a cache input. Capture requires explicit enablement and a nonempty +effective destination; directory configuration alone does not enable it. Requested +capture is required work: allocate before engine construction/preflight, record +incrementally, and finalize once before successful reporting. Capture failures +fail the invocation while retaining available evidence and primary causes. +Update the relative `latest` link at session start, never completion. + +Debug reuses prepared details and native evidence without another preparation/ +model call or parallel conversation ledger. Representations are private +diagnostics, not stable serialization contracts or exhaustive provider transcripts. +Use ConfigKit secret-aware inspection and preserve native credential omissions. +Do not directly serialize marked secrets, unrelated environment, or unselected +filesystem content. Do not heuristically scan or rewrite source, prompts, +responses, or arbitrary application text. Selected evidence may contain full +source/reference data, responses, and exchanges; it is not a sanitized bundle. +Operators own access and retention as for other application data. + +Writes are atomic where practical. Write, move, overwrite, and deletion targets +must be narrow and explicit. Never automatically delete output or requested debug +bundles. Cache cleanup remains explicit and recoverable. +[Operations](../operations.md) owns physical layout, provisioning, +retention, and recovery guidance. ## Platform And Distribution @@ -404,7 +381,9 @@ Notarius does not aim to provide: - an arbitrary workflow graph or general workflow language; - source-format or extraction-domain behavior in generic framework packages; -- provider-specific contracts exposed to modules; +- provider-specific transport contracts exposed to modules; - structural pipeline composition through ad hoc CLI flags; - implicit cross-stage behavior that bypasses the fixed pipeline; +- replacement kit APIs, duplicated schedulers, or compatibility parsers; +- a public debug serialization format or mandatory execution ledger; - abstractions introduced solely for hypothetical future complexity. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index e8926235..d76412b2 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1842,6 +1842,27 @@ complete final suite after documentation and remaining corrections. ## Stage 38: Record Successor Decisions And Align Architecture Policy +**Status:** Complete. Added sequential ADRs 0021–0024 for native kit/resource and +admission ownership, continuation and application assessment, unified accepted- +result caching, and artifact-first reporting/debug. Updated earlier status links +to distinguish full and partial supersession, including related version/profile, +registrar, reconciliation-fingerprint, and durable-diagnostic clauses. All changed +historical ADR titles, dates, and accepted bodies are byte-for-byte preserved; +ADRs 0012, 0017, and 0020 remain unchanged. Retained content-only assets, contextual +identity, typed/domain/source boundaries, process-warning classification, and +deployment-owned cache permissions. + +Aligned architecture policy with implemented native execution types, one worker +pool, independent native admission, complete candidate assessment, explicit +fallbacks, content-keyed reuse, native reports, and optional private debug evidence. +The adopted upstream baseline is unchanged and the local adaptation is identified. +Reviewed these invariants against the feature roadmap and current loading, +producer/assessment, worker, cache, publication, and recorder code. Local decision/ +policy links and anchors and `git diff --check` pass; existing pipeline, cache, +recorder, and configuration package tests pass (cached). No release/history rewrite, +new runtime contract, access policy, or permanent validation gate was introduced. +Canonical references and internal guides remain assigned to the following stages. + **Goal:** document deliberate boundary changes without rewriting their history. **Inspect/change:** `docs/policy/architecture.md`, ADRs 0005–0007, 0011, 0014,