Record CommandKit verification and migration plan
This commit is contained in:
@@ -26,7 +26,9 @@ selects native composition, shared loaders, and application preflight. Item 4
|
||||
selects native reporting, machine output, and outcomes. Item 5 selects conditional
|
||||
debug capture through the shared `--debug` flag, native diagnostics, and a run/
|
||||
operation bundle with a `latest` symlink. Capture depth, exact run-report fields,
|
||||
and verification remain open below.
|
||||
and other concrete record details remain open below. Item 6 selects verification,
|
||||
migration guidance, and incorporation into the master roadmap; that incorporation
|
||||
is still pending.
|
||||
|
||||
The PromptKit architecture and checkpoint decisions remain in force. This
|
||||
discussion develops the CommandKit portion of that migration and its interactions
|
||||
@@ -85,8 +87,9 @@ spending migration effort recreating existing CLI conventions.
|
||||
|
||||
Adopt the standard capability suite selected in item 1, including upstream
|
||||
commands and recommendations beyond the existing Notarius surface. Items 2–4
|
||||
select application inputs, composition, and reporting; the remaining discussion
|
||||
determines debug capture, verification, and concrete record details.
|
||||
select application inputs, composition, and reporting; items 5–6 select debug
|
||||
capture principles, verification, and migration. Concrete record details remain
|
||||
open as identified below.
|
||||
|
||||
Pipeline and domain code should remain independent of CommandKit. Native
|
||||
PromptKit observations supply execution facts; Notarius attaches application
|
||||
@@ -114,8 +117,8 @@ Two application needs already identified for deliberate treatment are:
|
||||
4. Complete command and reporting migration before final end-to-end verification
|
||||
and migration documentation.
|
||||
|
||||
This is the agreed sequencing direction, not a replacement phase plan. Concrete
|
||||
work units and their placement in the master roadmap follow the discussion below.
|
||||
Item 6 selects the replacement phase sequence for incorporation into the master
|
||||
roadmap. The master roadmap has not yet been updated.
|
||||
|
||||
## 1. Adoption Scope And Dependency Baseline
|
||||
|
||||
@@ -886,28 +889,136 @@ The current [debug lifecycle](../internal/state.md#debug-bundles) and
|
||||
|
||||
## 6. Verification, Migration, And Roadmap Incorporation
|
||||
|
||||
**Discussion / decision:** Pending.
|
||||
**Discussion / decision:** Agreed. Complete one coordinated native cutover, with
|
||||
CommandKit composition introduced early, verification focused on Notarius's
|
||||
integration and domain responsibilities, and practical release migration guidance.
|
||||
Incorporate these decisions into the existing master roadmap; another replacement
|
||||
document is unnecessary. Implementation and master-roadmap incorporation are pending.
|
||||
|
||||
- Which focused command and subprocess tests protect the new native command,
|
||||
output, outcome, receipt, and debug behavior? Which tests of superseded CLI
|
||||
contracts and parser/formatter implementation can retire?
|
||||
- Which representative assembled workflows prove that execution, inspection,
|
||||
and preflight use the same intended configuration without duplicating lower-level
|
||||
coverage? Keep the default suite offline and independent of real credentials.
|
||||
- How should CommandKit steps replace the master roadmap's deferred Phase 7,
|
||||
and which migration examples, documentation, and ADRs need coordinated updates?
|
||||
- What release-documentation guidance will show operators and subprocess callers
|
||||
how to migrate their workflows and integrations to the new conventions?
|
||||
- Reconcile older debug-sensitivity language, special-handling requirements, and
|
||||
heuristic redaction behavior/tests with item 5's clarified boundary. Do not
|
||||
reintroduce them as implicit requirements while migrating documentation or code.
|
||||
### Focused Verification Of Application Behavior
|
||||
|
||||
After discussion, reconcile the master roadmap's U1/U9/U10 entries, sequencing,
|
||||
removals, and acceptance criteria, including its treatment of existing inspection
|
||||
commands and machine-output compatibility. Carry forward the explicit non-goal
|
||||
of CLI backward compatibility and release migration guidance. Preserve other
|
||||
unresolved choices explicitly. Then mark this document as the discussion record
|
||||
and keep implementation status in the master roadmap.
|
||||
Use real CommandKit, ConfigKit, and PromptKit components in representative
|
||||
assembled workflows, with a fake at the provider boundary. Keep the default
|
||||
suite deterministic, offline, and independent of real credentials. Following the
|
||||
[testing policy](../policy/testing.md), assign each behavior an appropriate test
|
||||
owner and avoid duplicating upstream parser, renderer, or retry coverage.
|
||||
|
||||
Select a small set of workflows that collectively establish:
|
||||
|
||||
- Execution, inspection, and preflight resolve the same intended configuration
|
||||
and prompt definitions, including application overrides and companion sources.
|
||||
- Help, version, and completion work without application configuration; offline
|
||||
preflight makes no provider calls.
|
||||
- `run` and `pipeline run` reach the same application behavior, with the selected
|
||||
flags, supplied-value distinctions, and overrides mapped correctly.
|
||||
- Reports, diagnostics, and exit outcomes follow item 4, distinguishing domain
|
||||
rejection, command failure, and presentation failure after publication.
|
||||
- A downstream failure followed by resume reuses successful predecessors under
|
||||
the simpler checkpoint policy, with explicit recomputation and consumed-input
|
||||
changes handled as selected in the master roadmap.
|
||||
- Debug capture follows item 5's directory selection, reporting-mode independence,
|
||||
`latest` update timing, useful evidence, and allocation/write/finalization
|
||||
failure behavior.
|
||||
|
||||
Use subprocess tests where the actual executable boundary adds protection:
|
||||
stdout/stderr separation, process exit status, and representative machine output.
|
||||
Test domain interpretation, validation, continuation, and recovery at their
|
||||
appropriate package boundaries. Preserve meaningful native-execution acceptance
|
||||
checks from the master roadmap without repeating them throughout CLI tests.
|
||||
|
||||
### Retire Tests With Superseded Machinery
|
||||
|
||||
Review tests of the private adapter, global scheduler, manual continuation
|
||||
protocol, broad checkpoint fingerprints, and superseded CLI contracts for removal
|
||||
as those implementations retire. Where an application requirement survives, test
|
||||
that requirement through its new boundary. Do not mechanically reproduce old
|
||||
parser, formatter, or wrapper tests against the new implementation.
|
||||
|
||||
Avoid freezing complete upstream help output, incidental diagnostic wording, or
|
||||
every field of native debug snapshots. Debug verification should establish useful
|
||||
evidence and correct failure handling while allowing representations to evolve
|
||||
for convenience. Do not require exhaustive provider-attempt reconstruction or
|
||||
strict serialization deduplication.
|
||||
|
||||
Reconcile older debug-sensitivity language, special-handling requirements, and
|
||||
heuristic redaction behavior/tests with item 5. Avoid directly disclosing values
|
||||
marked as secrets by ConfigKit and preserve native omissions. Do not carry forward
|
||||
heuristic scrubbing of prompts, source content, or model output, or special
|
||||
debug-data handling requirements. Operators secure debug artifacts as they do
|
||||
ordinary application inputs and outputs.
|
||||
|
||||
### Migration Guidance And Documentation
|
||||
|
||||
Provide release guidance for operators and subprocess callers, with a practical
|
||||
old-to-new migration map covering:
|
||||
|
||||
- Commands and flags, including the native `run` alias, configuration inspection,
|
||||
and `--format`.
|
||||
- Application and PromptKit configuration changes, source discovery, profiles,
|
||||
and companion files.
|
||||
- Native backend admission, independent worker settings, and revised retry controls.
|
||||
- Checkpoint reuse, explicit recomputation, and the selected treatment of existing
|
||||
checkpoint files.
|
||||
- Machine reports, exit outcomes, and the distinction between successful command
|
||||
execution and accepted domain results.
|
||||
- Debug capture under `/tmp/notarius-debug`, `latest`, and the interaction between
|
||||
capture and reporting modes.
|
||||
|
||||
CLI backward compatibility remains a non-goal; migration guidance does not imply
|
||||
compatibility shims or preservation of old command behavior. Existing checkpoint
|
||||
and published-artifact compatibility remain explicit application decisions about
|
||||
stored work and consumer data, separate from CLI syntax.
|
||||
|
||||
Maintain runnable examples for embedded defaults and operator overrides. Update
|
||||
canonical CLI, configuration, operations, integration/consumer, and internal
|
||||
documentation as the corresponding behavior lands. Record major architectural
|
||||
changes in successor ADRs, preserving accepted decision text under the
|
||||
[documentation policy](../policy/documentation.md#architecture-decision-records).
|
||||
Keep future behavior in roadmap documents until implemented.
|
||||
|
||||
### Incorporation Into The Master Roadmap
|
||||
|
||||
Replace the deferred CommandKit Phase 7 with adoption throughout the existing
|
||||
master roadmap. Use this phase sequence:
|
||||
|
||||
1. Establish the agreed dependency baseline and remaining contract decisions.
|
||||
2. Build native application composition, shared loaders, standard commands, and
|
||||
the dedicated preflight package alongside native source assembly.
|
||||
3. Prove a representative native pipeline through the actual command surface,
|
||||
including reporting and useful debug capture.
|
||||
4. Migrate remaining roles and remove the adapter, scheduler, retry machinery,
|
||||
and custom CLI infrastructure.
|
||||
5. Complete checkpoint migration, durable records, and capture behavior.
|
||||
6. Verify the assembled application and finish migration documentation.
|
||||
|
||||
Each phase delivers a reviewable result with appropriate verification. Tests and
|
||||
documentation proceed throughout; the final phase checks integration and
|
||||
completeness. The phases do not require separate releases or an intermediate
|
||||
compatibility framework.
|
||||
|
||||
Resolve U1's dependency baseline and U10's CommandKit scope from items 1–4. Narrow
|
||||
U9 to remaining durable-field, accounting, and capture details within the selected
|
||||
boundaries. Reconcile removal inventory and acceptance criteria, including old
|
||||
inspection-command and machine-output compatibility assumptions. Preserve
|
||||
unresolved schema handling, role-specific retry/fallback policy, worker defaults,
|
||||
checkpoint transition choices, and other concrete choices until actually settled.
|
||||
|
||||
Once incorporated, the master roadmap owns implementation sequencing and status;
|
||||
both migration notepads remain discussion records. Do not maintain competing
|
||||
implementation checklists in the notepads.
|
||||
|
||||
### Completion Checks
|
||||
|
||||
Use focused checks while implementing and the existing repository-wide
|
||||
`go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` checks for the
|
||||
completed cutover. Check the affected maintained examples, documentation links,
|
||||
dependency attribution, and removal inventory as part of this migration.
|
||||
|
||||
Live provider smoke tests remain explicitly optional and outside the deterministic
|
||||
default suite. These change-specific migration checks do not create additional
|
||||
permanent release gates; publication continues to follow the
|
||||
[release policy](../policy/release.md). Recording these decisions does not claim
|
||||
that implementation acceptance checks have already passed.
|
||||
|
||||
## Evidence And Reading
|
||||
|
||||
|
||||
Reference in New Issue
Block a user