From ba9079ee7495189af22a155df08a8044915aad95 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 18 Nov 2026 01:10:44 +0000 Subject: [PATCH] Record algorithmic refactoring opportunities --- docs/roadmap/remediation.md | 158 ++++++++++++++++++++++++++++++++++-- 1 file changed, 153 insertions(+), 5 deletions(-) diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md index fa16dc2..0260e5e 100644 --- a/docs/roadmap/remediation.md +++ b/docs/roadmap/remediation.md @@ -12,9 +12,15 @@ An integration-focused follow-up reviewed the same implementation at command, preparation, publication, restore, and inspection workflows. Items 5–6 are additional findings from that pass; items 1–4 remain outstanding. +An algorithm-focused pass reviewed all twelve stages at +`583c5bdf1bc8f0af8ec2e257bcb03ecccbf283b1`. Items 7–10 are refactoring +opportunities for consideration. Complexity follows the code and the selected +ConfigKit v1.2.1 implementation; no operational timings or benchmarks were +collected. The pass preserves algorithms expressly chosen by the plan. + The native configuration architecture is substantially implemented. The items -below are the remaining required corrections; this review does not implement -their fixes. +below record the six required corrections and four additional algorithmic +opportunities. This document implements none of their fixes. ## 1. P2: Distinguish effective resource origins from superseded history @@ -198,6 +204,125 @@ the updated values. Include audio or previous-session selection as well as title Verify that restore still uses its selected immutable commit and that published objects are unchanged by the authoring update. +## 7. P3: Reuse enumerated source records and one inspection source index + +Stage: 9. + +Locations: `internal/config/effective_pipeline.go:57–75`, +`internal/config/pipeline_configkit.go:354–371`, and +`internal/config/artifact_catalog.go`, `declarationDefinitionSources`. + +`emptyImportSelections` enumerates `Result.Sources()` and then calls +`Result.Source(id)` for every enumerated record. `pipelineSourcePaths` does the +same lookup for every retained source. In ConfigKit v1.2.1, completed-result +source lookup scans the source slice. Enumerating S sources this way performs +Θ(S²) source-ID comparisons, although the enumerated records already contain +the required information, including empty-import evidence. + +The artifact catalog additionally rebuilds this source map for each declaration. +For A entries this introduces O(A S²) source-index work; with one imported file +per artifact, that component can become cubic in A. This is separate from the +native history queries and necessary output construction. + +Potential refactoring: consume the complete enumerated records directly. Build +the source-ID/path map once per inspection operation and share it with declaration +projection. This reduces empty-import enumeration to O(S + E), where E is the +returned selection evidence, and source-index work across the catalog to +expected O(S + A). Expanded members can also reuse the family-source lists +already retained in the family catalog. Keep native history and origin queries +as the provenance authority. + +Verification: temporary functional checks confirmed that enumerated records and +individual lookups have equal contents, and that direct enumeration preserves +source paths and nested empty-import selections. Preserve operator source/catalog +output, including composite and inherited contributors, when refactoring. + +## 8. P3: Reuse party metadata and per-character provenance within inspection + +Stage: 9, consuming Stage 5's imported party result. + +Locations: `internal/config/artifact_catalog.go:67–105` and +`internal/config/effective_pipeline.go`, `appendGeneratedArtifactSources`. + +`partyCharacterSources` calls `result.Metadata()` separately for Lookup and +History. ConfigKit v1.2.1 returns detached metadata by copying the complete +effective-path and field maps, including their origin records. This helper is +called for each generated artifact in the catalog and for each generated +artifact field in source inspection. The same character's provenance is +therefore repeatedly recomputed across fields and families. + +Let M be the copied party-metadata size and P the number of provenance requests. +The avoidable metadata-copy work and cumulative allocation are Θ(P M). Even +with one family and bounded-size character entries, M and P both grow with the +character count C, producing quadratic work. More families and fields multiply +the requests. This describes the provenance pass, not the complexity of the +whole command or unchanged party validation. + +Potential refactoring: obtain one native Metadata snapshot and source index per +inspection operation. Resolve each requested character's source list once and +reuse it for that operation. The full-metadata copy term becomes O(M), with +native per-character query costs and required output emission remaining. Keep +the cache local to the projection; a new persistent provenance service is +unnecessary. Apply the same snapshot reuse to campaign provenance helpers where +convenient, although their fixed field count is not a quadratic case. + +Verification: a temporary imported/inherited-roster check confirmed equivalent +source lists using shared native metadata and an enumerated source index. +Retain contributor accuracy, ordering, and secret-free inspection output. + +## 9. P3: Read the selected remote session without enumerating run history + +Stage: 7. + +Location: `internal/app/config_loader.go:194–228`, `findRemoteSessionConfig`. + +After selecting the committed prepared-session key, this function lists the +entire session prefix and scans it for that exact key. If H objects are retained +across historical runs, this adds Θ(H) listing/scan work and Θ(H) object-record +memory to every remote session load. The production S3 adapter paginates and +accumulates the complete listing. The scan predates the migration and was +retained in the new committed-object selection path. + +Potential refactoring: retrieve the exact selected key through the existing +`ObjectStore.Read` boundary, which returns metadata with the opened body. Use +that metadata for the existing size/generation checks and stream the body into +the command-owned temporary file while retaining checksum verification. The +object lookup/transfer then takes a constant number of requests independent of +H, plus the necessary O(B) work for B session bytes. Current-commit acquisition +and validation costs remain. Apply this after reconciling the selection policy +in item 6; no new backend interface is needed. + +Verification: the production S3 Read implementation performs one exact-key +GetObject and returns its size and generation. Preserve fake-store coverage for +missing objects, selected identity, checksum/size/generation mismatches, and +temporary-file cleanup. Confirm that unrelated historical keys are not enumerated. + +## 10. P3: Stream prepared-input archival copies + +Stage: 7. + +Location: `internal/stage/publish_sources.go:29–64`, +`collectPublishPreparedInputs`. + +After opening and verifying each prepared document, the new archival path reads +the whole file with `io.ReadAll` and passes the byte slice to +`writeBytesIfChanged`. For a document of B bytes this requires Θ(B) auxiliary +transfer memory. Expanded imported party documents can be larger than any one +authoring fragment. A full in-memory document is unnecessary for this file copy. + +Potential refactoring: preserve the current confined source opening and checksum +checks, then stream the opened descriptor into the run archive. The existing +`fileops.WriteReaderAtomicWithChecksum` already supplies reader-based atomic +replacement and hashing through `io.Copy`. Preserve unchanged-destination reuse +using the known checksum. Transfer time remains Θ(B), while auxiliary copy +memory becomes O(1) with a fixed-size buffer. This does not alter the plan's +separate canonical YAML rendering algorithm or total in-memory configuration size. + +Verification: preserve archived bytes/checksums, workspace file modes, source +validation, failure handling, and immutable publication behavior. Existing +reader-copy and prepared-session publication tests provide suitable behavioral +coverage; allocation or wall-clock thresholds are unnecessary. + ## Stage-by-stage disposition | Stage | Review result | @@ -206,11 +331,11 @@ objects are unchanged by the authoring update. | 2: Discovery/bootstrap | Native discovery, exclusive explicit selection, independent bootstrap, and shared diff bootstrap are present. | | 3: Resource bases | Native origin-based resolution and fallbacks are present; field-record cleanup remains in item 3. | | 4: Campaign/session imports | File contexts, contextless remote snapshots, native optional discovery, and detached document values are present; initialization still uses the snapshot path in item 5. | -| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present. | +| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present; the planned roster sort is retained. | | 6: Prepared inputs/resume | Canonical document materialization and matching resume evidence are present. | -| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; remote initialization and committed selection conflict in item 6. | +| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; item 6 is a correctness gap and items 9–10 are scalability opportunities. | | 8: Final-domain snapshots | Defaults-backed native snapshots, safe credential identities, rendering, and digest integration are present. | -| 9: Diff/provenance | Native diff and pointer paths are present; effective/history attribution needs item 1. | +| 9: Diff/provenance | Native diff and pointer paths are present; item 1 concerns attribution correctness and items 7–8 concern avoidable provenance work. | | 10: Examples | Destination-based definitions, directory imports, list operations, split roster, and secondary imports are demonstrated. | | 11: Documentation/policy | Current references cover the migration; finish attribution, source-role, and initialization/selection corrections above. | | 12: Final verification | Existing integration checks pass; add coverage for items 5–6. Full race validation remains unreliable because of item 2. | @@ -255,3 +380,26 @@ Using Go 1.27.4 at the follow-up revision: Only this remediation document changed. Diagnostic files remained outside the repository, and object storage used an in-memory fake. + +## Algorithm-focused review coverage and validation + +The stage-by-stage pass covered dependency/API substitutions (1), native +discovery/bootstrap (2), resource-base queries (3), secondary loading and +materialization data flow (4), roster ordering/rendering (5), prepare/resume +serialization (6), archive/remote/restore paths (7), finalized snapshots (8), +diff/provenance projection (9), examples (10), documentation (11), and assembled +verification (12). + +The chosen roster ordering, defaults-backed snapshot construction, native Diff, +and required content validation remain accepted. Canonical rendering is linear +in emitted data; the planned roster comparison sort is O(C log C) after deriving +ordering keys. Unchanged algorithms such as pairwise party-name validation and +prepare-input matching are not new findings from this migration-focused pass. + +Cost analysis followed the released module's public API wrappers through native +source lookup and metadata-copy implementations. Temporary functional Go-overlay +checks validated source-enumeration and shared-metadata equivalence, ordinarily +and under `-race`. Focused configuration, file-reader copy, S3 opened-object +metadata, native inspection, and prepared-session publication checks passed on +Go 1.27.4. No performance data, timing assertions, benchmark gates, or application +changes were introduced.