Files

60 lines
3.5 KiB
Markdown
Raw Permalink Normal View History

# Storage Internals
2026-05-31 02:00:24 +00:00
Audience: developers and LLM coding agents changing `internal/storage`, storage adapters, or storage-backed callers.
2026-05-31 02:00:24 +00:00
## Purpose
2026-05-31 02:00:24 +00:00
`internal/storage` defines backend-rooted logical file access, path validation, typed storage errors, traversal helpers, backend registration, managed deletion targets, and test fake storage behavior.
2026-05-31 02:00:24 +00:00
## Inputs And Outputs
2026-05-31 02:00:24 +00:00
Inputs are contexts, logical paths or prefixes, byte slices or readers, write options, walk options, delete options, and backend open configs. Outputs are file bytes, readers, `Entry` metadata, walk callbacks, boolean content checks, registered backends, and typed errors.
2026-05-31 02:00:24 +00:00
## Boundaries
Core packages depend on `internal/storage`, not concrete adapters. Adapter protocol behavior belongs in `internal/adapters/local`, `internal/adapters/ssh`, and `internal/adapters/s3`; external SSH/SFTP and S3 notes live under `docs/integrations/`.
2026-06-02 19:00:09 +00:00
Runtime backend construction and registration are owned by `internal/app`. The fake backend is for tests only.
2026-06-02 19:00:09 +00:00
## Config Fields Used
2026-06-02 19:00:09 +00:00
The storage package does not read config directly. App adapter wiring converts config fields into backend open config values.
2026-06-02 19:00:09 +00:00
## Adapters Used
2026-05-31 02:00:24 +00:00
Local, SSH/SFTP, and S3-compatible adapters implement `storage.Backend`. `internal/storage/fake` implements the same interface for tests.
## State And Manifest Behavior
2026-05-31 02:00:24 +00:00
2026-06-08 19:32:57 +00:00
Storage owns `.distributor.json` path helpers through `StateFileName`, `StatePath`, `ManagedOutputTargets`, and `ManagedBundleTargets`. It does not parse source manifests or destination state.
2026-05-31 04:06:48 +00:00
Logical paths are slash-separated and relative to a backend root. Prefix validation allows an empty prefix to mean the backend root; file path validation requires a non-empty path.
2026-05-31 04:06:48 +00:00
2026-06-08 19:32:57 +00:00
Reconcile-state callers use `Stat` to check whether managed output paths still exist and bounded recursive `Walk` to report unmanaged entries under a selected destination root. Prune execution callers use `DeleteManagedOutputs` for selected managed output records so `.distributor.json` is not part of the deletion target set. Storage does not decide whether entries are managed; callers compare entries against destination state.
2026-06-08 19:19:43 +00:00
## Skip And Resume Behavior
2026-05-31 16:53:37 +00:00
2026-06-08 19:32:57 +00:00
Storage has no publication skip policy. It supplies `HasAny` for unmanaged-content checks, `Stat` and `Walk` for state repair inspection, `DeleteManagedOutputs` target construction for managed output cleanup, `DeleteManagedBundle` target construction for normal replacement cleanup, and `DeletePrefix` semantics for explicit forced replacement.
2026-05-31 17:11:53 +00:00
## Failure Behavior
2026-05-31 04:06:48 +00:00
Storage errors use typed categories: not found, already exists, not empty, invalid path, conflict, permission, temporary, unsupported, and unknown. Callers should use helper predicates instead of matching strings. Traversal can stop cleanly with `ErrStopWalk`.
2026-05-31 02:00:24 +00:00
## Tests To Inspect
2026-05-31 02:00:24 +00:00
- `internal/storage/*_test.go`
- `internal/storage/fake/*_test.go`
2026-06-08 19:19:43 +00:00
- `internal/app/reconcile_state_test.go`
- `internal/adapters/local/*_test.go`
- `internal/adapters/ssh/*_test.go`
- `internal/adapters/s3/*_test.go`
2026-05-31 04:06:48 +00:00
## Architectural Invariants
2026-05-31 04:06:48 +00:00
- Logical paths are clean relative slash-separated paths confined to the backend root.
- Core packages never import concrete adapters.
- `storage.List` returns deterministic sorted entries.
2026-06-08 19:32:57 +00:00
- Managed output deletion targets are recorded outputs only; managed bundle deletion targets are recorded outputs plus `.distributor.json`.
- Prefix deletion is bounded to the requested logical prefix.
- Runtime registration remains app-owned.