Add report and scope benchmarks
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Performance implementation plan
|
||||
|
||||
Status: implementation in progress; Stage 1 is complete, Stages 2–11 have not started.
|
||||
Status: implementation in progress; Stages 1–2 are complete, Stages 3–11 have not started.
|
||||
|
||||
Date: 2026-10-09.
|
||||
|
||||
@@ -228,7 +228,7 @@ shows why later comparisons should report run variation.
|
||||
|
||||
## Stage 2: establish report and scope measurements
|
||||
|
||||
**Status:** not started. **Prerequisite:** Stage 1.
|
||||
**Status:** complete. **Prerequisite:** Stage 1.
|
||||
|
||||
**Files:** new `performance_reports_test.go` and evidence here. Production code
|
||||
stays unchanged in this stage.
|
||||
@@ -258,7 +258,57 @@ owner of actual library loading semantics.
|
||||
|
||||
**Completion:** PF-01, PF-04, and PF-06 have comparable baseline measurements.
|
||||
|
||||
**Evidence:** pending.
|
||||
**Evidence:** Fixtures were measured from source revision `aed4c59` with
|
||||
`performance_reports_test.go` in the working tree (committed with this evidence).
|
||||
Environment matched the first baseline: Go 1.26.8, linux/amd64, Linux
|
||||
6.17.0-41-generic, Intel Core i5-7260U CPU @ 2.20 GHz. Root package compilation
|
||||
passed with `GOWORK=off GOTOOLCHAIN=local GOPROXY=off GONOPROXY=none GOSUMDB=off go test -mod=readonly -run '^$' .`. The five-sample benchmark command passed:
|
||||
`GOWORK=off GOTOOLCHAIN=local GOPROXY=off GONOPROXY=none GOSUMDB=off go test -mod=readonly -run '^$' -bench '^BenchmarkPerformance(PromptReports|YAMLFraming|ConfigScopes|ConfigRegistration)$' -benchmem -benchtime=200ms -count=5 .`.
|
||||
Raw output is in `/tmp/commandkit-performance-reports-baseline.txt`.
|
||||
|
||||
Fixtures: PromptKit callbacks returned prebuilt inspection records directly;
|
||||
prompt reports used 4 and 128 input records, while profile `ExtraParams` used
|
||||
2 groups with 3 nested values each or 24 groups with 8 nested values each and a
|
||||
nested sequence. Each was measured as text and YAML through `App.Execute`.
|
||||
Native YAML fixtures contain unique short mapping keys and are 4,096 bytes with
|
||||
an ordinary comment preamble and 1,048,591 bytes with a `%YAML` directive
|
||||
preamble. Framing was measured directly; aggregate `config show --scope all`
|
||||
measured two documents of each selected fixture size/style, and single-scope
|
||||
display used the 4 KiB ordinary document. Scope operations used 2 or 32 scopes
|
||||
with the requested show/validate capabilities and cheap in-memory acquisition
|
||||
callbacks. Suite registration used 4 or 64 distinct selectors and a fresh app.
|
||||
The scope counts also covered discovery, named validation, and aggregate
|
||||
validation. Setup verified success and report shape outside timed loops; native
|
||||
fixtures, records, suites, and apps were prepared outside repeated loops.
|
||||
|
||||
Representative five-sample ranges (ns/op; B/op; allocs/op): prompt, 4 inputs,
|
||||
text 41,287–52,115; 26,160–26,161; 294, and YAML 88,991–104,671;
|
||||
74,730–74,733; 557. Prompt, 128 inputs, text 169,155–198,281;
|
||||
145,524–145,531; 1,171, and YAML 1,418,692–1,490,181; 1,393,508–1,393,568;
|
||||
6,039–6,040. Profile, small extra map, text 74,109–89,337; 70,914–70,918;
|
||||
462, and YAML 161,470–368,813; 162,688–162,717; 755. Profile, large extra map,
|
||||
text 1,277,396–2,583,950; 836,141–836,296; 3,027–3,028, and YAML
|
||||
1,756,571–2,377,499; 1,586,728–1,586,849; 5,878–5,879. Larger profile timings
|
||||
varied, while allocation counts and bytes were stable.
|
||||
|
||||
Direct framing measured 4 KiB ordinary at 5,883–6,842 ns/op, 11,056 B/op,
|
||||
3 allocs/op, and 1 MiB directive at 1,208,678–1,502,417 ns/op,
|
||||
2,539,520–2,539,523 B/op, 2 allocs/op. Aggregate show measured 4 KiB documents
|
||||
at 44,395–48,161 ns/op, 55,537–55,538 B/op, 211 allocs/op, and 1 MiB documents
|
||||
at 4,818,907–6,009,877 ns/op, 8,695,182–8,695,197 B/op, 215–216 allocs/op.
|
||||
Single-scope 4 KiB display measured 20,229–21,791 ns/op, 17,935–17,936 B/op,
|
||||
183 allocs/op. With 2 scopes, discovery measured 23,840–30,007 ns/op,
|
||||
21,161–21,162 B/op, 223 allocs/op; named validation 24,841–26,340 ns/op,
|
||||
21,817 B/op, 228 allocs/op; aggregate validation 25,231–31,126 ns/op,
|
||||
22,361–22,362 B/op, 237 allocs/op. With 32 scopes, discovery measured
|
||||
31,439–37,167 ns/op, 28,141–28,142 B/op, 257 allocs/op; named validation
|
||||
29,123–74,374 ns/op, 22,775–22,778 B/op, 228 allocs/op; aggregate validation
|
||||
45,548–63,471 ns/op, 29,781–29,782 B/op, 269 allocs/op. Registration with 4
|
||||
selectors measured 12,573–14,789 ns/op, 8,911 B/op, 104 allocs/op; with 64 it
|
||||
measured 91,675–105,548 ns/op, 82,817–82,818 B/op, 485 allocs/op. Large-report
|
||||
cost includes full output framing/delivery; no segmented-delivery comparison
|
||||
was selected. These measurements establish baselines and show timing variation;
|
||||
they are not fixed thresholds or speedup claims.
|
||||
|
||||
## Stage 3: render only the selected PromptKit format
|
||||
|
||||
|
||||
Reference in New Issue
Block a user