Files
promptkit/docs/roadmap/next-release-considerations.md

436 lines
24 KiB
Markdown
Raw Normal View History

# Feature Considerations Before The Next Release
Status: items 1 and 2 record accepted design decisions. Review these items before
the next release; this document does not authorize implementation or commit either
item to that release. Implemented schema support is documented in the
[format reference](../formats.md#schemas).
## 1. Enable One Structured-Output Repair Attempt By Default
2026-11-20 19:10:24 +00:00
**Accepted:** default to one repair attempt for every validation mode that
supports repair. This gives structurally invalid or empty output one automatic
correction opportunity without requiring each consumer to opt in.
| Validation mode | Default repair attempts |
| --- | ---: |
| `none` | 0 |
| `basic` | 1 |
| `json` | 1 |
| `json_schema` | 1 |
Use a presence-aware public input `RepairAttempts` field: omission or final null
selects the mode's default; explicit zero disables repairs. Accepting null is a
deliberate change from the current configuration contract. Authors can clear an
inherited budget with null through ordinary ConfigKit composition; PromptKit
interprets the final value without replaying that composition.
Retain explicit budgets of two or three
2026-11-20 19:10:24 +00:00
for eligible modes. Keep the existing rejection of positive budgets for `none`.
Choose one rather than two as the default; consumers can opt into another
correction opportunity when appropriate for their workloads.
Apply defaults after ConfigKit composition, preserving explicit values. Execution
and inspection use the resolved concrete budget. Apply the same mode defaults to
omitted runtime output-contract budgets; runtime contracts still replace the
configured contract completely. Prepared execution retains its resolved budget.
Confine presence awareness to inputs. A `*int` is the recommended Go input shape:
nil selects the default and a pointer to zero disables repair. Resolved inspection
and prepared details expose the concrete budget; execution uses a concrete integer.
Counters reporting repairs actually performed remain ordinary integers. Do not
propagate optional-value handling into execution or counters solely to accommodate
input defaulting.
2026-11-20 19:10:24 +00:00
Implementation planning must specify the presence-aware Go shape and migration
from the current integer field, and cover omission, null, and explicit zero across
2026-11-20 19:10:24 +00:00
configuration, typed helpers, and runtime contracts. Document that repair runs
only after failed validation and can add generation latency and usage. Existing
repair accounting, exhaustion, cancellation, and transient-retry semantics remain
unchanged; the change does not promise successful correction.
## 2. Consume Application-Resolved Configuration Under `promptkit`
**Accepted:** the application composes configuration once through ConfigKit and
passes the **entire resolved ConfigKit result** directly to PromptKit. PromptKit
locates the fixed root-level `promptkit` section and consumes only that section
for engine construction, ignoring sibling application settings. Inspection uses
the focused reporting semantics below. The application does not extract, reshape,
serialize, or reload a PromptKit-specific view. Do not add a configurable mounting
key.
The root-key migration supports this handoff; changing YAML paths alone does not
fulfill the feature. The architectural benefit is a single composition lifecycle
with native source context and provenance, rather than an application bridge
between separately loaded configurations.
### Current Boundary And Intended Consumer Workflow
Today, PromptKit accepts configuration sources and runs its own ConfigKit load.
Its typed root document rejects unrelated application fields. Audita, for example,
loads application settings separately, then uses their recorded main-source
context to assemble embedded and companion PromptKit sources for another load.
It does not currently hand PromptKit its resolved application configuration.
The target workflow is:
1. The application includes PromptKit's configuration definition/registration and
desired bundled or application sources in its ordinary ConfigKit setup.
2. ConfigKit composes and resolves the complete application configuration once,
applying each component's registered rules.
3. The application passes the whole resolved result to PromptKit, together with
runtime collaborators such as its HTTP client. PromptKit captures its effective
resources and constructs the engine without another configuration load.
File discovery, source selection, imports, layering, and application overrides
remain application policy. Embedded prompt selection and companion-file imports
join the original load; they do not disappear. The change removes context-transfer
and duplicate-loading machinery, not application-specific execution requests,
profile selection, required-resource checks, or diagnostics policy.
### ConfigKit v1.4.0 And Library-Owned Wiring
Adopt [ConfigKit v1.4.0](../../../ConfigKit/docs/releases/v1.4.0.md) as a
prerequisite for the configuration handoff. It releases the typed lookup and
focused inspection APIs required here. PromptKit currently selects v1.3.0;
make the dependency upgrade separately reviewable before the configuration
refactor. The Go consumer minimum remains 1.27.0.
The released lookup is:
```go
func (r *Result[T]) Value(path Path) (any, bool)
```
It selects an existing typed value from `Result.Config` by configuration
path (YAML/schema names). It performs no serialization, conversion, decoding,
defaulting, validation, or additional load. It is trusted execution access,
including resolved secrets; existing inspection remains redacted.
Accept ordinary Go value semantics. Struct values are copied, while maps, slices,
and pointers can share state with `Result.Config`; no new ConfigKit deep-copy or
frozen-execution guarantee is required. Return the actual terminal value without
automatic dereferencing. A found nil pointer differs from an unavailable path;
traversal through nil cannot succeed. A declared zero/defaulted field can exist
without YAML input, so input presence remains a metadata question. PromptKit checks
the selected section's type and handles absence according to its configuration
contract. Exact supported public settings value/pointer forms belong in planning.
Expose public PromptKit settings and a small typed registration helper using
existing ConfigKit APIs. The following is an illustrative consumer sketch, not
implemented API names or a complete runnable example:
```go
type AppConfig struct {
OutputDirectory string `yaml:"output_directory"`
PromptKit promptkit.Settings `yaml:"promptkit"`
}
definition := promptkit.Configure(
configkit.Definition[AppConfig]{},
func(c *AppConfig) *promptkit.Settings { return &c.PromptKit },
)
result, err := configkit.Load(source,
configkit.Options[AppConfig]{Definition: definition})
// Handle err before continuing.
engine, err := promptkit.NewFromConfig(result)
```
One compiler-checked field selector per library is accepted registration wiring,
not post-load extraction. The application passes the whole result to the engine.
PromptKit's helper composes its defaults, normalization, validation, sensitivity,
and sequence identities with existing application rules at full `/promptkit`
paths. Callback ordering stays explicit and application-controlled. Do not require
applications to duplicate PromptKit rules or use internal types. No ConfigKit
component registry, new lifecycle orchestration, or reflection-based registration
is needed for this design.
The proposed constructor can consume a PromptKit-owned interface such as:
```go
type Configuration interface {
Value(configkit.Path) (any, bool)
InspectSubtree(configkit.Path) (configkit.Inspection, bool)
Diagnostics() []configkit.Diagnostic
}
```
It selects `Value(configkit.Keys("promptkit"))`, checks its type, and captures
resources using native metadata and source contexts. Effective field origins are
authoritative; do not recover a missing effective location from historical evidence.
A serialized section loaded again is not an equivalent handoff.
Retain the inspection view selected at `/promptkit` for configuration reporting.
That view also supplies metadata and source contexts during capture, so the
constructor interface need not repeat those methods.
Load diagnostics come from the original result's `Diagnostics()` method; the
inspection view does not supply that method. Retain and forward those records
through PromptKit's configuration inspection facade. Concrete PromptKit API names
remain implementation-planning work.
Register desired bundled resources as ordinary underlays before composition;
engine construction must not inject another configuration layer into an already
resolved result. The sketch omits source assembly for brevity. Apply configured
repair defaults from item 1 during resolution; runtime request defaults remain
separate.
PromptKit must preserve its existing protection against caller mutation after
engine construction by capturing owned settings where needed. A shallow struct
copy alone does not detach maps and slices. Callers should hand off before mutating
resolved values in ways that invalidate their frozen inspection/provenance evidence.
Document this usage expectation rather than add mutation detection or demand a
new ConfigKit snapshot facility.
Retain source-loading construction as a convenience using the same registration
rules and engine-capture path: load once, then hand off the result. Do not maintain
two independent implementations of configuration semantics. Existing source
selection choices remain available subject to the accepted layout migration.
### Fixed Section And Clean Migration
Move all PromptKit-owned document settings beneath `promptkit`, including
`promptkit.backends`, `promptkit.profiles`, `promptkit.prompts`,
`promptkit.collections`, and `promptkit.schema_root`. Collection-local fields
remain inside their collection. Runtime collaborators are Go API inputs, not new
YAML fields.
Make a clean breaking migration with one canonical resolved location. Do not
support legacy root-level resource locations through aliases, automatic detection,
or a second merge/precedence pass. Runtime resource IDs and namespace selection
remain unchanged; the configuration wrapper is not part of a resource ID.
Preserve convenient standalone files. Individual resource files can still begin
with `id`, and collection files can still contain local `prompts`, `profiles`,
`backends`, and `schema_root` fields without a `promptkit` wrapper. Native ConfigKit
import destinations and PromptKit source helpers mount those fragments at the
appropriate nested location. Source-relative context follows the effective
contributing field, not the location of the application document or mount point.
ConfigKit-owned headers and directives retain their native semantics. The
application owns its configuration version policy. PromptKit's registration
preserves that policy; PromptKit-specific resource version validation is removed
as described below.
An absent `promptkit` section contributes no application-supplied PromptKit
settings; selected bundled underlays can still contribute defaults during the
original load. Old root-level keys such as `prompts` and `profiles` are neither
consumed nor specially rejected by PromptKit: they may be application settings.
Application-specific fields are accepted at the extension points below. The
application's own validation and ConfigKit's document/composition rules still
apply to the whole document. Migration guidance must explain that unmigrated
resources are not loaded and can therefore produce missing-resource errors when
selected.
### Application-Specific Fields
**Accepted:** allow additional application-specific fields on prompt, profile,
and backend definitions, on collection definitions, and at the `promptkit` section
level. Use ConfigKit's native string-keyed inline maps (`yaml:",inline"`) in the
public settings model to retain unmatched fields. Recognized fields keep their
typed decoding and semantic validation, including required-field checks.
Additional fields participate in ordinary ConfigKit composition and remain
available in the resolved application configuration and native configuration
inspection. Applications own their interpretation, validation, and sensitivity
rules. PromptKit ignores these fields during execution; they do not become model
parameters. `extra_params` retains its existing provider-parameter purpose.
Recommend application-specific grouping, such as an `audita` mapping, to reduce
future field-name collisions; do not require a naming prefix or wrapper. At these
extension points, a misspelled optional PromptKit field can be accepted as an
application field. Accept that tradeoff rather than introduce an extension registry
or custom unknown-field detection.
These extension points do not change the schemas of nested PromptKit-owned objects
such as output contracts. Maps beneath application-specific keys retain ConfigKit's
ordinary dynamic-value behavior. Standalone and application-owned loading use the
same settings model.
**Accepted inspection boundary:** retain application-specific fields in the
resolved configuration and ConfigKit-backed `InspectConfiguration()` reports.
`InspectPrompt`, `InspectProfile`, and `InspectBackend` continue to report
PromptKit-understood fields and omit application-specific fields. This keeps
resource reports simple: no additional metadata storage, copying, redaction
integration, or public report fields are needed for these extensions. Applications
can access their fields through the resolved configuration and configuration
inspection; execution objects need not carry them.
### Focused Configuration Inspection
**Accepted:** preserve resource listing and inspection, and provide configuration
reporting through ConfigKit v1.4.0's non-generic `Inspection` view:
```go
inspection, ok := result.InspectSubtree(configkit.Keys("promptkit"))
```
PromptKit's `InspectConfiguration()` forwards the view's capabilities without
loading another configuration or rebuilding history, redaction, or comparison.
Both application-owned loading and standalone construction use this reporting
path. This replaces the earlier inspection deferral.
The view is a reporting focus, not an isolation or access-control boundary.
Accept ConfigKit's native behavior:
- Effective output renders the selected subtree without enclosing application
mappings, using frozen completed-load values and existing redaction, including
sensitivity inherited from ancestors. Later settings mutation does not change
that evidence.
- Focused enumeration includes paths at or beneath `/promptkit`. Metadata remains
unfiltered, and explicit explanation/history queries use native application
paths and may query outside the focus. Do not rebase paths or add a separate
restriction layer.
- Sources and contextual evidence remain native and may describe other application
settings. Do not assign exclusive source ownership or trim ancestor history and
dependency evidence to manufacture an isolated report.
- Comparison uses ConfigKit's existing frozen-configuration comparison and retains
differences at, beneath, or above the focus. Sibling-only difference records are
excluded, but ancestor records may include changes outside PromptKit. A nonempty
report therefore does not necessarily establish that PromptKit settings changed.
Preserve native ordering, sequence-atomic comparison, and paired redaction;
do not implement a separate subtree comparison algorithm.
- Views compare the same native location even when the application Go root types
differ. Follow upstream's errors for mismatched scopes or unavailable views.
- A valid absent scope still yields a view, allowing history and added/removed
comparisons. Constructor success does not establish effective presence. Rendering
an absent effective value fails without output; explicit null renders normally.
Application-level inspection remains useful for the whole shared configuration.
The focused view adds convenience and supports existing integrations without
claiming exclusive access to PromptKit information. Resource APIs continue to
select and describe logical prompts, profiles, and backends independently of
configuration-tree paths.
### Resource Version Validation
**Accepted:** this migration gives up PromptKit-specific validation of `version`
fields in resource documents, including collection, prompt, and profile
definitions. This is an intentional simplification with a small practical cost,
not an unresolved compatibility requirement or a prerequisite for an upstream
version-policy feature.
The application retains ownership of ConfigKit's version policy. ConfigKit-owned
headers and directives follow that policy and native composition semantics;
PromptKit adds no separate resource-version checks. Apply this resource decision
to both application-owned loading and standalone construction. Update examples
and migration guidance accordingly, without preparsing or stripping headers or
adding a second load.
## Guidance For Implementation Planning
### Validate The Pattern Against PromptKit
The actual PromptKit prototype check is complete; see the
[integration assessment](configuration-integration-check.md) for evidence,
the smaller three-method handoff interface, and remaining implementation work.
The checklist below records what the check needed to establish and guides
implementation coverage; production implementation is not complete.
ConfigKit's released APIs and maintained
[one-library](../../../ConfigKit/examples/consumers/result_component.go) and
[two-library](../../../ConfigKit/examples/consumers/result_composition.go) examples
provide the integration starting point. They demonstrate typed registration,
whole-result handoff, owned settings capture, native inspection, and effective
source context without another load.
The release review passed PromptKit's full test suite against the published
v1.4.0 module using a temporary dependency file and Go 1.27.4. ConfigKit's focused
lookup, inspection, comparison, history, and component-example checks also passed.
This establishes compatibility with PromptKit's existing implementation. The
isolated prototype now also verifies the proposed handoff pattern; the production
handoff remains to be implemented.
The prototype demonstrates the typed registration helper,
one application-owned load, and whole-result engine construction. It also exercises
the standalone convenience path through the same rules. The checks cover:
1. **Public settings and callback composition.** Define a supported public settings
model suitable for ConfigKit decoding; existing typed source helpers are not
automatically that model. Preserve application callbacks, sensitivity, and
registrations when adding PromptKit's rules. Demonstrate explicit callback
ordering with a second representative component, without a registry. Keep
bundled underlay selection in the original load and specify its consumer API.
Include inline application-field maps at the accepted extension points and
preserve those values during PromptKit normalization.
2. **Per-load presence capture.** The current resolver captures input presence and
repair-budget presence in private maps used later during resource construction.
Determine which facts can come directly from final metadata and which need
per-load captured state. Do not rely on historical reconstruction or a shared
helper closure that associates one load's state with another result. Verify
independent loads using the same configured definition.
3. **Ownership and source context.** Capture mutable settings appropriately at
engine construction. Verify effective origins and filesystem contexts survive
mounting, inheritance, and overrides. Document mutation-before-handoff behavior;
no mutation-detection subsystem or ConfigKit deep-copy guarantee is required.
4. **Inspection facade and diagnostics.** Retain the native focused view and specify
public signatures for its operations and load-diagnostic forwarding. Obtain
diagnostics from the original result's `Diagnostics()` method through
PromptKit's consumer-owned interface. Expose focused enumeration
alongside native metadata and query operations. A PromptKit overview explanation
selects `/promptkit`; the native empty path selects the whole application.
Document what the interfaces do: their inputs, operations, outputs, and errors,
and how registration, loading, and engine construction fit together. Keep this
guidance concrete; hypothetical misuse does not call for registration tracking,
another validation lifecycle, or defensive caveats.
CommandKit neutrally exposes upstream PromptKit APIs. Any adapter updates are
routine downstream adoption, not a constraint on PromptKit's public API or a
prerequisite for completing this migration. PromptKit's implementation plan covers
its own resource, inspection, and diagnostic capabilities; CommandKit owns how
those are presented through CLI commands.
Specify the concrete registration, settings, and constructor shapes, source-helper
adaptations, and migration examples after these checks. Identify any remaining
concrete gap rather than infer a need for a larger upstream framework. Keep the
repair-default and handoff changes separately reviewable; the repair work need not
wait for the configuration refactor. Neither needs a new execution subsystem or
compatibility resolver.
### Focused Validation
- Mode defaults after composition, null clearing an inherited repair budget,
explicit zero, positive budgets, and rejection of positive budgets for `none`;
cover typed and runtime inputs as well as YAML.
- Concrete budgets in resource inspection and prepared snapshots; actual repair
counters retain their existing meaning.
- An application-owned type with siblings, typed component registration, one load,
and whole-result handoff with no post-load extraction or second load.
- Equivalent effective resources through registered application loading and
standalone convenience loading when supplied equivalent sources.
- Callback preservation and independent per-load state, including repeat loads
with the same definition and a representative second component.
- Mixed embedded/operator collections, source-relative message/schema references,
native inheritance and overrides, and effective field context at handoff.
- Unavailable paths, found nil values, zero/defaulted settings, and wrong section
types; source presence remains distinct from lookup availability.
- Missing application settings with bundled defaults enabled and disabled,
application-owned root keys resembling legacy settings, and additional fields
at each accepted extension point. Recognized fields retain type and semantic
validation; nested PromptKit-owned objects retain their existing schemas.
- Additional fields survive native composition, inheritance, normalization, and
configuration inspection, including application-defined sensitivity. They remain
available to application validation and do not enter provider requests merely
because they appear on a resource definition.
- Resource inspection omits application-specific fields while configuration
inspection retains them under ConfigKit's native sensitivity rules.
- Engine independence from later caller mutation of settings, including nested
maps and slices; trusted lookup does not replace redacted inspection.
- Resource listings, resource reports, and load diagnostics through PromptKit's
public APIs; configuration output and overview explanations focused on
`/promptkit`, with native paths and redaction preserved.
- Focused enumeration alongside unfiltered metadata; explicit
queries outside the focus and shared source/history context retain native behavior.
- Inspection remains frozen after settings mutation; absent, null, historical-only,
and added/removed sections follow upstream semantics. Sibling difference records
are excluded while ancestor differences remain; comparison works across different
application root types selecting the same scope.
The product direction, including focused inspection, removal of resource-version
validation, and the configuration-only inspection of application-specific fields,
is settled. No product decisions remain before implementation planning.
Concrete public API shapes, diagnostic forwarding, and verification of the actual
PromptKit integration are implementation-planning work.
## Further Candidates
Append additional pre-release considerations here as they arise.