436 lines
24 KiB
Markdown
436 lines
24 KiB
Markdown
# Feature Considerations Before The Next Release
|
|
|
|
Status: items 1 and 2 record accepted design decisions. Review these items before
|
|
the next release; this document does not authorize implementation or commit either
|
|
item to that release. Implemented schema support is documented in the
|
|
[format reference](../formats.md#schemas).
|
|
|
|
## 1. Enable One Structured-Output Repair Attempt By Default
|
|
|
|
**Accepted:** default to one repair attempt for every validation mode that
|
|
supports repair. This gives structurally invalid or empty output one automatic
|
|
correction opportunity without requiring each consumer to opt in.
|
|
|
|
| Validation mode | Default repair attempts |
|
|
| --- | ---: |
|
|
| `none` | 0 |
|
|
| `basic` | 1 |
|
|
| `json` | 1 |
|
|
| `json_schema` | 1 |
|
|
|
|
Use a presence-aware public input `RepairAttempts` field: omission or final null
|
|
selects the mode's default; explicit zero disables repairs. Accepting null is a
|
|
deliberate change from the current configuration contract. Authors can clear an
|
|
inherited budget with null through ordinary ConfigKit composition; PromptKit
|
|
interprets the final value without replaying that composition.
|
|
|
|
Retain explicit budgets of two or three
|
|
for eligible modes. Keep the existing rejection of positive budgets for `none`.
|
|
Choose one rather than two as the default; consumers can opt into another
|
|
correction opportunity when appropriate for their workloads.
|
|
|
|
Apply defaults after ConfigKit composition, preserving explicit values. Execution
|
|
and inspection use the resolved concrete budget. Apply the same mode defaults to
|
|
omitted runtime output-contract budgets; runtime contracts still replace the
|
|
configured contract completely. Prepared execution retains its resolved budget.
|
|
|
|
Confine presence awareness to inputs. A `*int` is the recommended Go input shape:
|
|
nil selects the default and a pointer to zero disables repair. Resolved inspection
|
|
and prepared details expose the concrete budget; execution uses a concrete integer.
|
|
Counters reporting repairs actually performed remain ordinary integers. Do not
|
|
propagate optional-value handling into execution or counters solely to accommodate
|
|
input defaulting.
|
|
|
|
Implementation planning must specify the presence-aware Go shape and migration
|
|
from the current integer field, and cover omission, null, and explicit zero across
|
|
configuration, typed helpers, and runtime contracts. Document that repair runs
|
|
only after failed validation and can add generation latency and usage. Existing
|
|
repair accounting, exhaustion, cancellation, and transient-retry semantics remain
|
|
unchanged; the change does not promise successful correction.
|
|
|
|
## 2. Consume Application-Resolved Configuration Under `promptkit`
|
|
|
|
**Accepted:** the application composes configuration once through ConfigKit and
|
|
passes the **entire resolved ConfigKit result** directly to PromptKit. PromptKit
|
|
locates the fixed root-level `promptkit` section and consumes only that section
|
|
for engine construction, ignoring sibling application settings. Inspection uses
|
|
the focused reporting semantics below. The application does not extract, reshape,
|
|
serialize, or reload a PromptKit-specific view. Do not add a configurable mounting
|
|
key.
|
|
|
|
The root-key migration supports this handoff; changing YAML paths alone does not
|
|
fulfill the feature. The architectural benefit is a single composition lifecycle
|
|
with native source context and provenance, rather than an application bridge
|
|
between separately loaded configurations.
|
|
|
|
### Current Boundary And Intended Consumer Workflow
|
|
|
|
Today, PromptKit accepts configuration sources and runs its own ConfigKit load.
|
|
Its typed root document rejects unrelated application fields. Audita, for example,
|
|
loads application settings separately, then uses their recorded main-source
|
|
context to assemble embedded and companion PromptKit sources for another load.
|
|
It does not currently hand PromptKit its resolved application configuration.
|
|
|
|
The target workflow is:
|
|
|
|
1. The application includes PromptKit's configuration definition/registration and
|
|
desired bundled or application sources in its ordinary ConfigKit setup.
|
|
2. ConfigKit composes and resolves the complete application configuration once,
|
|
applying each component's registered rules.
|
|
3. The application passes the whole resolved result to PromptKit, together with
|
|
runtime collaborators such as its HTTP client. PromptKit captures its effective
|
|
resources and constructs the engine without another configuration load.
|
|
|
|
File discovery, source selection, imports, layering, and application overrides
|
|
remain application policy. Embedded prompt selection and companion-file imports
|
|
join the original load; they do not disappear. The change removes context-transfer
|
|
and duplicate-loading machinery, not application-specific execution requests,
|
|
profile selection, required-resource checks, or diagnostics policy.
|
|
|
|
### ConfigKit v1.4.0 And Library-Owned Wiring
|
|
|
|
Adopt [ConfigKit v1.4.0](../../../ConfigKit/docs/releases/v1.4.0.md) as a
|
|
prerequisite for the configuration handoff. It releases the typed lookup and
|
|
focused inspection APIs required here. PromptKit currently selects v1.3.0;
|
|
make the dependency upgrade separately reviewable before the configuration
|
|
refactor. The Go consumer minimum remains 1.27.0.
|
|
|
|
The released lookup is:
|
|
|
|
```go
|
|
func (r *Result[T]) Value(path Path) (any, bool)
|
|
```
|
|
|
|
It selects an existing typed value from `Result.Config` by configuration
|
|
path (YAML/schema names). It performs no serialization, conversion, decoding,
|
|
defaulting, validation, or additional load. It is trusted execution access,
|
|
including resolved secrets; existing inspection remains redacted.
|
|
|
|
Accept ordinary Go value semantics. Struct values are copied, while maps, slices,
|
|
and pointers can share state with `Result.Config`; no new ConfigKit deep-copy or
|
|
frozen-execution guarantee is required. Return the actual terminal value without
|
|
automatic dereferencing. A found nil pointer differs from an unavailable path;
|
|
traversal through nil cannot succeed. A declared zero/defaulted field can exist
|
|
without YAML input, so input presence remains a metadata question. PromptKit checks
|
|
the selected section's type and handles absence according to its configuration
|
|
contract. Exact supported public settings value/pointer forms belong in planning.
|
|
|
|
Expose public PromptKit settings and a small typed registration helper using
|
|
existing ConfigKit APIs. The following is an illustrative consumer sketch, not
|
|
implemented API names or a complete runnable example:
|
|
|
|
```go
|
|
type AppConfig struct {
|
|
OutputDirectory string `yaml:"output_directory"`
|
|
PromptKit promptkit.Settings `yaml:"promptkit"`
|
|
}
|
|
|
|
definition := promptkit.Configure(
|
|
configkit.Definition[AppConfig]{},
|
|
func(c *AppConfig) *promptkit.Settings { return &c.PromptKit },
|
|
)
|
|
result, err := configkit.Load(source,
|
|
configkit.Options[AppConfig]{Definition: definition})
|
|
// Handle err before continuing.
|
|
engine, err := promptkit.NewFromConfig(result)
|
|
```
|
|
|
|
One compiler-checked field selector per library is accepted registration wiring,
|
|
not post-load extraction. The application passes the whole result to the engine.
|
|
PromptKit's helper composes its defaults, normalization, validation, sensitivity,
|
|
and sequence identities with existing application rules at full `/promptkit`
|
|
paths. Callback ordering stays explicit and application-controlled. Do not require
|
|
applications to duplicate PromptKit rules or use internal types. No ConfigKit
|
|
component registry, new lifecycle orchestration, or reflection-based registration
|
|
is needed for this design.
|
|
|
|
The proposed constructor can consume a PromptKit-owned interface such as:
|
|
|
|
```go
|
|
type Configuration interface {
|
|
Value(configkit.Path) (any, bool)
|
|
InspectSubtree(configkit.Path) (configkit.Inspection, bool)
|
|
Diagnostics() []configkit.Diagnostic
|
|
}
|
|
```
|
|
|
|
It selects `Value(configkit.Keys("promptkit"))`, checks its type, and captures
|
|
resources using native metadata and source contexts. Effective field origins are
|
|
authoritative; do not recover a missing effective location from historical evidence.
|
|
A serialized section loaded again is not an equivalent handoff.
|
|
Retain the inspection view selected at `/promptkit` for configuration reporting.
|
|
That view also supplies metadata and source contexts during capture, so the
|
|
constructor interface need not repeat those methods.
|
|
Load diagnostics come from the original result's `Diagnostics()` method; the
|
|
inspection view does not supply that method. Retain and forward those records
|
|
through PromptKit's configuration inspection facade. Concrete PromptKit API names
|
|
remain implementation-planning work.
|
|
|
|
Register desired bundled resources as ordinary underlays before composition;
|
|
engine construction must not inject another configuration layer into an already
|
|
resolved result. The sketch omits source assembly for brevity. Apply configured
|
|
repair defaults from item 1 during resolution; runtime request defaults remain
|
|
separate.
|
|
|
|
PromptKit must preserve its existing protection against caller mutation after
|
|
engine construction by capturing owned settings where needed. A shallow struct
|
|
copy alone does not detach maps and slices. Callers should hand off before mutating
|
|
resolved values in ways that invalidate their frozen inspection/provenance evidence.
|
|
Document this usage expectation rather than add mutation detection or demand a
|
|
new ConfigKit snapshot facility.
|
|
|
|
Retain source-loading construction as a convenience using the same registration
|
|
rules and engine-capture path: load once, then hand off the result. Do not maintain
|
|
two independent implementations of configuration semantics. Existing source
|
|
selection choices remain available subject to the accepted layout migration.
|
|
|
|
### Fixed Section And Clean Migration
|
|
|
|
Move all PromptKit-owned document settings beneath `promptkit`, including
|
|
`promptkit.backends`, `promptkit.profiles`, `promptkit.prompts`,
|
|
`promptkit.collections`, and `promptkit.schema_root`. Collection-local fields
|
|
remain inside their collection. Runtime collaborators are Go API inputs, not new
|
|
YAML fields.
|
|
|
|
Make a clean breaking migration with one canonical resolved location. Do not
|
|
support legacy root-level resource locations through aliases, automatic detection,
|
|
or a second merge/precedence pass. Runtime resource IDs and namespace selection
|
|
remain unchanged; the configuration wrapper is not part of a resource ID.
|
|
|
|
Preserve convenient standalone files. Individual resource files can still begin
|
|
with `id`, and collection files can still contain local `prompts`, `profiles`,
|
|
`backends`, and `schema_root` fields without a `promptkit` wrapper. Native ConfigKit
|
|
import destinations and PromptKit source helpers mount those fragments at the
|
|
appropriate nested location. Source-relative context follows the effective
|
|
contributing field, not the location of the application document or mount point.
|
|
|
|
ConfigKit-owned headers and directives retain their native semantics. The
|
|
application owns its configuration version policy. PromptKit's registration
|
|
preserves that policy; PromptKit-specific resource version validation is removed
|
|
as described below.
|
|
|
|
An absent `promptkit` section contributes no application-supplied PromptKit
|
|
settings; selected bundled underlays can still contribute defaults during the
|
|
original load. Old root-level keys such as `prompts` and `profiles` are neither
|
|
consumed nor specially rejected by PromptKit: they may be application settings.
|
|
Application-specific fields are accepted at the extension points below. The
|
|
application's own validation and ConfigKit's document/composition rules still
|
|
apply to the whole document. Migration guidance must explain that unmigrated
|
|
resources are not loaded and can therefore produce missing-resource errors when
|
|
selected.
|
|
|
|
### Application-Specific Fields
|
|
|
|
**Accepted:** allow additional application-specific fields on prompt, profile,
|
|
and backend definitions, on collection definitions, and at the `promptkit` section
|
|
level. Use ConfigKit's native string-keyed inline maps (`yaml:",inline"`) in the
|
|
public settings model to retain unmatched fields. Recognized fields keep their
|
|
typed decoding and semantic validation, including required-field checks.
|
|
|
|
Additional fields participate in ordinary ConfigKit composition and remain
|
|
available in the resolved application configuration and native configuration
|
|
inspection. Applications own their interpretation, validation, and sensitivity
|
|
rules. PromptKit ignores these fields during execution; they do not become model
|
|
parameters. `extra_params` retains its existing provider-parameter purpose.
|
|
|
|
Recommend application-specific grouping, such as an `audita` mapping, to reduce
|
|
future field-name collisions; do not require a naming prefix or wrapper. At these
|
|
extension points, a misspelled optional PromptKit field can be accepted as an
|
|
application field. Accept that tradeoff rather than introduce an extension registry
|
|
or custom unknown-field detection.
|
|
|
|
These extension points do not change the schemas of nested PromptKit-owned objects
|
|
such as output contracts. Maps beneath application-specific keys retain ConfigKit's
|
|
ordinary dynamic-value behavior. Standalone and application-owned loading use the
|
|
same settings model.
|
|
|
|
**Accepted inspection boundary:** retain application-specific fields in the
|
|
resolved configuration and ConfigKit-backed `InspectConfiguration()` reports.
|
|
`InspectPrompt`, `InspectProfile`, and `InspectBackend` continue to report
|
|
PromptKit-understood fields and omit application-specific fields. This keeps
|
|
resource reports simple: no additional metadata storage, copying, redaction
|
|
integration, or public report fields are needed for these extensions. Applications
|
|
can access their fields through the resolved configuration and configuration
|
|
inspection; execution objects need not carry them.
|
|
|
|
### Focused Configuration Inspection
|
|
|
|
**Accepted:** preserve resource listing and inspection, and provide configuration
|
|
reporting through ConfigKit v1.4.0's non-generic `Inspection` view:
|
|
|
|
```go
|
|
inspection, ok := result.InspectSubtree(configkit.Keys("promptkit"))
|
|
```
|
|
|
|
PromptKit's `InspectConfiguration()` forwards the view's capabilities without
|
|
loading another configuration or rebuilding history, redaction, or comparison.
|
|
Both application-owned loading and standalone construction use this reporting
|
|
path. This replaces the earlier inspection deferral.
|
|
|
|
The view is a reporting focus, not an isolation or access-control boundary.
|
|
Accept ConfigKit's native behavior:
|
|
|
|
- Effective output renders the selected subtree without enclosing application
|
|
mappings, using frozen completed-load values and existing redaction, including
|
|
sensitivity inherited from ancestors. Later settings mutation does not change
|
|
that evidence.
|
|
- Focused enumeration includes paths at or beneath `/promptkit`. Metadata remains
|
|
unfiltered, and explicit explanation/history queries use native application
|
|
paths and may query outside the focus. Do not rebase paths or add a separate
|
|
restriction layer.
|
|
- Sources and contextual evidence remain native and may describe other application
|
|
settings. Do not assign exclusive source ownership or trim ancestor history and
|
|
dependency evidence to manufacture an isolated report.
|
|
- Comparison uses ConfigKit's existing frozen-configuration comparison and retains
|
|
differences at, beneath, or above the focus. Sibling-only difference records are
|
|
excluded, but ancestor records may include changes outside PromptKit. A nonempty
|
|
report therefore does not necessarily establish that PromptKit settings changed.
|
|
Preserve native ordering, sequence-atomic comparison, and paired redaction;
|
|
do not implement a separate subtree comparison algorithm.
|
|
- Views compare the same native location even when the application Go root types
|
|
differ. Follow upstream's errors for mismatched scopes or unavailable views.
|
|
- A valid absent scope still yields a view, allowing history and added/removed
|
|
comparisons. Constructor success does not establish effective presence. Rendering
|
|
an absent effective value fails without output; explicit null renders normally.
|
|
|
|
Application-level inspection remains useful for the whole shared configuration.
|
|
The focused view adds convenience and supports existing integrations without
|
|
claiming exclusive access to PromptKit information. Resource APIs continue to
|
|
select and describe logical prompts, profiles, and backends independently of
|
|
configuration-tree paths.
|
|
|
|
### Resource Version Validation
|
|
|
|
**Accepted:** this migration gives up PromptKit-specific validation of `version`
|
|
fields in resource documents, including collection, prompt, and profile
|
|
definitions. This is an intentional simplification with a small practical cost,
|
|
not an unresolved compatibility requirement or a prerequisite for an upstream
|
|
version-policy feature.
|
|
|
|
The application retains ownership of ConfigKit's version policy. ConfigKit-owned
|
|
headers and directives follow that policy and native composition semantics;
|
|
PromptKit adds no separate resource-version checks. Apply this resource decision
|
|
to both application-owned loading and standalone construction. Update examples
|
|
and migration guidance accordingly, without preparsing or stripping headers or
|
|
adding a second load.
|
|
|
|
## Guidance For Implementation Planning
|
|
|
|
### Validate The Pattern Against PromptKit
|
|
|
|
The actual PromptKit prototype check is complete; see the
|
|
[integration assessment](configuration-integration-check.md) for evidence,
|
|
the smaller three-method handoff interface, and remaining implementation work.
|
|
The checklist below records what the check needed to establish and guides
|
|
implementation coverage; production implementation is not complete.
|
|
|
|
ConfigKit's released APIs and maintained
|
|
[one-library](../../../ConfigKit/examples/consumers/result_component.go) and
|
|
[two-library](../../../ConfigKit/examples/consumers/result_composition.go) examples
|
|
provide the integration starting point. They demonstrate typed registration,
|
|
whole-result handoff, owned settings capture, native inspection, and effective
|
|
source context without another load.
|
|
|
|
The release review passed PromptKit's full test suite against the published
|
|
v1.4.0 module using a temporary dependency file and Go 1.27.4. ConfigKit's focused
|
|
lookup, inspection, comparison, history, and component-example checks also passed.
|
|
This establishes compatibility with PromptKit's existing implementation. The
|
|
isolated prototype now also verifies the proposed handoff pattern; the production
|
|
handoff remains to be implemented.
|
|
|
|
The prototype demonstrates the typed registration helper,
|
|
one application-owned load, and whole-result engine construction. It also exercises
|
|
the standalone convenience path through the same rules. The checks cover:
|
|
|
|
1. **Public settings and callback composition.** Define a supported public settings
|
|
model suitable for ConfigKit decoding; existing typed source helpers are not
|
|
automatically that model. Preserve application callbacks, sensitivity, and
|
|
registrations when adding PromptKit's rules. Demonstrate explicit callback
|
|
ordering with a second representative component, without a registry. Keep
|
|
bundled underlay selection in the original load and specify its consumer API.
|
|
Include inline application-field maps at the accepted extension points and
|
|
preserve those values during PromptKit normalization.
|
|
2. **Per-load presence capture.** The current resolver captures input presence and
|
|
repair-budget presence in private maps used later during resource construction.
|
|
Determine which facts can come directly from final metadata and which need
|
|
per-load captured state. Do not rely on historical reconstruction or a shared
|
|
helper closure that associates one load's state with another result. Verify
|
|
independent loads using the same configured definition.
|
|
3. **Ownership and source context.** Capture mutable settings appropriately at
|
|
engine construction. Verify effective origins and filesystem contexts survive
|
|
mounting, inheritance, and overrides. Document mutation-before-handoff behavior;
|
|
no mutation-detection subsystem or ConfigKit deep-copy guarantee is required.
|
|
4. **Inspection facade and diagnostics.** Retain the native focused view and specify
|
|
public signatures for its operations and load-diagnostic forwarding. Obtain
|
|
diagnostics from the original result's `Diagnostics()` method through
|
|
PromptKit's consumer-owned interface. Expose focused enumeration
|
|
alongside native metadata and query operations. A PromptKit overview explanation
|
|
selects `/promptkit`; the native empty path selects the whole application.
|
|
|
|
Document what the interfaces do: their inputs, operations, outputs, and errors,
|
|
and how registration, loading, and engine construction fit together. Keep this
|
|
guidance concrete; hypothetical misuse does not call for registration tracking,
|
|
another validation lifecycle, or defensive caveats.
|
|
|
|
CommandKit neutrally exposes upstream PromptKit APIs. Any adapter updates are
|
|
routine downstream adoption, not a constraint on PromptKit's public API or a
|
|
prerequisite for completing this migration. PromptKit's implementation plan covers
|
|
its own resource, inspection, and diagnostic capabilities; CommandKit owns how
|
|
those are presented through CLI commands.
|
|
|
|
Specify the concrete registration, settings, and constructor shapes, source-helper
|
|
adaptations, and migration examples after these checks. Identify any remaining
|
|
concrete gap rather than infer a need for a larger upstream framework. Keep the
|
|
repair-default and handoff changes separately reviewable; the repair work need not
|
|
wait for the configuration refactor. Neither needs a new execution subsystem or
|
|
compatibility resolver.
|
|
|
|
### Focused Validation
|
|
|
|
- Mode defaults after composition, null clearing an inherited repair budget,
|
|
explicit zero, positive budgets, and rejection of positive budgets for `none`;
|
|
cover typed and runtime inputs as well as YAML.
|
|
- Concrete budgets in resource inspection and prepared snapshots; actual repair
|
|
counters retain their existing meaning.
|
|
- An application-owned type with siblings, typed component registration, one load,
|
|
and whole-result handoff with no post-load extraction or second load.
|
|
- Equivalent effective resources through registered application loading and
|
|
standalone convenience loading when supplied equivalent sources.
|
|
- Callback preservation and independent per-load state, including repeat loads
|
|
with the same definition and a representative second component.
|
|
- Mixed embedded/operator collections, source-relative message/schema references,
|
|
native inheritance and overrides, and effective field context at handoff.
|
|
- Unavailable paths, found nil values, zero/defaulted settings, and wrong section
|
|
types; source presence remains distinct from lookup availability.
|
|
- Missing application settings with bundled defaults enabled and disabled,
|
|
application-owned root keys resembling legacy settings, and additional fields
|
|
at each accepted extension point. Recognized fields retain type and semantic
|
|
validation; nested PromptKit-owned objects retain their existing schemas.
|
|
- Additional fields survive native composition, inheritance, normalization, and
|
|
configuration inspection, including application-defined sensitivity. They remain
|
|
available to application validation and do not enter provider requests merely
|
|
because they appear on a resource definition.
|
|
- Resource inspection omits application-specific fields while configuration
|
|
inspection retains them under ConfigKit's native sensitivity rules.
|
|
- Engine independence from later caller mutation of settings, including nested
|
|
maps and slices; trusted lookup does not replace redacted inspection.
|
|
- Resource listings, resource reports, and load diagnostics through PromptKit's
|
|
public APIs; configuration output and overview explanations focused on
|
|
`/promptkit`, with native paths and redaction preserved.
|
|
- Focused enumeration alongside unfiltered metadata; explicit
|
|
queries outside the focus and shared source/history context retain native behavior.
|
|
- Inspection remains frozen after settings mutation; absent, null, historical-only,
|
|
and added/removed sections follow upstream semantics. Sibling difference records
|
|
are excluded while ancestor differences remain; comparison works across different
|
|
application root types selecting the same scope.
|
|
|
|
The product direction, including focused inspection, removal of resource-version
|
|
validation, and the configuration-only inspection of application-specific fields,
|
|
is settled. No product decisions remain before implementation planning.
|
|
Concrete public API shapes, diagnostic forwarding, and verification of the actual
|
|
PromptKit integration are implementation-planning work.
|
|
|
|
## Further Candidates
|
|
|
|
Append additional pre-release considerations here as they arise.
|