In short. One call, services.AddAgentExperience(...), wires the whole loop with safe defaults, and each package
also registers its own services for a host that wires them by hand. The schema is
applied explicitly, on every deploy, by a PostgreSQL owner role; the application itself connects as a separate
application role that owns nothing and gets exactly the privileges the stores need. That two-role shape is the
supported deployment, and it is what makes the append-only guards and the single erasure path bind the application's
own credentials. Every call checks the host's AuthorizationContext before touching the database, and scope matching
is exact.
Supported: .NET 10 (net10.0 only); PostgreSQL 15, 16, 17 and 18; Microsoft.Agents.AI [1.22.0, 2.0.0). A host
on .NET 8 or 9 stays on 0.1.0-preview.3, the last release that targets net8.0 and net9.0. See
Compatibility evidence.
services.AddAgentExperience(...) (in AgentExperience.MicrosoftAgentFramework) registers Core, retrieval and the
injection provider with safe defaults, and returns a builder to choose the storage on. UseAgentExperience(provider) on
the agent builder then wires injection, capture, verification and storage. The
README quick start shows it whole. For development and tests:
services.AddAgentExperience(options =>
{
options.UseDevelopmentIdentity(); // one fixed local identity; refused outside Development/Test
options.Verify = ExperienceVerifiers.ToolSucceeded("run_tests"); // your own checks: without them nothing is stored
// run_tests reports failure as "exit=N" in its result: classify it, or ToolSucceeded sees every call as returned.
options.Capture = capture => capture.ClassifyToolResult = ExperienceToolResultClassifiers.ExitCode("run_tests");
}).UseInMemoryStorageForDevelopment();
await using var provider = services.BuildServiceProvider();
AIAgent agent = new ChatClientAgent(chatClient, name: "triage-ticket").AsBuilder().UseAgentExperience(provider).Build();In production the identity comes from your own authentication, and the storage is PostgreSQL:
services.AddAgentExperience(options =>
{
options.ResolveIdentity = (context, cancellationToken) => ...; // required: your authentication, never model output
options.TaskId = "triage-ticket"; // or ResolveTaskId, per invocation; default: the agent's name
options.Verify = (context, cancellationToken) => ...; // optional: without it nothing is stored
// Optional, each with a safe default:
// options.Sanitization = ...; AgentExperienceDefaults.Sanitization: no tool argument kept until allowlisted
// (AgentExperienceDefaults.SanitizationAllowing("ticketId") keeps one), results kept
// and cut by CaptureLimits, secret-named fields redacted
// options.CaptureLimits = ...; AgentExperienceDefaults.CaptureLimits: 10 attempts, 50 tool calls, 4,000 characters
// options.Injection = injection => ...; adjusts ExperienceInjectionOptions after the defaults
// options.Capture = capture => ...; adjusts ExperienceCaptureOptions after the defaults (OnCaptureFailure, ...)
// options.TimeProvider = ...; the clock; the container's, else the system clock (a different
// instance from a registered TimeProvider throws)
// options.IdentityTimeout = ...; how long ResolveIdentity may take; 5 seconds
// options.ReuseEvidence = ReuseEvidenceMode.SameTask; let verified reuse move confidence; Off by default
// options.ContradictOnFailure = true; with it, a failed run contradicts what it was given; false by default
// options.Embeddings = ...; Auto: semantic retrieval when an IEmbeddingGenerator is registered; Off; Required
// options.MinimumSemanticSimilarity = ...; the cosine a vector match needs; 0.5 (-1: no floor)
// options.EmbeddingGeneratorKey = ...; a keyed IEmbeddingGenerator to use; the unkeyed one by default
})
.UsePostgres(appConnectionString) // or .UsePostgres(dataSource), or .UseInMemoryStorageForDevelopment()
.WithPostgresVectors(); // optional (AgentExperience.Storage.Postgres.Vectors): semantic retrieval, see belowReuseEvidence and ContradictOnFailure are carried onto ExperienceFinalizationOptions (see
Letting reuse move confidence); ContradictOnFailure without
ReuseEvidence is refused, since it would do nothing. Set them here, not on a registered ExperienceFinalizationOptions.
One registered as an instance before AddAgentExperience is kept, with the two values carried onto a copy; one
registered through a factory or a type is refused; and one registered after AddAgentExperience replaces the
library's, so building an agent with UseAgentExperience throws when its values disagree with these.
What the library fills in, so the host does not:
| The one-call setup | Explicit wiring | |
|---|---|---|
| Who the run is for | ResolveIdentity returns an ExperienceIdentity(Authorization, Scope), used for capture, retrieval and finalization alike; null means no capture and no injection for that invocation |
Each resolver builds its own AuthorizationContext and Scope |
| Run description | TaskId (or ResolveTaskId; with neither, the agent's name reduced to an identifier, else "default"), the identity's scope, and DerivedTaskText |
ResolveRun |
| Retrieval request | The identity and DerivedTaskText; no text means no injection |
ResolveRequestAsync |
| Finalization request | Verify returns the required checks, the evidence and the artifact revision; the library adds a new ClosedVerificationRound (identifier from NewId), that revision as the current one, StorageDecision.Permit, the finalization time and the authorization. context.CreateEvidence(...) builds evidence bound to the round; evidence naming another round or revision is reported and nothing is stored. ExperienceVerificationContext.Create(...) builds a context for unit-testing Verify |
ResolveFinalizationAsync builds the whole FinalizeExperienceRequest |
The identity is resolved once per invocation, before capture, and injection reuses the answer (an identity, none, or a
failure); only an agent built without UseAgentExperience has injection ask on its own. Past IdentityTimeout the
invocation runs with no memory and the timeout is reported. A run keeps the identity it was opened under: an
invocation that continues it under another principal or scope is refused (reported at StartRun, and run uncaptured),
and a run the open-run bound closes is finalized under its own identity. Verify runs inside FinalizationTimeout,
which the one-call setup sets to 2 minutes rather than capture's 5 seconds, since checks often run tests; the caller
waits for it, so lower it, or verify out of band, if that is too long. Set it, like any capture option, through
options.Capture; a hook that sets the synchronous ResolveFinalization beside Verify is refused. The capture of
every agent built with UseAgentExperience is disposed with the container. Storage is chosen on the builder, once:
.UseInMemoryStorageForDevelopment() (AgentExperience.Storage.InMemory, with the same development-only guard as
AddAgentExperienceInMemoryStorageForDevelopment) or .UsePostgres(...) (AgentExperience.Storage.Postgres: the
data source, the record store and the candidate source). A second choice, or one beside an already-registered record
store or a different data source, throws. Without one, UseAgentExperience and
GetAgentExperienceContextProvider throw InvalidOperationException naming the choices. Nothing migrates the schema:
that stays an explicit step for the owner role (Applying the schema). Grants, reuse feedback,
encryption, signing and independence options are registered with the calls below, beside the one-call setup.
AddAgentExperience registers with TryAdd, so a service the host registered first is kept; calling it twice throws.
Semantic retrieval. When the container has a Microsoft.Extensions.AI IEmbeddingGenerator<string, Embedding<float>>
(registered before or after AddAgentExperience), the setup embeds each stored lesson's sanitized summary and retrieves
by meaning as well as by words; with none, nothing changes. The in-memory storage has an index built in. With
PostgreSQL, add .WithPostgresVectors() after UsePostgres and apply the vector schema as the owner role
(ExperienceVectorSchemaMigrator.MigrateAsync, after the base migration and before the privileges; see
Applying the schema). Without it, a PostgreSQL setup stays text-only and reports that once,
through OnCaptureFailure at stage Embeddings. A missing vector schema, or a generator that throws or times out,
never fails a run: that retrieval answers from text and says so (VectorFallback), and that record's indexing reports
the failure. options.Embeddings = AgentExperienceEmbeddings.Off keeps retrieval text-only whatever is registered, and
Required makes building an agent throw unless semantic retrieval is on. A vector match counts only at a cosine similarity
of at least options.MinimumSemanticSimilarity (0.5). Behaviour change for a host that already registers a generator:
task texts and lesson summaries start going to it on upgrade; Embeddings = Off is the opt-out. Records stored before,
or under a previous embedding model (whose vectors are then ModelMismatch, so retrieval answers from text alone), need a
re-index pass. The indexing guide has the details, the re-index snippet, and what
is sent to the provider.
The development identity. options.UseDevelopmentIdentity(tenant) sets ResolveIdentity to one fixed identity:
the tenant (local by default), an application and a project named after the registered IHostEnvironment's
ApplicationName, else the entry assembly (under a test runner, the runner's; app when it has no usable name), and
the principal local-developer with no roles, issued at each invocation by the configured clock. Every invocation then runs as the same caller in the
same scope. It runs only where the in-memory storage does, by the same rule: the environment named Development,
Test or Testing (case-insensitive), read from the registered IHostEnvironment, else DOTNET_ENVIRONMENT, else
ASPNETCORE_ENVIRONMENT, with nothing to check when none is set. Anywhere else, UseAgentExperience and
GetAgentExperienceContextProvider throw InvalidOperationException naming the rule, and under a Generic Host a hosted
service throws from StartAsync, so the host refuses to start; there is no override. Setting
ResolveIdentity afterwards replaces it.
The task ID. With neither TaskId nor ResolveTaskId set, each run is captured under the invoked agent's Name,
reduced to an identifier (lower case; ASCII letters, digits, . and _ kept; every run of other characters, dashes
included, one -; at most 100 characters, never fusing two words at the cut: Triage - Bot becomes triage-bot), and
under "default" when the agent has no usable name. Reading options.TaskId while it is unset still returns
"default", although runs use the agent's name; and distinct names can reduce to the same identifier (Triage Bot and
triage-bot), so their runs share it. With several agents, set TaskId or ResolveTaskId explicitly.
The README quick start is the setup above with a run_tests tool. What it leaves out:
- Which previews. It needs
0.1.0-preview.11or later. From0.1.0-preview.7to0.1.0-preview.10, setResolveIdentityas in the production setup above and also give the agentprovider.GetAgentExperienceContextProvider()inChatClientAgentOptions.AIContextProviders; earlier previews have only the explicit wiring. - What to install and supply.
AgentExperience.MicrosoftAgentFrameworkandAgentExperience.Storage.InMemory(--prerelease), plusMicrosoft.Extensions.DependencyInjectionforBuildServiceProviderif your app does not already have it.chatClientis anyMicrosoft.Extensions.AIIChatClient;RunTestsAsyncis your own test run, given to the agent as itsrun_teststool and returning text such asexit=0. - Where it runs. The in-memory storage is for development and tests only: nothing survives a restart. It and
UseDevelopmentIdentity()refuse any environment butDevelopment,TestorTesting, read from the host environment, elseDOTNET_ENVIRONMENT, elseASPNETCORE_ENVIRONMENT; a console app with none set needs nothing (see The development identity above). - When a run is stored. Only after your own check: see
Finalizing from the MAF adapter for what
ToolSucceededstores, as what, and what the next run's lesson names. - How a later run finds it. By the words (or, with an embedding generator, the meaning) of its task text, which is the user's latest message and is stored as the run's description: see Which lessons a run gets.
- The task ID. Each run is stored under the agent's name as its task ID (
triage-ticket;options.TaskIdoroptions.ResolveTaskIdsets another, and an unnamed agent's isdefault); see The task ID above. - One block.
UseAgentExperiencewires capture and injection both; an agent that also listsprovider.GetAgentExperienceContextProvider()inChatClientAgentOptions.AIContextProvidersstill gets exactly one block, and over aChatClientAgenteither wiring sees the replayed chat history and a run's own tools (see Injection on the agent, or on its options). - Nothing throws into the agent. A slow store or
ResolveIdentitymeans no memory for that run, reported throughoptions.Captureandoptions.Injectioncallbacks. - Argument values. No tool argument value is kept until you allowlist it
(
AgentExperienceDefaults.SanitizationAllowing("ticketId")), and secret-named fields are redacted (see Sanitization happens at capture).
The quick-start sample runs this wiring with a scripted model, twice on one ticket.
Verify decides whether a run is stored, and as what. Writing one by hand means building the RequiredChecks, the
evidence (context.CreateEvidence(...)) and the artifact revision. ExperienceVerifiers builds them from a signal the
host already has. Each member returns the exact delegate type of Verify, so options.Verify = ExperienceVerifiers.ToolSucceeded("run_tests"); works. A verifier decides only from facts the host's code produced (the
captured outcome of a tool call, or the host's own delegate), never from message text or model output. A verifier that
returns null stores nothing; one that throws stores nothing and is reported through OnCaptureFailure at stage
Finalization, like any Verify.
| Verifier | Pass | Fail (stored Quarantined) |
Not stored (null) |
Check ID, evidence kind, producer |
|---|---|---|---|---|
ToolSucceeded(toolName, checkId?, artifactRevision?) |
The run's last call to toolName (ordinal name match; ordered by attempt, then by call sequence within the attempt, across all attempts) returned |
That call failed: it threw, or ClassifyToolResult named an error class for it |
No call to toolName |
checkId (default toolName), ToolOutcome, tool:<toolName> |
Check(checkId, check, artifactRevision?) |
check returned true |
false |
null |
checkId, HostCheck, host |
Approval(checkId, approve, artifactRevision?) |
approve returned ExperienceApproval(approverId, Approved: true) |
Approved: false |
null; a blank ApproverId throws (reported, not stored) |
checkId, HumanApproval, the approver ID |
AllOf(verifiers…) |
Runs them in order; one verification with every check and all the evidence, so it passes only when every check does | Any check fails | The first one that returns null (the rest are not run). Duplicate check IDs, a null check or two different explicit revisions throw (reported, not stored) |
Those of its verifiers; the one explicit revision, else unversioned |
AnyOf(verifiers…) |
Runs them in order and returns the first verification that passed (every required check has Pass evidence of an accepted kind, none other), without running the rest |
None passes: the first verification returned | All return null |
That verification's |
ToolSucceeded decides from each captured call's name, sequence and Error, never from a result, so it works when
sanitization redacts results. The attempt order comes first because a call's sequence number counts only within its
attempt: a retry that finally passed is judged by its passing call. It relies on capture to mark failures: a tool that
reports failure in its result rather than by throwing counts as returned unless a classifier marks it failed, such as
ExperienceToolResultClassifiers.ExitCode("run_tests") (exit N for a non-zero exit code) or
HttpStatus("fetch_page") (HTTP N for a status of 400 or more), set through options.Capture (see
Tools that report failure in their result):
options.Verify = ExperienceVerifiers.AllOf(
ExperienceVerifiers.ToolSucceeded("run_tests", artifactRevision: buildId),
ExperienceVerifiers.Approval("reviewed", async (context, cancellationToken) =>
// Your approval flow, returning ExperienceApproval? (null: no decision, nothing stored).
await reviews.GetDecisionAsync(context.Run.RunId, cancellationToken),
artifactRevision: buildId));
options.Capture = capture => capture.ClassifyToolResult = ExperienceToolResultClassifiers.ExitCode("run_tests");Approval stores the approver ID verbatim as the evidence's producer, so pass an opaque identifier (a user's object
ID), not an email address or a name.
The revision. Every verifier's artifactRevision defaults to ExperienceVerifiers.UnversionedRevision
("unversioned"). Pass a real one, such as a build ID or a commit, when the artifact the agent works on changes between
runs, so a lesson is tied to the revision it was verified against.
UseAgentExperience adds the container's context provider at the agent level, through MAF's
AIAgentBuilder.UseAIContextProviders, unless the agent already holds any ExperienceContextProvider in
ChatClientAgentOptions.AIContextProviders (any one, deliberately, to avoid double injection) or an earlier
UseAgentExperience on the same pipeline already added one; then it adds nothing, and the agent gets exactly one block
either way. The provider is resolved only when it is added, when the agent is built. The two wirings inject the same block and keep the same session account. Over a
ChatClientAgent the agent level sees what the agent hands the provider on its options: the chat history the agent
replays, read the way the agent reads it (so a short follow-up is joined with an earlier turn's request), and the tools
a run adds through ChatClientAgentRunOptions followed by the agent's default tools (so with
ReceivingAgentCapabilities.UseRunTools a run-only tool counts). The derived task text and the block are the same,
byte for byte. The agent's rules apply: no history is read under a conversation ID, with per-service-call persistence,
when the agent would refuse the run (a history-provider conflict), or for a continued background response, which gets
no injection at all, as on the options. With no session the agent starts one, and the read does the same. Differences
remain:
- The default
InMemoryChatHistoryProvideris read without side effects, from the messages it stores. A reducer that runs before retrieval is therefore not run a second time, and an output filter is not applied: the join reads the history as stored, not the reduced or filtered view the model gets. The two differ only when the reducer or filter drops the earlier turn the follow-up is joined to, which a reducer that keeps the latest messages does not. - Any other history provider is asked the way the agent asks it, so it is read once more per run, with whatever cost
or side effect a read has (a remote store, say). That read is bounded at 500 milliseconds and by the caller's
cancellation; a read that fails or times out joins nothing, adds no failure to the run, and sets
agentexperience.history_readon theinjectspan. - Messages or tools that another context provider on the agent's options adds, or run options and sessions that middleware between the two replaces, are not seen.
- An inner agent that is not a
ChatClientAgentoffers the caller's messages only, and the tools it reports throughGetService<ChatOptions>()(none otherwise).
Where any of that matters, give the agent the provider on its options too:
AIAgent agent = new ChatClientAgent(chatClient, new ChatClientAgentOptions
{
Name = "triage-ticket",
AIContextProviders = [provider.GetAgentExperienceContextProvider()],
})
.AsBuilder().UseAgentExperience(provider).Build();options.InjectWithUseAgentExperience = false turns the agent-level injection off: an agent without the provider on its
options is then captured and verified but not injected, as UseAgentExperience alone did before it wired injection.
The one-call setup also registers AgentExperienceLessons, a read-only view of what is stored in one scope. Get it
with provider.GetAgentExperienceLessons() (or from the container), and list a scope with ListAsync(identity). After
the quick start, ListForDevelopmentIdentityAsync() lists what
UseDevelopmentIdentity() stored; it throws InvalidOperationException when the setup does not use that identity. For
each lesson, newest first, it gives:
- its experience ID, task ID, status, confidence, and when it was stored and last changed;
- the checks that verified it, each with its evidence kind and result (never the evidence's detail text);
BlockText, the Historical Reference block an agent in that scope would be shown for that record, rendered by the container's own context provider with its injection options (layout, failure detail, argument allowlist). It is the same text as an injected block but for two things: ranking depends on the task, so its ranking line says there is none (Matched: no ranking datain the compact layout), and the record stands alone as record 1, where a real block numbers it by rank. It is rendered even for a lesson retrieval would not use;IsEligible, withIneligibility(anExperienceLessonIneligibilitycode) andIneligibleReason(words) when it is not: whether retrieval would currently consider it, setting the task aside. The codes areStatus(quarantined, retired and so on),BelowConfidenceFloor,ModelAuthoredandContentUnconfirmed(when the provider excludes model-authored lessons),Expired(older than the policy's maximum age),OverByteBudget(its block alone does not fit the provider's byte budget) andUnrenderable(the record could not be rendered; the rest are still listed);LastInjectedAtandInjectionCount, a field on each lesson: when an agent in that scope was last shown the lesson, and how many injected blocks delivered it. It has three states: an instant; never (nulland 0 while the page'sInjectionsTrackedistrue); and not tracked (InjectionsTrackedisfalse: a store that does not track injections, or a PostgreSQL schema whose application role has not been granted0026's table yet), which the HTML shows as "not tracked" rather than "never". .NET, Python and Go agents all record their injections (the ports from contract 1.4), so in a mixed fleet "never" means no agent in the scope was shown it;ConsolidatedFrom, for a lesson a consolidation pass merged (ConsolidateAsync(identity)on the same view): how many source records it was merged from, each now superseded by it. See Consolidating repeated lessons.
ExperienceLessonsQuery narrows it: Statuses, Limit (the store's bounds, 1 to 500, default 50) and
IncludeIneligible (default true). The page's HasMore says whether more lessons match than it shows (or may: the
store answers at most 500 records per call). It reads through IExperienceRecordStore.QueryWithInjectionsAsync (the
page and each record's last injection in one bounded read; in PostgreSQL one statement) under the identity's own
authorization and scope, the boundary retrieval reads within, so lessons another scope lends through a sharing grant
are not listed; a record the store answers from another scope is left out and counted in OutOfScopeDropped. It never
writes. A store answer other than Found (Denied, say) is an empty page carrying that outcome, not an exception.
With IncludeMeasuredBenefit = true (default false), each lesson also gets MeasuredBenefit: what the
injection holdout measured for it, from one benefit estimate for the
page's scope over the context provider's benefit window and threshold (30 days and 20 runs per arm by default, whether
or not the benefit policy is on). A Hurts verdict there is why the policy omits
the lesson. It is null for a lesson no assigned run named, and for every lesson when the store does not record the
holdout or refused the read. Left off, nothing more is read and the page is unchanged.
Where "last injected" comes from. The context provider records it after it hands a block over, in the
background, so the model is called without waiting: it calls IExperienceRecordStore.RecordInjectionAsync with the
records the block delivered (one upsert in PostgreSQL, into 0026's experience_injections), bounded by a timeout of a
few seconds of its own, and abandons a store that ignores cancellation. It never throws and never delays or fails the
invocation; the outcome is the record_injection operation in telemetry (Recorded, NotTracked,
NotRecorded, or OutcomeUnknown for a timeout or cancellation, which may have landed). It can therefore lag an
injection by a moment. The store counts only records live in exactly the injecting scope, clamps the instant to its own
clock and saturates the count, so a lesson another scope borrows through a sharing grant and injects there is counted in
neither scope; its owner sees those deliveries in the grant access log instead.
A store opts in with IExperienceRecordStore.TracksInjections, and must override that, RecordInjectionAsync and
QueryWithInjectionsAsync together, or none of them (a decorator must forward all three): the defaults record nothing
and read as not tracked, and the store conformance suite fails a store that overrides only some
(RecordStoreConformanceTests.AssertInjectionTrackingIsConsistent). The in-memory and PostgreSQL stores track them.
After migrating to 0026, re-run ApplyApplicationRolePrivilegesAsync (it also runs before the migration: the table is
granted once it exists). Until then the application role cannot reach the table: injections report NotRecorded, the
view reads as not tracked, and the store reports it once through PostgresExperienceRecordStore.OnInjectionTrackingUnavailable.
In a fleet that also runs a Python or Go port, the application role's privileges are either the .NET call, or the
contract's two files in this order: sql/application-role-privileges.sql, then sql/application-role-privileges-1.4.sql
(the injection table's grants, which the first file's REVOKE ALL removes). Together the two files are exactly what the
.NET call grants, so the .NET call may also run after them. With the 1.0 file alone, every agent still injects, and
each injection is reported NotRecorded.
ExperienceLessonsHtml.Render(page) returns one self-contained HTML document: inline CSS, no script, no external
request, light and dark mode, a "development view" banner and the scope. Every value drawn from a record is
HTML-encoded, because lesson text is untrusted, and the page's content security policy allows no script.
The library maps no endpoint, and nothing in it enforces "development only". Map it yourself, behind your own authorization, with the identity from your own authentication; never take the scope from the query string. Exposing it publicly discloses every lesson in the scope. In an ASP.NET Core minimal API, mapped only in development:
if (app.Environment.IsDevelopment())
{
app.MapGet("/agent-experience/lessons", async (HttpContext http, CancellationToken cancellationToken) =>
{
ExperienceIdentity identity = MyIdentity.From(http.User); // your own mapping from the authenticated user
var page = await http.RequestServices.GetAgentExperienceLessons().ListAsync(identity, cancellationToken: cancellationToken);
http.Response.Headers.CacheControl = "no-store";
http.Response.Headers["X-Frame-Options"] = "DENY";
http.Response.Headers.ContentSecurityPolicy = "default-src 'none'; style-src 'unsafe-inline'; frame-ancestors 'none'";
return Results.Content(ExperienceLessonsHtml.Render(page), "text/html; charset=utf-8");
}).RequireAuthorization();
}The headers matter: no-store keeps the lessons out of browser and proxy caches, and X-Frame-Options with
frame-ancestors 'none' stops another site framing the page. The page's own content security policy is a <meta> tag,
and a <meta> policy cannot set frame-ancestors; only a response header can.
From a console app, write it to a file and open it:
var page = await provider.GetAgentExperienceLessons().ListForDevelopmentIdentityAsync(); // after the quick start
// or, with your own identity: .ListAsync(new ExperienceIdentity(authorization, scope))
File.WriteAllText("lessons.html", ExperienceLessonsHtml.Render(page));Since contract 1.14 the Python and Go ports render the same page, byte for byte (render_lessons_html(page),
ae.RenderLessonsHTML(page)), and serve it idiomatically: Python's lessons_html_response(lessons, identity) returns
a status, headers and body any ASGI or WSGI framework can send, and Go's ae.LessonsHTMLHandler is an http.Handler
that takes the host's identity function. Both send the headers above. As in .NET, the authorization is yours: map the
identity from your own authentication, and never expose the page publicly.
When a lesson was last injected is not shown in this version.
using AgentExperience.Core.DependencyInjection;
using AgentExperience.Storage.Postgres.DependencyInjection;
using AgentExperience.Storage.Postgres.Vectors.DependencyInjection; // optional: the vector channel
services.AddSingleton(NpgsqlDataSource.Create(connectionString));
services.AddAgentExperiencePostgresStore(); // IExperienceRecordStore
services.AddAgentExperiencePostgresCandidateSource(); // IExperienceCandidateSource
services.AddAgentExperiencePostgresGrantStore(); // IExperienceGrantStore, optional: only a host
// that shares records across scopes needs it
services.AddAgentExperiencePostgresReuseFeedbackStore(); // IExperienceReuseFeedbackStore, optional: only a
// host that records reuse feedback needs it
services.AddAgentExperiencePostgresGrantAccessLog( // IExperienceGrantAccessLog, optional: only a host
onNotRecorded: failure => logger.LogError( // that wants to know who read shared records
failure.Failure, "grant access row not written")); // needs it. Off unless wired.
services.AddAgentExperiencePostgresEncryption(keyStore); // optional: crypto-shredding for every component
services.AddAgentExperiencePostgresEmbeddingIndex(); // IExperienceEmbeddingIndex
services.AddAgentExperienceEmbeddingGenerator(); // IExperienceEmbeddingGenerator, over a registered
// IEmbeddingGenerator<string, Embedding<float>>
services.AddAgentExperienceCore(sanitizationOptions, captureLimits);
// -> ISanitizer, IExperienceCaptureService, IExperienceReflector,
// ExperienceLifecycleService (verifying independence), ExperienceFinalizationService
services.AddSingleton(new ExperienceIndependenceOptions // optional: the assessment token key, without
{ // which human evidence is refused; or the
AssessmentTokenKey = secrets.AssessmentTokenKey, // TrustHostSuppliedIdentifiers opt-out
});
services.AddSingleton(new ExperienceProvenanceSigningOptions( // optional: finalization signs what it vouches
secrets.ProvenanceSigningKeys, // for, and verification refuses records written
currentKeyId: "prov-2026-09")); // or edited outside it; keys from your secret
// store, never the database (confidence guide)
services.AddAgentExperienceIndexing(); // ExperienceIndexingService, and finalization's
// post-commit hook, in either registration order
services.AddAgentExperienceRetrieval(); // ExperienceRetrievalService
// Optional, each picked up in any registration order (register them as singletons):
// services.AddSingleton<IEnvironmentCompatibilityScorer>(myScorer); // grades preferred environment attributes
// services.AddSingleton(new ConfidenceDecayPolicy { HalfLives = ... }); // per-domain confidence decay when ranking
// services.AddSingleton<IExperienceConfidenceEngine>(myEngine); // replaces the default confidence heuristic
// -> defaults to RetrievalPolicy.Default and RankingWeights.Default; pass your own to override
// -> hybrid, because an index *and* a generator are registered; text-only, and flagged, if either is missing
services.AddAgentExperienceReuseFeedback(); // ExperienceReuseFeedbackService, over the ledger
// above and the lifecycle service
// Wired by hand, injection has no registration of its own: ExperienceContextProvider needs a per-host
// resolver and risk decision, so the host constructs it and adds it to ChatClientAgentOptions.AIContextProviders
// (see the injection guide). The one-call setup above registers one for you.The ports are registered independently: a host that only writes experience never has to register the search, one
that only reads never has to register the store, one that never shares a record across scopes never has to register
the grant store — the reads that honour grants do so in SQL either way — and one that never records reuse feedback
never has to register its ledger. Every registration uses TryAdd, so a host that has already registered its own
implementation keeps it. AgentExperience.Abstractions stays BCL-only; Core and the storage packages take only
Microsoft.Extensions.DependencyInjection.Abstractions (no container, no hosting) for these extensions.
The vector registrations and the vector migration are optional, and genuinely so: leave them out and everything
still works — finalization commits records with no indexing hook, and retrieval answers from text alone with
TextOnly set to NotConfigured. That is also why the embedding schema is not in the base package's script list:
CREATE EXTENSION vector needs a superuser, and a text-only deployment must never be made to run it for a feature
it has not enabled. See Indexing.
The MAF adapter can drive finalization for you: set FinalizationService and ResolveFinalizationAsync on
ExperienceCaptureOptions and every successfully captured invocation is finalized right after it is completed. See
Finalization. AssessmentTokenIssuer is deliberately not
registered: construct it in your review flow, where a person decides (see Confidence).
The one-call setup is a composition of these public pieces, and they keep working unchanged. Wire them by hand when
you need what the setup does not offer: a sanitization policy per tool, a different authorization for retrieval and
for storage, a storage decision other than Permit, or a round your review flow closes. This wires steps 1 to 6 for
one MAF agent over PostgreSQL: apply the schema, register the services, inject past lessons before each run, and
capture, verify and store each run after it. For a throwaway local database with a single role, skip the
ApplyApplicationRolePrivilegesAsync call, which refuses the role running it. You supply chatClient, two connection
strings, and EvidenceForAsync, which runs your own checks and turns them into Evidence for the round you closed.
using AgentExperience.Abstractions;
using AgentExperience.Core.Capture;
using AgentExperience.Core.DependencyInjection;
using AgentExperience.Core.Finalization;
using AgentExperience.Core.Retrieval;
using AgentExperience.Core.Sanitization;
using AgentExperience.Core.Verification;
using AgentExperience.MicrosoftAgentFramework;
using AgentExperience.MicrosoftAgentFramework.Injection;
using AgentExperience.Storage.Postgres;
using AgentExperience.Storage.Postgres.DependencyInjection;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.DependencyInjection;
using Npgsql;
// 1. Apply the schema, on every deploy, as the database owner role.
await using (var owner = NpgsqlDataSource.Create(ownerConnectionString))
{
await ExperienceSchemaMigrator.MigrateAsync(owner, CancellationToken.None);
await ExperienceSchemaMigrator.ApplyApplicationRolePrivilegesAsync(
owner, new ExperienceApplicationRoleOptions("agent_experience_app"), CancellationToken.None);
}
// 2. Register the services. The stores connect as the application role.
var toolPolicy = new SanitizationPolicy(
AllowedFieldNames: new HashSet<string> { "ticketId", "strategy" }, // kept
SecretFieldNames: new HashSet<string> { "apiKey" }, // redacted
MaxDepth: 4, MaxFieldCount: 50, MaxValueLength: 4_000, MaxFieldNameLength: 100);
var services = new ServiceCollection();
services.AddSingleton(NpgsqlDataSource.Create(appConnectionString));
services.AddAgentExperiencePostgresStore();
services.AddAgentExperiencePostgresCandidateSource();
services.AddAgentExperienceCore(
new SanitizationOptions(new Dictionary<string, SanitizationPolicy>
{
[SanitizationKinds.ToolArguments] = toolPolicy,
[SanitizationKinds.ToolResult] = toolPolicy,
}),
new CaptureLimits(MaxAttemptsPerRun: 10, MaxToolCallsPerAttempt: 50, MaxResultLength: 4_000, MaxErrorLength: 4_000));
services.AddAgentExperienceRetrieval();
await using var provider = services.BuildServiceProvider();
// 3. Scope and authority come from your own authentication, never from model output.
var scope = new Scope(TenantId: "contoso", ApplicationId: "support", ProjectId: "tickets");
var authorization = new AuthorizationContext(
TenantId: "contoso", PrincipalId: "svc-support-agent", Roles: [], IssuedAt: DateTimeOffset.UtcNow);
// 4. Before each run: retrieve applicable lessons and inject them as a labeled Historical Reference.
var injection = new ExperienceContextProvider(
provider.GetRequiredService<ExperienceRetrievalService>(),
provider.GetRequiredService<IExperienceRecordStore>(),
new ExperienceInjectionOptions
{
// The user's latest words, bounded; with none, return null to skip injection for this run.
// Async, with the invocation's token: look up the caller's authorization and scope here if you need I/O.
ResolveRequestAsync = (context, cancellationToken) => ValueTask.FromResult(context.DerivedTaskText is { } taskText
? new RetrieveExperienceRequest(authorization, scope, TaskText: taskText)
: null),
});
// 5. After each run: capture what happened, verify it with your checks, and store the lesson.
AIAgent agent = new ChatClientAgent(chatClient, new ChatClientAgentOptions { AIContextProviders = [injection] })
.AsBuilder()
.UseExperienceCapture(provider.GetRequiredService<IExperienceCaptureService>(), new ExperienceCaptureOptions
{
ResolveRun = context => new ExperienceRunDescriptor(
TaskId: "triage-ticket", Scope: scope, TaskDescription: context.DerivedTaskText),
FinalizationService = provider.GetRequiredService<ExperienceFinalizationService>(),
ResolveFinalizationAsync = async (context, cancellationToken) =>
{
var round = new ClosedVerificationRound(Guid.NewGuid(), ArtifactRevision: "build-42");
return new FinalizeExperienceRequest(
RunId: context.Run.RunId,
Authorization: authorization,
ClosedRound: round,
RequiredChecks: [new RequiredCheck("tests-pass", ExpectedKind: "TestResult")],
// Your own checks, never the model's word.
Evidence: await EvidenceForAsync(context.Run, round, cancellationToken),
CurrentArtifactRevision: "build-42",
StorageDecision: StorageDecision.Permit,
FinalizedAt: DateTimeOffset.UtcNow);
},
OnCaptureFailure = failure => Console.Error.WriteLine($"capture failed at {failure.Stage}"),
})
.Build();
var response = await agent.RunAsync("Ticket #4812: a refund is stuck on a lock. Triage it.");DerivedTaskText is the user's latest message (with the one before it, from the same input or the session's history,
when the latest is a short follow-up such as "and retry"), never injected context or a tool result, cleaned up and cut
to 512 UTF-16 code units (see Injection). Capture sees only the invocation's own input, so
there the follow-up is joined only to an earlier message in that input. It is the user's own words, unredacted: stored
as the run's TaskDescription it is stored as written, so a host whose prompts can hold sensitive data should redact it
first.
Schema comes in two calls, matching that split, and neither store ever migrates on its own:
// As the owner role, on every deploy. The stores themselves connect as the application role.
await ExperienceSchemaMigrator.MigrateAsync(ownerDataSource, cancellationToken); // 0001-0003, 0005-0019 and 0021-0023 (no 0014), always
await ExperienceVectorSchemaMigrator.MigrateAsync(ownerDataSource, cancellationToken); // 0004 and 0020, only with the vector channel
await ExperienceSchemaMigrator.ApplyApplicationRolePrivilegesAsync( // last, so it covers both
ownerDataSource,
new ExperienceApplicationRoleOptions("agent_experience_app") { AllowErasure = true },
cancellationToken);How the migrator behaves (journaling, locking, timeouts, permissions) and what each script does is in PostgreSQL schema.
With crypto-shredding on, run the owner-run authorship backfill after upgrading past 0021, and after any rolling
deploy that ran instances on an earlier build: a sealed record stored without its authorship flag is left out of
searches that exclude model-authored lessons until BackfillSealedAuthorshipAsync writes it. That job needs both the
owner role and the key store (the KEK and access to the wrapped keys) in one process: a deliberate, one-off
combination of custody, so run it from a short-lived operator process, not from the application. See
Backfilling authorship flags.
0022's single UPDATE reads every live plaintext payload inside the migrator's transaction and under its command
timeout (30 seconds by default). On a large plaintext table, run the migrator in a maintenance window, or over a data
source whose command timeout is raised for the migration; a script that times out rolls back and changes nothing.
0028 builds one index on experience_records (ix_experience_records_source_run, on tenant_id, source_run_id) with
a plain CREATE INDEX: the migrator runs each script in a transaction, and CREATE INDEX CONCURRENTLY cannot run in
one. The build takes a SHARE lock on experience_records, which blocks every write to it (reads go on) until the
index is built: imperceptible on a small table, a write outage on a large one, and it can also run past the command
timeout. On a large table, build the index out of band first, as the owner, and the script's own IF NOT EXISTS is then
a no-op:
CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_experience_records_source_run
ON agent_experience.experience_records (tenant_id, source_run_id);A concurrent build that fails leaves an invalid index under that name, which the script would accept: drop it
(DROP INDEX CONCURRENTLY agent_experience.ix_experience_records_source_run) and build it again. 0028 changes no
privilege.
Records are normally written by Core's ExperienceFinalizationService, which creates the record and commits its
initial lifecycle event; CreateAsync and CommitLifecycleEventAsync stay available for hosts that orchestrate that
themselves.
using AgentExperience.Abstractions;
using AgentExperience.Storage.Postgres;
using Npgsql;
await using var dataSource = NpgsqlDataSource.Create(applicationConnectionString);
IExperienceRecordStore store = new PostgresExperienceRecordStore(dataSource);
// Established by the host from its own authentication and authorization. Never built from request input.
var authorization = new AuthorizationContext(
TenantId: "tenant-1",
PrincipalId: "svc-support-agent",
Roles: ["experience:write"],
IssuedAt: DateTimeOffset.UtcNow,
ProjectId: "support"); // optional bound: this caller may only touch the "support" project
var created = await store.CreateAsync(authorization, record, cancellationToken);
var read = await store.GetAsync(authorization, record.Scope, record.ExperienceId, cancellationToken);
var page = await store.QueryAsync(
authorization,
new ExperienceRecordQuery(record.Scope, Statuses: [ExperienceStatus.Validated], Limit: 20),
cancellationToken);
// A lifecycle change: the event and the record's projection commit together, or neither does.
var commit = await store.CommitLifecycleEventAsync(
authorization,
record.Scope,
new LifecycleEvent(
EventId: eventId, // this call's idempotency key -- stable across retries
ExperienceRecordId: record.ExperienceId,
PriorStatus: ExperienceStatus.Candidate,
CurrentStatus: ExperienceStatus.Validated, // decided by Core, never by this adapter
Reason: "required checks passed",
Producer: "finalization",
OccurredAt: decidedAt,
ExpectedRevision: record.Revision), // must equal the stored revision
cancellationToken);
// commit.Revision is record.Revision + 1 when commit.Outcome is Committed.
// Finding records that could apply to a task. A separate, read-only port.
IExperienceCandidateSource search = new PostgresExperienceCandidateSource(dataSource);
var candidates = await search.SearchAsync(
authorization,
new ExperienceCandidateQuery(
Scope: record.Scope, // exact scope, applied in SQL
TaskText: "refund ticket stuck on a lock", // arbitrary text; no escaping needed
EligibleStatuses: [ExperienceStatus.Validated, ExperienceStatus.Reinforced],
MinimumConfidence: 0.5,
Limit: 50),
cancellationToken);
// candidates.Candidates is strongest match first, each with a Relevance in [0, 1].Neither the store nor the search disposes the data source. The host owns it.
AuthorizationContextis the authority, andScopeonly selects within it. The host must build the context from its own trusted identity and permission checks, never from model output or request payloads.TenantIdmust always match. A non-null bound (ApplicationId,ProjectId,TeamId,AgentId,UserId) must equal the request scope's field exactly. A null bound leaves that field unrestricted. A scope outside the context returnsDeniedbefore any connection opens.- Scope matching in SQL is exact, ordinal, and case-sensitive. A null optional scope field matches only null
(
IS NOT DISTINCT FROM) and never acts as a wildcard. Empty or whitespace scope values areInvalid. - A get for an ID that exists in another scope returns
NotFound, the same as a missing ID. A create with an existing ID returnsConflictwithout record data, whichever scope owns the existing record. - The store does not evaluate roles. Role-based decisions stay with the host.
Every port returns a structured result with an outcome rather than throwing for an expected refusal, so a host can tell "not allowed" from "not there" from "not stored".
| Situation | Result |
|---|---|
| Saved | Created |
Read (a query with no matches is still Found) |
Found |
| ID missing, or in another scope | NotFound |
| Scope outside the authorization context | Denied (no connection opened) |
| Malformed request | Invalid with every StoreValidationError(Path, Message) (no connection opened) |
| ID already exists in any scope | Conflict (stored row unchanged) |
| Lifecycle event and projection committed together | Committed |
Lifecycle ExpectedRevision ≠ the record's current Revision |
StaleRevision (nothing written) |
Lifecycle PriorStatus ≠ the record's stored Status |
StatusMismatch with the stored status (nothing written) |
| Supersession check ran | Allowed with the replacement's status, or RecordNotFound / ReplacementNotFound / Cycle (nothing written either way) |
| Lifecycle event names a replacement the commit transaction will not accept | ReplacementNotAllowed with the replacement's stored status (nothing written) |
| The record was erased (in its owning scope) | Deleted |
UPDATE or DELETE against a stored event row, or a grant revocation cleared, expiry extended, or disclosure level changed |
rejected by the database with SQLSTATE 42501, surfaced as ExperienceStoreException |
Candidate search ran (no text match is still Found) |
Found with the matching candidates, strongest match first |
Database or driver failure (NpgsqlException, SocketException, TimeoutException) |
throws ExperienceStoreException with the original as InnerException |
Stored row with an unsupported payload_version or an unreadable payload |
throws ExperienceStoreException |
| Caller cancellation | throws OperationCanceledException, unwrapped |
Validation messages never contain record payload content, and the store does not log.
A create whose acknowledgement was lost (cancelled or timed out after PostgreSQL committed it) returns Conflict
when retried. After a Conflict, call GetAsync in your own scope to check whether the stored record is yours.
For a single agent on one machine (a coding agent, a CLI, an MCP server), AgentExperience.Storage.Sqlite keeps
everything in one database file, with no server and no roles:
await ExperienceSqliteSchema.MigrateAsync("experience.db"); // journaled and idempotent: run it on every start
builder.Services
.AddAgentExperience(options => { /* ... */ })
.UseSqlite("experience.db");Or opt in to migrating on first use: .UseSqlite("experience.db", o => o.MigrateSchemaOnFirstUse = true). Nothing
migrates on its own otherwise: a store checks the schema version once, on its first operation, and refuses a missing or
older schema (saying to migrate) and one newer than the package. The migrator creates a missing file owner-only: on
Unix its directory 0700 and the file 0600, and SQLite gives the -wal and -shm files the file's permissions. An
existing file or directory keeps its permissions. The file is put in write-ahead-log mode, connections run with foreign keys on and a busy
timeout, and every write is one BEGIN IMMEDIATE transaction, so several processes may share the file: two commits
from the same revision still have exactly one winner, and a writer waits for another rather than failing.
It registers the record store, the full-text candidate source (FTS5, Porter-stemmed: closer to PostgreSQL's matching
than to the in-memory store's), the failure-signature source, the reuse-feedback ledger and the run store, each passing
the same conformance suite as PostgreSQL. With the run store, the capture service keeps each completed run in the file,
so a later process can finalize it when its verdict arrives (Finalizing later). SqliteExperienceRecordStore.DeleteAsync erases a record to a tombstone, removing its
search text and every row that named it. Every connection runs with secure_delete on, so freed content is zeroed in
the file; the full-text index is rewritten (FTS5 optimize) in the erasure's transaction, so the record's terms leave
its segments at once; and the write-ahead log is checkpointed and truncated after the commit, best effort -- a reader
still holding an older snapshot keeps the pre-erasure pages in the -wal file until a later checkpoint. Copies of the
file (backups, snapshots) are outside its reach, and the tombstone keeps the ID, scope, status, counters, revision and
instants, never content. A run save waits at most SqliteStorageOptions.RunSaveBusyTimeout (2 seconds) for a busy
file, so a completion is never held up behind another process's write.
What it does not do. No sharing grants or access log, no encryption or crypto-shredding, no row-level security or database roles (anything that can open the file can read and write all of it, so protect the file as you would the agent's other secrets), no vectors (retrieval is text and failure signature only) and no retention sweeps. For a team, a shared service or any of those, use PostgreSQL.
This is the supported deployment, and the one every guarantee in these guides about append-only history and erasure is stated for. It takes two PostgreSQL roles:
- an owner role, which owns the
agent_experienceschema and everything in it and runs the migrators. The application never holds its credentials; - an application role, which is what every store, search and ledger connects as. It owns nothing and holds exactly the privileges the stores need.
Why it matters: PostgreSQL's triggers do not bind a table's owner, who can ALTER TABLE … DISABLE TRIGGER, drop a
trigger or replace a function and then write freely; and the purge markers are custom settings any session can set.
If the application runs as the owner — which it does whenever it runs the migrator itself — the append-only guards
and the single erasure path are only as strong as the application's own code. With two roles they bind the
application's own credentials.
Once, as a superuser (rename to taste; the passwords are placeholders):
-- The owner: runs the migrators. Never configured in the application.
CREATE ROLE agent_experience_owner LOGIN PASSWORD 'change-me';
-- The application: what the stores connect as. Owns nothing.
CREATE ROLE agent_experience_app LOGIN PASSWORD 'change-me-too';
-- A database the owner owns. On an existing database, GRANT CREATE ON DATABASE ... TO agent_experience_owner
-- instead -- and the database's own owner must not be the application role either: a database owner can drop it.
CREATE DATABASE agent_experience OWNER agent_experience_owner;
-- 0010 and 0012 create functions whose SET clause names the two purge markers, and PostgreSQL 15+ lets a
-- non-superuser name a custom setting there only when granted SET on it. This is the one superuser-only step
-- the base schema has.
GRANT SET ON PARAMETER agent_experience.purge_authorized, agent_experience.access_purge_authorized
TO agent_experience_owner;
-- Only with the vector channel: pgvector is not a trusted extension, so a superuser creates it, in that database.
-- The vectors migrator's own CREATE EXTENSION IF NOT EXISTS is then a no-op.
\c agent_experience
CREATE EXTENSION IF NOT EXISTS vector;Do not make the application role a member of the owner role, of pg_write_all_data, of a superuser role, of
pg_write_server_files, pg_read_server_files or pg_execute_server_program, or of any role that owns something in
the schema, and do not grant it SET on session_replication_role, or (PostgreSQL 17 and later) membership in
pg_maintain or the MAINTAIN privilege on any table in the schema. The call below refuses each of these or fails
its verification. Do not give it CREATEROLE either: that is not checked, and a role that can create and grant roles
is an administrator.
As the owner, after MigrateAsync — and after ExperienceVectorSchemaMigrator.MigrateAsync, when the host uses the
vector channel, so the embedding table is covered:
await using var owner = NpgsqlDataSource.Create(ownerConnectionString);
await ExperienceSchemaMigrator.MigrateAsync(owner, cancellationToken);
await ExperienceVectorSchemaMigrator.MigrateAsync(owner, cancellationToken); // only with the vector channel
await ExperienceSchemaMigrator.ApplyApplicationRolePrivilegesAsync(
owner,
new ExperienceApplicationRoleOptions("agent_experience_app")
{
AllowErasure = true, // DeleteAsync, SweepExpiredAsync, PurgeExpiredAsync
AllowAccessLogPurge = false, // PurgeOlderThanAsync on the access log
AllowSealing = false, // SealPlaintextRecordsAsync, the crypto-shredding upgrade job
},
cancellationToken);All three opt-ins are off unless you set them. In one transaction, under the migrator's advisory lock, the call revokes every privilege the application role holds on the schema and on every table, sequence and function in it, then grants exactly this:
| Object | The application role gets |
|---|---|
schema agent_experience |
USAGE — never CREATE |
experience_records |
SELECT, INSERT, and UPDATE on status, revision, updated_at, reuse_confidence, supporting_validations, contradictions only |
experience_grants |
SELECT, INSERT, and UPDATE on revoked_at, revocation_reason only |
lifecycle_events, experience_grant_events, confidence_evidence, reuse_feedback, reuse_feedback_exposures, experience_grant_access |
SELECT, INSERT |
experience_injections (0026, when it exists) |
SELECT, INSERT, and UPDATE on last_injected_at, injection_count only — the injection upsert; never its record ID or scope columns, and no DELETE (erasure removes a row through 0026's trigger, as the owner) |
experience_holdout_assignments (0027, when it exists) |
SELECT, INSERT — the injection holdout's assignment rows; no UPDATE (the insert is ON CONFLICT DO NOTHING) and no DELETE (erasure removes rows through 0027's trigger, as the owner). Contract 1.8's sql/application-role-privileges-1.8.sql grants the same |
experience_provider_holdout_assignments (0029, when it exists) |
SELECT, INSERT — a measured memory provider's holdout rows; no UPDATE and no DELETE (erasure removes rows through 0029's trigger, as the owner). Contract 1.11's sql/application-role-privileges-1.11.sql grants the same |
experience_failure_signatures (0030, when it exists) |
SELECT, INSERT — each record's failure signatures, written with the record and searched by PostgresExperienceSignatureSource; no UPDATE and no DELETE (erasure removes rows through 0030's trigger, as the owner). .NET-only: no contract privileges file grants it |
experience_runs (0032, when it exists) |
SELECT, INSERT, DELETE, and UPDATE on finalized_at only — the run store's staging rows, saved at completion, marked finalized, deleted after finalization or a retention; never the run or its scope columns. .NET-only: no contract privileges file grants it |
experience_embeddings (when the vectors package created it) |
SELECT, INSERT, DELETE, and UPDATE on model_id, dimension, content_hash, source_revision, embedding, updated_at only — derived, rebuildable data, not a ledger, but never its record ID or scope columns |
purge_experience_record, purge_expired_grants |
EXECUTE only with AllowErasure |
purge_grant_access |
EXECUTE only with AllowAccessLogPurge |
seal_experience_record (0016) |
EXECUTE only with AllowSealing |
search_experience_text (0025, replacing 0024's) |
EXECUTE only with EnableRowLevelSecurity (see Enabling row-level security) |
schema_versions (the journal), its sequence, and anything else |
nothing |
Then it checks the role's effective privileges — which also see grants to PUBLIC, grants made by another
grantor, memberships and predefined roles — and throws ExperienceStoreException, rolling everything back, on any
difference: CREATE on the schema, a DELETE or TRUNCATE on a ledger, an UPDATE on any other column, a
TRIGGER or REFERENCES privilege, on PostgreSQL 17 and later MAINTAIN on a table (no trigger fires on
LOCK TABLE, CLUSTER, REINDEX or VACUUM), any privilege held WITH GRANT OPTION, a sequence, a
SECURITY DEFINER function in the schema the role could execute without an opt-in, or a missing grant. What must be
absent is checked on every role the application role is a member of, not only on the ones it inherits from, so a
privilege one SET ROLE away (a NOINHERIT role, or on PostgreSQL 16 and later a membership granted
WITH INHERIT FALSE) is caught too.
Before any of that it refuses a role that does not exist, an unmigrated schema, a superuser, the role running the
call, any role that is — or is a member of — the owner of the database, the schema, or anything in it, and any role
that is or reaches a superuser, one of the server-file roles, pg_maintain (PostgreSQL 17 and later), or SET on
session_replication_role. The message names the violation; nothing is changed.
Concurrent privileges calls against one database — parallel deploy jobs, say — are serialized by the migrator's
advisory lock, so they never run their transactions at once. Run the call when store traffic is low. While it
re-creates policies and switches row-level security it takes ACCESS EXCLUSIVE locks table by table, and live store
traffic that locks the same tables in another order can deadlock with it; PostgreSQL then rolls the call's
transaction back (40P01 deadlock detected). As a mitigation, not a cure, the call retries the whole transaction —
it is all-or-nothing and idempotent — up to three attempts in all, after a short jittered wait (50–150 ms, then
100–300 ms) that honours the caller's cancellation token. Only a deadlock is retried. A deadlock on the
third attempt throws ExperienceStoreException saying the call was a deadlock victim three times in a row; every
other failure is thrown exactly as before. The call writes nothing to any log, so that message is the only report of
a retry.
Why it is an API and not a migration. A migration runs once and is journaled, so it could never re-grant on an
object a later migration adds, and the role name is host configuration. Revoke-everything-then-grant-the-list is
idempotent, so running it on every deploy is what keeps the set exact: an object a later migration adds gets nothing
until this list names it, and a stray GRANT ALL an operator made is taken away on the next deploy. Between a
later script committing and this call running, a host's own ALTER DEFAULT PRIVILEGES for the application role
would be in effect — do not configure one.
Row locks and column grants. The store takes FOR UPDATE and FOR KEY SHARE locks on experience_records,
which PostgreSQL permits only to a role holding UPDATE on at least one column; the projection columns are that
column. The column list is also what closes the last hand-written erasure path: 0010's projection guard admits
one marked UPDATE, a live record into its tombstone, and a role that could run it by hand could erase a record's
payload while leaving every ledger row that names it. The application role cannot write deleted_at, payload,
task_id, created_at or source_run_id at all.
With the two roles, the application role — and so a bug, a careless script, or a compromised application holding its credentials:
- cannot
ALTER TABLE,DISABLE TRIGGER,DROP TRIGGER, drop a constraint, or replace or re-pin a guard function, because it owns nothing; and cannot create anything in the schema, because the verification fails onCREATEfrom any source; - cannot
UPDATE,DELETEorTRUNCATEany ledger, norDELETEorTRUNCATEa record or a grant, whatever purge marker it sets by hand — the privilege system refuses the statement (42501,permission denied for table …) before any trigger is consulted, so the marker gains it nothing; - cannot write a tombstone, or any column of a record other than the six the store moves, or of a grant other than its revocation;
- cannot call a purge function unless the host opted in — and when it did, what it gets is exactly that function's one bounded, scope-checked path.
What it cannot bind, stated precisely: the owner role and any superuser. The owner can disable or drop a trigger,
replace a function, or grant itself anything; a superuser bypasses privilege checks altogether. That is inherent in
PostgreSQL, not a gap in this schema: keep the owner's credentials out of the application and its configuration,
and treat them as the administrator credentials they are. The application role can also still INSERT a fabricated
row into a ledger — append-only is not authenticity — and anyone who can reach backups, replicas or the data
directory is out of reach entirely (see
What deletion does not reach).
A single-role deployment — the application role runs the migrator and so owns the tables — still works, and is fine for local development and tests. It gets none of the above: the application is the owner, so every guard is only as strong as its code. It is not a supported production deployment.
The two roles keep the application role away from the schema's guards. They do not stop a store statement whose own scope predicate is wrong from reading or writing another tenant's rows: that isolation rests on the predicates every statement carries. Row-level security is an optional second layer under them. With it on, PostgreSQL itself admits the application role only to the rows inside the authorization bounds of the operation it is running, so one predicate's mistake is cut back to what the host authorized.
It is off by default. Turn it on in the same privileges call, as the owner, after both migrators:
await ExperienceSchemaMigrator.ApplyApplicationRolePrivilegesAsync(
owner,
new ExperienceApplicationRoleOptions("agent_experience_app")
{
AllowErasure = true,
EnableRowLevelSecurity = true, // default false; false switches it back off
},
cancellationToken);What the call does. In the same transaction as the privileges, under the migrator's lock, it:
- checks what row-level security depends on, and refuses — changing nothing and enabling nothing — when a table's
policies are missing (run
MigrateAsyncand, with the vector channel,ExperienceVectorSchemaMigrator.MigrateAsyncfirst) or a table carries a policy the migrations did not create; whenexperience_grantsis missing or the application role may not read it; or when a default for anyagent_experience.auth_*setting is configured for the application role or the database (ALTER ROLE … SET,ALTER DATABASE … SET), which would admit rows to a statement that declared nothing; or when0025's text search function is missing; - re-creates the five helper functions and every canonical policy from the definitions the migrations were written
from, so a helper replaced or a policy altered by hand since the migration (
CREATE OR REPLACE FUNCTION …,ALTER POLICY … USING (true)) is put back before anything is enabled; - runs
ALTER TABLE … ENABLE ROW LEVEL SECURITYon every table the application role reads or writes by scope:experience_records(tombstones included),lifecycle_events,confidence_evidence,experience_grants,experience_grant_events,experience_grant_access,reuse_feedback,reuse_feedback_exposures,experience_injections,experience_holdout_assignments,experience_provider_holdout_assignments,experience_failure_signatures,experience_runs, andexperience_embeddingswhen the vectors package created it; - verifies the result like the privileges — each table enabled and not forced; each policy's name, command,
permissiveness, roles (
PUBLIC) and deparsed expressions the canonical ones; each helper's source, definition, volatility, settings and owner the canonical ones, and noneSECURITY DEFINER; the text search function exactly0025's (below); no role the application role can reach holdingBYPASSRLS— and rolls everything back on a difference.
It never uses FORCE ROW LEVEL SECURITY, and puts back a table forced by hand. With EnableRowLevelSecurity = false
the same call disables row-level security on those tables, so the setting is declarative. Enabling re-creates the
policies on every deploy, which takes a brief ACCESS EXCLUSIVE lock on each covered table; disabling alters only a
table whose state differs.
What the stores declare. Every store operation runs in a transaction and, first, declares its caller's
AuthorizationContext bounds with set_config(…, true): agent_experience.auth_tenant, auth_application,
auth_project, auth_team, auth_agent, auth_user, and the marker agent_experience.auth_set = 'on' — all seven,
every transaction, so nothing set earlier on the connection survives into an operation. A bound is = followed by
the value, or the empty string where the context leaves the field unrestricted, exactly as
AuthorizationContext.Permits reads a null bound; a setting that is unset or malformed admits nothing. The values
travel as parameters. The settings are transaction-local, so they end with the operation's transaction. The stores
declare whether or not row-level security is on: with it off nothing reads the settings, and the SQL a store sends is
what it was, inside a transaction. The grant access log, when a store audits a read, appends under the reader's own
authorization. Called directly through IExperienceGrantAccessLog.RecordAsync, which is handed none, it declares each
row's recipient scope, a null field of it unrestricted; what bounds those rows is the insert policy, which admits one
only about a live grant naming exactly its grant ID, record, owner and recipient.
Not supported with row-level security on: presetting any agent_experience.auth_* setting outside a store's own
transaction — as a connection-string option (Options=-c agent_experience.auth_set=on), a role or database default
(refused, as above), or a session-level SET on a pooled connection, especially one with No Reset On Close. The
stores do not rely on such a setting and override every one per transaction, but a statement that is not a store's —
yours — would run under it. The privileges call inspects role and database defaults only, case-insensitively; it does
not inspect server-level settings (postgresql.conf, ALTER SYSTEM) or the defaults of a separate login role that
SET ROLEs to the application role, and both are equally unsupported.
What the policies admit (0019, and 0020 for the embedding table). A row whose scope lies inside the declared
bounds. For reading only, also a record or embedding a live sharing grant shares with the declared bounds, by the
store's own grant rule: not revoked, not yet expired on clock_timestamp(), and the grant's owner columns equal to
the row's. A grant is read by its owner, and by its recipient only while it is live; it is issued and revoked by its
owner. An access row is read by either side and appended only by a reader inside the bounds, about a live grant whose
ID, record, owner and recipient are exactly the row's. Evidence rows follow their record, and exposure rows their
submission. Nothing is admitted while nothing is declared, and no policy admits a DELETE except of an embedding.
The policies are a superset of every store predicate, including the subtree sweeps' and the grant reads', so every
store statement returns exactly what it did before. The predicates stay in the SQL: this layer is defence in depth,
not a replacement.
The owner's functions apply the bounds too. purge_experience_record, purge_expired_grants,
purge_grant_access and seal_experience_record run as the owner, whom the policies do not bind, so each checks its
own scope argument first: while the calling transaction has declared bounds, a scope outside them is refused with
insufficient_privilege before anything is read. While row-level security is enabled, a caller that declared nothing
(or set the marker to anything but on) is refused as well, unless its login role is a member of the tables' owner or
holds BYPASSRLS — roles the policies do not bind either. With row-level security disabled, an undeclared caller is
unaffected, as before.
Text search keeps its index. Under the policies, @@ — the full-text match — is not leakproof, so
PostgreSQL would apply it only after the read policy and could not use the GIN indexes. So while row-level security is
enabled on experience_records and the application role may execute it, the text channel runs its search through
agent_experience.search_experience_text (0024, replaced by 0025), a SECURITY DEFINER function owned by the schema owner, and the
planner uses ix_experience_records_search and ix_experience_records_search_sealed exactly as with row-level security
off. With row-level security off the store sends its own statement, unchanged. There is no opt-out: enabling
row-level security requires the function, and the privileges call refuses to enable it without. The candidate source
detects which path applies once per data source and caches it until the next privileges call in the same process, or
for at most five minutes, so a privileges call made by another process takes effect within that time. A call that fails
because the function was revoked or dropped since falls back to the store's own statement under the policies; any
other error from inside the function is reported as a failure.
After upgrading to 0025, re-run the privileges call. 0025 makes text search match on any term and rank by
coverage (see How the text channel matches). The function gains an
argument, so the script drops 0024's and creates the new one, and dropping it drops the application role's
EXECUTE. Until ApplyApplicationRolePrivilegesAsync grants the new function, text searches with row-level security
on run the store's own statement under the policies: correct answers, without the GIN index. Deploys that already run
the privileges call after MigrateAsync need nothing more.
Its security review, recorded in full in 0024's header and restated for the replacement in 0025's:
- Why
SECURITY DEFINER. The barrier belongs to the policy, and the only roles it does not bind are the owner, a superuser and aBYPASSRLSrole. Running this one query as the owner, inside a function the application role can only call, drops the barrier for it alone; grantingBYPASSRLSor marking a built-in leakproof would drop it for every query, and neither is done. - What it trusts. The transaction-local
agent_experience.auth_*settings, and nothing else — exactly what the policies trust. A session that can run arbitrary SQL as the application role can declare any bounds and then read through the function exactly what it could read through the policies. - Why it cannot read more than the policies. It returns no row unless
agent_experience.auth_setison, and none unless the requested scope lies inside the declared bounds (rls_scope_admits,AuthorizationContext.Permits' reading of a null bound). That scope selects the permitting grant, so a grant whose ID and disclosure level it returns always has a recipient inside the bounds, onerls_grants_selectwould show the caller. Every row it returns satisfies the read policy's own admission — the same expression, from the same source, over the same helpers (rls_bound,rls_unbounded,rls_granted_keys), not a restatement of them — and then every predicate of the store's own search (scope or live grant, not a tombstone, statuses, confidence floor, the authorship exclusion withIS FALSE, the match, the ranking and the limit), built from the store's own SQL constants. It isSTABLE, only readsexperience_recordsandexperience_grants, and returns exactly the columns the store's statement selects. So its answers equal the store's own statement's with row-level security off, for the same data and authorization. - Hardened as
0013prescribes.search_pathpinned topg_catalog, pg_temp; every relation and every function of the schema schema-qualified;EXECUTErevoked fromPUBLICand granted to the application role only by this call, only while it enables row-level security. Unlike the helpers, the call never re-creates it: it verifies it —SECURITY DEFINER, PL/pgSQL,STABLE, not leakproof, thatsearch_pathalone, owned by the tables' owner, the canonical arguments, result and body byte for byte — and refuses to enable row-level security over one altered by hand. Put it back by running, as the owner, theCREATE OR REPLACE FUNCTIONstatement thatPostgresExperienceRecordSchema.GetScript(PostgresExperienceRecordSchema.TextSearchFunctionScriptName)returns. The check runs when the call enables row-level security: aCREATE OR REPLACEthe owner runs afterwards is used until the next privileges call refuses it. ASECURITY DEFINERfunction the role can execute without its opt-in — this one granted toPUBLIC, say — is refused like any other. - The grant table must stay readable. The function reads
experience_grantsas the owner, so the call requires the application role to holdSELECTon it while row-level security is on, and a grant the function shares is one the role could read itself. Revoking thatSELECTafterwards is unsupported drift, which the next privileges call refuses. - What it does not hide. It searches the table-wide GIN indexes, so a search's cost reflects how many rows match in
every tenant, not only the declared one: a weak timing and statistics side channel, part of
KL-17. A tenant-leading composite GIN index would need the
btree_ginextension, which the library does not require.
What it guards against, and what it does not. It guards against a mistake in a store's SQL: a predicate dropped, mistyped or composed wrong is cut back to the bounds the host authorized, and a write outside them fails. It does not:
- bind a compromised application role. The settings are ordinary session settings any session may set, so a host able to run arbitrary SQL as the application role can declare any tenant's bounds itself;
- enforce a grant's disclosure level. A grant admits the whole row; what of a borrowed record is shown (
LessonOnlyand the others) is decided in the library's code; - hide that a caller-chosen ID exists in another tenant. Record IDs are globally unique by design, so a create that
collides with another tenant's ID still fails as a collision (
Conflict). That is an accepted design trade: making them unique per scope would rekey every table. Finalization's own derived IDs mix in the scope, so they cannot collide across tenants; - check the record an exposure names. An exposure of an unknown record is kept, as
Unresolved, by design, so that feedback for a record erased since it was injected still records; and checking existence under row-level security would itself reveal that the record exists. The submission itself must lie inside the bounds.
Those boundaries are KL-17. The owner and superusers bypass row-level security, as they bypass every privilege.
What changes for anything else that connects. The policies apply to every role but the owner, not only to the
application role. A reporting role, or a script run as the application role, sees nothing until it declares bounds
the same way, and then only what they admit. Run cross-tenant maintenance as the owner. A hand-made role that may not
read experience_grants still reads its own scope through the stores' exact-scope fallback: the grant branch of a
policy answers "nothing is granted" when the grant table is missing or unreadable.
What it costs. About 0.7 ms per operation for the transaction and the declaration, on the benchmark machine. The
per-search cost with row-level security on was not re-measured after search_experience_text was added. Text search keeps its GIN index through search_experience_text (above); in 0.1.0-preview.6 and earlier it lost it under the policies
and scanned every live record of the declared tenant. The vector channel keeps its HNSW index. See
Row-level security for the plans and numbers measured when row-level
security shipped, before the function existed.
The role the application used to migrate as owns the database, the schema and every object in it. Keep it as the application role — its credentials do not change — and move ownership to a new owner.
Stop the application first, and run everything below through the next deploy in one maintenance window. The
moment ownership moves, the old role loses the implicit rights it had as owner: it cannot read or write a single
table until ApplyApplicationRolePrivilegesAsync grants the manifest, and an old build that still runs the migrator
with its credentials fails. As a superuser, connected to that database (the transfer also takes a brief
ACCESS EXCLUSIVE lock on each table):
CREATE ROLE agent_experience_owner LOGIN PASSWORD 'change-me';
GRANT SET ON PARAMETER agent_experience.purge_authorized, agent_experience.access_purge_authorized
TO agent_experience_owner;
DO $transfer$
DECLARE
obj record;
BEGIN
EXECUTE format('ALTER DATABASE %I OWNER TO %I', current_database(), 'agent_experience_owner');
EXECUTE format('ALTER SCHEMA agent_experience OWNER TO %I', 'agent_experience_owner');
FOR obj IN
SELECT c.oid::regclass AS name, c.relkind
FROM pg_class c
WHERE c.relnamespace = 'agent_experience'::regnamespace
AND c.relkind IN ('r', 'p', 'v', 'm', 'f', 'S')
AND NOT EXISTS (
SELECT 1 FROM pg_depend d
WHERE d.classid = 'pg_class'::regclass AND d.objid = c.oid AND d.deptype IN ('a', 'i'))
LOOP
EXECUTE format(
CASE WHEN obj.relkind = 'S' THEN 'ALTER SEQUENCE %s OWNER TO %I' ELSE 'ALTER TABLE %s OWNER TO %I' END,
obj.name, 'agent_experience_owner');
END LOOP;
FOR obj IN SELECT p.oid::regprocedure AS name FROM pg_proc p WHERE p.pronamespace = 'agent_experience'::regnamespace
LOOP
EXECUTE format('ALTER FUNCTION %s OWNER TO %I', obj.name, 'agent_experience_owner');
END LOOP;
END
$transfer$;Sequences owned by a column, and indexes, move with their table. The purge functions stay SECURITY DEFINER and now
run as the new owner; the EXECUTE that 0010 and 0012 granted to the old owner moves to the new owner with the
ownership, so the application role has none until the next step grants it. Then, still inside the window, run the
migrator with the owner's credentials and the privileges call after it, naming the old role as the application role,
and only then start the application again. From that deploy on, the migrator never runs with the application's
credentials again. The package's tests run exactly this SQL against a database migrated by its application role and
prove the old role can no longer disable a guard.