Skip to content

Publish AI logging settings from the API, with plan defaults and readback - #815

Closed
Brooooooklyn wants to merge 2 commits into
ai-logging/proxy-kvfrom
ai-logging/api-publish
Closed

Brooooooklyn wants to merge 2 commits into
ai-logging/proxy-kvfrom
ai-logging/api-publish

Conversation

@Brooooooklyn

@Brooooooklyn Brooooooklyn commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

The API is now the writer of the ROUTING KV key the proxy reads. It no
longer needs PLATFORM_AUTH_DATABASE_UNCACHED for AI logging.

  • publishProjectAiLogging locks the project row, then reads every input
    through a write statement (so cached Hyperdrive reads never feed a
    publish), and writes KV before the lock is released. A later publisher
    always writes last.
  • The operator POST orders the write by direction: a tighten writes in the
    same transaction and rolls back on a KV failure; a loosen, or any change
    with inherit, commits first and publishes in a second locked step.
    Failures are ai_logging_publish_failed / ai_logging_publish_pending.
  • Plan defaults live in platform_settings 'plans:ai-logging', edited
    through the plan routes and dashboard, and published in pages of 100.
    A user plan change publishes the account's projects one page per call;
    transfer, project create and delete keep the key in step.
  • Readback (project show, GET .../ai-logging, dashboard) returns the saved
    mode plus plan, effective mode, source, published value and a state:
    applied, pending, pending_plan_publish, drift or unknown.
  • An hourly sweep repairs drifted keys within a 400 KV operation budget.
  • The internal route 1.8.x proxies call is kept and answers from KV.

Refs #793.


Stack created with GitHub Stacks CLI • Give Feedback 💬

Stack

Redesign of #793 so AI works again on hosted Void Cloud. Merge bottom to top:

  1. Make AI work again on hosted Void Cloud: publish AI logging policy to KV #814 proxy reads ROUTING KV (a missing key means inherit). Merge with the rest of the stack (gh stack merge 817), not alone: Publish AI logging settings from the API, with plan defaults and readback #815 is what writes the key and Publish AI logging settings from the installer and show them in the CLI #816 copies saved settings on upgrade.
  2. Publish AI logging settings from the API, with plan defaults and readback #815 API publishing, plan defaults, readback, hourly sweep
  3. Publish AI logging settings from the installer and show them in the CLI #816 installer projection and CLI output
  4. Document AI Gateway logging policies, plan defaults and their readback #817 docs and design doc 0094 (platform/meta/design-docs/0094-ai-gateway-logging-policy.md)

Verification

  • Each layer: vp run check plus its package tests.
  • Stack top: vp test run — 804 files, 17,022 tests passed, 35 skipped.
  • The two PostgreSQL-only lock-wait tests (VOID_TEST_DATABASE_URL) run in CI.
  • Not run: live checks A/B (platform/scripts/ai-logging-live-check.ts) need a scratch Cloudflare account; see 0094.

🤖 Generated with Claude Code

Brooooooklyn and others added 2 commits November 13, 2026 20:59
Since 1.8.0 the proxy asked the API for each project's AI logging policy
on every AI call, and the API refused unless PLATFORM_AUTH_DATABASE_UNCACHED
was set. Hosted Void Cloud never sets it, so every hosted AI call failed
with 503 ai_logging_policy_unavailable.

The proxy now reads `ai-logging:<projectId>` from ROUTING KV (cacheTtl 30)
in /ai/run, /ai/provider and /ai/toMarkdown, after body validation:

- missing key: inherit (send no logging option, as in 1.7.0)
- off / metadata / full: apply the 1.8.0 gateway mapping, kept in one place
- KV error: 503 ai_logging_policy_unavailable with Retry-After, logged
- any other value: 503 ai_logging_policy_invalid, logged without the value

Caller-supplied logging keys are still stripped. /ai/models, unknown
paths and 400 bodies never read the key.

The modes, key names, parsers and resolver move to a new shared subpath,
@void/platform/ai-logging, used by the proxy, the API and the installer.
Adds a manual live-check script for the two Cloudflare behaviours the
binding mapping depends on.

Refs #793.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…back

The API is now the writer of the ROUTING KV key the proxy reads. It no
longer needs PLATFORM_AUTH_DATABASE_UNCACHED for AI logging.

- publishProjectAiLogging locks the project row, then reads every input
  through a write statement (so cached Hyperdrive reads never feed a
  publish), and writes KV before the lock is released. A later publisher
  always writes last.
- The operator POST orders the write by direction: a tighten writes in the
  same transaction and rolls back on a KV failure; a loosen, or any change
  with inherit, commits first and publishes in a second locked step.
  Failures are ai_logging_publish_failed / ai_logging_publish_pending.
- Plan defaults live in platform_settings 'plans:ai-logging', edited
  through the plan routes and dashboard, and published in pages of 100.
  A user plan change publishes the account's projects one page per call;
  transfer, project create and delete keep the key in step.
- Readback (project show, GET .../ai-logging, dashboard) returns the saved
  mode plus plan, effective mode, source, published value and a state:
  applied, pending, pending_plan_publish, drift or unknown.
- An hourly sweep repairs drifted keys within a 400 KV operation budget.
- The internal route 1.8.x proxies call is kept and answers from KV.

Refs #793.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Brooooooklyn
Brooooooklyn requested a review from a team as a code owner November 13, 2026 13:10
@Brooooooklyn
Brooooooklyn added this pull request to stack #818 November 13, 2026 13:10
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-13T13:14:02.356411Z 44d6cab PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 44d6cab2da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

return policy ? context.json(policy) : context.json({ error: 'project not found' }, 404);
let mode;
try {
mode = await readPublishedAiLogging(context.env.ROUTING, projectId);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve saved overrides when switching the legacy route to KV

When upgrading an installation with an existing project_ai_logging override, the old API has saved that override only in Postgres. Switching this legacy endpoint directly to KV therefore returns inherit until the hourly sweep publishes the missing key. A still-supported 1.8.x proxy consequently stops honoring an existing off or metadata policy and can log prompts and responses under the gateway default. A repository-wide search found no pre-cutover backfill of these rows; the new sweep repairs them only after deployment. Keep the authoritative legacy read until projection is guaranteed, or add a verified backfill before changing readers.

AGENTS.md reference: platform/AGENTS.md:L108-L109

Useful? React with 👍 / 👎.

Comment on lines +114 to +116
return (
previous !== next || (moved && modeFor(modesBefore, before, input.moved!.from) !== next)
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Transfer unfinished publish coverage to a removed plan's replacement

If team and pro both have an off default, but team still has an unfinished publish, removing team with replacement pro excludes pro here because the saved modes are equal. startPlanAiLoggingPublish then preserves only the pending team ID, while the removal moves all its accounts to pro. The next publish page finds no candidates and marks the run complete, leaving previously unpublished projects inheriting the gateway setting until hourly drift repair. Remap unfinished coverage from the removed plan to its replacement even when their saved defaults match.

Useful? React with 👍 / 👎.

context.env.ROUTING,
initialPlanDefault(context),
);
return { ...accounts, aiLoggingPublish };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update CLI completion checks for the separate AI logging publish

When an AI logging publish needs more than one page, this response can report reconciliation.complete: true while aiLoggingPublish.complete is false. The existing CLI's apply() in packages/void/src/cli/platform-plans-config.ts:321–350 checks only account reconciliation, so plans sync exits successfully after one page and also ignores AI logging publish errors. This affects, for example, changing the default plan when more than 100 projects belong to accounts on unknown plans. The documented command continues updates “until completion” (docs/guide/platform/administration/plans.md, Complete Pending Updates), but now leaves this work to the hourly sweep without warning. Update the CLI loop, progress detection, and error handling to inspect both results.

Useful? React with 👍 / 👎.

@Brooooooklyn
Brooooooklyn removed this pull request from stack #818 November 13, 2026 13:22
@Brooooooklyn

Copy link
Copy Markdown
Member Author

Merged into #814 as a single PR.

@Brooooooklyn
Brooooooklyn deleted the ai-logging/api-publish branch November 13, 2026 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant