Conversation
Add TypeSafe Jev as an optional backend for the tool_call classifier. The default provider stays "pi", so behavior does not change without configuration. Jev needs Oh My Pi 18.2.4 or newer and a TypeSafe credential. If either is absent, auto mode runs the existing classifier and reports why. - Ask one typed question per policy rule. Combine answers in code. - Escalate an unresolved review band to structured review. - Apply jev.onFailure after the gate: fall back, or fail closed. - Show the gate state in the status line and /automode status. Signed-off-by: Jauder Ho <jauderho@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
A proposal, opened for feedback rather than as a finished change.
It adds TypeSafe's Jev as an additional backend for the
tool_callclassifier. Jev answers typed questions with calibrated probabilities instead of emitting decision JSON, so the allow/block decision moves out of prompt parsing and into code.Nothing changes without configuration:
autoMode.classifierProviderdefaults to"pi"and that path is untouched.How it works
environment, loaded project instructions, and the classifier transcript. Rule lists stay out.hard_deny/soft_deny/allowrule, plus an explicit-authorization Noul and a severity Score. Jev evaluates them in parallel in one call, so the fan-out is close to free.CLASSIFIER_SYSTEM_PROMPTsemantics. Probabilities in a[reviewThreshold, threshold)band are unresolved and escalate to structured review, or block in"jev"mode.Only steps 13–14 of the enforcement order change. Permission rules, the deterministic hard-deny checks, path gates, and protected paths all still run first and are untouched.
Gating
Jev is inert unless three conditions hold: OMP ≥ 18.2.4 exports
TypeSafeJudge, atypesafecredential resolves, and config opts in. A miss runs the existing classifier and reports why through/automode configand the status line — it never blocks. Stock Pi is unaffected; the runtime check is a feature detect, so a future Pi judgment API needs one branch.Auth, transport, retries, and credential rotation are all OMP's (
TypeSafeJudge+ctx.modelRegistry.resolver("typesafe", …)). No new runtime dependency, and this extension never sees the key.Points I'd particularly like your view on
resolveJudge(). It falls back to thetiny/smolchat chain on failure, which would silently demote the safety classifier to an unvalidated model. Insteadjev.onFailurechooses between the user's own configured classifier and failing closed. Reasonable, or too clever?"auto"vs"jev-prefilter"are currently synonyms."auto"exists to express "the credential is the switch", but it may just be dead surface.docs/configuration.md) are principled but not measured against real traffic. They are documented as starting points. Happy to hold the defaults behind a stronger warning if you'd rather.jev-1.13does not treat state as hostile. That is why"jev-prefilter"is the recommended mode and why the caveat sits next to the standalone mode in the docs rather than being glossed.Notes
tests/jev.test.ts;tsc --noEmitclean.npm testlocally (nonodeon this machine) — the suite passes underbun'snode:testrunner, 324/326, with the one pre-existingpermissions.test.tsbackslash failure that also fails on a clean checkout there. CI is the real check.Happy to split this up, change defaults, or drop parts of it. Feedback welcome.