Skip to content

feat(classifier): add TypeSafe Jev classifier backend - #49

Open
FazalAAli wants to merge 1 commit into
czottmann:mainfrom
FazalAAli:feat/jev-classifier
Open

FazalAAli wants to merge 1 commit into
czottmann:mainfrom
FazalAAli:feat/jev-classifier

Conversation

@FazalAAli

Copy link
Copy Markdown

Summary

Adds TypeSafe Jev as a classifier backend. Setting classifierModel to typesafe/jev-latest classifies each action with one System One call instead of the two-stage LLM flow.

  • The configured hard_deny and soft_deny rules become the options of a Choice question, so custom rules work without changes.
  • Two Noul questions check for an ALLOW exception and direct user authorization.
  • The decision is made locally from the probabilities, with one tolerance of 0.2:
    • allow when P(no deny rule) ≥ 0.8
    • hard_deny when a hard-deny rule is the top match or hard-deny rules together reach ≥ 0.2; exceptions never override it
    • otherwise the top soft-deny rule blocks unless the exception or authorization probability is ≥ 0.8
  • The denial reason names the matched rule.
  • Missing TYPESAFE_API_KEY, HTTP errors, timeouts (classifierTimeoutMs) and malformed answers fail closed.
  • /automode model typesafe/jev-latest skips the Pi model registry and checks the API key instead.
  • Uses plain fetch; no new dependency.

Testing

  • npm run typecheck passes.
  • New tests/jev.test.ts covers the decision mapping and the HTTP call (request shape, auth header, HTTP error, missing key).
  • npm test: 2 failures in tests/hook-path-policy.test.ts, which fail the same way on main without this change.
  • Not yet tested against the live TypeSafe API.

Not included

  • The 0.2 tolerance is a constant, not a config setting.
  • Jev does not appear in the /automode model picker.

🤖 Generated with Claude Code

Setting classifierModel to typesafe/<model> classifies each action with
one System One call instead of the two-stage LLM flow. The configured
hard_deny and soft_deny rules become the options of a Choice question;
Noul questions cover ALLOW exceptions and direct user authorization.
Decisions are made locally from the returned probabilities, and request
errors, missing keys, and malformed answers fail closed.
@FazalAAli

Copy link
Copy Markdown
Author

Hi @czottmann ! Would love to get your feedback on this or #48 . I'd love to see this feature merged in

openrouter has also started offering jev from their api so if you'd like that implementation instead, I'd be happy to look into it.

Thanks for this awesome project!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant