Skip to content

Add Decision Models as Guardrails to Benchmarks & research - #169

Open
agairola wants to merge 1 commit into
hellogumbo:mainfrom
agairola:add-dm-as-guardrails
Open

agairola wants to merge 1 commit into
hellogumbo:mainfrom
agairola:add-dm-as-guardrails

Conversation

@agairola

@agairola agairola commented Oct 20, 2026 •

Copy link
Copy Markdown

Project: https://github.com/raxITlabs/decision-models-as-guardrails (leaderboard: https://dm-as-guardrails.raxitlabs.com, dataset: https://huggingface.co/datasets/raxITLabs/decision-models-as-guardrails)

Where Jev fits: Jev 1.13.0 and 1.14.0 are two of the 19 systems scored on 9,975 labelled guardrail checks; each check goes to Jev as a typed Noul question and code blocks at a probability of 0.5 or more (benchmark/goldrails_bench/systemone.py).

  • I added one object to data/projects.json and ran npm run validate
  • I did not commit README.md or site/index.html (CI regenerates them)
  • I am the author, or I linked where the project was announced

I work at raxIT Labs, which built and maintains the benchmark.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Summary by CodeRabbit

  • New Features
    • Added the “Decision Models as Guardrails” research project to the projects catalog, including its description, links, category, language, star count, and date added.

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1c3e66dd-6a85-4f28-8fa8-a7ca7758b9df
📥 Commits

Reviewing files that changed from the base of the PR and between 2e4f338 and ba9d36c.

📒 Files selected for processing: 1
  • data/projects.json

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

📝 Walkthrough

The projects catalog adds the Decision Models as Guardrails research project. Its entry includes repository and site details, a description, category, language, star count, and added date.

Changes: Project catalog entry

Layer / File(s) Summary
Add project catalog entry
data/projects.json
The catalog gains an entry for raxITlabs/decision-models-as-guardrails with project metadata and a description of its guardrail comparison scope.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Feature

Suggested reviewers: scald

Merge Risk: ⚪ Minimal · up to ba9d3

This catalog-only change introduces no identified merge-blocking risk.

Pre-merge checks: Passed 5
Check name Status Explanation
Description check Passed The description includes the required Project and Where Jev fits sections. It confirms the data file change, validation command, excluded generated files, and author or announcement requirement.
Title check Passed The title clearly identifies the main change: adding Decision Models as Guardrails to the benchmarks and research catalog.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant