Visualize LLM outputs against datasets, manually annotate results, and run automated evaluations to algorithmically optimize prompts.
-
Updated
Nov 22, 2025 - TypeScript
#
llm-evals
Here are 35 public repositories matching this topic...
An implementation of the Anthropic's paper and essay on "A statistical approach to model evaluations"
-
Updated
Oct 6, 2025 - Python
Create an evaluation framework for your LLM based app. Incorporate it into your test suite. Lay the monitoring foundation.
workshop llm llms llmops llm-eval llm-test llm-evaluation-framework llm-evaluation-metrics llm-monitoring llm-testing llm-evals
-
Updated
May 6, 2025 - Jupyter Notebook
A framework for evaluating large language models (LLMs) across a variety of tasks.
nlp benchmark ai evaluation-framework ai-evaluation llm llm-evaluation llm-as-a-judge g-eval llm-evals
-
Updated
Mar 18, 2026 - Python
Squeeze your model with pressure prompts to see if its behavior leaks.
reliability evaluation calibration alignment quality-assurance metamorphic-testing ai-safety trustworthiness hallucinations prompt-engineering llm-eval llm-evals
-
Updated
Mar 1, 2026 - Python
Codex-native autoresearch harness with structured worker/judge turns for optimizing anything you can measure.
-
Updated
Mar 21, 2026 - Python
Detecting Relational Boundary Erosion in AI systems. A framework for testing whether models maintain honest, calibrated, and appropriate boundaries.
python yaml calibration alignment metamorphic-testing model-evaluation ai-safety red-teaming prompt-injection hallucination-detection llm-evals evaluation-harness
-
Updated
Feb 22, 2026 - Python
Evaluation patterns, release gates, and anti-hallucination techniques for developer-focused AI workflows.
-
Updated
Mar 27, 2026 - Python
In this we evaluate the LLM responses and find accuracy
-
Updated
Jul 8, 2025 - Python
Local-first LLM evaluation runner with baselines, caching, markdown reports, and CI-friendly quality, latency, and cost gates.
-
Updated
Apr 13, 2026 - Python
Agentic research pipeline with local retrieval, structured evaluation, conditional revision, and traceable outputs using Groq.
python cli retrieval evaluation ai-agents groq llm prompt-engineering research-agent agentic-workflow llm-evals
-
Updated
Mar 23, 2026 - TypeScript
Portfolio site for AI-assisted software experiments, prototypes, and polished demos.
-
Updated
Apr 19, 2026 - HTML
CLI release gate for structured AI changes.
-
Updated
Mar 12, 2026 - TypeScript
Germany-based AI/backend engineer. Shipping production quality loops, build logs, and practical engineering notes.
-
Updated
Apr 1, 2026
Measure LLM output consistency from the command line.
python cli openai developer-tools ai-evaluation llm prompt-engineering prompt-testing reliability-testing llm-evals
-
Updated
Apr 6, 2026 - Python
Tool-calling reliability runtime — 25% to 90% benchmark lift. Published as @ai-sdk-tool/parser on npm.
benchmarking typescript ai-sdk tool-calling llm-evals agentic-systems review-pack reliability-runtime review-first
-
Updated
Apr 22, 2026 - TypeScript
Horizon-Eval: evaluation-integrity framework for portable long-horizon agent benchmarks, with QA gates, trajectory auditing, replayable run bundles, and safety-gap analysis.
evaluation alignment reproducibility trajectory-analysis ai-safety safety-engineering llm-evals agent-evals benchmark-integrity
-
Updated
Mar 21, 2026 - Python
Reproducible evaluation suite for LLM behavior research: epistemic pathology, delegated introspection, and temporal consciousness diagnostics
python alignment language-models reproducibility ai-safety interpretability research-engineering ai-alignment research-tools rlhf llm-evaluation behavioral-research llm-evals epistemic-alignment temporal-consciousness epistemic-pathology
-
Updated
Nov 16, 2025 - Python
Evaluation-first research kit for biology. Structured logs, reproducible specs, and lightweight validation for agentic workflows.
-
Updated
Dec 25, 2025 - Python
Improve this page
Add a description, image, and links to the llm-evals topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-evals topic, visit your repo's landing page and select "manage topics."