Skip to content

feat: Add SMK evaluator to TypeScript SDK#21

Open
adnanrhussain wants to merge 2 commits intomainfrom
ahussain/typescript_sdk_add_smk
Open

feat: Add SMK evaluator to TypeScript SDK#21
adnanrhussain wants to merge 2 commits intomainfrom
ahussain/typescript_sdk_add_smk

Conversation

@adnanrhussain
Copy link
Contributor

Summary

This PR implements the Subject Matter Knowledge evaluator to the TypeScript SDK

Documentation

See sdks/typescript/README.md

Testing

  • CI workflow should build successfully

@codecov
Copy link

codecov bot commented Mar 16, 2026

Codecov Report

❌ Patch coverage is 87.50000% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
sdks/typescript/src/evaluators/smk.ts 84.09% 7 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Subject Matter Knowledge (SMK) evaluator to the TypeScript SDK and wires it into the existing composite TextComplexityEvaluator, along with docs and tests.

Changes:

  • Introduces SmkEvaluator (Google Gemini) with prompt + Zod schema and functional API export.
  • Extends TextComplexityEvaluator to run vocabulary, sentence structure, and SMK in parallel and return subjectMatterKnowledge.
  • Adds unit + integration tests and updates SDK README + changelogs.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
sdks/typescript/tests/unit/evaluators/text-complexity.test.ts Updates unit tests to include SMK sub-evaluator behavior and 3-way parallelism expectations.
sdks/typescript/tests/unit/evaluators/smk.test.ts Adds unit tests for SMK evaluator validation, metadata, mapping, and error propagation.
sdks/typescript/tests/integration/smk.integration.test.ts Adds integration coverage for SMK across grades 3–12 using dataset-derived cases.
sdks/typescript/src/schemas/smk.ts Defines the structured SMK response schema and SmkInternal type.
sdks/typescript/src/prompts/subject-matter-knowledge/index.ts Adds prompt getters that inline shared prompt text assets.
sdks/typescript/src/index.ts Exposes SMK evaluator and internal types from the package entrypoint.
sdks/typescript/src/evaluators/text-complexity.ts Adds SMK as a third sub-evaluator and includes it in the returned result map.
sdks/typescript/src/evaluators/smk.ts Implements the SMK evaluator and functional evaluateSmk API.
sdks/typescript/src/evaluators/index.ts Re-exports SMK evaluator and functional API from the evaluators barrel.
sdks/typescript/README.md Documents the new SMK evaluator and updates Text Complexity docs to include SMK.
sdks/typescript/CHANGELOG.md Adds a release entry describing SMK and composite changes.
evals/prompts/subject-matter-knowledge/user.txt Adds the SMK user prompt template.
evals/prompts/subject-matter-knowledge/system.txt Adds the SMK system prompt (rubric + output contract).
evals/prompts/CHANGELOG.md Records the new SMK prompt addition.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

- curriculum_check: String explaining if the topics are "Standard/General" (typical for K-8) or "Specialized/High School" (typical for 9-12).
- assumptions_and_scaffolding: String analyzing what the author assumes the reader knows vs what is explained.
- friction_analysis: String discussing the gap between Concrete description and Abstract meaning.
- complexity_score: String (One of: slightly_complex, moderately_complex, very_complex, exceedingly_complex).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants