Open
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Pull request overview
Adds a new Subject Matter Knowledge (SMK) evaluator to the TypeScript SDK and wires it into the existing composite TextComplexityEvaluator, along with docs and tests.
Changes:
- Introduces
SmkEvaluator(Google Gemini) with prompt + Zod schema and functional API export. - Extends
TextComplexityEvaluatorto run vocabulary, sentence structure, and SMK in parallel and returnsubjectMatterKnowledge. - Adds unit + integration tests and updates SDK README + changelogs.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| sdks/typescript/tests/unit/evaluators/text-complexity.test.ts | Updates unit tests to include SMK sub-evaluator behavior and 3-way parallelism expectations. |
| sdks/typescript/tests/unit/evaluators/smk.test.ts | Adds unit tests for SMK evaluator validation, metadata, mapping, and error propagation. |
| sdks/typescript/tests/integration/smk.integration.test.ts | Adds integration coverage for SMK across grades 3–12 using dataset-derived cases. |
| sdks/typescript/src/schemas/smk.ts | Defines the structured SMK response schema and SmkInternal type. |
| sdks/typescript/src/prompts/subject-matter-knowledge/index.ts | Adds prompt getters that inline shared prompt text assets. |
| sdks/typescript/src/index.ts | Exposes SMK evaluator and internal types from the package entrypoint. |
| sdks/typescript/src/evaluators/text-complexity.ts | Adds SMK as a third sub-evaluator and includes it in the returned result map. |
| sdks/typescript/src/evaluators/smk.ts | Implements the SMK evaluator and functional evaluateSmk API. |
| sdks/typescript/src/evaluators/index.ts | Re-exports SMK evaluator and functional API from the evaluators barrel. |
| sdks/typescript/README.md | Documents the new SMK evaluator and updates Text Complexity docs to include SMK. |
| sdks/typescript/CHANGELOG.md | Adds a release entry describing SMK and composite changes. |
| evals/prompts/subject-matter-knowledge/user.txt | Adds the SMK user prompt template. |
| evals/prompts/subject-matter-knowledge/system.txt | Adds the SMK system prompt (rubric + output contract). |
| evals/prompts/CHANGELOG.md | Records the new SMK prompt addition. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| - curriculum_check: String explaining if the topics are "Standard/General" (typical for K-8) or "Specialized/High School" (typical for 9-12). | ||
| - assumptions_and_scaffolding: String analyzing what the author assumes the reader knows vs what is explained. | ||
| - friction_analysis: String discussing the gap between Concrete description and Abstract meaning. | ||
| - complexity_score: String (One of: slightly_complex, moderately_complex, very_complex, exceedingly_complex). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements the Subject Matter Knowledge evaluator to the TypeScript SDK
Documentation
See
sdks/typescript/README.mdTesting