feat(b2b_learner_records): serve /courses from mv_b2b_contract_courserun - #60
blarghmatey wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The generated CourseRun schema omits the contract’s int64 format and non-negative seat-limit constraint.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds the learner-records /courses endpoint backed by the contract/course-run materialized view.
Changes:
- Adds contract and course-run response modeling and queries.
- Supports contract filtering and deterministic pagination.
- Adds endpoint and schema coverage.
File summaries
| File | Description |
|---|---|
models.py |
Defines CourseRun. |
queries.py |
Queries and counts course runs. |
routers/organizations.py |
Exposes listCourses. |
tests/test_learner_records.py |
Tests filtering, ordering, serialization, and schema. |
Review details
Suppressed comments (1)
src/ol_analytics_api/tenants/b2b_learner_records/models.py:104
- The contract declares
seat_limitwithminimum: 0, but this field currently accepts negative values and omits that constraint from the generated schema. Add the non-negative bound so runtime validation and generated clients matchdocs/openapi/b2b-learner-records-v1.yaml:744-747.
seat_limit: int | None = Field(description="Null means uncapped, not zero.")
- Files reviewed: 4/4 changed files
- Comments generated: 1
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ked certificates as coded Copilot flagged CourseRun.contract_id on #60 as an unformatted integer. The contract declares it int64, and Enrollment.contract_id had the same gap, so a generated client could store it in 32 bits. certificate_is_revoked promised a revoked certificate reads passed. The derivation doesn't: a revoked certificate isn't certified, and completion_status then follows the grade. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6
…ked certificates as coded Copilot flagged CourseRun.contract_id on #60 as an unformatted integer. The contract declares it int64, and Enrollment.contract_id had the same gap, so a generated client could store it in 32 bits. certificate_is_revoked promised a revoked certificate reads passed. The derivation doesn't: a revoked certificate isn't certified, and completion_status then follows the grade. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6
* config: Bump pre-commit check versions * docs: spec the B2B learner records tenant and provider authorization The learner records design had only ever existed as untracked files, so nobody could review it or link to it. This commits the one-pager, the data-backing/tenancy doc and the draft OpenAPI contract as they stood, plus a proposal for the blocking open question: who authorizes a training provider to read an organization's learner data. The proposal separates the provider's credential (a Keycloak client, Pulumi-owned per the B2B onboarding substrate/tenant split) from the per-organization grant (a mitxonline record). The organization authorizes, MIT records and enforces it, and phase 1 lets the organization see and revoke grants before self-service authorization exists. It is marked pending sign-off. Nothing is implemented. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBTqmr3ki2pAAP1EeWH7wg * docs: record the learner-records access decision, fix the OpenAPI file Access to the M2M API is settled when the contract is signed, not mediated by mitxonline at request time, and the partner handles per-user authorization in its own LMS. That replaces the grant-record proposal: the per-contract Keycloak client is the whole record, with its organizations as a hardcoded claim and identity as the read-pii scope. The OpenAPI file was wrapped in markdown code fences, so it was not parseable YAML as committed. Stripping them makes it pass check-yaml and openapi-spec-validator. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBTqmr3ki2pAAP1EeWH7wg * docs(learner-records): type contract_id as an integer, fix stale design facts contract_id is mitxonline's ContractPage id, exposed by dim_contract as b2b_contract_id: an integer. The spec typed it as a string with a UUID example, the same mismatch #39 fixed in b2b_dashboard after production rejected input_value=695. A partner generating a client from this contract would have expected UUID strings. Also corrects three stale statements in the design doc: §4 still titled the provider-authorization question "Open" above its decision, the b2b_analytics MV count is eight rather than six, and the roster's manager column is userorganization_is_manager. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBTqmr3ki2pAAP1EeWH7wg * docs(learner-records): declare contract_id as int64 dim_contract.contract_id and the mitxonline source column b2b_contractpage.page_ptr_id are both BIGINT in the production warehouse, so the contract states the width a partner's client should use rather than leaving it to generator defaults. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBTqmr3ki2pAAP1EeWH7wg * docs(learner-records): address Copilot review of the spec Drops the read/read-pii scope split. The organization already holds its learners' identity because they are its employees or students, so redacting it protects nothing. A single per-organization export could not have honoured the split anyway. Records and exports always carry identity, and one scope remains. Pins down contract semantics the review found ambiguous: - Enrollment grain includes the contract, matching the backing model. No org/run pair in production links to more than one contract today (0 of 2468), so this adds no duplicates. - Every list endpoint has a unique order, so offset paging is stable. - /learners?contract_id= recomputes rollups from that contract's enrollments only. - Fixed-shape nullable fields are required, so omission is not a second way to say null. - The x-data-readiness claim is limited to the record schemas, and the security scheme says "per contracted integration", matching the decision doc. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBTqmr3ki2pAAP1EeWH7wg * docs(learner-records): settle the declined-learner question as out of scope The organization can see which learners declined by diffing its roster, and no response shape changes that. How it may use that knowledge is a contract term owned by legal and contracting, and issuing a client presumes those terms exist, so the spec drops the options list and keeps the per-learner record shape. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014zeAAspVVGJDPn944Gt9vz * feat(b2b_learner_records): add the learner-records tenant Serves /organizations/{id}/learners and /enrollments from the b2b_learner_records MVs to contracted partners over client credentials. It is a separate tenant because it has the opposite privacy posture from b2b_dashboard: records identify learners, so there is no k-anonymity floor, and access comes from a per-contract client's organization claim rather than an org-manager round-trip. Consent fails closed in the SQL. Every outcome column projects NULL until the upstream consent field exists, and the record models null them again so a query change can't leak one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6 * fix(b2b_learner_records): declare the OAuth2 scheme and read as_of first The tenant's OpenAPI published no security scheme, so a client generated from it would never obtain or send a token. Declare the client-credentials flow and learner-records:read on every operation. It stays documentation only, since APISIX validates the token. as_of was read after the records. A refresh landing between the queries would label old rows with the new refresh time, and a client passing that as_of as updated_since would skip the refreshed rows. Reading it first can only cause re-sends. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6 * Add field descriptions to the learner-records response schemas Learner, Enrollment and LearnerRecordsResponse now carry a Field description on every attribute, matching the descriptions already agreed in docs/openapi/b2b-learner-records-v1.yaml (falling back to a short description grounded in queries.py for fields the spec left undocumented as self-evident). FastAPI surfaces these in the OpenAPI schema the same way as the b2b_dashboard tenant, so API consumers and a documentation UI get the same self-documentation there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MGKU7sSqgn7rVhUrexWahp * docs(b2b_learner_records): say activity fields are null regardless of consent The schema descriptions read as if last_active_on, days_active and the activity counters were only consent-gated. They are hardcoded NULL in the queries until the activity fact is wired into the MVs, so a partner reading the contract would expect them to fill in once consent lands. completion_status also didn't say that not_started and in_progress come from the grade alone until then. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6 * feat(b2b_learner_records): make the consent default an environment toggle No learner consent field exists upstream yet, so the consent expression decides every record, and it was hardcoded FALSE. consent_fail_open (OL_ANALYTICS_API_B2B_LEARNER_RECORDS_CONSENT_FAIL_OPEN) now picks the literal. It defaults to false, so a deployment that never sets it still discloses nothing. When the consent field lands the expression becomes COALESCE(<field>, <literal>): a recorded decision wins, and the toggle only covers learners with none. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2joM6rRfCzWanQzAks1Yf * fix(b2b_learner_records): recompute learner rollups on #2669's definitions ol-data-platform#2669 changed mv_b2b_learner so completions and record_updated_on count every enrollment under the organization's contracts, and a learner whose seats were all reclaimed keeps a row with courses_enrolled = 0. Only courses_enrolled, the enrolled dates and `both` membership still look at active enrollments. The contract_id and include_inactive path still filtered enrollments to active before rolling up. A partner filtering by contract would then see fewer passed and certified courses than the default response for the same learner, and learners whose seats in that contract were reclaimed would disappear. It now applies the active filter only to the columns the view applies it to. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6 * fix(b2b_learner_records): publish contract_id as int64, describe revoked certificates as coded Copilot flagged CourseRun.contract_id on #60 as an unformatted integer. The contract declares it int64, and Enrollment.contract_id had the same gap, so a generated client could store it in 32 bits. certificate_is_revoked promised a revoked certificate reads passed. The derivation doesn't: a revoked certificate isn't certified, and completion_status then follows the grade. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6 * feat(b2b_dashboard): add a contract-scoped learner-progress endpoint (#59) * feat(b2b_dashboard): add a contract-scoped learner-progress endpoint MIT Learn's organization dashboard needs each learner's standing in each course run (#58). The learner-records tenant serves the same rows, but it authenticates with client credentials, which a browser can't hold. This adds GET /organizations/{org}/contracts/{contract}/learner-progress to the dashboard tenant, behind its existing org-manager and contract gates. The rows are individual learners, so they skip the anonymization floor and read mv_b2b_learner_enrollment from the b2b_learner_records database. Outcome fields are consent-gated with the same fail-closed default and consent_fail_open toggle as the learner-records tenant. Server-side search, completion_status filtering and sorting let the frontend page the table instead of fetching it whole. Activity (last_active_on, and the active / not-active split #58 asks for) stays null until learner-grain activity data exists in the warehouse. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2joM6rRfCzWanQzAks1Yf * fix(b2b_dashboard): send learner-progress timestamps as UTC StarRocks returns these as zone-less UTC strings, and the plain datetime fields sent them without an offset, so a browser would read them as local time and shift the displayed dates. The date-time fields and the envelope's as_of now carry UTC, matching the learner-records tenant. certificate_is_revoked also promised a revoked certificate reads passed. The derivation doesn't: completion_status follows the grade. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6 * Write the learner-progress descriptions for managers, not developers These descriptions can back dashboard help text, the same as #57's, and had the same problem: "Keycloak user id", "Not consent-gated", field names like outcomes_shared. They now describe each field in plain language and say which ones are hidden when a learner hasn't agreed to share their progress. The five fields that had no description now have one. The developer detail (the join key, which fields are consent-gated, how completion_status is derived, activity still pending) moves to the module docstring. A test holds every field to having a description that names no other field. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6 --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * fix(b2b_learner_records): suppress PLR0917 on _assemble's positional args Rule stabilized in ruff 0.16 (main's pinned version, via #61); this branch's own lockfile predates that. CI tests the PR merged into main, where the now-stable rule flags _assemble's 8 positional args same as PLR0913 already does. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BAEbxXmdu9jZdrmhPpUjKR --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
9dbbc8f to
b91fe9c
Compare
The contract declares seat_limit with minimum: 0, but the Pydantic field had no lower bound, so it validated a negative value and omitted the constraint from the generated schema. Flagged by Copilot review on #60. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019W22jyGzuYBwRZKk6XwujT
|
Addressed Copilot's suppressed finding on |
The spec's listCourses operation had no backing view, so #56 left it out. The new MV (ol-data-platform) includes contract runs nobody has enrolled in, which mv_b2b_learner_enrollment drops. It bypasses _assemble: the rows hold no personal data, so there is no consent projection and the withheld count is a literal 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EDJRuj6mDpGRqNbQSsXZ9o
The contract declares contract_id int64 and the column is a BIGINT. An unformatted integer lets a generated client store it in 32 bits. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6
The contract declares seat_limit with minimum: 0, but the Pydantic field had no lower bound, so it validated a negative value and omitted the constraint from the generated schema. Flagged by Copilot review on #60. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019W22jyGzuYBwRZKk6XwujT
60336cd to
2b5e3a7
Compare
What are the relevant tickets?
N/A. Serves
listCoursesfrom the spec in #55. Stacked on #56, which left/coursesout for lack of a backing view. Reads the MV added in mitodl/ol-data-platform#2684.Description (What does it do?)
Adds
GET /organizations/{organization_id}/coursesto the learner-records tenant. It takescontract_idandlimit/offset, is ordered by(contract_id, courserun_id), and readsmv_b2b_contract_courserun. The grant dependency is the router's, same as/learnersand/enrollments.queries.courses()builds its SQL without_assemble. The rows carry no personal data, so there's no consent projection, andoutcomes_withheld_countis a literal 0, which is what the envelope documents for endpoints with no consent gate.CourseRunfollows the spec schema: contract dates asdate, run start/end as UTCdate-time, and every field required, nulls included.How can this be tested?
uv run pytest: 237 passed. The new cases cover the view and ordering, the boundcontract_id, the envelope's zero withheld count, date serialization, andCourseRunin the required-field schema test. The existing client-credentials security test iterates every path, so it covers this one too.uv run ruff checkandruff format --checkon the tenant and its test file, plusuv run mypy src/ol_analytics_api/tenants/b2b_learner_records: clean.Additional Context
Measured on production data (see the data-platform PR):
contract_start_dateis null for every contract, null in the raw MITx Online extract as well, andcontract_end_dateis null for all but one. Clients will see nulls there until that's populated upstream.Checklist:
b2b_learner_records.mv_b2b_contract_courserunexists before deploying🤖 Generated with Claude Code
https://claude.ai/code/session_01EDJRuj6mDpGRqNbQSsXZ9o