Skip to content

feat(b2b_learner_records): serve /courses from mv_b2b_contract_courserun - #60

Open
blarghmatey wants to merge 3 commits into
mainfrom
feat/b2b-learner-records-courses
Open

blarghmatey wants to merge 3 commits into
mainfrom
feat/b2b-learner-records-courses

Conversation

@blarghmatey

Copy link
Copy Markdown
Member

What are the relevant tickets?

N/A. Serves listCourses from the spec in #55. Stacked on #56, which left /courses out for lack of a backing view. Reads the MV added in mitodl/ol-data-platform#2684.

Description (What does it do?)

Adds GET /organizations/{organization_id}/courses to the learner-records tenant. It takes contract_id and limit/offset, is ordered by (contract_id, courserun_id), and reads mv_b2b_contract_courserun. The grant dependency is the router's, same as /learners and /enrollments.

queries.courses() builds its SQL without _assemble. The rows carry no personal data, so there's no consent projection, and outcomes_withheld_count is a literal 0, which is what the envelope documents for endpoints with no consent gate. CourseRun follows the spec schema: contract dates as date, run start/end as UTC date-time, and every field required, nulls included.

How can this be tested?

  • uv run pytest: 237 passed. The new cases cover the view and ordering, the bound contract_id, the envelope's zero withheld count, date serialization, and CourseRun in the required-field schema test. The existing client-credentials security test iterates every path, so it covers this one too.
  • uv run ruff check and ruff format --check on the tenant and its test file, plus uv run mypy src/ol_analytics_api/tenants/b2b_learner_records: clean.
  • The SQL hasn't been run on StarRocks; the MV doesn't exist until the data-platform PR merges.

Additional Context

Measured on production data (see the data-platform PR): contract_start_date is null for every contract, null in the raw MITx Online extract as well, and contract_end_date is null for all but one. Clients will see nulls there until that's populated upstream.

Checklist:

🤖 Generated with Claude Code

https://claude.ai/code/session_01EDJRuj6mDpGRqNbQSsXZ9o

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The generated CourseRun schema omits the contract’s int64 format and non-negative seat-limit constraint.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds the learner-records /courses endpoint backed by the contract/course-run materialized view.

Changes:

  • Adds contract and course-run response modeling and queries.
  • Supports contract filtering and deterministic pagination.
  • Adds endpoint and schema coverage.
File summaries
File Description
models.py Defines CourseRun.
queries.py Queries and counts course runs.
routers/organizations.py Exposes listCourses.
tests/test_learner_records.py Tests filtering, ordering, serialization, and schema.
Review details

Suppressed comments (1)

src/ol_analytics_api/tenants/b2b_learner_records/models.py:104

  • The contract declares seat_limit with minimum: 0, but this field currently accepts negative values and omits that constraint from the generated schema. Add the non-negative bound so runtime validation and generated clients match docs/openapi/b2b-learner-records-v1.yaml:744-747.
    seat_limit: int | None = Field(description="Null means uncapped, not zero.")
  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/ol_analytics_api/tenants/b2b_learner_records/models.py Outdated
blarghmatey added a commit that referenced this pull request Sep 15, 2026
…ked certificates as coded

Copilot flagged CourseRun.contract_id on #60 as an unformatted integer. The
contract declares it int64, and Enrollment.contract_id had the same gap, so a
generated client could store it in 32 bits.

certificate_is_revoked promised a revoked certificate reads passed. The
derivation doesn't: a revoked certificate isn't certified, and
completion_status then follows the grade.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6
blarghmatey added a commit that referenced this pull request Sep 17, 2026
…ked certificates as coded

Copilot flagged CourseRun.contract_id on #60 as an unformatted integer. The
contract declares it int64, and Enrollment.contract_id had the same gap, so a
generated client could store it in 32 bits.

certificate_is_revoked promised a revoked certificate reads passed. The
derivation doesn't: a revoked certificate isn't certified, and
completion_status then follows the grade.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6
blarghmatey added a commit that referenced this pull request Sep 17, 2026
* config: Bump pre-commit check versions

* docs: spec the B2B learner records tenant and provider authorization

The learner records design had only ever existed as untracked files, so
nobody could review it or link to it. This commits the one-pager, the
data-backing/tenancy doc and the draft OpenAPI contract as they stood,
plus a proposal for the blocking open question: who authorizes a
training provider to read an organization's learner data.

The proposal separates the provider's credential (a Keycloak client,
Pulumi-owned per the B2B onboarding substrate/tenant split) from the
per-organization grant (a mitxonline record). The organization
authorizes, MIT records and enforces it, and phase 1 lets the
organization see and revoke grants before self-service authorization
exists. It is marked pending sign-off. Nothing is implemented.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EBTqmr3ki2pAAP1EeWH7wg

* docs: record the learner-records access decision, fix the OpenAPI file

Access to the M2M API is settled when the contract is signed, not
mediated by mitxonline at request time, and the partner handles
per-user authorization in its own LMS. That replaces the grant-record
proposal: the per-contract Keycloak client is the whole record, with
its organizations as a hardcoded claim and identity as the read-pii
scope.

The OpenAPI file was wrapped in markdown code fences, so it was not
parseable YAML as committed. Stripping them makes it pass check-yaml
and openapi-spec-validator.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EBTqmr3ki2pAAP1EeWH7wg

* docs(learner-records): type contract_id as an integer, fix stale design facts

contract_id is mitxonline's ContractPage id, exposed by dim_contract as
b2b_contract_id: an integer. The spec typed it as a string with a UUID
example, the same mismatch #39 fixed in b2b_dashboard after production
rejected input_value=695. A partner generating a client from this
contract would have expected UUID strings.

Also corrects three stale statements in the design doc: §4 still titled
the provider-authorization question "Open" above its decision, the
b2b_analytics MV count is eight rather than six, and the roster's
manager column is userorganization_is_manager.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EBTqmr3ki2pAAP1EeWH7wg

* docs(learner-records): declare contract_id as int64

dim_contract.contract_id and the mitxonline source column
b2b_contractpage.page_ptr_id are both BIGINT in the production
warehouse, so the contract states the width a partner's client should
use rather than leaving it to generator defaults.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EBTqmr3ki2pAAP1EeWH7wg

* docs(learner-records): address Copilot review of the spec

Drops the read/read-pii scope split. The organization already holds its
learners' identity because they are its employees or students, so
redacting it protects nothing. A single per-organization export could
not have honoured the split anyway. Records and exports always carry
identity, and one scope remains.

Pins down contract semantics the review found ambiguous:
- Enrollment grain includes the contract, matching the backing model.
  No org/run pair in production links to more than one contract today
  (0 of 2468), so this adds no duplicates.
- Every list endpoint has a unique order, so offset paging is stable.
- /learners?contract_id= recomputes rollups from that contract's
  enrollments only.
- Fixed-shape nullable fields are required, so omission is not a
  second way to say null.
- The x-data-readiness claim is limited to the record schemas, and the
  security scheme says "per contracted integration", matching the
  decision doc.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EBTqmr3ki2pAAP1EeWH7wg

* docs(learner-records): settle the declined-learner question as out of scope

The organization can see which learners declined by diffing its roster,
and no response shape changes that. How it may use that knowledge is a
contract term owned by legal and contracting, and issuing a client
presumes those terms exist, so the spec drops the options list and keeps
the per-learner record shape.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014zeAAspVVGJDPn944Gt9vz

* feat(b2b_learner_records): add the learner-records tenant

Serves /organizations/{id}/learners and /enrollments from the
b2b_learner_records MVs to contracted partners over client credentials.
It is a separate tenant because it has the opposite privacy posture from
b2b_dashboard: records identify learners, so there is no k-anonymity floor,
and access comes from a per-contract client's organization claim rather
than an org-manager round-trip.

Consent fails closed in the SQL. Every outcome column projects NULL until
the upstream consent field exists, and the record models null them again
so a query change can't leak one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6

* fix(b2b_learner_records): declare the OAuth2 scheme and read as_of first

The tenant's OpenAPI published no security scheme, so a client generated
from it would never obtain or send a token. Declare the client-credentials
flow and learner-records:read on every operation. It stays documentation
only, since APISIX validates the token.

as_of was read after the records. A refresh landing between the queries
would label old rows with the new refresh time, and a client passing that
as_of as updated_since would skip the refreshed rows. Reading it first can
only cause re-sends.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6

* Add field descriptions to the learner-records response schemas

Learner, Enrollment and LearnerRecordsResponse now carry a Field
description on every attribute, matching the descriptions already
agreed in docs/openapi/b2b-learner-records-v1.yaml (falling back to a
short description grounded in queries.py for fields the spec left
undocumented as self-evident). FastAPI surfaces these in the OpenAPI
schema the same way as the b2b_dashboard tenant, so API consumers and
a documentation UI get the same self-documentation there.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MGKU7sSqgn7rVhUrexWahp

* docs(b2b_learner_records): say activity fields are null regardless of consent

The schema descriptions read as if last_active_on, days_active and the
activity counters were only consent-gated. They are hardcoded NULL in the
queries until the activity fact is wired into the MVs, so a partner reading
the contract would expect them to fill in once consent lands. completion_status
also didn't say that not_started and in_progress come from the grade alone
until then.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6

* feat(b2b_learner_records): make the consent default an environment toggle

No learner consent field exists upstream yet, so the consent expression
decides every record, and it was hardcoded FALSE. consent_fail_open
(OL_ANALYTICS_API_B2B_LEARNER_RECORDS_CONSENT_FAIL_OPEN) now picks the
literal. It defaults to false, so a deployment that never sets it still
discloses nothing. When the consent field lands the expression becomes
COALESCE(<field>, <literal>): a recorded decision wins, and the toggle only
covers learners with none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2joM6rRfCzWanQzAks1Yf

* fix(b2b_learner_records): recompute learner rollups on #2669's definitions

ol-data-platform#2669 changed mv_b2b_learner so completions and
record_updated_on count every enrollment under the organization's
contracts, and a learner whose seats were all reclaimed keeps a row with
courses_enrolled = 0. Only courses_enrolled, the enrolled dates and `both`
membership still look at active enrollments.

The contract_id and include_inactive path still filtered enrollments to
active before rolling up. A partner filtering by contract would then see
fewer passed and certified courses than the default response for the same
learner, and learners whose seats in that contract were reclaimed would
disappear. It now applies the active filter only to the columns the view
applies it to.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6

* fix(b2b_learner_records): publish contract_id as int64, describe revoked certificates as coded

Copilot flagged CourseRun.contract_id on #60 as an unformatted integer. The
contract declares it int64, and Enrollment.contract_id had the same gap, so a
generated client could store it in 32 bits.

certificate_is_revoked promised a revoked certificate reads passed. The
derivation doesn't: a revoked certificate isn't certified, and
completion_status then follows the grade.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6

* feat(b2b_dashboard): add a contract-scoped learner-progress endpoint (#59)

* feat(b2b_dashboard): add a contract-scoped learner-progress endpoint

MIT Learn's organization dashboard needs each learner's standing in each
course run (#58). The learner-records tenant serves the same rows, but it
authenticates with client credentials, which a browser can't hold. This adds
GET /organizations/{org}/contracts/{contract}/learner-progress to the
dashboard tenant, behind its existing org-manager and contract gates.

The rows are individual learners, so they skip the anonymization floor and
read mv_b2b_learner_enrollment from the b2b_learner_records database. Outcome
fields are consent-gated with the same fail-closed default and
consent_fail_open toggle as the learner-records tenant. Server-side search,
completion_status filtering and sorting let the frontend page the table
instead of fetching it whole.

Activity (last_active_on, and the active / not-active split #58 asks for)
stays null until learner-grain activity data exists in the warehouse.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2joM6rRfCzWanQzAks1Yf

* fix(b2b_dashboard): send learner-progress timestamps as UTC

StarRocks returns these as zone-less UTC strings, and the plain datetime
fields sent them without an offset, so a browser would read them as local
time and shift the displayed dates. The date-time fields and the envelope's
as_of now carry UTC, matching the learner-records tenant.

certificate_is_revoked also promised a revoked certificate reads passed. The
derivation doesn't: completion_status follows the grade.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6

* Write the learner-progress descriptions for managers, not developers

These descriptions can back dashboard help text, the same as #57's, and had
the same problem: "Keycloak user id", "Not consent-gated", field names like
outcomes_shared. They now describe each field in plain language and say which
ones are hidden when a learner hasn't agreed to share their progress. The
five fields that had no description now have one.

The developer detail (the join key, which fields are consent-gated, how
completion_status is derived, activity still pending) moves to the module
docstring. A test holds every field to having a description that names no
other field.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

* fix(b2b_learner_records): suppress PLR0917 on _assemble's positional args

Rule stabilized in ruff 0.16 (main's pinned version, via #61); this
branch's own lockfile predates that. CI tests the PR merged into
main, where the now-stable rule flags _assemble's 8 positional args
same as PLR0913 already does.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BAEbxXmdu9jZdrmhPpUjKR

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Base automatically changed from feat/b2b-learner-records-tenant to main September 17, 2026 14:32
@blarghmatey
blarghmatey added this pull request to stack #63 September 17, 2026 14:32
@blarghmatey
blarghmatey force-pushed the feat/b2b-learner-records-courses branch from 9dbbc8f to b91fe9c Compare September 17, 2026 14:51
blarghmatey added a commit that referenced this pull request Sep 17, 2026
The contract declares seat_limit with minimum: 0, but the Pydantic field had
no lower bound, so it validated a negative value and omitted the constraint
from the generated schema. Flagged by Copilot review on #60.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019W22jyGzuYBwRZKk6XwujT
@blarghmatey

Copy link
Copy Markdown
Member Author

Addressed Copilot's suppressed finding on models.py:104 (not posted as an inline thread): fixed in 60336cd, CourseRun.seat_limit now enforces ge=0 to match the contract's minimum: 0. The int64-format thread (PRRT_kwDOTPlRi86iq6F2) was already resolved and stays that way.

blarghmatey and others added 3 commits September 17, 2026 15:56
The spec's listCourses operation had no backing view, so #56 left it out.
The new MV (ol-data-platform) includes contract runs nobody has enrolled
in, which mv_b2b_learner_enrollment drops.

It bypasses _assemble: the rows hold no personal data, so there is no
consent projection and the withheld count is a literal 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EDJRuj6mDpGRqNbQSsXZ9o
The contract declares contract_id int64 and the column is a BIGINT. An
unformatted integer lets a generated client store it in 32 bits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcdY1zQ7nhnVKXuftNjQa6
The contract declares seat_limit with minimum: 0, but the Pydantic field had
no lower bound, so it validated a negative value and omitted the constraint
from the generated schema. Flagged by Copilot review on #60.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019W22jyGzuYBwRZKk6XwujT
@blarghmatey
blarghmatey force-pushed the feat/b2b-learner-records-courses branch from 60336cd to 2b5e3a7 Compare September 17, 2026 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants