Sync Schools and Roles to Salesforce on Create and Update

[fspeirs]

Status

• Closes https://github.com/RaspberryPiFoundation/experience-cs/issues/1677

What's changed?

Infrastructure

• Adds a salesforce_connect database configuration in database.yml to connect to the rpf-heroku-connect Heroku Connect datastore (read/write via a separate PostgreSQL connection, with database_tasks: false so Rails migrations leave it alone).
• Adds a salesforce_connect Docker service (image: ghcr.io/raspberrypifoundation/heroku-connect) to docker-compose.yml for local development, with a named volume and a health check.
• Adds the SALESFORCE_CONNECT_* environment variables to .env.example and docker-compose.yml.
• Updates CI (.github/workflows/ci.yml) to spin up the heroku-connect service container, pass the Salesforce connection env vars, and add packages: read permission so the private image can be pulled.

Models

• Salesforce::Base — abstract Active Record base class that routes all reads and writes to the salesforce_connect database.
• Salesforce::School — maps to the salesforce.editor__c table (PK: editoruuid__c).
• Salesforce::Role — maps to the salesforce.contact_editor_affiliation__c table (PK: affiliation_id__c).
• Salesforce::Contact — maps to the salesforce.contact table (PK: pi_accounts_unique_id__c).

Jobs

• Salesforce::SalesforceSyncJob — base job class that:
  • Checks the SALESFORCE_ENABLED env var and discards the job (no retry) if it is not true.
  • Retries on SalesforceRecordNotFound with polynomial backoff, up to 10 attempts.
  • Truncates string values to respect Salesforce column limits (appending …).
  • Enqueues onto the new salesforce_sync queue.
• Salesforce::SchoolSyncJob — upserts a School record into salesforce.editor__c, mapping all address, status, and metadata fields. Defaults userorigin__c to 'for_education' when blank.
• Salesforce::RoleSyncJob — upserts non-student Role records into salesforce.contact_editor_affiliation__c. Skips student roles entirely.
• Salesforce::ContactSyncJob — looks up the Salesforce Contact by the school creator's user ID and syncs the creator_agree_to_ux_contact flag. Raises SalesforceRecordNotFound (triggering retry) if no Contact exists yet.

Model callbacks

• School — after_commit on create/update enqueues both SchoolSyncJob and ContactSyncJob.
• Role — after_commit on create/update enqueues RoleSyncJob.

GoodJob queue configuration

• Adds the salesforce_sync:10 queue to allow up to 10 concurrent Salesforce sync workers.

Rake tasks

• salesforce_sync:school — bulk-enqueues SchoolSyncJob for every School (for backfilling).
• salesforce_sync:role — bulk-enqueues RoleSyncJob for every Role (for backfilling).
• salesforce_sync:contact — bulk-enqueues ContactSyncJob for every School (for backfilling the UX contact flag).

Tests

• Full RSpec coverage for SchoolSyncJob, RoleSyncJob, and ContactSyncJob, including field mapping, skipping/discarding behaviour, and error cases.
• Model specs for School and Role verify that the correct jobs are enqueued after create/update.

Points for consideration

• Security: The salesforce_connect connection credentials are injected via environment variables. No secrets are committed to the repo.
• Performance: Each after_commit callback enqueues a background job (GoodJob), so there is no synchronous overhead on the request. Concurrency is capped per-record to avoid TOCTOU races.
• Salesforce Contact availability: ContactSyncJob requires the Salesforce Contact record to already exist (keyed by creator_id). If it does not exist yet, the job retries up to 10 times with polynomial backoff.

How to Test Locally

To test these changes locally, you can:

1. Log in as a user that has no school.
2. Go to For Education > Create your school account
3. Complete the sign-up form

Observe the GoodJob dashboard - you should see new SchoolSyncJob and RoleSyncJob jobs in the salesforce_sync queue. In the Rails Console, you can also inspect the number of Salesforce::School and Salesforce::Role objects that have been created - you should see them increase when you add a new school.

Steps to perform after deploying to production

After deploying, run the rake backfill tasks to sync existing data:

rails salesforce_sync:school
rails salesforce_sync:role
rails salesforce_sync:contact

[github-actions]

Test coverage

90.18% line coverage reported by SimpleCov.
Run: https://github.com/RaspberryPiFoundation/editor-api/actions/runs/23539068448

[Copilot]

Pull request overview

Adds Salesforce (Heroku Connect) synchronization for Schools and Roles by introducing a dedicated salesforce_connect database connection, Salesforce-backed ActiveRecord models, and GoodJob jobs/queues to sync on create/update (plus backfill rake tasks).

Changes:

• Add multi-database config (salesforce_connect) and local/CI Docker services to run Heroku Connect Postgres.
• Add Salesforce AR models (Salesforce::Base, Salesforce::School/Role/Contact) and sync jobs (SalesforceSyncJob + per-model jobs).
• Enqueue sync jobs from School/Role lifecycle callbacks and add specs/factories/rake tasks to validate & backfill.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
spec/models/school_spec.rb	Adds expectations that Salesforce sync jobs are enqueued on School create/update.
spec/models/role_spec.rb	Adds expectations that Salesforce sync job is enqueued on Role create/update.
spec/jobs/salesforce/school_sync_job_spec.rb	Verifies SchoolSyncJob field mappings, truncation, and feature-flag discard behavior.
spec/jobs/salesforce/role_sync_job_spec.rb	Verifies RoleSyncJob field mappings, student skip behavior, and feature-flag discard behavior.
spec/jobs/salesforce/contact_sync_job_spec.rb	Verifies ContactSyncJob updates the opt-in field and retries when contact missing.
spec/factories/salesforce/school.rb	Factory for Salesforce school records.
spec/factories/salesforce/role.rb	Factory for Salesforce role records.
spec/factories/salesforce/contact.rb	Factory for Salesforce contact records.
lib/tasks/salesforce_sync.rake	Adds rake tasks to enqueue backfill sync jobs for all Schools/Roles/Contacts.
docker-compose.yml	Adds salesforce_connect service and wires api dependency/env for local dev.
config/initializers/good_job.rb	Adds a dedicated salesforce_sync queue with concurrency.
config/database.yml	Adds salesforce_connect DB configuration for all envs (tasks disabled).
app/models/school.rb	Enqueues Salesforce School + Contact sync jobs on create/update commit.
app/models/role.rb	Enqueues Salesforce Role sync job on create/update commit.
app/models/salesforce/base.rb	Establishes Salesforce model base class connected to salesforce_connect.
app/models/salesforce/school.rb	Maps Salesforce school table and primary key.
app/models/salesforce/role.rb	Maps Salesforce role/affiliation table and primary key.
app/models/salesforce/contact.rb	Maps Salesforce contact table and primary key.
app/jobs/salesforce/salesforce_sync_job.rb	Adds shared job behaviors: concurrency keying, retries, discard flag, truncation helper.
app/jobs/salesforce/school_sync_job.rb	Implements School-to-Salesforce attribute mapping and save.
app/jobs/salesforce/role_sync_job.rb	Implements Role-to-Salesforce attribute mapping (skips student roles) and save.
app/jobs/salesforce/contact_sync_job.rb	Syncs school creator UX-contact opt-in to Salesforce contact.
.github/workflows/ci.yml	Adds GHCR permissions and Heroku Connect service container for tests.
.env.example	Documents Salesforce connect env vars for local development.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 4 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[zetter-rpf]

Is it important to handle deletions of schools and roles?
I think schools are unlikely to be deleted, but roles may be if a school requests it.

This might be something you want to consider in a seperate PR.

[zetter-rpf]

Is is possible to configure this so that the salesforce database isn't needed to run the app and tests?

When we spoke before I said it would be useful if docker wasn't required to run the app as we don't all use it in the team. I can currently boot the server and the console on this branch without the salesforce database running, but see errors when I run the tests:

Maybe this is because it's trying to run the sync job inline during the test run?

 bundle exec rspec spec/models/school_spec.rb:31
Run options: include {locations: {"./spec/models/school_spec.rb" => [31]}}

Randomized with seed 45890

School
  associations
    has many roles (FAILED - 1)

Failures:

  1) School associations has many roles
     Failure/Error: connects_to database: { writing: :salesforce_connect }
     
     ActiveRecord::NoDatabaseError:
       We could not find your database: salesforce_development. Available database configurations can be found in config/database.yml.
     
       To resolve this error:
     
       - Did you not create the database, or did you delete it? To create the database, run:
     
           bin/rails db:create
     
       - Has the database name changed? Verify that config/database.yml contains the correct database name.
     # ./app/models/salesforce/base.rb:7:in '<class:Base>'
     # ./app/models/salesforce/base.rb:4:in '<module:Salesforce>'
     # ./app/models/salesforce/base.rb:3:in '<top (required)>'
     # ./app/models/salesforce/school.rb:4:in '<module:Salesforce>'
     # ./app/models/salesforce/school.rb:3:in '<top (required)>'
     # ./app/jobs/salesforce/school_sync_job.rb:5:in '<class:SchoolSyncJob>'
     # ./app/jobs/salesforce/school_sync_job.rb:4:in '<module:Salesforce>'
     # ./app/jobs/salesforce/school_sync_job.rb:3:in '<top (required)>'
     # ./app/models/school.rb:177:in 'School#do_salesforce_sync'
     # ./spec/models/school_spec.rb:9:in 'block (2 levels) in <top (required)>'
     # ------------------
     # --- Caused by: ---
     # PG::ConnectionBad:
     #   connection to server at "::1", port 5432 failed: FATAL:  database "salesforce_development" does not exist
     #   ./app/models/salesforce/base.rb:7:in '<class:Base>'

Finished in 0.20151 seconds (files took 2.06 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/models/school_spec.rb:31 # School associations has many roles

Randomized with seed 45890

[fspeirs]

@zetter-rpf I've fully split out the Salesforce tests from the code.

If you run:

POSTGRES_HOST=localhost HOSTNAME=$(hostname) POSTGRES_USER=postgres POSTGRES_DB=choco_cake_test bundle exec rspec

You can successfully run the test suite locally without docker and without the Salesforce integration. In CI it will run inside docker and test the Salesforce integration directly.

[fspeirs]

@zetter-rpf I got a reply from the Salesforce team about deletions: it may be something they want to do in the future but the lack of syncing deletions is not a blocker for this work.