Zuora Source Connector (Edrolo)

An Airbyte source connector for Zuora, built and maintained by Edrolo.

Attribution & status. This is an independent, Edrolo-maintained connector, adapted from Airbyte's original source-zuora (which Airbyte has archived at version 0.1.3). It is not hosted, published, or maintained by Airbyte, and it is not part of the Airbyte connector registry. It has been modernized to run on airbyte-cdk 7.x with per-stream incremental state. Use it at your own discretion.

What it does

The connector extracts data from Zuora using the ZOQL Data Query API. It:

• Discovers streams dynamically — every queryable Zuora object (SHOW TABLES) becomes a stream; its schema is derived on the fly from DESCRIBE <object>. There is no hand-maintained schemas/ directory.
• Syncs incrementally per stream — each object carries its own cursor and state. The cursor resolves from the object's schema: updateddate if present, else createddate, else the stream is full-refresh only.
• Runs each query as an async job — submit → poll → download the JSONL result — via a small requests-based client (ZuoraQueryClient) with retries, backoff, and per-call timeouts.

Authentication

Zuora OAuth2 client credentials. You need a Zuora client_id / client_secret for an API user with Data Query permissions, and you must know which Zuora tenant endpoint your account lives on.

Configuration

Config conforms to source_zuora/spec.json:

Field	Required	Description
start_date	yes	Replication start date, YYYY-MM-DD.
tenant_endpoint	yes	Your Zuora tenant location (e.g. US Production, EU Production, US API Sandbox, …). See the spec for the full enum.
query_api	no	Data Query (default) or AQuA. Selects the extraction API — see Choosing the query API.
data_query	yes	Live (default) or Unlimited (the replicated Data Query store, ~12h freshness, for high-volume extraction). Applies to the Data Query API only.
client_id	yes	OAuth client ID (secret).
client_secret	yes	OAuth client secret (secret).
window_in_days	no	Size of each incremental date slice (default 90). Larger = fewer, bigger jobs.

Choosing the query API

query_api selects which Zuora API the connector extracts with. Data Query is the default and unchanged — set query_api only if you specifically want AQuA.

	Data Query	AQuA
API	POST /query/jobs (ZOQL)	POST /v1/batch-query/ (Export ZOQL)
Schema discovery	SHOW TABLES / DESCRIBE	GET /v1/describe (XML)
Download	JSONL	CSV
Objects (APAC sandbox)	188	119

Switching query_api on an existing connection is a breaking change. Create a new connection and re-sync instead. Two reasons:

1. The object sets differ. Of 188 Data Query objects and 119 AQuA objects, only 85 exist in both. 103 streams you have today are unavailable in AQuA (orders, user, attachment, chargemetrics, the audit* and extended* families, …) and 34 are AQuA-only (order, invoiceadjustment, invoicesplit, the journalentrydetail* family, …). Note orders is renamed to order.
2. Cursor values are formatted differently, so existing incremental state does not carry over.

Foreign keys under AQuA. Export ZOQL does not expose foreign keys as columns, so a plain select * returns none of them — InvoiceItem would drop from 76 fields to 36. The connector reconstructs them by selecting <Relationship>.Id for each entry in the object's <related-objects> metadata, which recovers about two-thirds of them (66 of 98 across 12 sampled core objects) under their Data Query names (accountid, billtocontactid, …). The remaining third has no corresponding relationship and is unavailable in Export ZOQL at all — mostly eInvoicing/eReporting fields, contact snapshots, and organizationid.

Local development

Requires Poetry and Python 3.10–3.13 (airbyte-cdk does not support 3.14).

poetry env use python3.13
poetry install

Create a secrets/config.json matching source_zuora/spec.json (the secrets/ directory is git-ignored). Then run the standard Airbyte connector commands:

poetry run source-zuora spec
poetry run source-zuora check    --config secrets/config.json
poetry run source-zuora discover --config secrets/config.json
poetry run source-zuora read     --config secrets/config.json --catalog integration_tests/configured_catalog.json

Testing

Unit tests use pytest + requests-mock (no live Zuora account needed):

poetry run pytest unit_tests/ -v

check, discover, and read against a real Zuora tenant require valid credentials in secrets/config.json. acceptance-test-config.yml is provided for running Airbyte's Connector Acceptance Tests if you wire them up in your own CI.

Building the image

Dependencies are managed with Poetry (pyproject.toml / poetry.lock) — there is no setup.py and no requirements.txt. The Dockerfile builds a standalone connector image FROM airbyte/python-connector-base:4.1.0 (Python 3.13) and installs the package so the container's entrypoint is the source-zuora command (it accepts spec / check / discover / read).

CI (recommended). .github/workflows/build-and-publish.yml runs the tests, then builds linux/amd64 and pushes to GHCR at ghcr.io/edrolo/airbyte-source-zuora:

• push to main → tags main and sha-<short>
• push a version tag vX.Y.Z → tags X.Y.Z, X.Y, and latest
• pull requests build the image to validate the Dockerfile but do not push

Cut a release image:

git tag v0.2.0 && git push origin v0.2.0

Local build (build for the cluster's arch — Airbyte job pods are typically amd64, so cross-build from Apple Silicon):

docker build --platform linux/amd64 -t ghcr.io/edrolo/airbyte-source-zuora:0.2.0 .
docker run --rm ghcr.io/edrolo/airbyte-source-zuora:0.2.0 spec        # sanity check → SPEC message
docker push ghcr.io/edrolo/airbyte-source-zuora:0.2.0

Bumping the base image: pick the newest python-connector-base tag whose Python is still < 3.14, rebuild, re-run spec/check, and re-pin the digest in both Dockerfile and metadata.yaml (they must match).

Installing on a self-hosted Airbyte (Kubernetes)

1. Publish the image to a registry your cluster can pull from (see above). GHCR packages are private by default — either make the package public, or configure a pull secret (step 2).

2. Private-registry pull secret (skip if the image is public). Create a secret in Airbyte's namespace and point the worker at it:

   kubectl create secret docker-registry regcred \
     --docker-server=ghcr.io \
     --docker-username=<github-user> \
     --docker-password=<ghcr-token-with-read:packages> \
     --namespace airbyte

   In your Helm values.yaml, then helm upgrade:

   worker:
     extraEnv:
       - name: JOB_KUBE_MAIN_CONTAINER_IMAGE_PULL_SECRET
         value: regcred

   (The exact key path can differ between Airbyte Helm chart V1 and V2.)

3. Register the connector in the Airbyte UI: Workspace Settings → Sources → New connector → Add a new Docker connector, then set:

   Field	Value
   Connector display name	Zuora (Edrolo)
   Docker full image name	ghcr.io/edrolo/airbyte-source-zuora
   Docker image tag	0.2.0
   Connector documentation URL	(optional)

   Airbyte pulls the image and runs spec to render the config form. Create a source from it using the config fields above.

Bump the image tag on every change — Airbyte caches connector images by tag, so reusing a tag can serve a stale image.

Architecture

File	Responsibility
source_zuora/source.py	SourceZuora(AbstractSource) + ZuoraObjectStream(Stream, CheckpointMixin) — discovery, per-stream state, slicing, cursor resolution/fallback.
source_zuora/zuora_backend.py	QueryBackend interface + get_backend() factory selecting on query_api.
source_zuora/zuora_client.py	ZuoraQueryClient — the Data Query backend: ZOQL submit/poll/download.
source_zuora/zuora_aqua_client.py	ZuoraAquaClient — the AQuA backend: Export ZOQL batch-query submit/poll/CSV download, FK reconstruction.
source_zuora/zuora_describe.py	Parsers for the XML Describe API (objects, export-context fields, relationships).
source_zuora/zuora_http.py	ZuoraHttpClient — shared retries/backoff/timeouts and auth-failure classification.
source_zuora/zuora_types.py	Zuora-type → JSON-schema-type mapping, shared by both backends.
source_zuora/zuora_auth.py	OAuth2 client_credentials authenticator.
source_zuora/zuora_endpoint.py	Tenant-endpoint → API base-URL mapping.
source_zuora/zuora_errors.py	AirbyteTracedException-based error taxonomy (config / transient / system).
source_zuora/zuora_excluded_streams.py	Objects to skip during discovery.

Known limitations & follow-ups

• Python 3.14 is not supported until airbyte-cdk supports it.
• check / discover / read cannot be verified without a live Zuora tenant.
• Minor deferred items: date-window boundaries overlap by one edge (deduplicated by primary key); the requests.Session is not explicitly closed.
• Under AQuA, roughly a third of foreign keys cannot be recovered (see Choosing the query API), and AQuA exposes fewer objects than Data Query.

Maintainers

Maintained by Edrolo. For issues with this connector, use this repository's issue tracker — not Airbyte's.