This is a project of the Apache Software Foundation and follows the ASF vulnerability handling process.

This document is for human security researchers, AI-assisted security researchers, and project maintainers handling vulnerability reports. Read it alongside the Apache Fineract Threat Model which governs the source banking application whose data this pipeline consumes.

Do not open a public GitHub issue for security vulnerabilities.

Report vulnerabilities privately using the ASF vulnerability reporting process:

• ASF Security team: security@apache.org
• Project mailing list (private): private@apache.org (Apache committers only)
• GitHub private reporting: Use the Security Advisories tab → "Report a vulnerability"

Include in your report:

• A description of the vulnerability and affected component
• Steps to reproduce or a proof-of-concept
• The potential impact and attack scenario
• Any suggested remediation

You will receive an acknowledgement within 72 hours. The ASF Security team coordinates disclosure timing with maintainers. Public disclosure happens only after a fix is available.

Source code repository: https://github.com/apache/fineract-business-intelligence

This document covers the fineract-business-intelligence data pipeline and BI layer:

• Analytics for microfinance operations: Branch managers and analysts view loan portfolio health, delinquency trends, PAR (Portfolio at Risk) metrics, and batch job status through Superset dashboards.
• Read-only extraction: The pipeline never writes to the Fineract source database. All mutations are on the analytics warehouse only.
• Incremental extraction: The extractor uses watermarks to extract only records modified since the last run, bounded by a COB (Close of Business) completion gate.

• Apache Fineract backend (Spring Boot, JDBC, Kafka, COB batch, core banking logic) — covered by the Fineract security team.
• Fineract source database internals — the extractor connects read-only; what happens inside the source DB is out of scope.
• Superset upstream vulnerabilities — bugs in the Apache Superset project should be reported to the Superset security team.
• Customer-facing portals — Superset is back-office only; bank customers are not users.

• Fineract source DB compromise: A compromised upstream DB will cause corrupted data to be extracted and propagated. Defence is Fineract's responsibility.
• Network-level DDoS against services: Volumetric DDoS mitigation is the operator's responsibility (CDN, cloud WAF, rate limiting at the load balancer).
• Physical access to the host machine: OS-level access is assumed to be already compromised.
• Supply-chain attack on Python/dbt/Superset packages: Dependency scanning (Dependabot, pip audit) is a downstream responsibility; this model assumes packages are not themselves compromised.

[Apache Fineract DB — fineract_reader role, SELECT only]
        |
        | PostgreSQL / pg8000 (TLS optional)
        v
[Extractor — Python service]
  - COB gate: verifies batch_job_execution before extracting
  - Watermark manager: incremental extraction by last_modified_on_utc
  - Replica lag check: aborts if replica is too far behind
        |
        | PostgreSQL / pg8000
        v
[Analytics Warehouse — raw schema]
        |
        | dbt SQL transformations
        v
[Analytics Warehouse — analytics schema (marts, dims, presentations)]
        |
        | PostgreSQL / psycopg2 (read-only reader role)
        v
[Apache Superset — dashboard layer]
        |
        | HTTPS (browser, TLS in production)
        v
[End user — branch manager, MFI analyst]

1. Extractor → Fineract source DB: Read-only replica user (fineract_reader). The bi_connector_source schema provides views that add synthetic created_on_utc / last_modified_on_utc columns for tables that lack them natively (m_office, m_currency, m_product_loan). The extractor queries these views, not the underlying tables directly.
2. Extractor → Analytics warehouse: analytics_loader role. Writes are scoped to the raw schema. The loader role has no access to analytics schema where the marts live.
3. dbt → Analytics warehouse: Runs as analytics_loader. Materialises models in analytics schema. Reads from raw; writes to analytics.
4. Superset → Analytics warehouse: analytics_reader role. SELECT on analytics schema only. Cannot read raw, meta, or other schemas.
5. CI runner → fineract-db container: Uses development-only bootstrap credentials (root / skdcnwauicn2ucnaecasdsajdnizucawencascdca). These are not production credentials and appear in plain text in .env.example and ci.yml deliberately — they are seeded into an ephemeral container and discarded after each run.

• Docker Engine and Docker Compose are installed and up to date.
• The host is not shared with untrusted workloads (no multi-tenant container platform without network isolation).
• The Docker socket is not exposed to the public network.

• In local dev: all services are on a private Docker bridge network, not reachable from outside the host by default.
• In production: the warehouse and Superset must be placed behind a firewall; neither should be reachable from the public internet without TLS termination and access control.
• All browser-to-Superset communication must be over TLS in production. The default compose.yaml exposes Superset on HTTP port 8088 only.

• The watermark manager (meta.watermarks) serialises extraction windows per tenant and source table. Concurrent extractor runs for the same tenant would cause undefined watermark state. The pipeline is designed for one run at a time per tenant.

• The extractor uses datetime.now(timezone.utc) for pipeline state timestamps. Clock skew between the extractor host and the Fineract DB host can affect the extract_lookback_seconds window. A skew larger than REPLICA_LAG_THRESHOLD_SECONDS (default: 300 s) will cause the replica lag check to abort the run.

• Extraction batch size: Controlled by extract_batch_size in AppConfig; batches are committed incrementally. Very large tables will produce many small commits — this is by design.
• Pipeline interval: Default PIPELINE_INTERVAL_SECONDS=3600. No rate limiting beyond this interval is enforced by the pipeline itself.
• Superset query limits: Not configured by this project. Operators must configure Superset's ROW_LIMIT and async query timeouts to prevent runaway queries from loading the warehouse.

• Fineract DB admin: Full access to the source DB is assumed to be already authorised; this model cannot defend against it.
• Host OS / hypervisor attacker: Physical or OS-level access is assumed compromised.
• Browser zero-day exploiter: Superset UI browser attacks via unpatched engine vulnerabilities are not modelled.
• Supply-chain attacker (PyPI / npm): Compromise of a dependency package is out of scope; addressed by dependency scanning tooling.

• Read-only source access — the extractor connects to the Fineract DB as a SELECT-only replica user (fineract_reader). No INSERT, UPDATE, or DELETE is possible on the source database from this pipeline.
• Role separation in the warehouse — three distinct roles: analytics_admin (DDL), analytics_loader (write to raw), analytics_reader (SELECT on analytics only). Superset uses analytics_reader and cannot reach raw or meta schemas.
• Parameterised queries throughout the extractor — all source queries use %s placeholders via pg8000; column and table names are quoted with _quote_identifier() which double-quotes and escapes the identifier. No raw string interpolation of source data into SQL.
• COB gate — the extractor refuses to run if no COMPLETED batch job execution is found within the configured lookback window, preventing extraction of inconsistent mid-batch state.
• Replica lag check — aborts extraction if the source replica is too far behind the primary, preventing stale data from entering the warehouse silently.
• Watermark-based incremental extraction — extraction windows are bounded by last_modified_on_utc cursors stored in meta.watermarks, preventing unbounded full-table scans after initial backfill.
• Pinned GitHub Actions — all workflow action references are pinned to full commit SHAs to prevent supply-chain attacks via mutable version tags.
• persist-credentials: false on all checkout steps to limit credential exposure on CI runners.
• Least-privilege CI permissions — workflow-level permissions: contents: read.
• Concurrency controls — CI workflow uses cancel-in-progress: true to prevent duplicate runs accumulating on the same branch.
• Zizmor audit — all Actions workflow files are audited by Zizmor on every change for unpinned refs, script injection, and excessive permissions.

• TLS between services — Docker Compose uses an internal bridge network without TLS. Container-to-container credentials and query results are transmitted in plaintext. Production deployments must add TLS or mTLS between all services.
• Docker socket isolation — the extractor service mounts /var/run/docker.sock in compose.yaml. This gives the extractor container full Docker control over the host (equivalent to root). In production, remove this mount unless the pipeline management scripts genuinely require it; if required, use Docker's authorisation plugin or rootless Docker instead.
• Superset MFA — multi-factor authentication for Superset is not configured. Enforce at the Superset layer in production (AUTH_TYPE, LDAP, or SAML integration).
• Superset SQL Lab restriction — SQL Lab is not disabled by default. An analyst with SQL Lab access can run arbitrary SELECT queries against the warehouse reader connection. Restrict SQL Lab access to trusted roles in production Superset configuration.
• Warehouse encryption at rest — not configured in the Docker Compose setup. Production PostgreSQL must use filesystem-level or Transparent Data Encryption.
• Audit log for pipeline runs — no tamper-resistant audit trail of extractor runs or dbt executions beyond container logs and the meta.pipeline_state table (which is mutable by the loader role).
• Network egress restrictions — the extractor has no outbound firewall rules. In production, restrict egress to the warehouse host and source DB host only.
• Secret manager integration — credentials are passed via environment variables. Production deployments should use a secret manager (HashiCorp Vault, AWS Secrets Manager, Kubernetes Secrets) rather than plain .env files.
• CI credential rotation — the development PostgreSQL password (skdcnwauicn2ucnaecasdsajdnizucawencascdca) appears in plain text in .env.example and ci.yml. This is intentional: it is a development-only seed credential for an ephemeral CI container containing no real data. It must not be used in any non-CI deployment. See §10 for the production credential requirement.

• Warehouse loader role abuse: If WAREHOUSE_LOADER_PASSWORD is leaked, an attacker can corrupt or delete raw extraction data. The analytics schema is protected (reader role only), but raw data integrity is lost.
• Superset metadata DB access via admin credentials: If WAREHOUSE_ADMIN_PASSWORD is leaked, an attacker can read all Superset dashboard configs, database connection strings, and user session data stored in the metadata DB.
• Docker socket escalation: A compromised extractor process with access to /var/run/docker.sock can spawn privileged containers and escape to the host. This is a known Docker architecture limitation; see §9 mitigation above.
• Fineract source DB poisoning: If the upstream Fineract DB is compromised before extraction, malicious data will propagate through the pipeline to dashboards. The pipeline has no integrity-verification layer on source data.

1. Replace all .env.example credentials before any non-local deployment. Every password in .env.example is a placeholder — none may be used in production.

2. Remove the Docker socket mount from the extractor service in compose.yaml unless the pipeline management scripts genuinely require Docker access. If required, use the minimum necessary Docker API scope.

3. Terminate TLS in front of Superset. The default compose.yaml exposes Superset on HTTP port 8088 only.

4. Set SUPERSET_SECRET_KEY to a cryptographically random value. The .env.example placeholder (change_me_before_production_use_a_random_32_byte_hex) must be replaced. A compromised secret key allows session forgery for all Superset users.

   python -c "import secrets; print(secrets.token_hex(32))"

5. Restrict network access — neither the warehouse nor the Fineract source DB should be reachable from the public internet. Place them on private subnets.

6. Disable or restrict Superset SQL Lab for analyst roles. SQL Lab gives direct query access to the warehouse reader connection.

7. Enable Superset MFA for admin accounts.

8. Use a secret manager for all credentials. Do not use plain .env files in production.

9. Rotate credentials on a regular schedule and after any personnel change.

10. Monitor pipeline state via the meta.pipeline_state table and container logs, or integrate with a log aggregation platform for failure alerting.

11. Run pip audit against the extractor's dependencies before production deployment to detect known CVEs in Python packages.

1. Using .env.example credentials in production. The bootstrap password (skdcnwauicn2ucnaecasdsajdnizucawencascdca) is public, appears in the repository, and is known to anyone who has read this file. Any deployment using it is exploitable by any reader of this repository.

2. Exposing Superset on HTTP without TLS. Port 8088 in compose.yaml is HTTP only. All session cookies, credentials, and dashboard data are transmitted in plaintext to any network observer.

3. Leaving the Docker socket mounted in production. The compose.yaml mount of /var/run/docker.sock into the extractor container is a development convenience for running docker compose commands from within the pipeline. In a production deployment, this grants the extractor (and any code it executes) full Docker host control.

4. Deploying compose.yaml directly to production without changes. The file is optimised for local development: no TLS, hardcoded dev credential defaults, volume paths suited to local developer machines, and the Docker socket mount. It is not a production-ready deployment manifest.

5. Enabling SQL Lab for all Superset users. Analysts who should only see pre-built dashboards gain the ability to run arbitrary queries against the warehouse if SQL Lab is not restricted by role.

6. Running multiple extractor instances for the same tenant simultaneously. The watermark manager uses per-table, per-tenant rows in meta.watermarks but does not hold a distributed lock. Concurrent runs will produce undefined watermark state and may load duplicate or missing rows.

The following changes should trigger a revision of this threat model:

1. Write-back to Fineract source DB — if any component is granted INSERT/UPDATE privileges on the Fineract source DB, the read-only source trust boundary is broken and the entire model must be reconsidered.
2. Addition of a public-facing API layer — e.g., a REST API exposing warehouse data directly. This introduces a new trust boundary and attack surface not covered here.
3. Multi-tenant SaaS deployment — if a single warehouse instance serves multiple organisations, tenant isolation in the raw and analytics schemas must be formally modelled (currently assumed single-tenant per warehouse instance).
4. New extraction targets — adding extraction of PII beyond what is currently extracted (client names, dates of birth, account numbers) requires a PII inventory review and potentially a GDPR/data-protection impact assessment.
5. Streaming / CDC extraction — replacing batch watermark extraction with Change Data Capture (Debezium, logical replication) introduces replication slot management, new failure modes, and potential for replay attacks.
6. Superset embedded analytics — if Superset is embedded in an external web application via the Embedded SDK, the browser trust boundary changes and CSRF/iframe protections must be modelled.
7. Removal of the COB gate — the _ensure_cob_completed check is a data-integrity gate, not just a scheduling convenience. Removing it without an equivalent guarantee creates a window where partially-committed Fineract batches are extracted.
8. New CVE affecting pg8000, psycopg2, dbt-core, or Apache Superset that cannot be cleanly routed to one of the §13 dispositions — this indicates a MODEL-GAP and requires model revision.

Use these dispositions when evaluating a security report before escalating to the ASF Security team.

The following automated security gates run on every pull request and push via GitHub Actions. All workflows pin action references to full commit SHAs and use persist-credentials: false.

Workflow: .github/workflows/ci.yml
Trigger: push on any branch; pull_request on main and develop

Two-stage pipeline:

• Stage 1 — compilation checks: Verifies dbt parse succeeds (all model references resolve, env var defaults are present), that the Extractor Python package installs cleanly, and that the Apache Superset bootstrap assets script is syntactically valid.
• Stage 2 — smoke tests: Brings up the full Docker Compose stack, seeds the Fineract schema, creates bi_connector_source views, runs the extractor in backfill mode, runs dbt run, and asserts 12 data-integrity invariants against the warehouse.

Security-relevant properties of the CI workflow:

• permissions: contents: read only — no write, deploy, or secret access.
• cancel-in-progress: true — prevents credential or resource accumulation from orphaned runs.
• External Docker network (fineract_default) created before stack startup; not inherited from a previous run.
• Stale volumes cleaned up with docker compose down -v before each run to prevent data leakage between runs.
• No pull_request_target trigger — untrusted PR code cannot access repository secrets.

Workflow: .github/workflows/zizmor.yml
Trigger: push/PR on any branch when .github/workflows/** files change

Zizmor audits all GitHub Actions workflow YAML files for security misconfigurations, including:

• Unpinned action references (all actions must be pinned to a full commit SHA)
• Script injection via github.event.* context variables in run: steps
• Excessive workflow permissions
• Dangerous pull_request_target patterns

All action references in this repository are pinned to SHA hashes. Tag-based references (@v3, @v4) are not permitted and will fail the Zizmor check.

Results are uploaded to GitHub's Security tab as SARIF for review.

Workflow: .github/workflows/codeql.yml
Trigger: push/PR to main; weekly scheduled scan
Languages: python, actions

GitHub CodeQL performs deep static analysis on the Python extractor code and on the GitHub Actions workflow files. It catches:

• SQL injection patterns in Python DB code
• Command injection in shell run: steps
• Insecure use of subprocess, eval, exec
• Workflow-level injection via untrusted inputs

Results appear in GitHub's Security tab.

Workflow: .github/workflows/apache-rat.yml

Scans every file for an Apache License 2.0 header. Fails if any project-owned file is missing a header. Prevents unlicensed third-party code from entering the repository.

Workflow: .github/workflows/asf-allowlist-check.yml

Verifies that all dependencies are on the ASF-approved licence allowlist. Blocks GPL, LGPL, AGPL, and other ASF-incompatible licences.

Workflow: .github/workflows/license-check.yml

Per-file Apache header enforcement across source files using a custom script.

The PGPASSWORD in ci.yml is intentionally plain text. It is the PostgreSQL bootstrap password for an ephemeral container seeded with synthetic test data. It is not a secret; it is the same value as SOURCE_BOOTSTRAP_PASSWORD in .env.example which is already public in the repository. Any reader of this repository already has this value.

Generate a secure SUPERSET_SECRET_KEY for production:

python -c "import secrets; print(secrets.token_hex(32))"

All contributors — human or AI-assisted — must satisfy the following before a PR can merge.

PRs that touch the files below require explicit maintainer sign-off on the security implications before merge:

• When you add a feature that changes the attack surface (see §12 for the trigger list), update this document in the same PR.
• When CI flags a finding you believe is a false positive, document the rationale in the PR description. Do not suppress linter rules or audit findings silently.
• When a new CVE in pg8000, dbt-core, or Apache Superset is published, open a tracking issue within 48 hours and reference the relevant §13 disposition.

This is an Apache Software Foundation project. Every file and every dependency must be compatible with the ASF Licensing Policy.

Every project-owned source file must carry the full Apache License 2.0 header. Apache RAT and the license-check workflow enforce this on every PR. Files legitimately excluded from RAT (generated assets, lockfiles, binary images) must be listed in .rat-excludes with a documented justification.

To add a header to a new file, copy the block from any existing file of the same type.

Before adding any Python package to extractor/requirements.txt or any dbt package to dbt/packages.yml, verify its licence is in the ASF Category A list:

The following are blocked:

.asf.yaml enforces the following on the main branch:

Do not add open questions here. Raise sensitive questions via the ASF security reporting process (§1). Raise non-sensitive questions on the mailing list, in GitHub Discussions, or in a PR comment.