Skip to content

fix(aikido): icanbwell namespace, numpy 2.5.2, patch 4 vulnerable transitive deps - #160

Merged
vanshikajtg merged 14 commits into
mainfrom
fix/aikido-vulnerabilities
Sep 8, 2026
Merged

vanshikajtg merged 14 commits into
mainfrom
fix/aikido-vulnerabilities

Conversation

@vanshikajtg

@vanshikajtg vanshikajtg commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

What this does

Two independent Aikido fixes:

  1. Moves the Spark base image off the imranq2 Docker Hub namespace onto the
    icanbwell-owned one.
  2. Bumps the transitively-pinned numpy to 2.5.2.

Findings fixed

Issue Rule / CVE File Severity Open since
17993914 Unapproved registry usage spark.Dockerfile 60 medium SLA 2025-01-18
648392668 AIKIDO-2026-125890 in numpy Pipfile.lock 25 low —

1. Base image namespace

-FROM imranq2/helix.spark:3.5.5.0-slim     # spark.Dockerfile
+FROM icanbwell/helix.spark:3.5.5.0-slim

The tag is unchanged, so the Spark version baked into the image still matches this repo's
pyspark = "==3.5.5" pin. pre-commit.Dockerfile already used
public.ecr.aws/docker/library/python:3.12-alpine and is untouched.

Why this is safe: a no-op for image content. 3.5.5.0-slim is byte-for-byte
identical in both namespaces —
sha256:e2c4762e38e3f57bfa99afdd68621c0be46eb373475c5578113c0e683b193126 — verified
anonymously against the Docker Hub registry API on 2026-09-08. An OCI image-index digest
hashes the whole index including every child manifest digest, so an identical index digest
proves the entire image graph is identical. The image stays multi-arch (linux/amd64 +
linux/arm64) and public and anonymously pullable, so no registry credentials are
introduced and fork PRs keep building.

Why not the private ECR. CIE-8032's ratified target is the private services ECR. That
was implemented on this branch and then deliberately reverted: this repo is public and
runs on ubuntu-latest, so a private ECR needs a credential (static IAM key or OIDC) that
does not exist today, and fork PRs would receive no credential at all. Measured
confirmation that no policy change can avoid this: ECR Private has no anonymous
authentication path — GET /v2/ returns 401 with
Www-Authenticate: Basic realm=..., service="ecr.amazonaws.com", and an unsigned
GetAuthorizationToken returns MissingAuthenticationTokenException. A repository policy
granting pull to cluster nodes is an authorization rule; it cannot let a GitHub-hosted
runner authenticate. (This is exactly why helix.pipelines needs no secret — it runs
runs-on: main, a self-hosted runner inside the cluster, and inherits the node identity
ambiently.)

This is therefore the interim option from CIE-8032, not the ratified target — it closes
the finding but does not complete the ticket, and does not block moving to the ECR later.

How we know the rule keys on the namespace, not on Docker Hub — a controlled comparison
inside a single repo on a single branch: SparkAutoMapper/pre-commit.Dockerfile pulls
python:3.12-slim from Docker Hub and has no "Unapproved registry usage" finding,
while its spark.Dockerfile pulls imranq2/... and does. Both files carry the root.io and
Alpine rules, proving both FROM lines are parsed. Across the org the correlation is 10/10.
Aikido's remediation text ("publish them to public ECR") does not describe what actually
closes the finding — helix.pipelines cleared it via the private ECR.


2. numpy 2.5.2

numpy is purely transitive here — absent from [packages], [dev-packages] and
[pipenvsetup]. It arrives via sparkautomapper==3.0.4, whose requirement is
numpy>=1.15 with no upper bound, so 2.5.2 satisfies it; pyspark==3.5.5 declares no
numpy constraint. An independent uv pip compile --python-version 3.12 of [packages]
selects numpy 2.5.2 on its own, so this is the resolver-correct target rather than just
Aikido's assertion. Pipfile is deliberately untouched; only Pipfile.lock's
default.numpy entry changes.

The 66 hashes are the complete distinct sha256 set published for numpy 2.5.2 on PyPI, and
are byte-identical to the set already committed for this CVE in SparkAutoMapper and
SparkPipelineFramework. pipenv passes these to pip under --require-hashes, so a wrong or
short list is a hard install failure.

The marker moves python_version >= '3.11' → >= '3.12' because numpy 2.5.2 declares
requires_python: >=3.12. This repo pins [requires] python_version = "3.12", so the
marker stays true. Worth a reviewer's eye: numpy is the only marker-gated entry in this
lock, so nothing else independently proves 3.12 at image-build time. If the base image
resolved to 3.11, pipenv would silently skip numpy and sparkautomapper would fail at
import — which make tests in CI would catch loudly.

_meta.hash is deliberately not recomputed

This lock is already out of sync with its Pipfile on origin/main (stores
0b5a6f4d…, Pipfile computes 6e214300…), and has been since PR #146: commit
8d371973 "Fix Pipfile" flattened a malformed [packages.spark] sub-table without
re-locking. I walked all 36 commits touching Pipfile.lock — every one back to 2021 is in
sync except that tip.

The consequences are real but separate from this CVE, and recomputing the hash here
would silently bury them:

  • pyspark = "==3.5.5" and py4j = "==0.10.9.7" are in [packages] but absent from lock
    default
    (they appear only in develop, pulled in transitively).
  • Lock default carries a spurious spark==0.2.1 — an abandoned 2006 PyPI project
    ("a Super-Small, Super-Fast, Super-Easy web framework"), unrelated to Apache Spark. It is
    being installed into this image today.
  • [dev-packages] sparkdataframecomparer = "==2.0.14" while the lock has ==2.0.16,
    because sparkautomapper==3.0.4 hard-requires 2.0.16. As written the Pipfile is
    genuinely unsatisfiable.

That last point is also why a real pipenv lock was not run: it would rewrite ~1400 lines
and bump unrelated transitives, and it may simply fail on the unsatisfiable constraint.
This deserves its own PR referencing #146.

The drift breaks nothing today: every pipenv invocation in this repo is pipenv sync
(spark.Dockerfile:12, pre-commit.Dockerfile:8, Makefile:40), which sets
ignore_pipfile/skip_lock and never compares _meta.hash
(pipenv/routines/install.py:281-284). There is no --deploy, pipenv verify or
pipenv check anywhere in the repo.


Still open after this PR

Neither is fixable by a code change, and neither depends on a credential:

Issue Rule Why not fixed here
506323050, 506323051 Not sourced from root.io ECR mirror Accepts only .../root-mirror/*. helix.spark is b.well's own image, so Root.io will not mirror it. Needs the Aikido allowlist update tracked as CIE-8032 ask #3. (The ticket names CUSTOM-RULE-558; the rules actually firing are 559 and 2300.)
626292990 Use Alpine base images Keys on the repository name, not the tag — fires on images that genuinely are Alpine. Needs a false-positive exception.

Testing

CI on this PR is the first real build-and-test on this branch (earlier runs failed at an
AWS-credentials step that has since been removed). No Docker daemon was available locally,
so neither the image build nor the numpy install was exercised on this machine. The lock
edit was validated by a byte-exact JSON round-trip (only the numpy block and no other
"version" line appears in the diff, and _meta.hash is untouched), but the install itself
must be confirmed by make up / make tests in CI.

🤖 Generated with Claude Code

vanshikajtg and others added 10 commits September 8, 2026 02:58
Aikido issue 17993914 (CUSTOM-RULE-559 "Unapproved registry usage", group 3838154,
severity 60, SLA overdue since 2025-01-18).

Per the final CIE-8032 decision (comment 484133) helix.spark is published to the
services-account private ECR, and icanbwell/helix.spark on Docker Hub is now private.
Follows the one migrated precedent, helix.pipelines#5301 / commit 905cc650.

Verified with aikido_full_scan: this FROM clears CUSTOM-RULE-559.

CUSTOM-RULE-2300 (issue 506323051) still fires and is knowingly left open. helix.spark
is b.well's OWN image, built from spark:3.5.5-java17-python3; Root.io mirrors upstream
public images, so it will never live under root-mirror/*. The private services ECR does
not satisfy 2300 either (measured). That needs a rule exception, not a Dockerfile
change - the same disposition accepted for helix.pipelines in PR #5325.

CUSTOM-RULE-2576 (issue 626292990, "use Alpine base images") also still fires. No
Alpine build of helix.spark exists, and rebasing a JVM+pyspark image onto musl is not
viable; the rule also fires on images that ARE Alpine, so it clears nothing.

The 3.5.5.0-slim TAG is deliberately unchanged: the tag's Spark version must match the
pyspark==3.5.5 pin in Pipfile (Python half vs JVM half), and -slim is load-bearing
because of the `mv /opt/minimal_entrypoint.sh` step below.

PREREQUISITE: only 3.5.1.11-slim is proven present in that ECR repo. 3.5.5.0-slim is
NOT confirmed - helix.spark's deploy_docker-slim.yml pushes to Docker Hub only, with no
ECR push step, so a human must copy the tag across or this build fails on pull. Not
verifiable here: no AWS credentials in this environment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Hardens the CKV_DOCKER_3 fix from PR #159 (Aikido issue 3776016, now closed).

- Replaces the bare `USER 1001` with a real named user
  (`addgroup`/`adduser`, the Alpine equivalents of groupadd/useradd) so HOME and
  shell lookups resolve instead of pointing at a UID with no passwd entry.
- `git config safe.directory` moves from --global to --system. That RUN executes as
  root, so --global wrote /root/.gitconfig, which the non-root runtime user can never
  read - doubly so because pre-commit-hook sets HOME=/tmp. --system (/etc/gitconfig)
  is world-readable, so the setting now actually applies. This was latent rather than
  active: CI never tripped git's dubious-ownership check because the runner UID
  already matches the workspace owner.

pre-commit-hook keeps its runtime `--user "$(id -u):$(id -g)" -e HOME=/tmp` override
from PR #157, which is REQUIRED: pre-commit's formatters rewrite files in the
bind-mounted /sourcecode, so the container UID must match the host owner. Both halves
of the non-root fix stay in place together.

BASE IMAGE INTENTIONALLY UNCHANGED - stays on
public.ecr.aws/docker/library/python:3.12-alpine (set by CIE-5665). That clears
CUSTOM-RULE-559 but not CUSTOM-RULE-2300 (issue 506323050), which accepts only
root-mirror/*. 2300 is therefore left OPEN here as a rule-exception request rather
than a code change. The flip to
856965016623.dkr.ecr.us-east-1.amazonaws.com/root-mirror/python:3.12-alpine3.22 was
scan-verified to clear both rules and is recorded in an in-file comment, but was NOT
adopted: that tag's existence in the mirror is unverifiable here, no other repo in the
org uses it, and every working root-mirror precedent is a PRIVATE repo on a
self-hosted runner. Keeping public.ecr.aws also means `make run-pre-commit` needs no
registry credentials, so linting still works for forks and outside contributors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…k image

spark.Dockerfile now pulls helix.spark from
856965016623.dkr.ecr.us-east-1.amazonaws.com, so docker must be logged in there before
`make up` builds it. Modelled on helix.pipelines commit 905cc650 (`ecr-login` Makefile
target + configure-aws-credentials/amazon-ecr-login in CI).

- Makefile: new `ecr-login` target (AWS_SERVICES_PROFILE ?= services). It no-ops when
  $CI is set, because a named local SSO profile does not exist on a GitHub runner -
  there the workflow steps do the login. devdocker/build/up depend on it.
  `run-pre-commit` deliberately does NOT: pre-commit.Dockerfile stays on public.ecr.aws
  and needs no credentials.
- build_and_test.yml: assume the services role via OIDC and run amazon-ecr-login
  (skip-logout: true, matching the precedent) AFTER the pre-commit step and before
  `make up`. Step order is deliberate - lint/format feedback keeps working even while
  the ECR prerequisites below are unmet. Adds `permissions: id-token: write`.
- CONTRIBUTING.md: document `aws sso login --profile services`, and state plainly that
  `make run-pre-commit` needs no AWS access while `make up`/`make tests` do.

BLOCKER - this CI change CANNOT WORK AS COMMITTED. helix.pipelines does the same thing
from a self-hosted `runs-on: main` runner; that is not available to a PUBLIC repo
(a fork PR could execute on it), and no GitHub OIDC trust policy for
icanbwell/SparkAutoMapper.FHIR exists on
arn:aws:iam::856965016623:role/services-k8s-service-account-github-actions-runner
(none found in cie.terraform). A human must provision that trust - preferably as a new
pull-only role, since the existing one is heavily over-privileged for a base-image
pull from a public repo. The steps are committed rather than omitted so the failure
surfaces at an explicit, diagnosable auth step instead of an opaque "pull access
denied" inside docker build.

Worth re-raising on CIE-8032 before that trust is built: publishing helix.spark to a
PUBLIC ECR (public.ecr.aws/...) also clears CUSTOM-RULE-559, needs no CI auth at all,
and would fix all six public consumer repos at once instead of one OIDC trust each.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
helix.spark publishes only to Docker Hub, so the base-image tags this repo
needs must be copied into the services ECR before the new FROM lines build.
Lists the exact tags, their expected digests, and a manifest-preserving
crane copy (these images are multi-arch; docker pull/push would lose an arch).

Refs CIE-8032.
The six Spark repos had diverged onto three different variable names plus
two hardcoded ARNs. Two of them hardcoded
services-k8s-service-account-github-actions-runner, which is the k8s
service-account role: it is trusted for a cluster service account, not for
GitHub OIDC, and carries broader rights than an image pull needs. Those two
would have failed even after OIDC was provisioned.

All six now read vars.AWS_SERVICES_ECR_ROLE_ARN with no fallback, so one
variable configures every repo and a missing value fails loudly instead of
silently attempting the wrong role.

Refs CIE-8032.
Replaces the GitHub-OIDC approach with static access keys held in GitHub org
secrets (SERVICES_ECR_ACCESS_KEY_ID / SERVICES_ECR_SECRET_ACCESS_KEY), matching
the pattern helix.pipelines already uses for the dev account and the ~58 other
workflows in the org.

Why not OIDC: no GitHub Actions OIDC provider or role exists anywhere in
icanbwell's IaC (0 org-wide references to token.actions.githubusercontent.com),
so it would have meant new IAM infrastructure via a cie.terraform PR. Why not
helix.pipelines' role-to-assume: that works only because it runs on the
self-hosted 'main' runner and inherits the pod's IRSA identity. These repos are
PUBLIC on GitHub-hosted runners, with no ambient identity, and moving a public
repo onto self-hosted runners would let fork PRs execute untrusted code on
b.well infrastructure.

No credential material is committed -- the workflow references
${{ secrets.* }} and the values live only in GitHub's encrypted secret store.

Also drops the 'permissions: contents: read / id-token: write' block these
workflows had gained. id-token is no longer needed, and the block was a
regression: origin had no permissions block, so declaring one silently removed
'checks: write' that dorny/test-reporter needs to publish its check run
(SparkDataFrameComparer) -- it would have failed once CI got past auth.

Account choice: services (856965016623), per cie.terraform/aws_config where
856965016623 is bwell-services and 875300655693 is bwell-dev. A base image
shared by six libraries and every environment is a platform artifact, and
CIE-8032 already provisioned the ECR repo there (cie.terraform#2053).

Refs CIE-8032.
Reverts the CIE-8032 registry move and all of its supporting plumbing: the
Dockerfile FROM lines return to imranq2/helix.spark (tags never changed), and the
Makefile ecr-login target, workflow AWS-credential/ECR-login steps and the
README/CONTRIBUTING ECR sections are removed.

Reason: the move made CI depend on credentials that do not exist. It needed a
read-only IAM user in account 856965016623 plus two GitHub org secrets, which
requires services-account admin access nobody on this change has, and separately
needed four multi-arch tags copied into that ECR because helix.spark publishes
only to Docker Hub. All four PRs failed at the AWS Credentials step. The
migration is deferred to whoever owns the base image and the credential story.

Consequence, stated plainly: CUSTOM-RULE-559 (unapproved registry) stays OPEN,
along with CUSTOM-RULE-2300 and CUSTOM-RULE-2576. Those three were always going
to need Aikido allowlist/exception work rather than a code change in this repo -
only 559 was ever fixable here.

Non-ECR security fixes in this branch are retained.

Refs CIE-8032.
Clears Aikido "Unapproved registry usage" (rule group 3838154), open since
January 2025. The rule keys on the `imranq2` namespace, so moving to the
icanbwell-owned namespace mandated by CIE-8032 closes it.

This is a no-op for image content: every tag involved is digest-identical
between the two namespaces, verified anonymously against the Docker Hub
registry API on 2026-09-08:

  3.5.5.0-slim             sha256:e2c4762e38e3f57bfa99afdd68621c0be46eb373475c5578113c0e683b193126
  3.5.5.0-precommit-slim   sha256:0a972955724aca975fe124f6d4310c27003f9dcce839fdb5b93c9d3044d2b6b8
  3.5.1.11-slim            sha256:be956d9e2f3d3a2a032aa670f007dd14930dccb840beed3339cf21a93b8fa447
  3.5.1.11-precommit-slim  sha256:f3165b952b4b01a6e2d5eb062ed76da7a9e89566550cf2e0a727b5cfe2042145

Tags are unchanged, so the baked-in Spark version still matches each repo's
pyspark pin. The images remain public and anonymously pullable, so no
registry credentials are needed and fork PRs keep building -- unlike the
private-ECR approach this replaces.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes Aikido issue 648392668 (numpy 2.3.2 -> 2.5.2).

numpy is a purely TRANSITIVE dependency here -- it is not in `[packages]`,
`[dev-packages]` or `[pipenvsetup]`. It arrives via `sparkautomapper==3.0.4`,
whose requirement is `numpy>=1.15` with no upper bound, so 2.5.2 satisfies it.
`pyspark==3.5.5` declares no numpy constraint. An independent
`uv pip compile --python-version 3.12` of `[packages]` selects numpy 2.5.2 on
its own, so this is the resolver-correct target and not just Aikido's
assertion. `Pipfile` is therefore deliberately untouched.

The 66 hashes are the complete distinct sha256 set published for numpy 2.5.2
on PyPI, and are byte-identical to the set already committed for this CVE in
SparkAutoMapper and SparkPipelineFramework. pipenv passes these to pip under
--require-hashes, so a wrong or short list is a hard install failure.

The marker moves from `python_version >= '3.11'` to `>= '3.12'` because
numpy 2.5.2 declares `requires_python: >=3.12`. This repo pins
`[requires] python_version = "3.12"`, so the marker stays true.

`_meta.hash` is deliberately NOT recomputed. This lock is already out of sync
with its Pipfile on origin/main (stores 0b5a6f4d…, Pipfile computes
6e214300…), and has been since PR #146: commit 8d37197 "Fix Pipfile"
flattened a malformed `[packages.spark]` sub-table without re-locking. The
consequences are real but separate from this CVE -- `pyspark`/`py4j` are in
`[packages]` yet missing from lock `default`, and lock `default` carries a
spurious `spark==0.2.1` (an abandoned 2006 web framework, unrelated to Apache
Spark). Recomputing the hash here would silently bury that. It deserves its
own PR referencing #146.

The drift breaks nothing today: every pipenv call in this repo is
`pipenv sync` (spark.Dockerfile:12, pre-commit.Dockerfile:8, Makefile:40),
which sets ignore_pipfile/skip_lock and never compares `_meta.hash`. There is
no `--deploy`, `pipenv verify` or `pipenv check` anywhere in the repo.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vanshikajtg vanshikajtg changed the title CIE-8032: move helix.spark base image to the private services ECR fix(aikido): icanbwell base-image namespace + numpy 2.5.2 Sep 8, 2026
vanshikajtg and others added 4 commits September 8, 2026 16:26
Point the helix.spark FROM lines at
856965016623.dkr.ecr.us-east-1.amazonaws.com/helix.spark, keeping every tag
unchanged. This is the CIE-8032 ratified registry target.

Clears CUSTOM-RULE-559 (unapproved registry). It does NOT clear CUSTOM-RULE-2300
(root.io mirror), which accepts only <acct>.dkr.ecr.../root-mirror/* — helix.spark
is not on the root.io allowlist and cannot be mirrored without a rootio-terraform
change plus Root.io publishing the tags.

KNOWN: this breaks CI on this repo. ECR Private has no anonymous auth path
(GET /v2/ -> 401; unsigned ecr:GetAuthorizationToken -> MissingAuthenticationToken),
and these workflows run on ubuntu-latest, which has no AWS identity. The repository
policy granting pull to cluster nodes is authorization, not authentication -- it is
why helix.pipelines needs no secret (self-hosted runner inside EKS, ambient node
role) and why it does not help a GitHub-hosted runner. Fork PRs cannot be fixed
with a secret at all. CI will stay red until the runner/credential story is in
place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Dockerfiles now pull helix.spark from the private services ECR, and the
build fails on GitHub-hosted runners with:

  856965016623.dkr.ecr.us-east-1.amazonaws.com/helix.spark:...: Unauthorized

ECR Private has no anonymous access path -- a pull needs a token from
ecr:GetAuthorizationToken, a SigV4-signed AWS API call, and `ubuntu-latest`
has no AWS identity to sign with. Measured: an unauthenticated GET on the
registry returns 401 with
`Www-Authenticate: Basic ... service="ecr.amazonaws.com"`, and an unsigned
GetAuthorizationToken returns MissingAuthenticationTokenException. No
repository-policy change fixes this: a policy grants *authorization* to an
already-authenticated caller, it does not provide a way to authenticate.

So `build_and_test` moves to `runs-on: main` -- the in-house Actions Runner
Controller scale set on the services-ue1 EKS cluster
(cie-base-eks/config/arc-runner-set/values.yaml, runnerScaleSetName: "main",
containerMode dind). Those pods run under the arc-runner-set Kubernetes
service account, which is IRSA-mapped to an AWS role, so the job has an
ambient AWS identity and can assume the services role with no stored
credentials. This is the same mechanism helix.pipelines uses to pull this
exact image, copied from its test_pipeline_pr.yml.

Fork pull requests are deliberately excluded. The runner pods' IAM role
grants, on resources = ["*"]: ssm:Get*/Describe*/List* (every SSM parameter
in the account), ECR write (PutImage, InitiateLayerUpload, UploadLayerPart,
CompleteLayerUpload), eks:Describe*/List*, plus s3:* on many platform
buckets. All four scale sets share the same service account, so there is no
lower-privilege pool to use instead. This repo is PUBLIC, so `pull_request`
runs code written by whoever opened the PR. GitHub withholds secrets from
fork PRs, but that does not help here -- the exposure is the pod's ambient
identity, which the job can reach directly. helix.pipelines can use this
pattern unconditionally only because it is a private repo.

Fork PRs therefore run `fork_pr_checks` on ubuntu-latest instead. Every
Docker image here comes from the private ECR, so they cannot build or test at
all; they get a native `black --check` (which needs no Docker) and an
explanatory notice. `.idea` is excluded because it holds IntelliJ file
templates that are not valid Python -- .pre-commit-config.yaml excludes them
the same way. Verified black passes on all six repos with that exclusion.

Other changes:

- `install required packages`: the ARC runner image is a bare
  ghcr.io/actions/actions-runner:latest with no make and no compiler, so every
  make target would fail with "command not found". helix.pipelines does the
  same.
- `aws-actions/configure-aws-credentials` and `aws-actions/amazon-ecr-login`
  are third-party actions, and Aikido's pinning rule fires on those, so both
  are pinned to commit SHAs rather than floating tags -- otherwise this commit
  would have introduced two new findings while fixing others.
- `actions/checkout@v2|v3` -> v4 and `actions/setup-python@v3|v4` -> v5 where
  present: those versions run on Node runtimes GitHub has removed, which fails
  on a current runner image.

STILL BLOCKED, needs an org admin: GitHub blocks public repositories from
org-level self-hosted runners unless "Allow public repositories" is enabled on
the runner group. Until that is set, `build_and_test` will queue rather than
start. The API returns 403 (must be an org admin) so this could not be checked
from here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ed CI

Reverts the two commits that moved the base image to the private services ECR
and, consequently, moved CI onto the in-house ARC runner.

Why both: the `runs-on: main` change existed only to authenticate to the
private ECR. With the image public again it is unnecessary, and leaving it in
place would be actively worse than reverting -- GitHub blocks public
repositories from org-level self-hosted runners unless the runner group
enables it, so the job never gets a runner. Measured after pushing it: all six
repos' `build_and_test` jobs sat `queued` with `runner_name: unassigned` for
45 minutes and were never picked up, despite the pool keeping a warm idle
runner. Those runs do not fail, they stall until GitHub cancels them at 24h.
That also removes the need for an org-admin permission change, and removes
fork PRs' exposure to a runner identity holding `ssm:Get*` on `resources=["*"]`.

Back to `FROM icanbwell/helix.spark:<tag>`, which:

- clears the same Aikido finding. Rule group 3838154 keys on the `imranq2`
  namespace, not on Docker Hub -- verified by a controlled comparison inside a
  single repo: SparkAutoMapper's `pre-commit.Dockerfile` pulls
  `python:3.12-slim` FROM DOCKER HUB and has no 3838154 finding, while its
  `spark.Dockerfile` pulled `imranq2/...` and did. Both files carry the root.io
  and Alpine rules, so both FROM lines are demonstrably parsed. Org-wide the
  correlation is 10/10.
- is a no-op for image content. All four tags are digest-identical across the
  two namespaces (e.g. 3.5.1.11-slim = sha256:be956d9e...a93b8fa447), verified
  anonymously against the Docker Hub registry API. An OCI index digest hashes
  the whole index including every child manifest, so an identical index digest
  proves the entire image graph is identical.
- needs no credentials and keeps fork PRs building, because it is public and
  anonymously pullable (verified with an anonymous token whose JWT `sub` is
  empty).

Tags are unchanged throughout, so the Spark version baked into each image
still matches that repo's pyspark pin.

This is CIE-8032's interim option rather than its ratified target (the private
services ECR), and should be described that way -- it closes rule 559 without
completing the ticket. Rules 2300 and 2576 remain open either way; no registry
move clears them.

Note this revert also drops the `actions/checkout@v2|v3 -> v4` and
`setup-python@v3|v4 -> v5` bumps that were bundled into the runner commit.
Those were worth doing on their own merits (v2/v3 run on Node runtimes GitHub
has removed) and can be re-applied in a separate commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Applies the same dependency CVE fixes already made in SparkPipelineFramework
to the repos that ship the same vulnerable versions. Aikido has NOT filed
findings for most of these -- its open-source coverage is stale on 5 of the 6
Spark repos -- but the locked versions are the same ones it flags in
SparkPipelineFramework, so the exposure is real regardless of whether a
dashboard entry exists. Fixing them now also stops a batch of new findings
appearing the next time Aikido re-syncs.

Versions chosen are the MINIMUM patched release for each CVE, to keep the
change as small as possible, except aiohttp which is pinned to 3.13.5 -- the
highest release that still works with aioresponses (3.14.0 made
`stream_writer` a required kwarg of ClientResponse.__init__, which no released
aioresponses supports; upstream fix pnuckowski/aioresponses#288 is still open).

  aiohttp        -> 3.13.5     urllib3         -> 2.7.0
  requests       -> 2.34.0     pymysql         -> 1.1.3
  pymongo        -> 4.16.0     fastjsonschema  -> 2.22.2
  pyathena       -> 3.35.4     numpy           -> 2.5.2

Also bumps aiosignal 1.3.2 -> 1.4.0 where aiohttp moved: this is not a CVE
fix, it is required because aiohttp 3.13.5 declares `aiosignal>=1.4.0`.
Verified aiohttp 3.13.5's other seven requirements are already satisfied by
the locked transitives (aiohappyeyeballs 2.6.1, attrs 25.3.0, frozenlist
1.7.0, multidict 6.4.4, propcache 0.3.2, yarl 1.20.1, async-timeout 5.0.1).

**Lock-only by design.** Every one of these packages is transitive -- none is
declared in any Pipfile -- so no Pipfile changes are needed and `_meta.hash`
is deliberately left untouched in all repos. That is load-bearing for
SparkAutoMapper.FHIR, whose lock hash is already drifted from a pre-existing
bug introduced in PR #146; recomputing it here would silently bury that.
Verified after the change: 5 repos still IN SYNC, SparkAutoMapper.FHIR still
DRIFTED (unchanged), and no `_meta.hash` value differs from before.

Markers were tightened where the new release requires a newer Python (e.g.
numpy 2.5.2 declares requires_python >=3.12, so its marker moves from
`python_version >= '3.11'` to `>= '3.12'`). Markers are only ever tightened,
never loosened, and every repo pins `[requires] python_version = "3.12"`, so
each tightened marker remains true -- checked repo by repo. A marker that
evaluated false would cause pipenv to silently SKIP the package.

The `pipenvsetup` category is included. It is developer tooling installed with
`--skip-lock`, so those pins are inert at install time, but vulnerable
versions should not sit in the file the scanner parses.

Hashes are the complete distinct sha256 set PyPI publishes for each release;
pipenv passes them to pip under --require-hashes, so a wrong or short list is
a hard install failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vanshikajtg vanshikajtg changed the title fix(aikido): icanbwell base-image namespace + numpy 2.5.2 fix(aikido): icanbwell namespace, numpy 2.5.2, patch 4 vulnerable transitive deps Sep 8, 2026
@vanshikajtg
vanshikajtg marked this pull request as ready for review September 8, 2026 12:57
@vanshikajtg
vanshikajtg merged commit 1bfe380 into main Sep 8, 2026
4 checks passed

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants