fix(docker): install cuDNN 8 compat libraries in the CUDA Docker image

[basil-k-aji-dev]

Fixes #2050

The CUDA base image is pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtime, so it ships cuDNN 9. CTranslate2 — WhisperX and faster-whisper — links cuDNN 8, and per the note in backend/core/cudnn8.py its absence aborts the backend process outright rather than raising (#1371).

scripts/setup.py side-loads the cuDNN 8 libraries for source installs, but the Dockerfile never did. Docker users therefore lose every CTranslate2 ASR engine, which surfaces as the demo synthesis timing out and:

WARNING [core.cudnn8] CTranslate2 ASR engines unavailable: CUDA is active but
libcudnn_ops_infer.so.8 cannot be loaded

Change

Install the same nvidia-cudnn-cu12==8.9.7.29 shim during the image build.

Two details worth flagging:

The target is derived from sys.prefix, not hardcoded. backend/core/cudnn8.py::compat_dirs() searches <sys.prefix>/lib/pythonX.Y/site-packages/cudnn8_compat/nvidia/cudnn/lib, and sys.prefix differs between the conda-based CUDA image and the ROCm venv. Deriving it keeps the install and the lookup in agreement if the base image's Python version or layout moves. I verified the computed target is exactly the parent of the directory compat_dirs() looks in.

A post-install assert fails the build if no .so.8 libraries landed. Without it, a silently empty install would reappear as this same runtime warning, which is what made the original report hard to place.

Guarded to GPU_FLAVOR=cuda — ROCm does not use cuDNN — and --no-deps keeps the install to the cuDNN wheels alone, leaving the base image's torch stack untouched, consistent with the existing torch-clobber guard directly above.

Quality gates

• Cross-platform parity: no change to default behavior on any desktop platform; this only affects the CUDA container build, and the ROCm path is explicitly skipped.
• Engine back-compat: additive. Already-installed engines need no reinstall or re-download — engines that were previously unusable in Docker start working.
• i18n / DB schema / security posture: not touched, no user-facing strings, no migration, no new HTTP surface.
• Local-first: a build-time package install from PyPI, the same mechanism as the existing uv pip install steps in this Dockerfile. No new runtime outbound calls.

Testing

docker build --check -f deploy/Dockerfile . passes — the only warning is InvalidBaseImagePlatform, because I am on arm64 and the base image is amd64.

What I could not run: I could not build the image or run uv run pytest backend/ -x -q on this machine — it is an arm64 Raspberry Pi, and the CUDA base image is amd64-only. So the install step itself is unexercised here and I would rely on CI for it. What I did verify directly is the path logic, since that is the part most likely to be wrong:

Dockerfile target : <sys.prefix>/lib/pythonX.Y/site-packages/cudnn8_compat
compat_dirs()     : <sys.prefix>/lib/pythonX.Y/site-packages/cudnn8_compat/nvidia/cudnn/lib
target is the parent of the searched lib dir: True

The pinned version matches the one scripts/setup.py already installs, so it is not a new dependency choice.

The reporter in #2050 confirmed a hardcoded variant of this fix works in a derived image; this PR generalises it and adds the verification step.

The CUDA Docker image now installs and verifies nvidia-cudnn-cu12==8.9.7.29 in the cudnn8_compat path required by CTranslate2-based WhisperX and faster-whisper engines, with documentation and regression coverage added. The install runs only for GPU_FLAVOR=cuda and uses --no-deps. Full image builds and backend tests were not run because the available machine is arm64 while the CUDA image is amd64.

[greptile-apps]

No actionable merge-blocking issue was identified.

Summary

Installs isolated cuDNN 8 compatibility libraries in CUDA containers, with a build-time presence check.

• Adds regression coverage for CUDA installation, ROCm exclusion, and empty-install failure.
• Updates Docker documentation and the changelog.
• Full image builds and runtime GPU inference were not validated in this review.

_{Reviews (2) · Last reviewed commit: "fix: integrate current main and finish r..."}

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 1b6799f8-0974-43f6-aed9-c17cd987968c

📥 Commits

Reviewing files that changed from the base of the PR and between a628ba1 and 4273f81.

📒 Files selected for processing (3)

• CHANGELOG.md
• docs/install/docker.md
• tests/test_docker_cudnn8_compat.py

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

The Docker runtime stage installs pinned cuDNN 8 compatibility libraries for CUDA images. It derives the target site-packages path and verifies the installed .so.8 files. ROCm images skip this step. Tests, documentation, and the changelog cover the behavior.

Changes

CUDA cuDNN compatibility

Layer / File(s)	Summary
Runtime cuDNN installation, verification, and validation deploy/Dockerfile, tests/test_docker_cudnn8_compat.py	The CUDA runtime stage installs nvidia-cudnn-cu12==8.9.7.29 into cudnn8_compat without dependencies and verifies the .so.8 libraries. Tests validate CUDA installation, ROCm exclusion, target isolation, and failure for an empty target.
Installation documentation and release note docs/install/docker.md, CHANGELOG.md	The Docker documentation describes the isolated cuDNN 8 compatibility directory and the ROCm exclusion. The unreleased changelog records the CUDA container fix.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: debpalash

Merge Risk: ⚪ Minimal · up to 4273f

CUDA images gain the required cuDNN 8 compatibility libraries while ROCm images remain unaffected. No actionable merge risk is identified from the supplied evidence.

🚥 Pre-merge checks | ✅ 7 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (2 skipped: 2 …	Write docstrings for the functions missing them to satisfy the coverage threshold.
Local-First Guarantee	⚠️ Warning	The CUDA image build adds a required outbound PyPI dependency fetch at deploy/Dockerfile:117-120: uv pip install ... nvidia-cudnn-cu12==8.9.7.29 has no offline or local-artifact source. This viola…	Supply the cuDNN 8 compatibility libraries from a vendored or otherwise locally available build artifact, or use a base-image artifact that already contains them. The CUDA image build must not require a PyPI request.

✅ Passed checks (7 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title uses Conventional Commit format with the required scope (fix(docker): ...). The issue reference appears in the body as Fixes #2050``.
Description check	✅ Passed	The description is detailed and covers the change, rationale, issue impact, testing, limitations, and release considerations. It does not use every template heading or include checklist selections, bu…
Linked Issues check	✅ Passed	Issue #2050 requires cuDNN 8 compatibility libraries in the CUDA Docker image. deploy/Dockerfile installs nvidia-cudnn-cu12==8.9.7.29 into the sys.prefix-derived cudnn8_compat path only for `G…
Out of Scope Changes check	✅ Passed	The Dockerfile change, focused test, Docker troubleshooting entry, and changelog entry all support the cuDNN 8 compatibility objective in issue #2050. No unrelated change is established by the reviewe…
Cross-Platform Default Parity	✅ Passed	PASS. The PR changes only the Linux Docker image build in deploy/Dockerfile, where the install runs under the existing GPU_FLAVOR=cuda guard. It does not change native macOS, Windows, or Linux des…
I18n Completeness (21 Locales)	✅ Passed	No frontend files changed in the reviewed range. The patch adds no frontend t('...') keys and no frontend hardcoded user-facing strings. The repository contains 21 locale files, but no locale comparis…
Backward Compatibility	✅ Passed	The pull request only adds an isolated cuDNN 8 library install under the interpreter site-packages path. It does not modify omnivoice_data, database schema, migrations, engine configuration, or mode…

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (2 skipped: 2 unsupported.)

Full details: Local-First Guarantee

Explanation

The CUDA image build adds a required outbound PyPI dependency fetch at deploy/Dockerfile:117-120: uv pip install ... nvidia-cudnn-cu12==8.9.7.29 has no offline or local-artifact source. This violates the check because outbound traffic is limited to opt-in GitHub Issues reporting and HuggingFace model downloads; no accounts, API keys, or telemetry were added.

• Fix all pre-merge checks with AI

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[basil-k-aji-dev]

Thanks — title updated to fix(docker): ....

On the local-first warning, I think it is being applied to the wrong layer, but I would rather lay out the reasoning than quietly dismiss it.

The gate in CONTRIBUTING reads:

Local-first: no new outbound calls except GitHub Issues (opt-in reporting) and HuggingFace model downloads. Never log or persist secrets or absolute home paths.

That sits among runtime properties — logging, persisted secrets, home paths — so I read it as constraining what the running application does on a user's machine, not what the image build fetches. This change adds nothing at runtime; it only puts files into the image.

At build time the Dockerfile already reaches the network in four places:

• deploy/Dockerfile:20 — bun install --frozen-lockfile (npm registry)
• deploy/Dockerfile:48 — apt-get install (Debian archives)
• deploy/Dockerfile:64 — python3 -m pip install uv (PyPI)
• deploy/Dockerfile:90 — uv pip install --constraint deploy/torch-constraints.txt . (PyPI)

My line 119 is the same mechanism as 64 and 90, pinned to the exact version scripts/setup.py already installs, so it is not a new outbound channel or a new dependency decision — it is the step that was missing relative to the source install.

Vendoring the wheel as the bot suggests would mean carrying ~700 MB of platform-specific CUDA binaries in-tree, which seems a worse trade than one pinned install alongside the four that already run.

Happy to be overruled — if you would prefer it vendored, sourced from a prebuilt artifact, or made opt-in behind a build arg, say which and I will rework it.