Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Keep Linux shell scripts and container files on Unix line endings even when cloned on Windows.
*.sh text eol=lf
Dockerfile* text eol=lf
*.yml text eol=lf
*.yaml text eol=lf
35 changes: 27 additions & 8 deletions .github/workflows/gpu-static-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ name: CPU/GPU direct-runner static check
on:
pull_request:
paths:
- ".gitattributes"
- "Dockerfile.cpu"
- "Dockerfile.gpu"
- "docker-compose.yml"
Expand All @@ -18,6 +19,7 @@ on:
push:
branches: [main]
paths:
- ".gitattributes"
- "Dockerfile.cpu"
- "Dockerfile.gpu"
- "docker-compose.yml"
Expand Down Expand Up @@ -77,25 +79,42 @@ jobs:
cmp mock-release/lfm_aux.pt mock-downloaded/lfm_aux.pt
cmp mock-release/lfm_dspark.pt mock-downloaded/lfm_dspark.pt

- name: Validate GPU shell entrypoint
run: bash -n scripts/test_gpu.sh
- name: Validate Linux shell entrypoint and line endings
run: |
bash -n scripts/test_gpu.sh
python - <<'PY'
from pathlib import Path
data = Path('scripts/test_gpu.sh').read_bytes()
assert b'\r\n' not in data, 'scripts/test_gpu.sh must be LF, not CRLF'
assert data.startswith(b'#!/usr/bin/env bash\n')
PY
grep -Fq '*.sh text eol=lf' .gitattributes
grep -Fq "sed -i 's/\\r$//' /app/scripts/test_gpu.sh" Dockerfile.gpu

- name: Validate Docker Compose CPU and GPU services
run: |
docker compose --profile cpu --profile gpu config -q
docker compose --profile cpu config --services | grep -qx test-cpu
docker compose --profile gpu config --services | grep -qx test-gpu
docker compose --profile cpu config | grep -Fq 'image: dflash-mini-lab-test-cpu:latest'
docker compose --profile gpu config | grep -Fq 'image: dflash-mini-lab-test-gpu:latest'

- name: Verify zero-install Docker paths
run: |
grep -q 'torch torchvision' Dockerfile.cpu
grep -q 'download.pytorch.org/whl/cpu' Dockerfile.cpu
grep -q 'COPY run_gpu.py run_cpu.py' Dockerfile.cpu
# Keep the known-working Dockerfile installation style. Do not require
# version pins or a dedicated CPU wheel index for the CPU image.
grep -Fq 'python -m pip install torch torchvision' Dockerfile.cpu
Comment on lines +104 to +106

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore the CPU-only wheel assertion

The updated check now explicitly accepts pip install torch torchvision without the CPU wheel index, even though the CPU Compose path is documented as using CPU-only PyTorch (README.md:67). On Linux, the default PyPI distribution can pull CUDA-enabled PyTorch and its NVIDIA dependencies, substantially bloating the CPU image while this workflow remains green. Keep asserting the /whl/cpu index, and update Dockerfile.cpu accordingly rather than weakening the validation.

Useful? React with 👍 / 👎.

grep -Fq 'python -m pip install numpy transformers' Dockerfile.cpu
grep -Fq 'COPY run_gpu.py run_cpu.py ./' Dockerfile.cpu
! grep -q 'pip install \.' Dockerfile.cpu
grep -q 'torch torchvision' Dockerfile.gpu
grep -q 'download.pytorch.org/whl/cu132' Dockerfile.gpu
grep -q 'COPY run_gpu.py' Dockerfile.gpu

# GPU keeps the same style but must use the CUDA 13.2 wheel index.
grep -Fq 'python -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cu132' Dockerfile.gpu
grep -Fq 'python -m pip install numpy transformers' Dockerfile.gpu
grep -Fq 'COPY run_gpu.py ./run_gpu.py' Dockerfile.gpu
! grep -Fq 'pip install install' Dockerfile.gpu
! grep -q 'pip install \.' Dockerfile.gpu

grep -q 'gpus: all' docker-compose.yml
grep -q 'test-cpu:' docker-compose.yml
grep -q 'test-gpu:' docker-compose.yml
Expand Down
8 changes: 6 additions & 2 deletions Dockerfile.gpu
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,18 @@ ENV PYTHONDONTWRITEBYTECODE=1 \
WORKDIR /app

RUN python -m pip install --upgrade pip \
&& python -m pip install install torch torchvision --index-url https://download.pytorch.org/whl/cu132 \
&& python -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cu132 \
&& python -m pip install numpy transformers

COPY src ./src
COPY real_benchmarks ./real_benchmarks
COPY scripts ./scripts
COPY run_gpu.py ./run_gpu.py
RUN chmod +x /app/scripts/test_gpu.sh

# Windows checkouts can provide CRLF shell scripts. Normalize inside the Linux image
# as a second line of defense in addition to .gitattributes.
RUN sed -i 's/\r$//' /app/scripts/test_gpu.sh \
&& chmod +x /app/scripts/test_gpu.sh

RUN mkdir -p /cache /app/lfm-artifacts /app/gpu-reports

Expand Down
2 changes: 2 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ services:

test-cpu:
profiles: ["cpu"]
image: dflash-mini-lab-test-cpu:latest
build:
context: .
dockerfile: Dockerfile.cpu
Expand Down Expand Up @@ -41,6 +42,7 @@ services:

test-gpu:
profiles: ["gpu"]
image: dflash-mini-lab-test-gpu:latest
build:
context: .
dockerfile: Dockerfile.gpu
Expand Down