[ci] Retry Ollama install and propagate download failures - #891
Conversation
|
The 2 failing checks are unrelated to this change — the Ollama install step this PR touches actually succeeds in both failing jobs (
This PR hardens the install step against transient download 404s; it doesn't address the |
The upstream Ollama installer probes for the .tar.zst release asset with a silent HEAD request and, on any failure of that probe, falls back to a .tgz that current releases no longer publish. A single transient network error therefore produces a hard 404 and fails the integration-test jobs before any test runs. Retry the install up to three times with a linear backoff. Reading the exit status after "curl ... | sh" also captured the status of sh rather than curl, so a failed download left sh reading empty stdin and exiting zero. The install step then "succeeded" with no Ollama present. Download the installer to a file and check curl directly, under set -euo pipefail.
a929394 to
3aa0ca8
Compare
|
Rebased onto current One correction to the CI triage above: the That leg should therefore be clean on this run. The |
Linked issue: #889
Purpose of change
The
it-java,it-python, andcross-languageCI jobs install a local Ollama server viatools/start_ollama_server.sh, which pipeshttps://ollama.com/install.shintosh. That upstream installer probes for theollama-linux-${ARCH}.tar.zstrelease asset with a silentcurl --fail --silent --head, and on any failure of that probe it falls back to a.tgzasset that current releases no longer publish (.tgzreturns404). A single transient network error on the probe therefore turns into a hard failure, before any test compiles or runs:This retries the install up to three times with a linear backoff, so a transient failure recovers instead of failing the job. The underlying
.tar.zstasset does exist, so a retry is a genuine remedy rather than another roll of the dice.It also fixes a latent bug in the wrapper. The old script read
ret=$?aftercurl -fsSL ... | sh, which captures the exit status ofsh, notcurl. With nopipefail, a failed download leftshreading empty stdin and exiting0, so the step "succeeded" with no Ollama installed and the failure surfaced later somewhere less obvious. The installer is now downloaded to a file andcurlis checked directly, underset -euo pipefail.Tests
Verified locally by exercising both paths against stubbed install targets: the script recovers and exits
0when a transient failure clears before the retries are exhausted, and it exits1after three failed attempts against a persistent404.bash -npasses; the CI lint job exercises the script's shell checks.The full CI matrix on this PR exercises the change end-to-end — every
it-java/it-python/cross-languagejob runs this installer.API
No public API changes.
Documentation
doc-not-needed