Skip to content

A local model's download resumes where it stopped, instead of starting over - #189

Merged
shinyobjectz merged 1 commit into
mainfrom
resume-downloads
Nov 19, 2026
Merged

shinyobjectz merged 1 commit into
mainfrom
resume-downloads

Conversation

@shinyobjectz

Copy link
Copy Markdown
Owner

Found while testing #188 on Windows: SAO's setup failed 110 MB into the 1.4 GB DiT graph with curl's error 56 (schannel: server closed abruptly). --retry doesn't retry that error, and fetch() deleted the .part, so every rerun of a 2 GB install started again from zero. On a flaky connection, that install might never finish.

  • The .part is now kept across tries and runs, and resumed: curl -C -, or a Range request in the urllib fallback.
  • Each run makes up to 8 tries, with a short pause between them.
  • A file that's already whole (the server answers 416) counts as done.
  • The SHA-256 check still runs on the finished file; a mismatch deletes it.

Tested against Hugging Face with t5-base's tokenizer.json, on the Mac (curl 8.7) and on Windows (curl 8.21), through both curl and urllib: a 300 KB head resumed to the whole file with a matching SHA-256, and a second call on the whole file left it unchanged. A live check is running on Windows: SAO's setup rerun on the 110 MB .part that the failure left. Gates pass: size, dupes, layers, names, one-version, no-void, taxonomy and rask check.

🤖 Generated with Claude Code

…g over

On Windows (11-18) SAO's setup failed 110 MB into the 1.4 GB DiT graph: curl's error 56 (schannel: server closed
abruptly), which --retry doesn't retry, and fetch() threw the .part away, so every rerun began again from zero. Now
the .part is kept across tries and runs and resumed (curl -C -, or a Range request in the urllib fallback), with up to
8 tries a run; a file already whole (416) counts as done, and the SHA-256 check still runs on the finished file.

Tested against Hugging Face with t5-base's tokenizer.json, on the Mac and on Windows, curl and urllib: a 300 KB head
resumed to the whole file (SHA-256 matching), and a second call on the whole file left it as it was.
@shinyobjectz
shinyobjectz merged commit 1039099 into main Nov 19, 2026
4 checks passed
rosaldo pushed a commit to rosaldo/raea that referenced this pull request Nov 19, 2026
…first time one needs it

On Windows raea's own decoders read H.264 without B-frames (openh264) and AV1, not what phones and cameras write:
H.264 with B-frames, HEVC, VP9. With no FFmpeg installed (as on nearly every Windows PC), raea media probe, raea
check and a render's <Video> refused such a clip ("no picture or sound raea can read"). Found testing 0.1.8-rc6's
raea.exe on Windows with orbit.mp4 (H.264 High, B-frames).

- raea_io_assets::ffmpeg_tool: ffmpeg or ffprobe from the PATH, else raea's own in $RAEA_TOOLS_CACHE/ffmpeg/bin.
  On Windows raea's own is fetched there once: GyanD's essentials build 9.0.2 (the one winget's
  Gyan.FFmpeg.Essentials installs) from its GitHub release, 114 MB, checked against GitHub's SHA-256 digest
  for it, unpacked by Windows' tar (ffmpeg.exe, ffprobe.exe and the LICENSE, ~210 MB). Nothing is bundled.
  Off Windows the error names the package manager's command.
- The fetch shares uv's code (fetch_archive, from fetch_uv), which now resumes across curl's tries
  (-C - --retry-all-errors, as shinyobjectz#189 does for the models).
- The ffmpeg backend runs that binary everywhere. Only opening a file named as a video fetches; sound,
  encoding and asking what FFmpeg can do never do, so a silent clip tried on every backend, or an mp3, never
  downloads 114 MB.
- raea-app-media opens a video through every backend in order (raea_io_media::preferred), as renders already
  did, instead of the platform's alone: media probe, the change and picture commands, and raea check's
  native reader. probe's refusal now says what each backend said.

Checked on the Mac with no FFmpeg on PATH and the portable backend first: B-frame H.264 and HEVC probe
through VideoToolbox (they were refused before), and VP9 is refused with each backend's reason, ending in the
install line; VP9 probes once FFmpeg is on PATH. The zip's layout and extraction were checked with bsdtar.
The Windows fetch itself is proved on the release's raea.exe on Windows. Tests: raea-io-assets, raea-io-ffmpeg,
raea-app-media and raea-app-check-read (parity against FFmpeg) pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant