Skip to content

fix: stream ZIP uploads into isolated temporary files - #2

Open
harshitethic wants to merge 4 commits into
mainfrom
fix/unique-upload-tempfiles
Open

harshitethic wants to merge 4 commits into
mainfrom
fix/unique-upload-tempfiles

Conversation

@harshitethic

Copy link
Copy Markdown
Owner

Problem

ZIP uploads were buffered fully in memory and written to a predictable temporary path derived from the uploaded filename:

/tmp/repopilot-{filename}

Two concurrent uploads using the same filename could therefore write to the same temp file. One request could overwrite or delete the file while the other request was still processing it.

The endpoint also allocated up to ~50 MB per upload in one Python bytes object before writing anything to disk, which scales poorly with concurrent requests.

Fix

  • use NamedTemporaryFile(delete=False) so every request receives a unique temp path;
  • stream UploadFile into the temp file in 1 MiB chunks instead of buffering the entire archive;
  • enforce the existing 50 MB cap while streaming;
  • delete partially written temp files when streaming fails;
  • preserve the existing final cleanup after successful extraction/analysis.

Tests

Adds coverage for:

  • two uploads receiving distinct temp files even with the same logical filename;
  • the streamed bytes being preserved correctly;
  • oversized uploads raising the expected validation error;
  • partially written temp files being removed after an oversized upload is rejected.

This keeps the public upload API unchanged while making concurrent uploads deterministic and reducing per-request memory pressure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant