archive: extract safe symlinks on old Python#2576
Merged
Merged
Conversation
The daily release-tox-drift job crashed while fetching sources with: source error: unsafe archive member (link/special): osism-release-a2e4875/1.0.0/ceph.yml _safe_extract downloads each source repo as a GitHub tarball and extracts it. When tarfile.data_filter is available (Python 3.11.4+ / 3.12+) it extracts with filter="data", which permits relative symlinks that stay inside the extraction directory. The manual fallback for older Pythons, however, rejected every symlink member outright, so it was stricter than the data filter it was meant to emulate. The Zuul node runs Python 3.11.2, which predates the data_filter backport, so it takes the fallback path. The release repo -- now one of the fetched sources -- contains ~80 legitimate relative symlinks (per-version ceph.yml and openstack.yml aliases), so extraction aborted on the first one and the whole check exited non-zero. Local runs and the release-tox-drift-test job never caught this because their interpreters have data_filter. Make the fallback emulate the data filter: permit symlinks and hardlinks whose target resolves inside the extraction root, and keep rejecting links that escape (absolute or via ..) and any other special member. The final extractall is made version-robust (filter="fully_trusted" where supported, plain extractall otherwise) so it behaves identically on old and new interpreters. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Roger Luethi <luethi@osism.tech>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The daily
release-tox-driftjob (periodic-daily pipeline) has been failing since the release repo was added as a fetched source. It crashes during source fetching, not on detected drift:_safe_extractdownloads each source repo as a GitHub tarball and extracts it. Whentarfile.data_filteris available (Python 3.11.4+ / 3.12+) it extracts withfilter="data", which permits relative symlinks that stay inside the extraction directory. The manual fallback for older Pythons rejected every symlink member outright — stricter than thedatafilter it was meant to emulate.The Zuul node runs Python 3.11.2, which predates the
data_filterbackport, so it takes the fallback path. The release repo contains ~80 legitimate relative symlinks (per-versionceph.yml/openstack.ymlaliases), so extraction aborted on the first one. Local runs andrelease-tox-drift-testnever caught this because their interpreters havedata_filter.Fix
Make the fallback emulate the
datafilter: permit symlinks/hardlinks whose target resolves inside the extraction root, keep rejecting links that escape (absolute or via..) and any other special member. The finalextractallis made version-robust (filter="fully_trusted"where supported, plainextractallotherwise) so it behaves identically on old and new interpreters.Verification
data_filterpath and extracting an archive built from the real release tree now succeeds (6 symlinks resolved) where it previously raised.🤖 Generated with Claude Code