Repository navigation
fix(rest): keep table tokens off the catalog session - #3976
jaideeppyne wants to merge 1 commit into
Conversation
commit_table wrote a table-scoped Authorization header into the live session mapping, so later requests from the same RestCatalog carried it. Send that token on the commit request only. Also skip fsspec's S3FileSystem instance cache when a custom signer is registered, so two catalogs cannot overwrite each other's before-sign handler. Closes apache#3970
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
This is still relevant and up to date with main. I'd appreciate a review whenever someone has time, thanks! |
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
Rationale for this change
RestCatalog.commit_tableassigned intoself._session.headers, which is the session's own mapping. A table-scoped token then stuck on the catalog session for later requests.Session.auth would also overwrite a per-request
Authorizationheader with the catalog token, so the table token is applied through a request-local auth callable and the session headers are left unchanged.A second, related leak: fsspec caches
S3FileSysteminstances by constructor kwargs. Custom S3 signers are registered after construction under a fixed event id, so a second catalog with the same client kwargs replaced the first catalog's signer.skip_instance_cacheis set when a signer is in use.Closes #3970
Are these changes tested?
Unit tests in
tests/catalog/test_rest.pyandtests/io/test_fsspec.py.Are there any user-facing changes?
Bug fix. Catalog sessions no longer retain a table-scoped token after
commit_table. FileIO instances with custom S3 signers are no longer shared across catalogs via fsspec's instance cache.