Skip to content

Handle BucketAlreadyExists races and bump arsenal for MPU afterEach fix#6222

Open
benzekrimaha wants to merge 4 commits into
development/9.3from
improvement/CLDSRV-919-afterEach-flakiness-internal-error-fix
Open

Handle BucketAlreadyExists races and bump arsenal for MPU afterEach fix#6222
benzekrimaha wants to merge 4 commits into
development/9.3from
improvement/CLDSRV-919-afterEach-flakiness-internal-error-fix

Conversation

@benzekrimaha

@benzekrimaha benzekrimaha commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix MPU shadow-bucket creation races in cloudserver callers and bump arsenal to 8.4.20 (ARSN-613).

ARSN-613 makes MongoClientInterface.createBucket() return BucketAlreadyExists on concurrent duplicates instead of silent idempotence. Cloudserver must handle that contract:

  • freshStartCreateBucket: continue addToUsersBucket / transient cleanup when the metastore entry already exists.
  • getMPUBucket: on BucketAlreadyExists, fetch and return the existing MPU shadow bucket.

This fixes the intermittent afterEach failure in mpuVersion.js ("We encountered an internal error") when ListMultipartUploads races on MPU shadow bucket creation during test cleanup.

Paired with Arsenal PR #2672 (ARSN-613).

Issue: CLDSRV-919

@bert-e

bert-e commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Hello benzekrimaha,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request TBA
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@bert-e

bert-e commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Incorrect fix version

The Fix Version/s in issue CLDSRV-919 contains:

  • None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

  • 9.4.0

Please check the Fix Version/s of CLDSRV-919, or the target
branch of this pull request.

@benzekrimaha benzekrimaha force-pushed the improvement/CLDSRV-919-afterEach-flakiness-internal-error-fix branch 2 times, most recently from ab63d85 to 2cae9bc Compare July 8, 2026 15:31
@benzekrimaha benzekrimaha changed the base branch from development/9.4 to development/9.3 July 8, 2026 15:31
@benzekrimaha benzekrimaha changed the title CLDSRV-919: bump arsenal to 8.5.7 for MPU afterEach flake fix Bump arsenal with MPU afterEach flakiness fix Jul 8, 2026
@scality scality deleted a comment from bert-e Jul 8, 2026
@scality scality deleted a comment from bert-e Jul 8, 2026
@bert-e

bert-e commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

@scality scality deleted a comment from bert-e Jul 8, 2026
@bert-e

bert-e commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

The following options are set: create_integration_branches

@bert-e

bert-e commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

The following options are set: create_pull_requests, create_integration_branches

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.94%. Comparing base (055ae1b) to head (bda521d).
✅ All tests successful. No failed tests found.

Additional details and impacted files

Impacted file tree graph

Files with missing lines Coverage Δ
lib/api/apiUtils/bucket/bucketCreation.js 93.49% <100.00%> (+0.10%) ⬆️
lib/services.js 87.63% <100.00%> (+0.19%) ⬆️

... and 2 files with indirect coverage changes

@@               Coverage Diff                @@
##           development/9.3    #6222   +/-   ##
================================================
  Coverage            84.93%   84.94%           
================================================
  Files                  206      206           
  Lines                13366    13374    +8     
================================================
+ Hits                 11353    11360    +7     
- Misses                2013     2014    +1     
Flag Coverage Δ
file-ft-tests-null-compat 62.96% <40.00%> (-5.95%) ⬇️
kmip-ft-tests 28.28% <40.00%> (-0.01%) ⬇️
mongo-v0-ft-tests 69.59% <70.00%> (-0.04%) ⬇️
mongo-v1-ft-tests 69.60% <70.00%> (-0.02%) ⬇️
multiple-backend 36.75% <30.00%> (-0.02%) ⬇️
s3c-ft-tests-v0 64.09% <40.00%> (-0.03%) ⬇️
s3c-ft-tests-v0-null-compat 64.15% <40.00%> (-0.03%) ⬇️
s3c-ft-tests-v1 64.07% <40.00%> (-0.03%) ⬇️
sur-tests 35.81% <40.00%> (-0.83%) ⬇️
sur-tests-inflights 37.65% <40.00%> (-0.02%) ⬇️
unit 70.52% <100.00%> (+0.01%) ⬆️
utapi-v2-tests 34.54% <40.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@benzekrimaha benzekrimaha requested a review from tcarmet July 8, 2026 15:39
ARSN-613 makes Mongo createBucket return BucketAlreadyExists instead of
silent idempotence. Continue bucket creation follow-up when a concurrent
call already created the metastore entry (freshStartCreateBucket), and
fetch the existing MPU shadow bucket when getMPUBucket races.

Issue: CLDSRV-919
@benzekrimaha benzekrimaha force-pushed the improvement/CLDSRV-919-afterEach-flakiness-internal-error-fix branch from 2cae9bc to 6178ad8 Compare July 9, 2026 15:30
@benzekrimaha benzekrimaha changed the title Bump arsenal with MPU afterEach flakiness fix Handle BucketAlreadyExists races and bump arsenal for MPU afterEach fix Jul 9, 2026
@scality scality deleted a comment from bert-e Jul 9, 2026
@scality scality deleted a comment from bert-e Jul 9, 2026
@scality scality deleted a comment from bert-e Jul 9, 2026
@scality scality deleted a comment from bert-e Jul 9, 2026
@scality scality deleted a comment from bert-e Jul 9, 2026
@scality scality deleted a comment from bert-e Jul 9, 2026
@bert-e

bert-e commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

Pull in ARSN-613 (paired with this cloudserver change) so MPU shadow
bucket creation returns BucketAlreadyExists on concurrent races.

Issue: CLDSRV-919
@benzekrimaha benzekrimaha force-pushed the improvement/CLDSRV-919-afterEach-flakiness-internal-error-fix branch from 71b7d71 to 0a09689 Compare July 9, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants