Fix pip.installed state managed pre-release upgrades

[kurtrwall]

What does this PR do?

What issues does this PR fix or reference?

Fixes #68525

Previous Behavior

Python packages installed in an environment would not upgrade to their pre-release version when any exist in the index.

New Behavior

Pre-release versions of packages managed by pip.installed state are upgraded when using upgrade=True and pre_releases=True.

Merge requirements satisfied?

[NOTICE] Bug fixes or features added to Salt require tests.

• Docs
• Changelog - https://docs.saltproject.io/en/master/topics/development/changelog.html
• Tests written/updated

Commits signed with GPG?

No

[kurtrwall]

I've been able to trace the bug back to the inception of the code that uses pip index versions, so I based the PR on the v3004.x branch, but I'm unsure if that was the appropriate one.

I'm also having a bit of trouble with the tests, but I didn't want it to block the contribution.

[twangboy]

We do bug fixes on the olders supported branch where the bug exists. There will be no more releases of 3004 and 3005. The oldest supported branch is 3006.x. Please rebase this PR on 3006.x.
Also, please create a changelog for this PR.
https://docs.saltproject.io/en/3006/topics/development/changelog.html

[kurtrwall]

Rebased to 3006.x and refactored the test a bit to utililze the new fixtures and provide more coverage.

[kurtrwall]

Hey all, thanks for the quick review of this PR.

Out of curiousity, how does the release flow work? Does the bugfix get released to all major versions, i.e. 3006.+1, 3007.+1?

[twangboy]

Looks like you still have pre-commit failiures.

[kurtrwall]

I'm an on-prem gitlab user by trade, so I'm unsure of how these checks can be run/verified on my end. I think they've been corrected based on my testing, but apologies if my ignorance has prolonged this.

Does this look okay now? I've followed the contributing documentation, but if there's something else I should be doing to verify these changes so we can get this into a release, I'm all ears.

Thanks.

[twangboy]

Just install pre-commit (pip install pre-commit)
Add your changed files (git add .)
And run pre-commit (pre-commit)

[kurtrwall]

Okay, sounds like I had the right setup for the most recent push. Glad to see it's approved! Thank you.

[twangboy]

still need to see passing tests

[kurtrwall]

Looking at the error logs from the tests, I could see that the versions returned didn't include any prerelease versions at all, leading me to believe that the reliance on PyPI's typing-extensions package versions is a bigger problem than I previously thought.

My latest commit mocks the specific invocations of __salt__["cmd.run_all"] that get package versions to remove the reliance on PyPI. That said, it does complicate the testing code more, albeit for (hopefully) passing tests.

A better implementation would be only patch __salt__["cmd.run_all"] for the specific test, but given the guidance around the configure_loader_modules() fixure, I wasn't sure how to actually accomplish that (not for lack of trying).

All said, I believe this could be acceptable, hence the submission. However, if there is a way to constrain the mocking to the test function instead of the entire module, I'd love to refactor to clean it up.

Thanks!

[kurtrwall]

It seems like the only failing test was due to a timeout, is there a way to rerun just the one test?

[kurtrwall]

Thanks for the approval @twangboy! I'm sure we'd like to get this merged and on the way to release as quickly as possible. I do notice there are still some checks failing, so I just wanted to offer I'm ready to help get things resolved. That said, I'll be unavailable starting next week until June. Thanks again!