Repository navigation
Fix: mulder state failures on CI (jammy PPA, home path, root setup) - #245
digitalisx wants to merge 4 commits into
Conversation
|
Hello @calebevans, |
add-apt-repository on Ubuntu 22.04 silently writes an empty PPA .list file when /etc/apt/sources.list is missing, so PPA packages such as dotnet-sdk-9.0 (ppa:dotnet/backports) cannot be located afterwards. Co-authored-by: ParkJeongKyun <44119972+ParkJeongKyun@users.noreply.github.com> Co-authored-by: KIM EUN WOO <148983524+rlo-lo@users.noreply.github.com> Co-authored-by: Ryu_harudia <145317670+Ryu-jae-kwong@users.noreply.github.com> Co-authored-by: Donggyun Jo <90193658+Refrain4@users.noreply.github.com>
The setup and ensurepath steps hardcoded /home/<user>/.local/bin, which does not exist for root (/root) or users with non-default homes. Co-authored-by: ParkJeongKyun <44119972+ParkJeongKyun@users.noreply.github.com> Co-authored-by: KIM EUN WOO <148983524+rlo-lo@users.noreply.github.com> Co-authored-by: Ryu_harudia <145317670+Ryu-jae-kwong@users.noreply.github.com> Co-authored-by: Donggyun Jo <90193658+Refrain4@users.noreply.github.com>
pipx installs into ~/.local/bin, which is not on PATH for the salt cmd.run environment, so 'mulder setup' failed with 'not found'. Co-authored-by: ParkJeongKyun <44119972+ParkJeongKyun@users.noreply.github.com> Co-authored-by: KIM EUN WOO <148983524+rlo-lo@users.noreply.github.com> Co-authored-by: Ryu_harudia <145317670+Ryu-jae-kwong@users.noreply.github.com> Co-authored-by: Donggyun Jo <90193658+Refrain4@users.noreply.github.com>
mulder refuses to run 'setup' as root by design, so applying the state with sift_user=root (as CI does) always failed. Co-authored-by: ParkJeongKyun <44119972+ParkJeongKyun@users.noreply.github.com> Co-authored-by: KIM EUN WOO <148983524+rlo-lo@users.noreply.github.com> Co-authored-by: Ryu_harudia <145317670+Ryu-jae-kwong@users.noreply.github.com> Co-authored-by: Donggyun Jo <90193658+Refrain4@users.noreply.github.com>
77423e9 to
ddb659b
Compare
|
@digitalisx thank you! I don't have any permissions in this repo, but I will follow it and reach out to someone if needed. |
|
I prematurely merged in a state we only had done limited testing on. Thank you for submitting this. It is much appreciated and love to see others contributing. We'll get this reviewed as soon as we can. @digitalsleuth can you take a quick look? I have tests running now. |
|
Will do, just getting in the door and will start shortly. |
|
Hi @calebevans , did you happen to come across an error which spawned this conversation? I want to make sure I'm reviewing and addressing all of the points of interest before providing a potential solution, which may or may not address your issue. |
|
Hello @digitalsleuth, This issue and PR raised by me https://github.com/teamdfir/sift-saltstack/actions/runs/39260879739/job/118087938739 After investigating the root cause, I've submitted a fix (PR #245) that also prevents the same class of failure from recurring. |
|
Hi @digitalisx , Thanks for the heads up. The issues you're experiencing only occur during the CI tests, however don't occur in the actual SIFT environment. The root cause for these issues is that, especially when testing only on the As the When properly refreshed, the The The problem with All in all, this will fix the issues appearing during CI tests, and any issues during installation. I'm working on testing the fixes across both arm64 and amd64 architectures, as well as from 22-26.04 versions of Ubuntu. |
|
Thank you for reply @digitalsleuth, |
|
There's nothing to do right now, as I'm in the middle of testing some changes, but I'll keep you posted once the tests are finished. Thanks! |
|
Just as a heads up, the Ubuntu PPA launchpadcontent.net seems to be experiencing some network issues (returning 503 errors), which are preventing my tests from being successful. I'm still monitoring the situation, but I believe I have a viable fix for the issues identified here. More to follow. |
|
So the issues appear to be stemming from overload, due to the recently identified / disclosed PHP vulnerabilities and people updating their packages. I'll try my testing again later when it should be less busy. |
|
Hi @digitalisx , while investigating the issue you reported, I also identified a few other areas which needed addressing and, while your changes were effective, the changes which needed to be addressed were updated during my testing. Because of the additional changes, I've created a new PR #247 which encompasses the changes you were looking to make, though in a different way. |
|
Thank you for effort @digitalsleuth, |
|
#247 has been merged, closing this PR. Thanks again for bringing this to our attention! |
Summary
Fixes the
sift.python3-packages.mulderCI failure from #241 (e.g. run 39260879739). Fixing the first problem exposed three more, so this PR fixes all four:dotnet-sdk-9.0not found on 22.04. On jammy,ubuntu-universe/ubuntu-multiversedelete/etc/apt/sources.list. When that file is missing, jammy'sadd-apt-repositorystill printsAdding deb entry to .../dotnet-ubuntu-backports-jammy.list, but the.listfile it leaves is empty (0 bytes) andsources.listgets only the header comment. So the PPA is never really added. A 2x2 probe (add-apt-repository vs saltpkgrepo.managed, with and withoutsources.list) reproduces it with plainadd-apt-repositorytoo, so Salt is not the cause. The fix is to keep an empty (comment-only)/etc/apt/sources.listinstead of removing it./home/<user>/.local/bincwd. This path does not exist forroot(CI) or for users whose home is somewhere else. The fix resolves the home directory the same waysift/config/user/*.slsalready does.mulder: not found.cwddoes not put~/.local/binonPATH, so the fix calls the pipx-installed binary by its full path.mulder setuprefuses to run as root (by design). The fix skips the setup step whensift_userisroot;pipx ensurepathstill runs.Testing
tests.ymlon the fork: allmulder,ubuntu-universeandubuntu-multiversejobs pass on 22.04/24.04 × Salt 3006/3007.sift.python3-packages.mulderwithsift_user=rootandsansforensicson 22.04/24.04 (Salt 3007): 0 failures. The non-root runs include a successfulmulder setup --yes.Notes
Coauthored: @ParkJeongKyun @rlo-lo @Ryu-jae-kwong @Refrain4
This work was supported by the Best of the Best (BoB) Program,
a cybersecurity talent development program operated by the
Korea Internet & Security Agency (KISA).