gh-152936: Make privileged functions available on Android

[sabamdarif]

os.chroot,initgroups,setegid,seteuid,setgid.... are currently blocked on Android.
Because the underlying C functions crash the process when called by a non-root user on android, rather than returning an error.
So i added getuid() != 0 check (under #ifdef __ANDROID__) before each call, so it will return PermissionError for non-root just like normal Unix

• Fixes Make privileged functions available on Android #152936

• Issue: Make privileged functions available on Android #152936

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[sabamdarif]

it's my first pr here so i don't know much about the NEWS entry, if it's not a problem can anyone tell me if i do need to add any NEWS entry ?

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #33498151 | 📁 Comparing 27e854c against main (423ae0f)

  🔍 Preview build  

60 files changed · ± 60 modified

± Modified

• deprecations/index.html
• deprecations/pending-removal-in-3.21.html
• extending/first-extension-module.html
• extending/newtypes_tutorial.html
• faq/programming.html
• howto/descriptor.html
• library/ast.html
• library/codecs.html
• library/datetime.html
• library/dbm.html
• and 50 more...

[Aniketsy]

it's my first pr here so i don't know much about the NEWS entry, if it's not a problem can anyone tell me if i do need to add any NEWS entry ?

thanks for the PR, as per my thought yes, this will need NEWS entry but you can wait for the review and then you can add NEWS entry ( to get 100 % sure 😀)

[sabamdarif]

i have another question for the Android test failure should i update the test_posix.py ? to do something like if support.is_android and os.getuid() != 0: so it will not failed on PermissionError ? or should i need to update something on my patch so the existing test pass

[sabamdarif]

why did the Tests / Sanitizers / TSan (free-threading) (pull_request) failed i don't get it ?

[robertkirkman]

We can see that the cause of one of the test failures probably is that the Python test suite has interpreted the unblocking of setresuid() as an indication that it should be usable as non-root user, even though we want to use it as root user and continue to show an error for non-root users;

cpython/Lib/test/test_os/test_posix.py

Lines 99 to 105 in 0621639

	@unittest.skipUnless(hasattr(posix, 'setresuid'),
	'test needs posix.setresuid()')
	def test_setresuid(self):
	current_user_ids = posix.getresuid()
	self.assertIsNone(posix.setresuid(*current_user_ids))
	# -1 means don't change that value.
	self.assertIsNone(posix.setresuid(-1, -1, -1))

I would say that yes, the test should probably be modified to check for Android platform, and in the case of Android platform, only test that function if the test is running as root user;

however, what I find concerning is that the CI here only produced a failure for x86_64, and not for aarch64, even though in my own separate test, both x86_64 and aarch64 architectures of Android should be returning the same result from the same function:

I wonder if that suggests there is a deeper problem with the tests or an inconsistency somewhere in the test implementation between x86_64 and aarch64.

[sabamdarif]

@robertkirkman yes, let see what others are saying about this . I still didn't get any other comment from any of the core maintainers

[robertkirkman]

Skipping tests: GitHub Actions does not support the Android emulator on this platform.

Ah I see, the aarch64 build does not run the test suite in CI, so if it did, it would also have that error.

cpython/Platforms/Android/__main__.py

Lines 844 to 852 in 881a13a

	if (
	"GITHUB_ACTIONS" in os.environ
	and (platform.system(), platform.machine()) != ("Linux", "x86_64")
	):
	print(
	"Skipping tests: GitHub Actions does not support the Android "
	"emulator on this platform."
	)
	else:

It's possible in theory, in some projects, to implement testing aarch64 Android platform in CI, but in this case, based on my understanding of CPython and Android cumulatively so far, in order to implement that in CPython CI, one of the two following blockers would have to be overcome:

1. the current official upstream build.gradle.kts would have to be split in some way so that all portions of it related to Android NDK and all portions of it related to running the managed emulator (technically virtual machine) are in two separate files or at least run separately, on two separate runners - the NDK portion on the GitHub Actions x86 runner and the emulator portion on the GitHub Actions ARM runner.

cpython/Platforms/Android/testbed/app/build.gradle.kts

Lines 119 to 130 in 881a13a

	ndk.abiFilters.addAll(abis.values)
	externalNativeBuild.cmake.arguments(
	"-DPYTHON_PREFIX_DIR=" + if (inSourceTree) {
	// AGP uses the ${} syntax for its own purposes, so use a Jinja style
	// placeholder.
	"$PYTHON_CROSS_DIR/{{triplet}}/prefix"
	} else {
	prefixes[0]
	},
	"-DPYTHON_VERSION=$pythonVersion",
	"-DANDROID_SUPPORT_FLEXIBLE_PAGE_SIZES=ON",
	)

cpython/Platforms/Android/testbed/app/build.gradle.kts

Lines 266 to 270 in 881a13a

	val systemImage = listOf(
	"system-images",
	"android-${apiLevel.get()}",
	systemImageSource.get(),
	abi.get(),

2. Alternatively, another way to do that would be to obtain or build an unofficial build of the Android NDK for running on aarch64 GNU/Linux which could be used to build the testbed and run it in the managed emulator (technically virtual machine) contiguously on the same aarch64 GNU/Linux runner.

both of these blockers arise because Google has never officially released any prebuilt Android NDK archives for the platform aarch64 GNU/Linux. If they do in the future some time, that would change.

I don't know why the aarch64 macos runner that's currently used to build the aarch64 target Android testbed in CI isn't able to run the managed emulator (technically virtual machine), so I've just described this situation from the perspective I know, which is that of what would be required in the case of using entirely GNU/Linux runners for it.

cpython/.github/workflows/build.yml

Lines 337 to 349 in 881a13a

	build-android:
	name: Android (${{ matrix.arch }})
	needs: build-context
	if: needs.build-context.outputs.run-android == 'true'
	timeout-minutes: 60
	strategy:
	fail-fast: false
	matrix:
	include:
	- arch: aarch64
	runs-on: macos-26
	- arch: x86_64
	runs-on: ubuntu-24.04

it is understandable that all of this work is probably not worth it until there is a very important need to run the Android testbed for aarch64 in CI here.

[mhsmith]

It would be very useful to run the tests on aarch64 on GitHub Actions, but the macOS runner is apparently already a virtual machine, so it can't run the emulator.

another way to do that would be to obtain or build an unofficial build of the Android NDK for running on aarch64 GNU/Linux which could be used to build the testbed and run it in the managed emulator (technically virtual machine) contiguously on the same aarch64 GNU/Linux runner.

But the Android emulator isn't available for aarch64 Linux either. From this discussion, it sounds like it is possible to build it from source, but do you know of any downloadable build which is suitable for production use?

[mhsmith]

can anyone tell me if i do need to add any NEWS entry ?

It's a change in behavior, so there should be a NEWS entry in the "Library" section. I think it's enough for it to list the functions and say that they are now available on Android. It doesn't need to go into details of exactly how this was done.

should i update the test_posix.py ? to do something like if support.is_android and os.getuid() != 0: so it will not failed on PermissionError ? or should i need to update something on my patch so the existing test pass

On Linux, apparently the set...id functions all allow an unprivileged process to set any of the IDs to any of the saved, effective or real IDs (i.e. the values returned by getres...id), or in some cases, -1 to leave the ID unchanged. You should test whether this is true on Android as well, and if so, update the code to allow the function to run under those circumstances. Then the test will pass.

[mhsmith]

why did the Tests / Sanitizers / TSan (free-threading) (pull_request) failed i don't get it ?

Those tests are unreliable; you can ignore them for now.

[robertkirkman]

But the Android emulator isn't available for aarch64 Linux either. From beeware/briefcase#449 (comment), it sounds like it is possible to build it from source, but do you know of any downloadable build which is suitable for production use?

yes, that's completely right; not only would an aarch64 build of the Android NDK probably be required, also an aarch64 build of the emulator would be, though the part of the emulator that I believe would need to be compiled is the part that is mostly formed from QEMU, and doesn't take as long to compile as the entire AVD image would take to build (which I think wouldn't be required), nor as long as the Android NDK would take to build. I agree, though, that both a custom emulator QEMU and a custom NDK are probably too complicated and involve too many unofficial steps for them to be worth using for CPython unless a pressing need for running the aarch64 Android tests in CI is found.

On Linux, apparently the set...id functions all allow an unprivileged process to set any of the IDs to any of the saved, effective or real IDs (i.e. the values returned by getres...id), or in some cases, -1 to leave the ID unchanged. You should test whether this is true on Android as well, and if so, update the code to allow the function to run under those circumstances. Then the test will pass.

I am not sure exactly what the matrix of crashing vs not crashing looks like for this beyond "some of the functions here cause crashing when called as non-root user", but I can confirm that not all of the functions appear to be able to crash the program, only certain ones or combinations of arguments with certain ones, since I just now checked these several and saw these results, on Samsung Galaxy S8+ SM-G955F with Android 16:

does not crash

#include <stdio.h>
#include <unistd.h>

int main(void) {
    printf("%d\n", setresuid(-1, -1, -1));
}

does not crash

#include <stdio.h>
#include <unistd.h>

int main(void) {
    uid_t r, e, s;
    getresuid(&r, &e, &s);
    printf("%d\n", setresuid(r, e, s));
}

does not crash

#include <stdio.h>
#include <unistd.h>

int main(void) {
    printf("%d\n", setresuid(0, 0, 0));
}

crashes with Unknown signal 31

#include <stdio.h>
#include <unistd.h>

int main(void) {
    printf("%d\n", setgid(0));
}

crashes with Unknown signal 31

#include <stdio.h>
#include <unistd.h>

int main(void) {
    printf("%d\n", chroot("/"));
}

for this reason, if others see similar results and we assume that all recent Android versions produce the same matrix of whether each function will cause a crash, it does seem like some, maybe including setresuid(), could be entirely unblocked, and then tested here as non-root user, while others such as chroot() and setgid() would need to remain blocked for non-root users, but I haven't tested and am not sure of what decisions should be made for all other functions or all possible combinations of argument values.

[robertkirkman]

I have just finished creating a custom ROM of the Android Studio Android Virtual Device Android Emulator that has the popular KernelSU implementation of root for Android preconfigured into it. For all non-root tests of this PR this is not important, but for anyone interested in testing the root-only functionality, expand to read further.

more information

While it is not at all viable for CI anywhere since it is a custom ROM that takes many hours to compile, it might be useful if someone ever wants to manually test the root-only functionality of this PR, or the root-only functionality of the real-world reverse dependency of this PR (potential chroot managers for Android written in Python), but doesn't have any rooted bare metal Android devices to verify the functionality on that have one of the two leading implementations of root for Android, KernelSU or Magisk, installed, as opposed to only the vanilla AVD's adb root, which it is difficult and tedious to run Termux and Termux programs from properly without knowing a series of obscure commands. having KernelSU installed allows granting root permission to the Termux App from the KernelSU manager app in exactly the same workflow and root implementation that many users of rooted bare metal Android use.

https://github.com/robertkirkman/kernelsu-goldfish-builder

I made this because I was frustrated with the limited options available to most developers who don't have rooted Android for manual testing of root-only Android apps or code. It enables claims like "these functions don't crash the program if run as root user" to be validated objectively by outside testers.

[sabamdarif]

@robertkirkman can you test it https://github.com/sabamdarif/termux-packages/actions/runs/28925155859 and see if posix.setresuid(-1, -1, -1) works for you or not , i test tested it and it didn't actually crash , can you verify that ?

[robertkirkman]

Yes, for me that doesn't crash, when I test that build.

[picnixz]

@mhsmith FTR, once we are in the beta cycle, we can't introduce features anymore. Only bug fixes are considered backportable.

[picnixz]

Please also add a What's New entry for Python 3.16.

[mhsmith]

I can confirm that not all of the functions appear to be able to crash the program

I found the same thing when experimenting with ctypes on API level 36:

>>> c = ctypes.CDLL(None, use_errno=True)
>>> c.setresuid(0, 0, 0)
-1
>>> os.strerror(ctypes.get_errno())
'Operation not permitted'

On this emulator, when passing all arguments as 0, setresuid and seteuid return an error, while setuid and setreuid crash the process.

I can't see any logical reason for this difference, so I guess it may be an oversight which they'll fix in a future version of Android. In which case, it's probably still safer for us to keep the checks in all the functions.

[mhsmith]

Based on the logcat message, the crash is evidently a deliberate security feature:

10:08:54.234 DEBUG            A  signal 31 (SIGSYS), code 1 (SYS_SECCOMP), fault addr --------
10:08:54.234 DEBUG            A  Cause: seccomp prevented call to disallowed arm64 system call 146

This occurs even for calls which are supposed to be allowed in unprivileged processes, such as setuid(getuid()).

So I think we should disregard what I said before about checking the argument values, keep blocking all the functions when getuid() != 0, and update the failing test instead.