|
| 1 | +# iOS: the standard library and the payload inside the app bundle |
| 2 | + |
| 3 | +SPEC L-11, issue #59, ROADMAP §13.1. |
| 4 | + |
| 5 | +`Python.xcframework` carries the interpreter and headers and no standard library, and an installed |
| 6 | +app is launched with no `PYTHONHOME`. Until now only the simulator *test* binary had a stdlib |
| 7 | +(`extractIosSimulatorStdlib` + `SIMCTL_CHILD_PYTHONHOME`), and that recipe does not carry over to an |
| 8 | +app: a `PYTHONHOME` on this workspace's external volume parks the sandboxed app in `open()` at 0% CPU |
| 9 | +(`python-multiplatform/src/iosMain/README.md`). This page records how an app now carries its own |
| 10 | +prefix and payload, why, and how to check it. |
| 11 | + |
| 12 | +## What happens |
| 13 | + |
| 14 | +**Gradle** (`python-multiplatform/build.gradle.kts`, rules in the plugin's `IosPythonHomeLayout`). |
| 15 | +One task per slice, `stageIosPythonHome_<sdk>_<arch>` (`iphoneos_arm64`, `iphonesimulator_arm64`, |
| 16 | +`iphonesimulator_x86_64`), writes `python-multiplatform/build/python-ios-home/<sdk>-<arch>/`: |
| 17 | + |
| 18 | +| Path | From `Python.xcframework/` | |
| 19 | +|---|---| |
| 20 | +| `home/lib/python<X.Y>/` | `lib/python<X.Y>/` (pure Python, every slice) merged with `<slice>/lib-<arch>/python<X.Y>/` (`lib-dynload/*.so`, `_sysconfigdata__ios_*`) | |
| 21 | +| `dylib-Info-template.plist` | `build/iOS-dylib-Info-template.plist` (input to the Xcode phase; not shipped) | |
| 22 | + |
| 23 | +Not copied: `__pycache__`, `libpython*.dylib`, and the top-level packages `test` (36 MB and 1,641 of |
| 24 | +the 2,407 files of the 3.14 stdlib), `idlelib`, `tkinter`, `turtledemo` (no `_tkinter` on iOS) and |
| 25 | +`ensurepip` (pip cannot run in an app). `stageIosPythonHomeForXcode` reads `EFFECTIVE_PLATFORM_NAME` |
| 26 | +and `ARCHS` from the environment Xcode gives a Run Script phase, stages only that slice and prints: |
| 27 | + |
| 28 | +``` |
| 29 | +PYTHON_HOME_DIR=<abs>/python-multiplatform/build/python-ios-home/iphonesimulator-arm64/home |
| 30 | +PYTHON_DYLIB_INFO_TEMPLATE=<abs>/python-multiplatform/build/python-ios-home/iphonesimulator-arm64/dylib-Info-template.plist |
| 31 | +``` |
| 32 | + |
| 33 | +A universal `ARCHS` (`arm64 x86_64`) is refused with the fix (`ONLY_ACTIVE_ARCH=YES` or `ARCHS=arm64`): |
| 34 | +one prefix holds one `lib-dynload`. Upstream's `utils.sh` has the same limit without saying so. |
| 35 | + |
| 36 | +**Xcode** (`tools/xcode/install-python.sh`, a Run Script phase after Copy Bundle Resources and before |
| 37 | +Embed Frameworks; `ENABLE_USER_SCRIPT_SANDBOXING = NO`). |
| 38 | + |
| 39 | +1. Runs Gradle as above. The output is captured, then parsed, then checked to be a directory — a |
| 40 | + failing Gradle stops the phase and never leaves `rsync` an empty source (issue #59's comment). |
| 41 | +2. `rsync -a --delete` the prefix to `<app>/python-multiplatform-home/`. |
| 42 | +3. The payload, if configured, to `<app>/python/` (`--delete`, without `__pycache__`): |
| 43 | + `PYTHON_PAYLOAD_DIR=<dir>` (a directory whose contents are the payload root — the app's own Python |
| 44 | + sources, or toolchain's `build/pythonStaging/ios/python`), or `PYTHON_PAYLOAD_TASK=<gradle task>` |
| 45 | + that prints `PYTHON_PAYLOAD_DIR=` under `-q` (toolchain's `:app:stagePythonBundleIosForXcode`). |
| 46 | + Neither set: no payload, and an old `<app>/python/` is removed. |
| 47 | +4. Every `.so` under both directories becomes `Frameworks/<dotted.module>.framework/<dotted.module>` |
| 48 | + with an `Info.plist` from the template, a `<name>.fwork` placeholder where the `.so` was (its |
| 49 | + content is the framework binary's path relative to the app), and a `.origin` back-reference; |
| 50 | + each framework is signed with `EXPANDED_CODE_SIGN_IDENTITY` when there is one. |
| 51 | + |
| 52 | +**Run time** (`python-multiplatform/src/iosMain/.../env/PackagedPythonHome.ios.kt`). |
| 53 | +`Python3.initialize()` calls `applyPackagedPythonHome()` after the `PYTHONHOME` pre-flight check |
| 54 | +(L-3) and before `Py_Initialize()`. `IosPythonHome.resolve()` takes the first of: |
| 55 | + |
| 56 | +1. `PYTHONHOME` in the environment — nothing else happens. The simulator test task and an Xcode |
| 57 | + scheme variable keep working as before. |
| 58 | +2. `<NSBundle.mainBundle.resourcePath>/python-multiplatform-home`, only if that directory exists. |
| 59 | + |
| 60 | +A bundled prefix is checked with `PythonHomeCheck` (an `IllegalStateException` naming the path and |
| 61 | +"the app bundle's resource directory"), converted with `Py_DecodeLocale` (UTF-8 on Apple platforms) |
| 62 | +and passed to `Py_SetPythonHome`. `PythonPayload` then finds `<resourcePath>/python` and puts it on |
| 63 | +`sys.path`, as before. |
| 64 | + |
| 65 | +## Why this design |
| 66 | + |
| 67 | +**Upstream's framework rule, not loose `.so` files.** iOS does not load a binary that is not inside a |
| 68 | +framework on a device, and App Store validation rejects one. CPython 3.13+ on iOS has |
| 69 | +`importlib.machinery.AppleFrameworkLoader`: for `foo.cpython-314-iphoneos.fwork` it reads the file, |
| 70 | +joins it to `dirname(sys.executable)` and loads that binary. The script is upstream's |
| 71 | +`Python.xcframework/build/utils.sh` `install_dylib`, with two changes: the template comes from the |
| 72 | +Gradle staging (so the phase needs no path into the archive), and signing is skipped when Xcode |
| 73 | +exports no identity instead of failing. The simulator would load a loose `.so`; the device would not, |
| 74 | +so the simulator run uses the device's layout. |
| 75 | + |
| 76 | +**`python-multiplatform-home/`, not upstream's `python/`.** Upstream's testbed sets |
| 77 | +`config.home = <resourcePath>/python`. Here `python/` is the consumer's payload root |
| 78 | +(`PythonPayload.PAYLOAD_ROOT`, L-6), and the payload copy is `rsync --delete`: one directory for |
| 79 | +both would let each copy delete the other. The name is the one desktop already uses (L-9). |
| 80 | + |
| 81 | +**Staged per slice by Gradle, copied by a thin script.** The stdlib merge (shared tree + one |
| 82 | +`lib-<arch>` tree) and the exclusions are pure functions with unit tests |
| 83 | +(`GP/IosPythonHomeLayoutTest.kt`); the script only copies, wraps and signs. The selection lives in the |
| 84 | +Gradle plugin because the library build script already imports the plugin's public layout objects |
| 85 | +(`CPythonIncludeLayout`) and a build script cannot be unit-tested. |
| 86 | + |
| 87 | +**`Py_SetPythonHome`, not `setenv`.** Same reasoning as desktop: it sets CPython's path |
| 88 | +configuration for this process only, and it is checked by the same `PythonHomeCheck` immediately |
| 89 | +before. `PyConfig.home` is the non-deprecated route, but the library carries no `PyConfig` layout. |
| 90 | + |
| 91 | +## Wiring an app |
| 92 | + |
| 93 | +The sample's project is `iosApp/iosApp.xcodeproj` (product `PythonDemo.app`, bundle id |
| 94 | +`org.thisisthepy.python.multiplatform.demo`). Its phase: |
| 95 | + |
| 96 | +```sh |
| 97 | +set -e |
| 98 | +cd "$SRCROOT/.." |
| 99 | +export PYTHON_PAYLOAD_DIR="$SRCROOT/../sample/python/src/main" |
| 100 | +/bin/bash tools/xcode/install-python.sh |
| 101 | +``` |
| 102 | + |
| 103 | +It replaced two phases: "Install Target Specific Python Standard Library", which rsynced the slice's |
| 104 | +`lib/` — only `libpython3.14.dylib` — into `<app>/lib/`, and "Prepare Python Binary Modules", which |
| 105 | +did the framework wrapping from a template copied in as a resource. `iosApp/iosApp/dylib-Info-template.plist` |
| 106 | +is no longer copied into the app; the file itself is still in the project. |
| 107 | + |
| 108 | +`sample/src/iosMain/app.xcodeproj` is not wired: it still references `../../dist/toolchain/...` and |
| 109 | +a `ComposeApp` framework that this build does not produce, and has never been built here. |
| 110 | + |
| 111 | +A consumer outside this repository cannot call `:python-multiplatform:stageIosPythonHomeForXcode`: |
| 112 | +the iOS archive download lives in this repository's build script, not in the published plugin. That |
| 113 | +is the next step if consumers are to get this without copying the sample. |
| 114 | + |
| 115 | +## Checking it (completion criterion of #59) |
| 116 | + |
| 117 | +The sample app on the simulator imports a stdlib module, a stdlib extension module and a payload |
| 118 | +module from the installed bundle, launched with **no** `SIMCTL_CHILD_*` variable. All commands from |
| 119 | +the repository root; Gradle needs JDK 21 (`JAVA_HOME`), and Xcode's phases inherit it. |
| 120 | + |
| 121 | +```bash |
| 122 | +export JAVA_HOME=/Users/ibrew/Library/Java/JavaVirtualMachines/jdk-21.0.12+8/Contents/Home |
| 123 | +df -h / # the installed app (~60 MB) goes into CoreSimulator on the internal disk |
| 124 | + |
| 125 | +# 0. Stage once by hand and look at it |
| 126 | +EFFECTIVE_PLATFORM_NAME=-iphonesimulator ARCHS=arm64 \ |
| 127 | + ./gradlew -q :python-multiplatform:stageIosPythonHomeForXcode > .tmp/stage.out 2>&1; echo "EXIT=$?" |
| 128 | +cat .tmp/stage.out # two lines: PYTHON_HOME_DIR=..., PYTHON_DYLIB_INFO_TEMPLATE=... |
| 129 | +ls python-multiplatform/build/python-ios-home/iphonesimulator-arm64/home/lib/python3.14/lib-dynload | head |
| 130 | +test ! -e python-multiplatform/build/python-ios-home/iphonesimulator-arm64/home/lib/python3.14/test && echo "test/ excluded" |
| 131 | +# must fail, naming ONLY_ACTIVE_ARCH: |
| 132 | +EFFECTIVE_PLATFORM_NAME=-iphonesimulator ARCHS="arm64 x86_64" \ |
| 133 | + ./gradlew -q :python-multiplatform:stageIosPythonHomeForXcode > .tmp/stage-universal.out 2>&1; echo "EXIT=$?" |
| 134 | + |
| 135 | +# 1. Frameworks the Xcode project links against |
| 136 | +./gradlew :sample:prepareIosFrameworks --console=plain > .tmp/prepare.log 2>&1; echo "EXIT=$?" |
| 137 | + |
| 138 | +# 2. Build the app (flags from the last verified run, ROADMAP history 2026-08-13: actool cannot |
| 139 | +# compile the app icon against this machine's simulator runtimes, and -target avoids |
| 140 | +# -destination's eligibility check) |
| 141 | +(cd iosApp && xcodebuild -project iosApp.xcodeproj -target iosApp -configuration Debug \ |
| 142 | + -sdk iphonesimulator ARCHS=arm64 ONLY_ACTIVE_ARCH=YES SYMROOT="$PWD/build" \ |
| 143 | + ASSETCATALOG_COMPILER_APPICON_NAME="" \ |
| 144 | + CODE_SIGN_IDENTITY="-" CODE_SIGN_STYLE=Manual DEVELOPMENT_TEAM="" build) \ |
| 145 | + > .tmp/xcodebuild.log 2>&1; echo "EXIT=$?" |
| 146 | +grep -E "Installing the Python|Wrapped|error:" .tmp/xcodebuild.log |
| 147 | + |
| 148 | +APP=iosApp/build/Debug-iphonesimulator/PythonDemo.app |
| 149 | +ls "$APP/python-multiplatform-home/lib/python3.14/os.py" "$APP/python/example_py/__init__.py" |
| 150 | +ls "$APP/python-multiplatform-home/lib/python3.14/lib-dynload" | grep -c '\.fwork$' # 67 for 3.14 |
| 151 | +find "$APP/python-multiplatform-home" "$APP/python" -name '*.so' | wc -l # 0 |
| 152 | +ls -d "$APP/Frameworks/_json.framework" "$APP/Frameworks/Python.framework" |
| 153 | +codesign --verify --deep --strict "$APP" && echo SIGNED_OK |
| 154 | + |
| 155 | +# 3. Install and launch with no SIMCTL_CHILD_* variable |
| 156 | +UDID=$(xcrun simctl list devices booted | grep -oE '[0-9A-F-]{36}' | head -n 1); echo "$UDID" |
| 157 | +env | grep '^SIMCTL_CHILD_' && echo "unset these first" # must print nothing |
| 158 | +xcrun simctl uninstall "$UDID" org.thisisthepy.python.multiplatform.demo |
| 159 | +xcrun simctl install "$UDID" "$APP" |
| 160 | +xcrun simctl launch --console-pty --terminate-running-process "$UDID" \ |
| 161 | + org.thisisthepy.python.multiplatform.demo > .tmp/launch.log 2>&1 & |
| 162 | +LAUNCH=$! |
| 163 | +# wait for the DEMO block (Monitor/until-loop; no sleep in this shell), then: |
| 164 | +kill "$LAUNCH" |
| 165 | +grep '^DEMO' .tmp/launch.log |
| 166 | +``` |
| 167 | + |
| 168 | +Pass: `DEMO 1 runtime` names `3.14.x · sys.platform=ios`, and `DEMO 8 bundle` is a `tuple:` whose |
| 169 | +five entries are the payload's greeting as JSON (`"Hello from usage-example's Python payload 0.1.0"`), |
| 170 | +`_json.__file__` ending `PythonDemo.app/python-multiplatform-home/lib/python3.14/lib-dynload/_json.cpython-314-iphonesimulator.fwork`, |
| 171 | +`example_py.__file__` ending `PythonDemo.app/python/example_py/__init__.py`, `sys.prefix` ending |
| 172 | +`PythonDemo.app/python-multiplatform-home`, and `sys.executable` ending `PythonDemo.app/PythonDemo`. |
| 173 | + |
| 174 | +Fail shapes and what they mean: |
| 175 | + |
| 176 | +| Seen | Meaning | |
| 177 | +|---|---| |
| 178 | +| No `DEMO` lines, 0% CPU (`sample <pid>` parked in `open$NOCANCEL`) | the app read a path outside its container — check `env` for a stray `PYTHONHOME` | |
| 179 | +| `Fatal Python error: Failed to import encodings module` | no `python-multiplatform-home/` in the installed app; the phase did not run | |
| 180 | +| `DEMO 8 bundle | ModuleNotFoundError: No module named '_json'` | the `.fwork` did not resolve; compare `sys.executable`'s directory with the app directory | |
| 181 | +| `DEMO 8 bundle | ModuleNotFoundError: No module named 'example_py'` | `<app>/python/` missing: `PYTHON_PAYLOAD_DIR` not set in the phase | |
0 commit comments