Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 24 additions & 31 deletions Doc/using/ios.rst
Original file line number Diff line number Diff line change
Expand Up @@ -194,13 +194,21 @@ To add Python to an iOS Xcode project:
adjusting paths as needed. Ensure that this folder is associated with your
app target.

4. Select the app target by selecting the root node of your Xcode project, then
4. Create folders to store your third-party dependencies (e.g., packages you've
installed from PyPI). You'll need a separate dependencies folder for iOS
devices and iOS simulators. In the following instructions, we'll assume
you've used folders named ``app_packages.iphoneos`` and
``app_packages.iphonesimulator``, located in the root of your project; you
can use any other location by adjusting paths as needed. These folders
*should not* be associated with the app target.

5. Select the app target by selecting the root node of your Xcode project, then
the target name in the sidebar that appears.

5. In the "General" settings, under "Frameworks, Libraries and Embedded
6. In the "General" settings, under "Frameworks, Libraries and Embedded
Content", add ``Python.xcframework``, with "Embed & Sign" selected.

6. In the "Build Settings" tab, modify the following:
7. In the "Build Settings" tab, modify the following:

- Build Options

Expand All @@ -216,7 +224,7 @@ To add Python to an iOS Xcode project:

* Quoted Include In Framework Header: No

7. Add a build step that processes the Python standard library, and your own
8. Add a build step that processes the Python standard library, and your own
Python binary dependencies. In the "Build Phases" tab, add a new "Run
Script" build step *before* the "Embed Frameworks" step, but *after* the
"Copy Bundle Resources" step. Name the step "Process Python libraries",
Expand All @@ -228,11 +236,18 @@ To add Python to an iOS Xcode project:
set -e
source "$PROJECT_DIR/Python.xcframework/build/utils.sh"
install_python Python.xcframework app
if [ "$EFFECTIVE_PLATFORM_NAME" = "-iphonesimulator" ]; then
APP_PACKAGES_DIR="$PROJECT_DIR/app_packages.iphonesimulator/"
else
APP_PACKAGES_DIR="$PROJECT_DIR/app_packages.iphoneos/"
fi
rsync -au --delete "$APP_PACKAGES_DIR" "$CODESIGNING_FOLDER_PATH/app_packages"
install_site_packages Python.xcframework app_packages

If you have placed your XCframework somewhere other than the root of your
project, modify the path to the first argument.

8. Add Objective C code to initialize and use a Python interpreter in embedded
9. Add Objective C code to initialize and use a Python interpreter in embedded
mode. You should ensure that:

* UTF-8 mode (:c:member:`PyPreConfig.utf8_mode`) is *enabled*;
Expand All @@ -241,36 +256,14 @@ To add Python to an iOS Xcode project:
* Signal handlers (:c:member:`PyConfig.install_signal_handlers`) are *enabled*;
* System logging (:c:member:`PyConfig.use_system_logger`) is *enabled*
(optional, but strongly recommended; this is enabled by default);
* :envvar:`PYTHONHOME` for the interpreter is configured to point at the
``python`` subfolder of your app's bundle; and
* The :envvar:`PYTHONPATH` for the interpreter includes:

- the ``python/lib/python3.X`` subfolder of your app's bundle,
- the ``python/lib/python3.X/lib-dynload`` subfolder of your app's bundle, and
- the ``app`` subfolder of your app's bundle
* The interpreter's home direectory (:c:member:`PyConfig.home`) is configured
to point at the ``python`` subfolder of your app's bundle; and
* The interpreter's module search paths (:c:member:`PyConfig.home`) includes
the ``app`` subfolder of your app's bundle.

Your app's bundle location can be determined using ``[[NSBundle mainBundle]
resourcePath]``.

Steps 7 and 8 of these instructions assume that you have a single folder of
pure Python application code, named ``app``. If you have third-party binary
modules in your app, some additional steps will be required:

* You need to ensure that any folders containing third-party binaries are
either associated with the app target, or are explicitly copied as part of
step 7. Step 7 should also purge any binaries that are not appropriate for
the platform a specific build is targeting (i.e., delete any device binaries
if you're building an app targeting the simulator).

* If you're using a separate folder for third-party packages, ensure that
folder is added to the end of the call to ``install_python`` in step 7, and
as part of the :envvar:`PYTHONPATH` configuration in step 8.

* If any of the folders that contain third-party packages will contain ``.pth``
files, you should add that folder as a *site directory* (using
:meth:`site.addsitedir`), rather than adding to :envvar:`PYTHONPATH` or
:attr:`sys.path` directly.

Testing a Python package
------------------------

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
The iOS build utility script now contains better support for integrating
third-party code into a project's site-packages folder. The testbed has
been modified to use this script, ensuring that sub-interpreters can find
all installed third-party dependencies.
34 changes: 27 additions & 7 deletions Platforms/Apple/testbed/Python.xcframework/build/utils.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
# -----
# set -e
# source "$PROJECT_DIR/Python.xcframework/build/utils.sh"
# install_python Python.xcframework app app_packages
# install_python Python.xcframework app
# install_site_packages Python.xcframework app_packages
# -----

# Copy the standard library from the XCframework into the app bundle.
Expand Down Expand Up @@ -161,20 +162,39 @@ process_dylibs () {
#
# Accepts 1 or more arguments:
# 1. The path, relative to the root of the Xcode project, where the Python
# XCframework can be found. If the XCframework is in the root of the project,
# 2+. The path of a package, relative to the root of the packaged app, that contains
# library content that should be processed for binary libraries.
# XCframework can be found.
# 2+. The path of any package, relative to the root of the Xcode project, that contains
# user-space Python code that should be processed for binary libraries.
install_python() {
PYTHON_XCFRAMEWORK_PATH=$1
shift

install_stdlib "$PYTHON_XCFRAMEWORK_PATH"
PYTHON_VER=$(ls -1 "$CODESIGNING_FOLDER_PATH/python/lib" | grep -E "^python3\.\d+[a-z]*$")
echo "Install Python $PYTHON_VER standard library extension modules..."
process_dylibs "$PYTHON_XCFRAMEWORK_PATH" python/lib/$PYTHON_VER/lib-dynload
PYTHON_LIBDIR=$(ls -1 "$CODESIGNING_FOLDER_PATH/python/lib" | grep -E "^python3\.\d+[a-z]*$")

echo "Install Python $PYTHON_LIBDIR standard library extension modules..."
process_dylibs "$PYTHON_XCFRAMEWORK_PATH" python/lib/$PYTHON_LIBDIR/lib-dynload

for package_path in "$@"; do
echo "Installing $package_path extension modules ..."
process_dylibs "$PYTHON_XCFRAMEWORK_PATH" "$package_path"
done
}

# The entry point for post-processing the site packages for a Python XCframework.
#
# Accepts 2 arguments:
# 1. The path, relative to the root of the Xcode project, where the Python
# XCframework can be found.
# 2. The path, relative to the root of the Xcode project, that contains
# the site-packages content for the app.
install_site_packages() {
PYTHON_XCFRAMEWORK_PATH=$1
APP_PACKAGES_PATH=$2
PYTHON_LIBDIR=$(ls -1 "$CODESIGNING_FOLDER_PATH/python/lib" | grep -E "^python3\.\d+[a-z]*$")

echo "Install Python $PYTHON_LIBDIR site-packages extension modules..."
rm -rf "$CODESIGNING_FOLDER_PATH/python/lib/$PYTHON_LIBDIR/site-packages"
mv "$CODESIGNING_FOLDER_PATH/$APP_PACKAGES_PATH" "$CODESIGNING_FOLDER_PATH/python/lib/$PYTHON_LIBDIR/site-packages"
process_dylibs "$PYTHON_XCFRAMEWORK_PATH" python/lib/$PYTHON_LIBDIR/site-packages
}
77 changes: 10 additions & 67 deletions Platforms/Apple/testbed/TestbedTests/TestbedTests.m
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,9 @@ @implementation TestbedTests
- (void)testPython {
const char **argv;
int exit_code;
int failed;
PyStatus status;
PyPreConfig preconfig;
PyConfig config;
PyObject *app_packages_path;
PyObject *method_args;
PyObject *result;
PyObject *site_module;
PyObject *site_addsitedir_attr;
PyObject *sys_module;
PyObject *sys_path_attr;
NSArray *test_args;
NSString *python_home;
NSString *path;
Expand Down Expand Up @@ -115,73 +107,24 @@ - (void)testPython {
return;
}

NSLog(@"Initializing Python runtime...");
status = Py_InitializeFromConfig(&config);
if (PyStatus_Exception(status)) {
XCTFail(@"Unable to initialize Python interpreter: %s", status.err_msg);
PyConfig_Clear(&config);
return;
}

// Add app_packages as a site directory. This both adds to sys.path,
// and ensures that any .pth files in that directory will be executed.
site_module = PyImport_ImportModule("site");
if (site_module == NULL) {
XCTFail(@"Could not import site module");
return;
}

site_addsitedir_attr = PyObject_GetAttrString(site_module, "addsitedir");
if (site_addsitedir_attr == NULL || !PyCallable_Check(site_addsitedir_attr)) {
XCTFail(@"Could not access site.addsitedir");
return;
}

path = [NSString stringWithFormat:@"%@/app_packages", resourcePath, nil];
NSLog(@"App packages path: %@", path);
wtmp_str = Py_DecodeLocale([path UTF8String], NULL);
app_packages_path = PyUnicode_FromWideChar(wtmp_str, wcslen(wtmp_str));
if (app_packages_path == NULL) {
XCTFail(@"Could not convert app_packages path to unicode");
return;
}
PyMem_RawFree(wtmp_str);

method_args = Py_BuildValue("(O)", app_packages_path);
if (method_args == NULL) {
XCTFail(@"Could not create arguments for site.addsitedir");
return;
}

result = PyObject_CallObject(site_addsitedir_attr, method_args);
if (result == NULL) {
XCTFail(@"Could not add app_packages directory using site.addsitedir");
return;
}

// Add test code to sys.path
sys_module = PyImport_ImportModule("sys");
if (sys_module == NULL) {
XCTFail(@"Could not import sys module");
return;
}

sys_path_attr = PyObject_GetAttrString(sys_module, "path");
if (sys_path_attr == NULL) {
XCTFail(@"Could not access sys.path");
return;
}

path = [NSString stringWithFormat:@"%@/app", resourcePath, nil];
NSLog(@"App path: %@", path);
wtmp_str = Py_DecodeLocale([path UTF8String], NULL);
failed = PyList_Insert(sys_path_attr, 0, PyUnicode_FromString([path UTF8String]));
if (failed) {
status = PyWideStringList_Insert(&config.module_search_paths, 0, wtmp_str);
if (PyStatus_Exception(status)) {
XCTFail(@"Unable to add app to sys.path");
return;
}
PyMem_RawFree(wtmp_str);

NSLog(@"Initializing Python runtime...");
status = Py_InitializeFromConfig(&config);
if (PyStatus_Exception(status)) {
XCTFail(@"Unable to initialize Python interpreter: %s", status.err_msg);
PyConfig_Clear(&config);
return;
}

// Ensure the working directory is the app folder.
chdir([path UTF8String]);

Expand Down
3 changes: 2 additions & 1 deletion Platforms/Apple/testbed/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ def expand_template(location: Path, abiflags: str) -> None:
replacement = abiflags.upper()
for path in location.rglob("*"):
if (
path.suffix in {".m", ".pbxproj", ".plist"}
path.suffix in {".m", ".pbxproj", ".plist", ".sh"}
and path.is_file()
and not path.is_symlink()
):
Expand Down Expand Up @@ -330,6 +330,7 @@ def clone_testbed(
"__main__.py",
"TestbedTests",
"Testbed.lldbinit",
"prepare_python_code.sh",
f"{platform}Testbed",
f"{platform}Testbed.xcodeproj",
f"{platform}Testbed.xctestplan",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
/* End PBXCopyFilesBuildPhase section */

/* Begin PBXFileReference section */
60214C9030AD4273006FA640 /* prepare_python_code.sh */ = {isa = PBXFileReference; lastKnownFileType = text.script.sh; path = prepare_python_code.sh; sourceTree = "<group>"; };
607A66122B0EFA380010BFC8 /* iOSTestbed.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = iOSTestbed.app; sourceTree = BUILT_PRODUCTS_DIR; };
607A662D2B0EFA3A0010BFC8 /* iOSTestbedTests.xctest */ = {isa = PBXFileReference; explicitFileType = wrapper.cfbundle; includeInIndex = 0; path = iOSTestbedTests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
607A664A2B0EFB310010BFC8 /* Python{{abiflags}}.xcframework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.xcframework; path = "Python{{abiflags}}.xcframework"; sourceTree = "<group>"; };
Expand Down Expand Up @@ -110,6 +111,7 @@
children = (
60FE0EFB2E56BB6D00524F87 /* iOSTestbed.xctestplan */,
607A664A2B0EFB310010BFC8 /* Python{{abiflags}}.xcframework */,
60214C9030AD4273006FA640 /* prepare_python_code.sh */,
607A66142B0EFA380010BFC8 /* iOSTestbed */,
607A66302B0EFA3A0010BFC8 /* TestbedTests */,
607A66132B0EFA380010BFC8 /* Products */,
Expand Down Expand Up @@ -250,7 +252,7 @@
);
runOnlyForDeploymentPostprocessing = 0;
shellPath = /bin/sh;
shellScript = "set -e\nsource \"$PROJECT_DIR/Python{{abiflags}}.xcframework/build/utils.sh\"\ninstall_python Python{{abiflags}}.xcframework app app_packages\n";
shellScript = "set -e\nsource \"$PROJECT_DIR/prepare_python_code.sh\"\n";
showEnvVarsInLog = 0;
};
/* End PBXShellScriptBuildPhase section */
Expand Down
5 changes: 5 additions & 0 deletions Platforms/Apple/testbed/prepare_python_code.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh
set -e
source "$PROJECT_DIR/Python{{abiflags}}.xcframework/build/utils.sh"
install_python "Python{{abiflags}}.xcframework" app
install_site_packages "Python{{abiflags}}.xcframework" app_packages
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
/* End PBXCopyFilesBuildPhase section */

/* Begin PBXFileReference section */
60214C9030AD4273006FA640 /* prepare_python_code.sh */ = {isa = PBXFileReference; lastKnownFileType = text.script.sh; path = prepare_python_code.sh; sourceTree = "<group>"; };
6077B3802E82A4BE00E3D6A3 /* tvOSTestbed.xctestplan */ = {isa = PBXFileReference; lastKnownFileType = text; path = tvOSTestbed.xctestplan; sourceTree = "<group>"; };
EE7C8A1C2DCD6FF3003206DB /* Python{{abiflags}}.xcframework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.xcframework; path = Python{{abiflags}}.xcframework; sourceTree = "<group>"; };
EE989E4E2DCD6E780036B268 /* tvOSTestbed.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = tvOSTestbed.app; sourceTree = BUILT_PRODUCTS_DIR; };
Expand Down Expand Up @@ -98,6 +99,7 @@
children = (
6077B3802E82A4BE00E3D6A3 /* tvOSTestbed.xctestplan */,
EE7C8A1C2DCD6FF3003206DB /* Python{{abiflags}}.xcframework */,
60214C9030AD4273006FA640 /* prepare_python_code.sh */,
EE989E502DCD6E780036B268 /* tvOSTestbed */,
EE989E682DCD6E7A0036B268 /* TestbedTests */,
EE989E4F2DCD6E780036B268 /* Products */,
Expand Down Expand Up @@ -236,7 +238,7 @@
);
runOnlyForDeploymentPostprocessing = 0;
shellPath = /bin/sh;
shellScript = "set -e\n\nsource \"$PROJECT_DIR/Python{{abiflags}}.xcframework/build/utils.sh\"\n\ninstall_python Python{{abiflags}}.xcframework app app_packages\n";
shellScript = "set -e\nsource \"$PROJECT_DIR/prepare_python_code.sh\"\n";
showEnvVarsInLog = 0;
};
/* End PBXShellScriptBuildPhase section */
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
/* End PBXCopyFilesBuildPhase section */

/* Begin PBXFileReference section */
60214C9030AD4273006FA640 /* prepare_python_code.sh */ = {isa = PBXFileReference; lastKnownFileType = text.script.sh; path = prepare_python_code.sh; sourceTree = "<group>"; };
6077B3D62E82E60C00E3D6A3 /* visionOSTestbed.xctestplan */ = {isa = PBXFileReference; lastKnownFileType = text; path = visionOSTestbed.xctestplan; sourceTree = "<group>"; };
607A66122B0EFA380010BFC8 /* visionOSTestbed.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = visionOSTestbed.app; sourceTree = BUILT_PRODUCTS_DIR; };
607A662D2B0EFA3A0010BFC8 /* TestbedTests.xctest */ = {isa = PBXFileReference; explicitFileType = wrapper.cfbundle; includeInIndex = 0; path = TestbedTests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
Expand Down Expand Up @@ -110,6 +111,7 @@
children = (
6077B3D62E82E60C00E3D6A3 /* visionOSTestbed.xctestplan */,
EEE9C80C2DAB5ECA0056F8C6 /* Python{{abiflags}}.xcframework */,
60214C9030AD4273006FA640 /* prepare_python_code.sh */,
607A66142B0EFA380010BFC8 /* visionOSTestbed */,
607A66302B0EFA3A0010BFC8 /* TestbedTests */,
607A66132B0EFA380010BFC8 /* Products */,
Expand Down Expand Up @@ -250,7 +252,7 @@
);
runOnlyForDeploymentPostprocessing = 0;
shellPath = /bin/sh;
shellScript = "set -e\n\nsource \"$PROJECT_DIR/Python{{abiflags}}.xcframework/build/utils.sh\"\n\ninstall_python Python{{abiflags}}.xcframework app app_packages\n";
shellScript = "set -e\nsource \"$PROJECT_DIR/prepare_python_code.sh\"\n";
showEnvVarsInLog = 0;
};
/* End PBXShellScriptBuildPhase section */
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
/* End PBXCopyFilesBuildPhase section */

/* Begin PBXFileReference section */
60214C9030AD4273006FA640 /* prepare_python_code.sh */ = {isa = PBXFileReference; lastKnownFileType = text.script.sh; path = prepare_python_code.sh; sourceTree = "<group>"; };
60F7C0062EB0000000A1B2C3 /* watchOSTestbed.xctestplan */ = {isa = PBXFileReference; lastKnownFileType = text; path = watchOSTestbed.xctestplan; sourceTree = "<group>"; };
60F7C0072EB0000000A1B2C3 /* Python{{abiflags}}.xcframework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.xcframework; path = Python{{abiflags}}.xcframework; sourceTree = "<group>"; };
60F7C0082EB0000000A1B2C3 /* watchOSTestbed.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = watchOSTestbed.app; sourceTree = BUILT_PRODUCTS_DIR; };
Expand Down Expand Up @@ -98,6 +99,7 @@
children = (
60F7C0062EB0000000A1B2C3 /* watchOSTestbed.xctestplan */,
60F7C0072EB0000000A1B2C3 /* Python{{abiflags}}.xcframework */,
60214C9030AD4273006FA640 /* prepare_python_code.sh */,
60F7C00B2EB0000000A1B2C3 /* watchOSTestbed */,
60F7C00C2EB0000000A1B2C3 /* TestbedTests */,
60F7C0102EB0000000A1B2C3 /* Products */,
Expand Down Expand Up @@ -236,7 +238,7 @@
);
runOnlyForDeploymentPostprocessing = 0;
shellPath = /bin/sh;
shellScript = "set -e\n\nsource \"$PROJECT_DIR/Python{{abiflags}}.xcframework/build/utils.sh\"\n\ninstall_python Python{{abiflags}}.xcframework app app_packages\n";
shellScript = "set -e\nsource \"$PROJECT_DIR/prepare_python_code.sh\"\n";
showEnvVarsInLog = 0;
};
/* End PBXShellScriptBuildPhase section */
Expand Down
Loading