Skip to content

Support PGO in cross-compiles #113123

@rossburton

Description

@rossburton

Feature or enhancement

Proposal:

The current makefile rules for profiled guided optimisation don't support cross at all:

cpython/Makefile.pre.in

Lines 691 to 692 in fb4cb7c

@ # FIXME: can't run for a cross build
$(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK)

This is understandable, as the point is to run the target code and on the host this isn't generally possible.

However, qemu-user does let us run target code on the build host with only a little fuss and various build tools allow the use of this to run target binaries in limited cases, such as when probing capabilities or running tests. For example, Meson's cross files support an exe_wrapper script (https://mesonbuild.com/Cross-compilation.html) and CMake has a CMAKE_CROSSCOMPILING_EMULATOR variable (https://cmake.org/cmake/help/latest/variable/CMAKE_CROSSCOMPILING_EMULATOR.html).

I propose adding something similar for the PGO execution. Strawman proposal:

  1. Add --with-cross-runner to configure to let the user specify a wrapper script that can be used to run target binaries on the build host.
  2. Wrap the execution of the just built python3 when generating the PGO training data with this script

Prior art to demonstrate that this is at least feasible can be found in the Yocto Project at https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/python/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch. We generate the pgo-wrapper script (which simply calls qemu-user with arguments) during the build.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildThe build process and cross-buildtype-featureA feature request or enhancement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions