Skip to content

py/dynruntime.mk: Let natmods be built with Clang.#19308

Open
agatti wants to merge 8 commits into
micropython:masterfrom
agatti:natmod-clang
Open

py/dynruntime.mk: Let natmods be built with Clang.#19308
agatti wants to merge 8 commits into
micropython:masterfrom
agatti:natmod-clang

Conversation

@agatti

@agatti agatti commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR modifies the build rules for native modules in order to remove the dependence on GCC for creating native MPY files.

Whilst the Unix port of MicroPython can be built with Clang by overriding the CC variable, natmods require a bit more work. GCC builds compilers that are tailored for a single architecture, but Clang takes the opposite approach, so a single binary may target more than one architecture. Architecture selection is, by definition, not compatible between those two compilers.

These changes attempt to make things easier to handle when using Clang. Native modules can now be built with something like this: make CC=clang ARCH=<arch> CFLAGS_EXTRA='--target=<clang-target-arch>'. So, for example building an x86 native module the command line will look something like this: make CC=clang ARCH=x86 CFLAGS_EXTRA='--target=i686-unknown-linux-gnu'.

Clang and GCC, however, have different tolerances for deviations from the chosen C standard. Whilst GCC doesn't really mind whether a typedef is defined multiple times as long as it is defined to the same value, Clang does raise a warning which is then interpreted as an error.

Unfortunately #ifdef/#ifndef does not work with typedefs, and the way native modules are built meant that py/mpconfig.h would first include the native module's generated configuration file and then proceed with the rest of the configuration. However, both files attempt to provide aliases for both mp_int_t and mp_uint_t, and that doesn't really work. Thus, the only sane way to work around it is to rely on the presence of a definition that indicates that mp_int_t and mp_uint_t are already there to begin with, letting builds proceed on both GCC and Clang.

Testing

All natmods in examples/natmod were attempted to be built for x86 using the command line mentioned in the summary section. Results are as follows:

  • btree: fails to compile because __bt_close, __bt_sync, and bt_meta do not have a prototype
  • deflate: fails to link because memset is not defined builds on both x86 and x64
  • features0: builds on x86, x64, and armv7m
  • features1: builds on x86, x64, and armv7m
  • features2: builds on x86, x64, and armv7m
  • features3: builds on x86, x64, and armv7m
  • features4: builds on x86, x64, and armv7m
  • framebuf: fails to link because memset is not defined builds on both x86 and x64
  • heapq: builds on x86, x64, and armv7m
  • random: builds
  • re: fails to link because memset is not defined builds on both x86 and x64.

Importing features0 in a Clang-built x86 interpreter, though, crashes due to "something" happening when calling mp_call_function_0. The same happens on an x86 interpreter built with GCC, and on x64 too. Fun for the whole architecture family. With Clang the entry trampoline gets emitted incorrectly, so that had to be patched somehow.

The same procedure was done successfully without overriding CC, CFLAGS_EXTRA, and LDFLAGS_EXTRA, still building with ARCH=x86. Then natmods were built for xtensawin but specifying CROSS=xtensa-esp32s3-elf- to make sure the new compiler was picked up correctly.

Trade-offs and Alternatives

None I can see, although the crashes and link failures need to be sorted out first.

Generative AI

I did not use generative AI tools when creating this PR.


This is marked as draft due to the obvious shortcomings of the PR in its current state. Still working on it, tools/mpy_ld.py doesn't like something about the object files emitted by clang - maybe it's the section ordering or something else, but at least I have a rough idea on where to look.

@agatti agatti marked this pull request as draft June 4, 2026 13:49
@agatti agatti added py-core Relates to py/ directory in source tools Relates to tools/ directory in source, or other tooling labels Jun 4, 2026
@codecov

codecov Bot commented Jun 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.47%. Comparing base (75555f4) to head (f91697e).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #19308   +/-   ##
=======================================
  Coverage   98.47%   98.47%           
=======================================
  Files         176      176           
  Lines       22845    22845           
=======================================
  Hits        22497    22497           
  Misses        348      348           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown

Code size report:

Reference:  samd/mphalport: Run events at least once in mp_hal_delay_ms. [af38ee1]
Comparison: examples/natmod/re: Fix building with Clang toolchains. [merge of f91697e]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
      esp32:    +0 +0.000% ESP32_GENERIC
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@agatti

agatti commented Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

deflate, framebuf, and re need to use LINK_RUNTIME=1, along with a modification to py/dynruntime.mk to also look into libc.a if Clang is the compiler being used [1].

Still that's not enough, as then link fails even with all the symbols being present:

make CC=clang ARCH=x64 V=1                                                                                                                                                                                                                                                                                                                       
/bin/mkdir -p build/                                                                                                                                                                                                                                                                                                                                                                      
GEN build/deflate_x64.config.h                                                                                                                                                                                                                                                                                                                                                            
python3 ../../../tools/mpy_ld.py '-vvv' --arch x64 --preprocess -o build/deflate_x64.config.h deflate.c                                                                                                                                                                                                                                                                                   
CC deflate.c                                                                                                                                                                                                                                                                                                                                                                              
clang -I. -I../../.. -std=c99 -Os -Wall -Werror -DNDEBUG -DNO_QSTR -DMICROPY_ENABLE_DYNRUNTIME -DMP_CONFIGFILE='<build/deflate_x64.config.h>' -fpic -fno-common -U_FORTIFY_SOURCE  -fno-stack-protector -DMICROPY_FLOAT_IMPL=MICROPY_FLOAT_IMPL_DOUBLE  -o build/deflate.o -c deflate.c                                                                                                   
LINK build/deflate.o                                                                                                                                                                                                                                                                                                                                                                      
python3 ../../../tools/mpy_ld.py '-vvv' --arch x64 --qstrs build/deflate_x64.config.h -l/usr/lib/gcc/x86_64-linux-gnu/13/libgcc.a -l/usr/lib/x86_64-linux-gnu/libm.a -l/usr/lib/x86_64-linux-gnu/libc.a -o build/deflate_x64.native.mpy build/deflate.o                                                                                                                                   
qstr vals: DeflateIO, GZIP, RAW, ZLIB, deflate                                                                                                                                                                                                                                                                                                                                            
Loading /usr/lib/gcc/x86_64-linux-gnu/13/libgcc.a                                                                                                                                                                                                                                                                                                                                         
Loading /usr/lib/x86_64-linux-gnu/libm-2.39.a                                                                                                                                                                                                                                                                                                                                             
Loading /usr/lib/x86_64-linux-gnu/libmvec.a                                                                                                                                                                                                                                                                                                                                               
Loading /usr/lib/x86_64-linux-gnu/libc.a                                                                                                                                                                                                                                                                                                                                                  
Skippping weak dependency: _dl_find_object_init
...
Skippping weak dependency: _nl_C_LC_MONETARY
using /usr/lib/x86_64-linux-gnu/libc.a:memset.o
using /usr/lib/x86_64-linux-gnu/libc.a:memset-evex-unaligned-erms.o
using /usr/lib/x86_64-linux-gnu/libc.a:memset-sse2-unaligned-erms.o
using /usr/lib/x86_64-linux-gnu/libc.a:memset-erms.o
using /usr/lib/x86_64-linux-gnu/libc.a:dl-support.o
LinkError: /usr/lib/x86_64-linux-gnu/libc.a:dl-support.o: .data non-empty
make: *** [../../../py/dynruntime.mk:255: build/deflate_x64.native.mpy] Error 1

Edit: memset needs to be provided by the module, or the dependency chain will end up with an unsupported condition.


[1]

diff --git i/py/dynruntime.mk w/py/dynruntime.mk
index 26f2fc720..4c5b9655f 100644
--- i/py/dynruntime.mk
+++ w/py/dynruntime.mk
@@ -148,7 +148,7 @@ else
 $(error architecture '$(ARCH)' not supported)
 endif
 
-ifeq ($(firstword $(shell $(CC) --version)),clang)
+ifeq ($(findstring clang,$(shell $(CC) --version)),clang)
 CROSS =
 endif
 
@@ -199,6 +199,10 @@ LIBM_PATH := $(PICOLIBC_ROOT)/$(PICOLIBC_ARCH)/$(PICOLIBC_ABI)/$(LIBM_NAME)
 endif
 endif
 MPY_LD_FLAGS += $(addprefix -l, $(LIBGCC_PATH) $(LIBM_PATH))
+ifeq ($(findstring clang,$(shell $(CC) --version)),clang)
+LIBC_PATH := $(realpath $(shell $(CROSS)$(CC) $(CFLAGS) --print-file-name=libc.a))
+MPY_LD_FLAGS += $(addprefix -l, $(LIBC_PATH))
+endif
 endif
 ifneq ($(MPY_EXTERN_SYM_FILE),)
 MPY_LD_FLAGS += --externs "$(realpath $(MPY_EXTERN_SYM_FILE))"

@agatti agatti force-pushed the natmod-clang branch 2 times, most recently from 3ca75d4 to 6adecfb Compare June 4, 2026 15:04
@agatti

agatti commented Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

Even more fun, on x86 at least, clang will put mpy_init as the first function in the binary for features0 - which for some reason messes up the trampoline generation! That may be the trigger for the crash. The simplest fix is to not generate the trampoline if mpy_init is the first function, but maybe the trampoline is there to stay...

image

@agatti

agatti commented Jun 5, 2026

Copy link
Copy Markdown
Contributor Author

On x86 and x64 things seem to work with not emitting the trampoline in certain cases - although this will be revisited before this PR is marked as ready for review.

For cross-compilation, things are a bit more involved. The way Clang seems to work is that you need to cross-compile a runtime package for your target first, and then install it in a predefined place (/usr/lib/clang-runtimes/${TRIPLE} in my case). This is necessary otherwise not even assert.h will be resolved.

If you don't have any idea how this works, like I do, or already have a GCC cross-compiler installed, you can add --sysroot=/usr/${GCC_TRIPLE} to CFLAGS_EXTRA. There's also https://llvm.org/docs/HowToCrossCompileBuiltinsOnArm.html and https://compiler-rt.llvm.org/ that should provide more information for the technically inclined reader.

That said, when building examples/natmod/features0 with make CC=clang CFLAGS_EXTRA='--sysroot=/usr/arm-none-eabi --target=armv7m-none-eabi' LDFLAGS_EXTRA='--target=armv7m-none-eabi' ARCH=armv7m then this happens:

$ make CC=clang CFLAGS_EXTRA='--sysroot=/usr/arm-none-eabi --target=armv7m-none-eabi' LDFLAGS_EXTRA='--target=armv7m-none-eabi' ARCH=armv7m
CC features0.c
LINK build/features0.o
Traceback (most recent call last):
  File "/micropython/examples/natmod/features0/../../../tools/mpy_ld.py", line 1684, in <module>
    main()
    ~~~~^^
  File "/micropython/examples/natmod/features0/../../../tools/mpy_ld.py", line 1680, in main
    do_link(args)
    ~~~~~~~^^^^^^
  File "/micropython/examples/natmod/features0/../../../tools/mpy_ld.py", line 1565, in do_link
    link_objects(env, len(native_qstr_vals))
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/micropython/examples/natmod/features0/../../../tools/mpy_ld.py", line 1325, in link_objects
    do_relocation_text(env, sec.addr, r)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/micropython/examples/natmod/features0/../../../tools/mpy_ld.py", line 768, in do_relocation_text
    assert 0, (r_info_type, s.name, s.entry, env.arch.name)
           ^
AssertionError: (96, 'mp_fun_table', Container({'st_name': 155, 'st_value': 0, 'st_size': 0, 'st_info': Container({'bind': 'STB_GLOBAL', 'type': 'STT_NOTYPE'}), 'st_other': Container({'local': 0, 'visibility': 'STV_DEFAULT'}), 'st_shndx': 'SHN_UNDEF'}), 'EM_ARM')
make: *** [../../../py/dynruntime.mk:251: build/features0.native.mpy] Error 1

96 is R_ARM_GOT_PREL, which is currently not implemented. So more tools/mpy_ld.py changes are needed. Oh well.

@agatti agatti force-pushed the natmod-clang branch 2 times, most recently from ab69b67 to 11b8580 Compare June 5, 2026 05:11
@agatti

agatti commented Jun 5, 2026

Copy link
Copy Markdown
Contributor Author

ArmV6 and ArmV7 now sort of build, with some hacks of course. Using Arm's latest stable Clang toolchain (https://github.com/ARM-software/LLVM-embedded-toolchain-for-Arm/releases/tag/release-19.1.5):

Module ArmV6-M [1] ArmV7-M [2]
btree Fails to compile Fails to compile
deflate Builds and links Builds and links
features0 Builds and links Builds and links
features1 Builds and links Builds and links
features2 Builds and links Fails to link
features3 Builds and links Builds and links
features4 Builds and links Builds and links
framebuf Builds and links Builds and links
heapq Builds and links Builds and links
random Builds and links Fails to link
re Builds and links Builds and links

features2 and random won't link for armv7m due to floating point code bringing in fp_mode.c.obj from libclang_rt.builtins.a, and said object file has static BSS variables. This may or may not be toolchain-dependent, so I'll have to leave it at that.

[1] make CC=$LLVMROOT/bin/clang CFLAGS_EXTRA='--target=armv6-none-eabi' ARCH=armv6m
[2] make CC=$LLVMROOT/bin/clang CFLAGS_EXTRA='--target=armv7-none-eabi' ARCH=armv7m

agatti added 3 commits June 5, 2026 07:33
This commit modifies the build rules for native modules in order to
remove the dependence on GCC for creating native MPY files.

Whilst the Unix port of MicroPython can be built with Clang by
overriding the `CC` variable, natmods require a bit more work.  GCC
builds compilers that are tailored for a single architecture, but Clang
takes the opposite approach, so a single binary may target more than one
architecture.  Architecture selection is, by definition, not compatible
between those two compilers.

These changes attempt to make things easier to handle when using Clang.
Native modules can now be built with something like this:

make CC=clang ARCH=<arch> CFLAGS_EXTRA='--target=<clang-target-arch>'

So, for example building an x86 native module the command line will look
something like this:

make CC=clang ARCH=x86 CFLAGS_EXTRA='--target=i686-unknown-linux-gnu'

Clang and GCC, however, have different tolerances for deviations from
the chosen C standard.  Whilst GCC doesn't really mind whether a typedef
is defined multiple times as long as it is defined to the same value,
Clang does raise a warning which is then interpreted as an error.

Unfortunately #ifdef/#ifndef does not work with typedefs, and the way
native modules are built meant that `py/mpconfig.h` would first include
the native module's generated configuration file and then proceed with
the rest of the configuration.  However, both files attempt to provide
aliases for both `mp_int_t` and `mp_uint_t`, and that doesn't really
work.  Thus, the only sane way to work around it is to rely on the
presence of a definition that indicates that `mp_int_t` and `mp_uint_t`
are already there to begin with, letting builds proceed on both GCC and
Clang.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit lets `tools/mpy_ld.py` to not emit an entry point trampoline
if `mpy_init` is placed at the very beginning of the text segment.

These changes are currently a workaround to make natmods work when built
with Clang for x86/x64, as there will be garbage between the entry point
jump and `mpy_init` shortly after.  This is not a proper fix since the
root cause wasn't found yet, and this may break on other architectures,
but in the meantime it is enough to get things going.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit introduces support for the R_ARM_GOT_PREL relocation, found
in the text segment of certain Arm native modules.

Until now this was not needed, since the only supported compiler for
linking native modules was GCC, which did not seem to ever generate such
a relocation.

With the recent work in making Clang a supported compiler as well, it
was quickly found out that such a compiler actually does generate such a
relocation type.  This relocation seem to be enough to at least make
`examples/natmod/features0` link with Clang targeting `armv7m`, and then
run the output native module on an appropriate interpreter running under
the QEMU MPS2-AN500 target.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
agatti added 2 commits June 6, 2026 04:29
This commit fixes building the `btree` module using Clang rather than
using GCC, on x86, x64, and ArmV7 targets.

The Clang standard library (libc) implementation of `memset` for x86 and
x64 depends on functions that have a non-empty data section, which is not
currently supported.  Therefore we provide our own `memset`
implementation that is good enough to let linking succeed for `x86` and
`x64` targets.  On a more general note, Clang also required some
additional flags to disable an extra warning that GCC did not seem to
raise.

On Arm targets, `memset` is not a builtin of the compiler toolchain, so
it has to be fetched from the runtime support library.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes building the `deflate` module using Clang rather than
using GCC, on x86, x64, and ArmV7 targets.

The Clang standard library (libc) implementation of `memset` for x86 and
x64 depends on functions that have a non-empty data section, which is not
currently supported.  Therefore we provide our own `memset`
implementation that is good enough to let linking succeed for `x86` and
`x64` targets.

On Arm targets, `memset` is not a builtin of the compiler toolchain, so
it has to be fetched from the runtime support library.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
@agatti agatti force-pushed the natmod-clang branch 2 times, most recently from 6044fb8 to 575df98 Compare June 6, 2026 02:32
@agatti

agatti commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Prebuilt LLVM toolchains support is still a mess, I've tried almost a dozen toolchains, when I should have probably built my own from the beginning. Anyway, here's what I found:

Qualcomm's toolchain won't build examples/natmod/btree: if you really need examples/natmod/btree on armv7m use Arm's toolchain with --target=armv7m-none-eabi.

The results are as follows:

  • rv32imc natmods clear all tests when run on QEMU/VIRT_RV32.
  • armv6m natmods haven't been tested, all I can rely on are emulated targets right now.
  • armv7m natmods clear the tests when run on QEMU/MPS2_AN385, except for random since its floating point support uses static .bss variables.
  • rv64imc still needs a proper toolchain, however I'd expect to also work as rv32imc does.

@agatti agatti marked this pull request as ready for review June 6, 2026 02:48
@agatti

agatti commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

This is the best I can do at the moment. Docs are missing, I'll add them to this PR once the code changes are reviewed.

If needed I can split the R_ARM_GOT_PREL relocation handler into its own PR, since in theory that could be encountered when mixing object files built with Clang in an otherwise GCC-compiled natmod.

I've also encountered R_ARM_THM_JUMP19 when attempting to make ArmV7 floating point support work, only to end up with the usual static .bss entries issue. If somebody else encounters that relocation I'll add that (and also fix arm's thumb_b relocations handler, since it ignores J1 and J2 bits, so longer positive jumps may be incorrect).

agatti added 3 commits June 9, 2026 06:45
This commit fixes building the `features2` module using Clang rather than
using GCC.

Parts of the floating point support code for Clang may end up in libc.a
rather than its builtins support library.  This is the case for the
armv7m target, so we have to force linking symbols from libc.a in that
case.  Depending on the toolchain, this may or may not build
successfully, but the location of the `roundf` symbol should not move
across toolchains.

This makes it work on armv6m, rv32imc, and possibly on armv7m too.  The
latter depends on whether Clang's builtins support library not relying
on features that are unsupported by `tools/mpy_ld.py`.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes building the `framebuf` module using Clang rather than
using GCC, on x86, x64, and ArmV7 targets.

The Clang standard library (libc) implementation of `memset` for x86 and
x64 depends on functions that have a non-empty data section, which is not
currently supported.  Therefore we provide our own `memset`
implementation that is good enough to let linking succeed for `x86` and
`x64` targets.

On Arm targets, `memset` is not a builtin of the compiler toolchain, so
it has to be fetched from the runtime support library.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit fixes building the `re` module using Clang rather than using
GCC, on x86, x64, and ArmV7 targets.

The Clang standard library (libc) implementation of `memset` for x86 and
x64 depends on functions that have a non-empty data section, which is not
currently supported.  Therefore we provide our own `memset`
implementation that is good enough to let linking succeed for `x86` and
`x64` targets.

On Arm targets, `memset` is not a builtin of the compiler toolchain, so
it has to be fetched from the runtime support library.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
@agatti

agatti commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

For rv64imc I've finally found a working toolchain. Microchip's toolchain for PIC64 works for all natmods except for features2, but this may be related to some compiler flags that need to be tweaked for hardware floating point. Still better than nothing, as all other natmods work without modifications.

Using this toolchain, all rv64imc natmod tests pass for QEMU/VIRT_RV64.

This can be downloaded from https://www.microchip.com/en-us/products/microprocessors/64-bit-mpus/pic64-hpsc/toolchain (pick "LLVM/GCC" and use $TOOLCHAIN_ROOT/P64hTools-1.8.0-x86_64-linux-ubuntu22/riscv64-unknown-elf-gnu-toolsuite-1.8.0/bin/clang as the compiler).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

py-core Relates to py/ directory in source tools Relates to tools/ directory in source, or other tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant