Skip to content

Python binaries built on Fedora 43 segfaulting in test_frame_pointer_unwind test #146479

New issue
New issue
Closed
Closed
Python binaries built on Fedora 43 segfaulting in test_frame_pointer_unwind test#146479
Labels
OS-linuxtestsTests in the Lib/test dirtype-crashA hard crash of the interpreter, possibly with a core dump

Description

@ferdnyc
ferdnyc
opened on Mar 26, 2026

Even after #145499 I'm seeing test_frame_pointer_unwind segfaults on Fedora 43 x86_64 after building Python using system GCC 15.2.1 (wrapped with automatic ccache), configured with --enable-shared but without --fno-omit-frame-pointer. Exact commands used:

mkdir _build
cd _build
../configure --prefix=$(pwd)/dist --enable-shared
make -j8
make test -j8

Output for failing test:

0:00:54 load avg: 5.11 [ 81/501/1] test_frame_pointer_unwind worker non-zero exit code (Exit code -11 (SIGSEGV))
test_manual_unwind_respects_frame_pointers (test.test_frame_pointer_unwind.FramePointerUnwindTests.test_manual_unwind_respects_frame_pointers) ... Fatal Python error: Segmentation fault

Current thread 0x00007f52c3cfdbc0 [python] (most recent call first):
  File "/home/ferd/devel/cpython/Lib/test/test_frame_pointer_unwind.py", line 161 in setUp
  File "/home/ferd/devel/cpython/Lib/unittest/case.py", line 610 in _callSetUp
  File "/home/ferd/devel/cpython/Lib/unittest/case.py", line 663 in run
  File "/home/ferd/devel/cpython/Lib/unittest/case.py", line 723 in __call__
  File "/home/ferd/devel/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/ferd/devel/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/ferd/devel/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/ferd/devel/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/ferd/devel/cpython/Lib/unittest/runner.py", line 257 in run
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/single.py", line 84 in _run_suite
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/single.py", line 42 in run_unittest
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/single.py", line 162 in test_func
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/single.py", line 118 in regrtest_runner
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/single.py", line 165 in _load_run_test
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/single.py", line 210 in _runtest_env_changed_exc
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/single.py", line 319 in _runtest
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/single.py", line 348 in run_single_test
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/worker.py", line 99 in worker_process
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/worker.py", line 134 in main
  File "/home/ferd/devel/cpython/Lib/test/libregrtest/worker.py", line 138 in <module>
  File "<frozen runpy>", line 87 in _run_code
  File "<frozen runpy>", line 196 in _run_module_as_main

Current thread's C stack trace (most recent call first):
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _Py_DumpStack+0x2e [0x7f52c38810fe]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x29d5cc [0x7f52c389d5cc]
  Binary file "/lib64/libc.so.6", at +0x1a290 [0x7f52c32e3290]
  Binary file "/home/ferd/devel/cpython/_build/build/lib.linux-x86_64-3.15/_testinternalcapi.cpython-315-x86_64-linux-gnu.so", at +0x4b02 [0x7f52b41d1b02]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at PyObject_Vectorcall+0x56 [0x7f52c36675f6]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _Py_VectorCallInstrumentation_StackRefSteal+0x76 [0x7f52c37d87b6]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _PyEval_EvalFrameDefault+0x4cae [0x7f52c37e2e8e]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x1ec544 [0x7f52c37ec544]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x6aec8 [0x7f52c366aec8]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _PyEval_EvalFrameDefault+0x63f4 [0x7f52c37e45d4]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x1ec544 [0x7f52c37ec544]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x68e23 [0x7f52c3668e23]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x69025 [0x7f52c3669025]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x11e649 [0x7f52c371e649]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _PyObject_MakeTpCall+0xbc [0x7f52c366739c]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _Py_VectorCall_StackRefSteal+0x5f [0x7f52c37d866f]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _PyEval_EvalFrameDefault+0x58bf [0x7f52c37e3a9f]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x1ec544 [0x7f52c37ec544]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x6aec8 [0x7f52c366aec8]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _PyEval_EvalFrameDefault+0x6768 [0x7f52c37e4948]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x1ec544 [0x7f52c37ec544]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x68e23 [0x7f52c3668e23]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x69025 [0x7f52c3669025]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x11e649 [0x7f52c371e649]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _PyObject_MakeTpCall+0xbc [0x7f52c366739c]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _Py_VectorCallInstrumentation_StackRefSteal+0x76 [0x7f52c37d87b6]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _PyEval_EvalFrameDefault+0x4cae [0x7f52c37e2e8e]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x1ec544 [0x7f52c37ec544]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x6aec8 [0x7f52c366aec8]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at _PyEval_EvalFrameDefault+0x63f4 [0x7f52c37e45d4]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x1ec544 [0x7f52c37ec544]
  Binary file "/home/ferd/devel/cpython/_build/libpython3.15.so.1.0", at +0x68e23 [0x7f52c3668e23]
  <truncated rest of calls>

Extension modules: _testinternalcapi (total: 1)

Originally posted by @ferdnyc in #144741

I posted a bunch more info in followup comments on #144741, but I have no idea if any of its is useful or even coherent, so I'll refrain from burdening this report with it. It's there if anyone is interested.

Linked PRs

Activity

  1. akanil18 commented on Mar 26, 2026

    @akanil18
    akanil18
    on Mar 26, 2026

    Hi @ferdnyc, I'd like to investigate this issue and work on a fix. I'll start looking into the test_frame_pointer_unwind segfaults. Feel free to assign this to me!

  2. aisk
    added
    testsTests in the Lib/test dir
    OS-linux
    type-crashA hard crash of the interpreter, possibly with a core dump
    on Mar 27, 2026

  3. vstinner commented on Mar 27, 2026

    @vstinner
    vstinner
    on Mar 27, 2026
    Member

    _testinternalcapi.manual_frame_pointer_unwind() is a new function (added to Python 3.15). Its test_frame_pointer_unwind is also fresh and needs fine tuning.

    For example, if Python is built with gcc -O0 (on Fedora 43), the test fails with:

    AssertionError: 102 not less than or equal to 1 : unexpected Python frames counted on x86_64 with env {'PYTHON_JIT': '0'}

    The test expects that the function fails to unwind whereas it just works well with gcc -O0.

    Using gcc -O2, the test does crash. manual_unwind_from_fp() is called with frame_pointer=0x7fffffffbdb0 which is invalid: in manual_unwind_from_fp(), frame_pointer[1] fails with Cannot access memory at address 0x8000000000000008 in gdb.

    Using gcc -O2 -fno-omit-frame-pointer, it just works fine, the test pass successfully.

    Unwinding is really complex on an optimized program. Using -fno-omit-frame-pointer, it works reliably.

    Maybe for now, we can skip the test on x86-64 Linux if -fno-omit-frame-pointer option is missing, to avoid crashes.

    cc @diegorusso

  4. vstinner commented on Mar 27, 2026

    @vstinner
    vstinner
    on Mar 27, 2026
    Member

    I also tested clang (21.1.8) on Fedora 43 x86-64:

    • clang -O0: test fails with AssertionError: 102 not less than or equal to 1 : unexpected Python frames counted on x86_64 with env {'PYTHON_JIT': '0'}
    • clang -O2: test pass successfully (this one is surprising!)
    • clang -O2 -fno-omit-frame-pointer: test pass successfully

  5. ferdnyc commented on Mar 28, 2026

    @ferdnyc
    ferdnyc
    on Mar 28, 2026
    ContributorAuthor

    Maybe for now, we can skip the test on x86-64 Linux if -fno-omit-frame-pointer option is missing, to avoid crashes.

    That makes the most sense to me. I mean, unwinding isn't expected to really work on x86_64 if frame pointers are omitted, right?

    (Edt: In fact, you say: It's expected not to work. But testing a low-level feature like stack unwinding that may or may not be usable in the current runtime is a quagmire, and testing its absence without causing segfaults is (clearly) even messier. Not sure it's a good use of the tests' time trying to confirm that it's not working, in the scenarios where we know it's at least not reliable, whether or not it might work.)

  6. vstinner commented on Mar 31, 2026

    @vstinner
    vstinner
    on Mar 31, 2026
    Member

    Oh. I can no longer reproduce the crash on my Fedora 43. I tried the command (gcc -O3):

    ./configure CFLAGS="-O3" && make clean && make && ./python -m test -v test_frame_pointer_unwind

  7. vstinner commented on Mar 31, 2026

    @vstinner
    vstinner
    on Mar 31, 2026
    Member

    Oh. I can no longer reproduce the crash on my Fedora 43. I tried the command (gcc -O3):

    I forgot that the bug only triggers if Python is built with --enable-shared.

    Python built in release mode

    I built Python with various GCC and Clang optimization levels, with and without --enable-shared, and I looked at _frame_pointers_expected() output and I counted the number of Python frames.

    • GCC

      • gcc -O0: True, 102 Python frames
      • gcc -Og: True, 1 Python frame
      • gcc -O1: True, 2 Python frames
      • gcc -O2: False, 1 Python frame
      • gcc -O3: False, 1 Python frame

    • GCC --enable-shared

      • gcc -O0, --enable-shared: True, 102 Python frames
      • gcc -Og, --enable-shared: True, 1 Python frame
      • gcc -O1, --enable-shared: True, 2 Python frames
      • gcc -O2, --enable-shared: crash
      • gcc -O3, --enable-shared: crash

    • clang

      • clang -O0: True, 102 Python frames
      • clang -Og: True, 2 Python frames
      • clang -O1: True, 1 Python frame
      • clang -O2: False, 1 Python frame
      • clang -O3: False, 1 Python frame

    • clang --enable-shared

      • clang -O0, --enable-shared: True, 102 Python frames
      • clang -Og, --enable-shared: True, 2 Python frames
      • clang -O1, --enable-shared: True, 1 Python frame
      • clang -O2, --enable-shared: "crash", 1 Python frame
      • clang -O3, --enable-shared: "crash", 1 Python frame

    So only gcc -O2 and gcc -O3 does crash with --enable-shared.

  8. vstinner
    added 2 commits that reference this issue on Mar 31, 2026

    pythongh-146479: Fix test_frame_pointer_unwind for different compiler…

    588dacb

    pythongh-146479: Skip test_frame_pointer_unwind for gcc -O3 --enable-…

    3e443b5

  9. vstinner commented on Mar 31, 2026

    @vstinner
    vstinner
    on Mar 31, 2026
    Member

    Python built in debug mode

    I also ran some tests with Python built in debug mode (--with-pydebug).

    • GCC

      • gcc -O0, --with-pydebug: True, 102 Python frames
      • gcc -O3, --with-pydebug: True, 51 Python frames

    • GCC --enable-shared

      • gcc -O0, --with-pydebug --enable-shared: True, 102 Python frames
      • gcc -O3, --with-pydebug --enable-shared: True, 50 Python frames

    On Linux, --with-pydebug implies -fno-omit-frame-pointer. The test works well using -fno-omit-frame-pointer on x86-64.

  10. vstinner
    added a commit that references this issue on Apr 1, 2026

    gh-146479: Skip test_frame_pointer_unwind for gcc -O3 --enable-shared (…

    b38cfb7

  11. vstinner commented on Apr 1, 2026

    @vstinner
    vstinner
    on Apr 1, 2026
    Member

    The issue should be fixed by #147947. Thanks for your bug report.

  12. vstinner
    closed this as completedon Apr 1, 2026

  13. ferdnyc commented on Apr 2, 2026

    @ferdnyc
    ferdnyc
    on Apr 2, 2026
    ContributorAuthor

    Confirmed, test_frame_pointer_unwind still runs, but no longer crashes, so clearly it's skipping the dodgy bits. Thanks!

  14. clin1234
    added a commit that references this issue on Apr 16, 2026

    pythongh-146479: Skip test_frame_pointer_unwind for gcc -O3 --enable-…

    e85958f
  15. ljfp
    added a commit that references this issue on Apr 25, 2026

    pythongh-146479: Skip test_frame_pointer_unwind for gcc -O3 --enable-…

    09ddb81
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    OS-linuxtestsTests in the Lib/test dirtype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions