Skip to content

test_gdb.test_jit.test_bt_unwinds_from_inside_jit_executor fails on aarch64 with gdb 17.1 (Ubuntu 26.04 toolchain) #160631

Closed

Description

@sendaoYan

Bug report

Description

test_bt_unwinds_from_inside_jit_executor (Lib/test/test_gdb/test_jit.py) intermittently fails in CI on the aarch64 job of .github/workflows/jit.yml (ubuntu-26.04-arm, GCC 15.2.0, gdb 17.1-2ubuntu1). The same test passes on x86_64 and on older aarch64 toolchains.

Root cause

The test's helper FINISH_TO_JIT_EXECUTOR unwinds from a breakpoint at builtin_id up to the synthetic JIT frame py::jit:executor by repeatedly executing gdb's finish (up to MAX_FINISH_STEPS = 20 times):

for _ in range(20):
    frame = gdb.selected_frame()
    if frame is not None and frame.name() == target:
        break
    gdb.execute('finish')

The frame chain from the breakpoint is:

builtin_id
_PyObject_VectorcallTstate
_Py_VectorCallInstrumentation_StackRefSteal
_PyEval_EvalFrameDefault
_PyEval_EvalFrame          # inlined into _PyEval_Vector
py::jit:executor           # (target, never reached)

_PyEval_EvalFrame (Include/internal/pycore_ceval.h:122) is a thin wrapper that GCC inlines into _PyEval_Vector. When the repeated finish stops on this inlined frame, the next finish cannot set a return breakpoint for an inline frame, so gdb falls back to single-stepping. With the debug info produced by GCC 15 (Ubuntu 26.04), gdb 17.1 single-steps out of the inline frame and the inferior runs all the way to exit instead of stopping at _PyEval_Vector's call site. The loop then finds no selected frame, and the test errors:

Value returned is $3 = None
[Inferior 1 (process 22263) exited normally]
...
Python Exception <class 'gdb.error'>: No frame is currently selected.

Why this is not a JIT bug

The failure reproduces identically with PYTHON_JIT=0 (the JIT never runs, yet the same runaway finish occurs). The JIT executor, jit_unwind.c, and the synthetic .eh_frame/CFI data are not involved — it is purely a gdb-17.1 + GCC-15 inline-frame issue exposed by the test's repeated-finish approach.

Repro

Build CPython on an Ubuntu 26.04 aarch64 environment with the CI toolchain (gcc 15.2.0, clang-21, gdb 17.1-2ubuntu1):

LLVM_VERSION=21 ./configure --enable-experimental-jit
make -j$(nproc)
./python -m test test_gdb.test_jit -v
# -> test_bt_unwinds_from_inside_jit_executor ... ERROR  (100% reproducible here)

Minimal repro without the test harness (uses the same finish loop; frames.gdb prints each selected frame):

# frames.gdb
set pagination off
set confirm off
set breakpoint pending yes
break builtin_id
run
python
import gdb
for i in range(20):
    try:
        f = gdb.selected_frame(); name = f.name()
    except gdb.error as e:
        print("ITER %d NO_FRAME (%s)" % (i, e)); break
    print("ITER %d selected=%r" % (i, name))
    if name == "py::jit:executor": break
    gdb.execute("finish")
end
bt 8
PYTHON_JIT=0 gdb --batch -nx \
  --init-eval-command 'add-auto-load-safe-path /path/to/python-gdb.py' \
  -x frames.gdb --args ./python -S Lib/test/test_gdb/gdb_jit_sample.py

Output on the failing toolchain:

ITER 4 selected='_PyEval_EvalFrame'
[Inferior 1 (process ...) exited normally]

A control binary built with GCC 11 against the same gdb 17.1 walks the full 20 frames and reaches ITER 19 selected=None without exiting.

Notes

  • The failure is environment-sensitive: the same ubuntu-26.04-arm image and reported gcc/gdb versions passed on some runs (e.g. 2026-10-21) and failed on others (2026-10-05, 2026-10-22), but it reproduces deterministically with the Ubuntu 26.04 toolchain described above.
  • The test and its helper predate this change; the regression is in the debug info emitted by GCC 15 on aarch64, not in the JIT.

Suggested fix

Make the test helper robust to inlined frames rather than relying on finish from them. Options:

  1. Detect the inlined _PyEval_EvalFrame frame and step up to the real _PyEval_Vector frame before issuing finish (e.g. iterate gdb.newest_frame().older()).
  2. Instead of repeated finish, place a breakpoint on _PyEval_Vector (or the JIT entry) and continue to it.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Activity

  1. sendaoYan commented on Oct 25, 2026

    @sendaoYan
    ContributorAuthor

    This is closely related to — and likely a duplicate of — #157965, which reports the same test_bt_unwinds_from_inside_jit_executor failure on aarch64 Release builds with an identical error signature (the finish loop runs the inferior to exit from _PyEval_EvalFrame → No frame is currently selected).

    The diagnoses converge on the same underlying cause: a gdb finish bug on aarch64, not a CPython/JIT bug.

    #157965 attributes it to a linker-generated veneer being mistaken for a function call (layout-dependent, intermittent).
    This issue adds a deterministic repro with the Ubuntu 26.04 toolchain (gcc 15.2.0 + gdb 17.1-2ubuntu1) and isolates the inlined _PyEval_EvalFrame frame as the trigger: finish cannot set a return breakpoint for an inline frame, so gdb single-steps through the caller and runs to exit. It also reproduces with PYTHON_JIT=0, and a gcc-11-built binary against the same gdb 17.1 passes — confirming a gdb/compiler-debug-info issue rather than the JIT.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions