Skip to content

GH-126910: Allow native profilers and debuggers to unwind through JIT frames#143548

Open
brandtbucher wants to merge 5 commits intopython:mainfrom
brandtbucher:jit-unwind
Open

GH-126910: Allow native profilers and debuggers to unwind through JIT frames#143548
brandtbucher wants to merge 5 commits intopython:mainfrom
brandtbucher:jit-unwind

Conversation

@brandtbucher
Copy link
Member

@brandtbucher brandtbucher commented Jan 8, 2026 •
edited by bedevere-app bot
Loading

Most unwinders will optimistically fall back to frame pointers if DWARF unwind info isn't present for a given frame. This patch enables frame pointers in JIT code on x86-64 Linux, by setting them in the "shim" frame and reserving the frame pointer register in the rest of the JIT code. @Fidget-Spinner can confirm that this works for the unwinders in https://github.com/pablogsal/cpython-unwind.

Perf impact is neutral.

@brandtbucher brandtbucher requested a review from pablogsal January 8, 2026 07:03
@brandtbucher brandtbucher changed the title Allow native profilers and debuggers to unwind through JIT frames GH-126910: Allow native profilers and debuggers to unwind through JIT frames Jan 8, 2026
@bedevere-app bedevere-app bot mentioned this pull request Jan 8, 2026
Open
@pablogsal
Copy link
Member

pablogsal commented Jan 8, 2026

This patch enables frame pointers in JIT code on x86-64 Linux

What's the situation in aarch64?

@Fidget-Spinner
Copy link
Member

Fidget-Spinner commented Jan 8, 2026

Can confirm this works for everything except GNU backtrace, and for that one I think we're just missing debug info.

@pablogsal
Copy link
Member

pablogsal commented Jan 9, 2026

BTW we should check gdb and lldb as well but my guess is that those have the biggest set of tricks under the sleeve so I assume this won't be a problem....

@Fidget-Spinner
Copy link
Member

Fidget-Spinner commented Jan 13, 2026

PYTHON_JIT=0

[GNU backtrace]
Total frames: 56
Showing first 10 frames:
  # 0  /home/ken/Documents/GitHub/cpython-unwind/stackunwind.cpython-315-x86_64-linux-gnu.so(+0x23f4) [0x7b3bfc4bc3f4]
  # 1  ./python(PyObject_Vectorcall+0x43) [0x5706adb32893]
  # 2  ./python(_Py_VectorCallInstrumentation_StackRefSteal+0x176) [0x5706adc5d8e6]
  # 3  ./python(_PyEval_EvalFrameDefault+0x23be) [0x5706adc60a9e]
  # 4  ./python(+0x2223c6) [0x5706adc5d3c6]
  # 5  ./python(PyObject_Vectorcall+0x43) [0x5706adb32893]
  # 6  ./python(_Py_BuiltinCallFastWithKeywords_StackRefSteal+0x146) [0x5706adc5dd46]
  # 7  ./python(_PyEval_EvalFrameDefault+0x2e5c) [0x5706adc6153c]
  # 8  ./python(+0x2223c6) [0x5706adc5d3c6]
  # 9  ./python(PyObject_Vectorcall+0x43) [0x5706adb32893]
  ... (46 more frames)

[libunwind]
Total frames: 54
Showing first 10 frames:
  # 0  PyObject_Vectorcall+0x43
  # 1  _Py_VectorCall_StackRefSteal+0x15d
  # 2  _PyEval_EvalFrameDefault+0x4a89
  # 3  _PyEval_Vector+0x276
  # 4  PyObject_Vectorcall+0x43
  # 5  _Py_BuiltinCallFastWithKeywords_StackRefSteal+0x146
  # 6  _PyEval_EvalFrameDefault+0x2e5c
  # 7  _PyEval_Vector+0x276
  # 8  PyObject_Vectorcall+0x43
  # 9  _Py_BuiltinCallFastWithKeywords_StackRefSteal+0x146
  ... (44 more frames)

[libdw (DWARF)]
Total frames: 54
Showing first 10 frames:
  # 0  PyObject_Vectorcall
  # 1  _Py_VectorCall_StackRefSteal
  # 2  _PyEval_EvalFrameDefault
  # 3  _PyEval_Vector
  # 4  PyObject_Vectorcall
  # 5  _Py_BuiltinCallFastWithKeywords_StackRefSteal
  # 6  _PyEval_EvalFrameDefault
  # 7  _PyEval_Vector
  # 8  PyObject_Vectorcall
  # 9  _Py_BuiltinCallFastWithKeywords_StackRefSteal
  ... (44 more frames)

[Manual frame pointers]
Total frames: 2
Showing first 2 frames:
  # 0  0x5706adb32893
  # 1  0x7ffeb5887bc8

======================================================================
  Remote Stack Unwinding Test (libunwind-ptrace)
======================================================================

[Parent] Attaching to child process (PID: 50223)...
[Child 50223] Ready for sampling

[libunwind-ptrace (remote)]
Total frames: 16
Showing first 15 frames:
  # 0  0x00007b3bfc0eca7a in __clock_nanosleep + 0x5a (sp=0x00007ffeb5887180)
  # 1  0x00005706add904cc in time_sleep + 0x9c (sp=0x00007ffeb5887200)
  # 2  0x00005706adb32893 in PyObject_Vectorcall + 0x43 (sp=0x00007ffeb5887240)
  # 3  0x00005706adc5d8e6 in _Py_VectorCallInstrumentation_StackRefSteal + 0x176 (sp=0x00007ffeb5887260)
  # 4  0x00005706adc60a9e in _PyEval_EvalFrameDefault + 0x23be (sp=0x00007ffeb5887310)
  # 5  0x00005706adc5d0f6 in PyEval_EvalCode + 0x156 (sp=0x00007ffeb5887550)
  # 6  0x00005706add05bc6 in run_mod + 0x3f6 (sp=0x00007ffeb5887610)
  # 7  0x00005706add0325e in _PyRun_SimpleFileObject + 0x35e (sp=0x00007ffeb5887670)
  # 8  0x00005706add02cb3 in _PyRun_AnyFileObject + 0x73 (sp=0x00007ffeb58876e0)
  # 9  0x00005706add2a3f1 in pymain_run_file + 0x171 (sp=0x00007ffeb5887710)
  #10  0x00005706add29d09 in Py_RunMain + 0x689 (sp=0x00007ffeb58877d0)
  #11  0x00005706add29ffa in pymain_main + 0x14a (sp=0x00007ffeb5887850)
  #12  0x00005706add2a04d in Py_BytesMain + 0x2d (sp=0x00007ffeb5887ac0)
  #13  0x00007b3bfc02a1ca in __libc_start_call_main + 0x7a (sp=0x00007ffeb5887af0)
  #14  0x00007b3bfc02a28b in __libc_start_main_alias_2 + 0x8b (sp=0x00007ffeb5887b90)
  ... (1 more frames)

[elfutils native (remote)]
Total frames: 16
Showing first 15 frames:
  # 0  clock_nanosleep@GLIBC_2.2.5 (clock_nanosleep.c:78)
  # 1  time_sleep (0x5706add904cb)
  # 2  PyObject_Vectorcall (0x5706adb32892)
  # 3  _Py_VectorCallInstrumentation_StackRefSteal (0x5706adc5d8e5)
  # 4  _PyEval_EvalFrameDefault (0x5706adc60a9d)
  # 5  PyEval_EvalCode (0x5706adc5d0f5)
  # 6  run_mod (0x5706add05bc5)
  # 7  _PyRun_SimpleFileObject (0x5706add0325d)
  # 8  _PyRun_AnyFileObject (0x5706add02cb2)
  # 9  pymain_run_file (0x5706add2a3f0)
  #10  Py_RunMain (0x5706add29d08)
  #11  pymain_main (0x5706add29ff9)
  #12  Py_BytesMain (0x5706add2a04c)
  #13  __libc_start_call_main (libc_start_call_main.h:58)
  #14  __libc_start_main@@GLIBC_2.34 (libc-start.c:360)
  ... (1 more frames)

[Parent] Detaching and terminating child...

======================================================================
  Tests Complete
======================================================================

PYTHON_JIT=1

======================================================================
  Local Stack Unwinding Tests
======================================================================

Testing various unwinding methods on the same deep call stack...

[GNU backtrace]
Total frames: 8
Showing first 8 frames:
  # 0  /home/ken/Documents/GitHub/cpython-unwind/stackunwind.cpython-315-x86_64-linux-gnu.so(+0x23f4) [0x7ed57fec43f4]
  # 1  ./python(PyObject_Vectorcall+0x43) [0x5999ea29e893]
  # 2  ./python(_Py_VectorCallInstrumentation_StackRefSteal+0x176) [0x5999ea3c98e6]
  # 3  ./python(_PyEval_EvalFrameDefault+0x23be) [0x5999ea3cca9e]
  # 4  ./python(+0x2223c6) [0x5999ea3c93c6]
  # 5  ./python(PyObject_Vectorcall+0x43) [0x5999ea29e893]
  # 6  ./python(_Py_BuiltinCallFastWithKeywords_StackRefSteal+0x146) [0x5999ea3c9d46]
  # 7  [0x7ed57febf61d]

[libunwind]
Total frames: 64
Showing first 10 frames:
  # 0  PyObject_Vectorcall+0x43
  # 1  _Py_VectorCall_StackRefSteal+0x15d
  # 2  _PyEval_EvalFrameDefault+0x4a89
  # 3  _PyEval_Vector+0x276
  # 4  PyObject_Vectorcall+0x43
  # 5  _Py_BuiltinCallFastWithKeywords_StackRefSteal+0x146
  # 6  <unknown>
  # 7  _PyEval_EvalFrameDefault+0x6a4b
  # 8  _PyEval_Vector+0x276
  # 9  PyObject_Vectorcall+0x43
  ... (54 more frames)

[libdw (DWARF)]
Total frames: 64
Showing first 10 frames:
  # 0  PyObject_Vectorcall
  # 1  _Py_VectorCall_StackRefSteal
  # 2  _PyEval_EvalFrameDefault
  # 3  _PyEval_Vector
  # 4  PyObject_Vectorcall
  # 5  _Py_BuiltinCallFastWithKeywords_StackRefSteal
  # 6  <unknown>
  # 7  _PyEval_EvalFrameDefault
  # 8  _PyEval_Vector
  # 9  PyObject_Vectorcall
  ... (54 more frames)

[Manual frame pointers]
Total frames: 2
Showing first 2 frames:
  # 0  0x5999ea29e893
  # 1  0x7ffdfea57f28

======================================================================
  Remote Stack Unwinding Test (libunwind-ptrace)
======================================================================

[Parent] Attaching to child process (PID: 50282)...
[Child 50282] Ready for sampling

[libunwind-ptrace (remote)]
Total frames: 16
Showing first 15 frames:
  # 0  0x00007ed57faeca7a in __clock_nanosleep + 0x5a (sp=0x00007ffdfea574e0)
  # 1  0x00005999ea4fc4cc in time_sleep + 0x9c (sp=0x00007ffdfea57560)
  # 2  0x00005999ea29e893 in PyObject_Vectorcall + 0x43 (sp=0x00007ffdfea575a0)
  # 3  0x00005999ea3c98e6 in _Py_VectorCallInstrumentation_StackRefSteal + 0x176 (sp=0x00007ffdfea575c0)
  # 4  0x00005999ea3cca9e in _PyEval_EvalFrameDefault + 0x23be (sp=0x00007ffdfea57670)
  # 5  0x00005999ea3c90f6 in PyEval_EvalCode + 0x156 (sp=0x00007ffdfea578b0)
  # 6  0x00005999ea471bc6 in run_mod + 0x3f6 (sp=0x00007ffdfea57970)
  # 7  0x00005999ea46f25e in _PyRun_SimpleFileObject + 0x35e (sp=0x00007ffdfea579d0)
  # 8  0x00005999ea46ecb3 in _PyRun_AnyFileObject + 0x73 (sp=0x00007ffdfea57a40)
  # 9  0x00005999ea4963f1 in pymain_run_file + 0x171 (sp=0x00007ffdfea57a70)
  #10  0x00005999ea495d09 in Py_RunMain + 0x689 (sp=0x00007ffdfea57b30)
  #11  0x00005999ea495ffa in pymain_main + 0x14a (sp=0x00007ffdfea57bb0)
  #12  0x00005999ea49604d in Py_BytesMain + 0x2d (sp=0x00007ffdfea57e20)
  #13  0x00007ed57fa2a1ca in __libc_start_call_main + 0x7a (sp=0x00007ffdfea57e50)
  #14  0x00007ed57fa2a28b in __libc_start_main_alias_2 + 0x8b (sp=0x00007ffdfea57ef0)
  ... (1 more frames)

[elfutils native (remote)]
Total frames: 16
Showing first 15 frames:
  # 0  clock_nanosleep@GLIBC_2.2.5 (clock_nanosleep.c:78)
  # 1  time_sleep (0x5999ea4fc4cb)
  # 2  PyObject_Vectorcall (0x5999ea29e892)
  # 3  _Py_VectorCallInstrumentation_StackRefSteal (0x5999ea3c98e5)
  # 4  _PyEval_EvalFrameDefault (0x5999ea3cca9d)
  # 5  PyEval_EvalCode (0x5999ea3c90f5)
  # 6  run_mod (0x5999ea471bc5)
  # 7  _PyRun_SimpleFileObject (0x5999ea46f25d)
  # 8  _PyRun_AnyFileObject (0x5999ea46ecb2)
  # 9  pymain_run_file (0x5999ea4963f0)
  #10  Py_RunMain (0x5999ea495d08)
  #11  pymain_main (0x5999ea495ff9)
  #12  Py_BytesMain (0x5999ea49604c)
  #13  __libc_start_call_main (libc_start_call_main.h:58)
  #14  __libc_start_main@@GLIBC_2.34 (libc-start.c:360)
  ... (1 more frames)

[Parent] Detaching and terminating child...

======================================================================
  Tests Complete
======================================================================

@diegorusso
Copy link
Contributor

diegorusso commented Jan 14, 2026 •
edited
Loading

This patch enables frame pointers in JIT code on x86-64 Linux

What's the situation in aarch64?

Replied here: #126910 (comment)

diegorusso
diegorusso requested changes Mar 13, 2026
View reviewed changes
Copy link
Contributor

@diegorusso diegorusso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get this enabled for AArch64 as well?

@bedevere-app
Copy link

bedevere-app bot commented Mar 13, 2026

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@chris-eibl
Copy link
Member

chris-eibl commented Mar 13, 2026

This would also work for Windows 32bit. I've done successful experiments with @diegorusso's #144137 back then.

I just don't know how much we're willing to do wrt jit and Windows 32bit?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@diegorusso diegorusso diegorusso requested changes

@pablogsal pablogsal Awaiting requested review from pablogsal

@savannahostrowski savannahostrowski Awaiting requested review from savannahostrowski savannahostrowski is a code owner

Assignees

No one assigned

Labels

awaiting changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@brandtbucher @pablogsal @Fidget-Spinner @diegorusso @chris-eibl