gcmodule: GIL drop during collection in debug mode may result in unexpected behavior with multiple threads

Bug report

Bug description:

By default, garbage collection is performed without dropping the GIL. However, when debug mode is enabled using gc.set_debug, functions PySys_WriteStderr and PySys_FormatStderr are used to print debug information, potentially leading to GIL drop. In a multi-threaded application, this behavior can result in the execution of code in other threads while the garbage collector (GC) is in the process of collection.

gc.DEBUG_STATS seems to be safe, only possibly disrupting GC's debug info output and hypothetically causing issues for callbacks in gc.callbacks if they exist.

gc.DEBUG_COLLECTABLE can cause segmentation fault.

Code causing segmentation fault

import gc
import threading
from time import sleep
from weakref import ref


class Object:
    pass


RANGE = 10000


def main_thread():
    for _ in range(RANGE):
        gc.collect()
        sleep(0)


def side_thread_segmentation_fault():
    for _ in range(RANGE):
        references = []
        for _ in range(20):
            _object = Object()
            _list = [_object]
            _object._list = _list
            references.append(ref(_object))
        collected = gc.collect()
        if collected == 0 and any(_object() for _object in references):
            # GC started in the main thread and has dropped the GIL
            # somewhere in the middle of `gc_collect_main`
            for reference in reversed(references):
                if reference() is not None:
                    _object = reference()
                    _list = _object._list
                    _list.remove(_object)
                    del _object._list
                sleep(0)
        sleep(0)


gc.set_debug(gc.DEBUG_STATS | gc.DEBUG_COLLECTABLE)

t = threading.Thread(target=side_thread_segmentation_fault)
t.start()
main_thread()
t.join()

Can PySys_WriteStderr and PySys_FormatStderr be replaced with functions that don't drop the GIL?

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Linked PRs

gh-112564: Fix UAF for gc debug print #160662

From my macOS with the latest commit, there was no issue

gc: collectable <function 0x1030fb600>
gc: collectable <function 0x1030fb7e0>
gc: collectable <tuple 0x103120a60>
gc: collectable <cell 0x103120a30>
gc: done, 1203 unreachable, 0 uncollectable, 0.0043s elapsed

@corona10

I saved the code causing segmentation fault from the bug description to the file segfault.py and ran it a couple of times

for i in {1..20}
do
     ./python segfault.py 2>/dev/null && echo $?
done

Here, ./python is the binary built today from the main branch.
The result is:

0
0
0
0
Segmentation fault (core dumped)
0
0
0
0
0
0
Segmentation fault (core dumped)
0
0
0
0
0
Segmentation fault (core dumped)
Segmentation fault (core dumped)
0

If I run the script manually without redirecting stderr to /dev/null (./python segfault.py), it segfaults 10 out of 10 times. I believe this is due to the faster writing to /dev/null compared to printing to the terminal. @corona10, the same might be true for your machine; if it's fast enough, it could collect all objects within a time less than the switch interval. In that case, increasing the number of objects from 20 to 50 (or perhaps 70, 100, or even more) can help reproduce the bug. On my machine, changing the number of objects from 20 to 50 results in segfault 17 out of 20 times in the bash for loop with redirecting stderr.

I tested this on a local CPython 3.14.6 build and haven't been able to reproduce the crash so far. I ran a 50M-iteration stress test covering the affected keyword-call path with argument corruption checks, and all iterations completed successfully.

I'll keep narrowing down the conditions needed to trigger the failure.

Here is an improved version of the reproduction script that tests various execution parameters. From my testing, modifying sys.setswitchinterval() seems to have the most significant impact on triggering the race condition.

My current setup:
OS: Fedora Linux 44
Kernel: 7.2.7-200.fc44.x86_64 (64-bit)
Processor: Intel Core Ultra 7 155H

I tested this across multiple Python runtimes, including the system Python v3.14.7 and several versions managed via uv (v3.9 through v3.13). Interestingly, Python 3.9 and 3.10 last slightly longer before reproducing the failure compared to newer versions.
Also, the issue doesn't appear on free-threaded builds of Python.

Code

import gc
import sys
import threading
from time import sleep
from weakref import ref


class CircularReferenceObject:
    def __init__(self):
        self._list = [self]

    def remove_circular_reference(self):
        self._list.remove(self)
        del self._list


def main_thread(*, number_of_collections):
    for _ in range(number_of_collections):
        gc.collect()
        sleep(0)


def side_thread_segmentation_fault(*, number_of_collections, number_of_objects):
    for _ in range(number_of_collections):
        weak_refs = []
        for _ in range(number_of_objects):
            _object = CircularReferenceObject()
            weak_refs.append(ref(_object))
        collected = gc.collect()
        if collected == 0 and any(_object() for _object in weak_refs):
            # GC started in the main thread and has dropped the GIL
            # somewhere in the middle of `gc_collect_main`
            for reference in reversed(weak_refs):
                if reference() is not None:
                    _object = reference()
                    _object.remove_circular_reference()
                sleep(0)
        sleep(0)


gc.set_debug(gc.DEBUG_STATS | gc.DEBUG_COLLECTABLE)


for number_of_collections in (1_000, 5_000, 10_000, 20_000):
    for number_of_objects in (20, 50, 100, 200, 500, 1_000, 2_000):
        for switch_interval in (1, 1e-3, 1e-6, 1e-9, 1e-12):
            print(
                f"\tNumber of objects:     {number_of_objects}\n"
                f"\tNumber of collections: {number_of_collections}\n"
                f"\tSwitch interbal:       {switch_interval}\n"
            )

            sys.setswitchinterval(switch_interval)

            t = threading.Thread(
                target=side_thread_segmentation_fault,
                kwargs={
                    "number_of_collections": number_of_collections,
                    "number_of_objects": number_of_objects,
                },
            )
            t.start()
            main_thread(number_of_collections=number_of_collections)
            t.join()

I've raised a fix for this and added a reliable reproduction leveraging Stderr so that we can free the object during the debug loop

Navigation Menu

Repository navigation

gcmodule: GIL drop during collection in debug mode may result in unexpected behavior with multiple threads #112564

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

gcmodule: GIL drop during collection in debug mode may result in unexpected behavior with multiple threads #112564

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

corona10 commented on Dec 3, 2023

olutra commented on Dec 3, 2023

BHUVANSH855 commented on Sep 24, 2026

olutra commented on Sep 25, 2026

johng commented on Oct 25, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions