Repository navigation
_SelectorSocketTransport.writelines is missing a flow control check allowing writes to fill memory until exhausted #127655
Description
Bug report
Bug description:
This is the public issue for GHSA-fw89-6wjj-8j95
CPython versions tested on:
3.12
Operating systems tested on:
Linux, macOS
Linked PRs
- gh-127655: Ensure
_SelectorSocketTransport.writelinespauses the protocol if needed #127656 - [3.13] gh-127655: Ensure
_SelectorSocketTransport.writelinespauses the protocol if needed (GH-127656) #127663 - [3.12] gh-127655: Ensure
_SelectorSocketTransport.writelinespauses the protocol if needed (GH-127656) #127664
Activity
picnixz commentedon Dec 6, 2024
@bdraco I put all security labels but does it affect 3.9/3.10/3.11? (can't check for now)
bdraco commentedon Dec 6, 2024
Its 3.12+ only since
writelineswas implemented in asyncio selector transport in 3.125 remaining items
kumaraditya303 commentedon Dec 6, 2024
Thanks @bdraco
I assume aiohttp would be affected by this too if it uses writelines. If possible do share any performance numbers you have of how much impact the zero copy writes have made to aiohttp.
bdraco commentedon Dec 6, 2024
@kumaraditya303
aio-libs/aiohttp#10126 is the PR from a few hours ago that turned off zero copy writes which was released as part of
aiohttp3.11.10The codspeed report shows the degraded performances however its not so great because we were pushing the CI pretty hard at the time and it wasn't keeping up.
codspeedis set to only show changes >8% on GH so you'd have to look at the report directly to get a better idea.This run https://codspeed.io/aio-libs/aiohttp/runs/675241d947498039ef29339c is probably the best one to look at for aiohttp 3.11.x. For
aiohttpmaster branch https://codspeed.io/aio-libs/aiohttp/branches/disable_zero_copy as its going to show the difference thememcpymakes.The relevant benchmarks that show a real difference are (keep in mind we are benchmarking disabling zero copy writes):
test_one_hundred_get_requests_with_1024_chunked_payload+3% (writelinesis a bit worse for tiny payloads)test_one_hundred_get_requests_with_512kib_chunked_payload-21% (writelinesis a lot better for large payloads)test_one_hundred_get_requests_with_30000_chunked_payload-4% (writelinesis a slightly better for medium size payloads)Note that the margin of error is ~1-3% with the benchmarking (but if you have experience and a bit of wetware memory of previous runs, you can look at the flame-graphs you can usually tell if its error or actual change). Based on that, I'd say the above seems accurate and the error rate here is likely <1% between runs.
Also benchmarks above were run with 3.12. We switched to benchmarking with 3.13 now that the first patch release, 3.13.1 is out a few hours after these runs. https://codspeed.io/aio-libs/aiohttp/runs/67524ceb47498039ef2933c9 is the first run with Python 3.13 for aiohttp 3.11 and shows a few unrelated (to this issue) improvements. More details in aio-libs/aiohttp#10131
[3.13] gh-127655: Ensure `_SelectorSocketTransport.writelines` pauses…
[3.12] gh-127655: Ensure `_SelectorSocketTransport.writelines` pauses…
bdraco commentedon Dec 6, 2024
Closing issue as linked PRs are merged
00445: CVE-2024-12254: Ensure _SelectorSocketTransport.writelines pau…
main/python3: patch CVE-2024-12254
main/python3: patch CVE-2024-12254
main/python3: patch CVE-2024-12254
00445: CVE-2024-12254: Ensure _SelectorSocketTransport.writelines pau…
core/python3: patch CVE-2024-12254
00445: CVE-2024-12254: Ensure _SelectorSocketTransport.writelines pau…
pythongh-127655: Ensure `_SelectorSocketTransport.writelines` pauses …