Skip to content

Pass the input beyond UINT_MAX bytes to zlib - #167

Open
hsbt wants to merge 1 commit into
masterfrom
claude/zstream-run-uint-max-eb20ee
Open

hsbt wants to merge 1 commit into
masterfrom
claude/zstream-run-uint-max-eb20ee

Conversation

@hsbt

@hsbt hsbt commented Oct 24, 2026

Copy link
Copy Markdown
Member

Zlib::Deflate.deflate and Zlib::GzipWriter#write silently drop everything after the first 4GiB - 1 bytes of their input, and Zlib::Inflate.inflate raises Zlib::BufError on compressed input of that size. avail_in is a uInt, and the zstream gave zlib only that much and discarded the rest.

src = "a" * (2**32 + 1)
Zlib::Inflate.inflate(Zlib::Deflate.deflate(src)).bytesize
# expected: 4294967297
# actual:   4294967295

The rest is now handed to zlib as avail_in runs out, and the flush mode is passed only with the last part, so Z_FINISH no longer ends the stream early.

I didn't add a test because it needs over 12GiB of memory. I checked Zlib::Deflate.deflate, Zlib.gzip and Zlib::Inflate.inflate with 4GiB + 1MiB of input on x64-mswin64_140 (ruby 4.1.0dev 3a55e5c6d3), and all three round-trip with this change.

Generated with Claude Code

avail_in is a uInt, and the input past the first UINT_MAX bytes was discarded. Zlib::Deflate.deflate and Zlib::GzipWriter#write silently lost the tail of a longer input, and Zlib::Inflate.inflate raised Zlib::BufError on longer compressed input.

  src = "a" * (2**32 + 1)
  Zlib::Inflate.inflate(Zlib::Deflate.deflate(src)).bytesize
  # expected: 4294967297
  # actual:   4294967295

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant