gh-145607: Ensure BIG_DATA has two compressed blocks in test_bz2#145730
Merged
hugovk merged 1 commit intopython:mainfrom Mar 10, 2026
Merged
gh-145607: Ensure BIG_DATA has two compressed blocks in test_bz2#145730hugovk merged 1 commit intopython:mainfrom
hugovk merged 1 commit intopython:mainfrom
Conversation
emmatyping
added
skip news
needs backport to 3.13
Member
Author
|
Oh also I ran the test in a loop with random file orderings for a number of minutes to ensure that this actually fixes the bug. |
Member
Author
|
!buildbot s390x RHEL9 |
|
🤖 New build scheduled with the buildbot fleet by @emmatyping for commit f9e9d59 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F145730%2Fmerge The command will test the builders whose names match following regular expression: The builders matched are:
|
|
Thanks @emmatyping for the PR, and @hugovk for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14. |
Member
|
Thank you! |
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Mar 10, 2026
pythonGH-145730) (cherry picked from commit 19676e5fc28bdee8325a062a31ddeee60960cf75) Co-authored-by: Emma Smith <emma@emmatyping.dev>
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Mar 10, 2026
pythonGH-145730) (cherry picked from commit 19676e5fc28bdee8325a062a31ddeee60960cf75) Co-authored-by: Emma Smith <emma@emmatyping.dev>
|
GH-145733 is a backport of this pull request to the 3.14 branch. |
|
GH-145734 is a backport of this pull request to the 3.13 branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
BIG_DATAvariable intest_bz2will sometimes only have one block of compressed data instead of the two it is intended to contain due to the value depending on the order of listing files. To fix this, we stream in input data and check for multiple block header magic strings to ensure that there are at least 2 block headers.This is an unfortunate hack, but the bz2 module doesn't expose flushing in calls to
BZ2Compressor.compress()so we cannot directly generate two blocks. I'm also not certain if block headers can span the output to.compress(), so conservatively I have the code scan the entire set of data, which should only need to happen a few times at test setup time. There's probably a few ways to optimize this code, but I wanted to keep it simple to get it in before the alpha tomorrow. I specifically did not want to use BZ2File as I wanted the module setup code to be as minimal as possible in case the implementation is broken.