X Tutup
Skip to content

properly add equals() and compare fields#3218

Merged
pongad merged 2 commits intogoogleapis:masterfrom
pongad:add-equals
May 1, 2018
Merged

properly add equals() and compare fields#3218
pongad merged 2 commits intogoogleapis:masterfrom
pongad:add-equals

Conversation

@pongad
Copy link
Copy Markdown
Contributor

@pongad pongad commented May 1, 2018

No description provided.

@pongad pongad requested a review from garrettjonesgoogle May 1, 2018 19:20
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label May 1, 2018
Copy link
Copy Markdown
Contributor

@garrettjonesgoogle garrettjonesgoogle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add tests?

@pongad
Copy link
Copy Markdown
Contributor Author

pongad commented May 1, 2018

@garrettjonesgoogle PTAL

Copy link
Copy Markdown
Contributor

@garrettjonesgoogle garrettjonesgoogle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pongad pongad merged commit cfb0d01 into googleapis:master May 1, 2018
@pongad pongad deleted the add-equals branch May 1, 2018 20:56
suztomo pushed a commit that referenced this pull request Mar 9, 2026
chingor13 pushed a commit that referenced this pull request Mar 30, 2026
## Description
feat: *breaking behavior* rewrite Storage.blobAppendableUpload to be non-blocking and have improved throughput (#3231)

Rewrite internals of BlobAppendableUpload to provide non-blocking write calls, and it take advantage of grpc async message handling.

When `AppendableUploadWriteableByteChannel#write(ByteBuffer)` is called, an attempt will be made to enqueue the bytes in the outbound queue to GCS.
If there is only enough room to partially consume the bytes provided in the `ByteBuffer` the write call will return early specifying the number of bytes actually consumed.

As acknowledgements come in from gcs, enqueued messages will be evicted freeing space in the outbound queue. Thereby allowing more bytes to be consumed and enqueued.

Given appendable objects are still in private preview I can't quote any metrics here, however preliminary benchmarking of several million objects across a range of sizes show across the board throughput improvments.

Because the channel's write call is now non-blocking, if you want to block your application until the full buffer is consumed some new helper methods have been added in StorageChannelUtils to provide blocking behavior.

A new method `MinFlushSizeFlushPolicy#withMaxPendingBytes(long)` has been added to allow limiting the number of pending outbound bytes. The default values is 16MiB, but can be configured lower if necessary.

## Release Notes

BEGIN_COMMIT_OVERRIDE

BEGIN_NESTED_COMMIT
feat: *breaking behavior* rewrite Storage.blobAppendableUpload to be non-blocking and have improved throughput (#3231)
END_NESTED_COMMIT

BEGIN_NESTED_COMMIT
feat: add StorageChannelUtils to provide helper methods to perform blocking read/write to/from non-blocking channels (#3231)
END_NESTED_COMMIT

BEGIN_NESTED_COMMIT
feat: add MinFlushSizeFlushPolicy#withMaxPendingBytes(long) (#3231)
END_NESTED_COMMIT

BEGIN_NESTED_COMMIT
fix: update BlobAppendableUploadConfig and FlushPolicy.MinFlushSizeFlushPolicy to default to 4MiB minFlushSize and 16MiB maxPendingBytes (#3249)
END_NESTED_COMMIT

BEGIN_NESTED_COMMIT
fix: make FlushPolicy${Min,Max}FlushSizeFlushPolicy constructors private (#3217)
END_NESTED_COMMIT

END_COMMIMT_OVERRIDE

## Sub PRs
This PR is made of up the following PRs, in sequence
1. #3217
2. #3218 
3. #3219
4. #3220
5. #3221
6. #3222
7. #3223
8. #3224 
9. #3225 
10. #3226 
11. #3227 
12. #3228 
13. #3229 
14. #3230 
15. #3235 
16. #3236 
17. #3241
18. #3242
19. #3246
20. #3248
21. #3249
22. #3252
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

X Tutup