Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: port DefaultBlobWriteSessionConfig to work with HttpStorageOptions #2472

Merged
merged 2 commits into from
Mar 28, 2024

Conversation

BenWhitehead
Copy link
Collaborator

No description provided.

@BenWhitehead BenWhitehead added the owlbot:ignore instruct owl-bot to ignore a PR label Mar 27, 2024
@BenWhitehead BenWhitehead requested a review from a team as a code owner March 27, 2024 15:23
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: storage Issues related to the googleapis/java-storage API. labels Mar 27, 2024
new LazyWriteChannel<>(
() -> {
final Map<StorageRpc.Option, ?> optionsMap = opts.getRpcOptions();
BlobInfo.Builder builder = info.toBuilder().setMd5(null).setCrc32c(null);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to set Md5 and CRC32C to null here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is replicated to follow the pattern of all the other write methods, for example

BlobInfo.Builder builder = blobInfo.toBuilder().setMd5(null).setCrc32c(null);

The reasoning why this is done, is due to the fact if the checksum is present in the object GCS will enforce it, and we only want to enforce it if the option crc32cMatch() or md5Match() are provided. At which point the files will be set back to non-null in the following line with opts.blobInfoMapper().apply(builder).build().

The usage pattern this enables is the read-modify-write of an object, i.e.

// assume storage.createFrom(info, Path.get("/file/on/disk/1")); created the current generation
Blob gen1 = storage.get(BlobId.of("bucket", "object"));
Blob gen2 = storage.createFrom(gen1, Paths.get("/file/on/disk/2"));

@BenWhitehead BenWhitehead merged commit e5772a4 into main Mar 28, 2024
22 of 23 checks passed
@BenWhitehead BenWhitehead deleted the write-acceleration/m4/json-backport/2/chunked branch March 28, 2024 01:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/java-storage API. owlbot:ignore instruct owl-bot to ignore a PR size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants