Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reinitialize the com.google.protobuf.UnsafeUtil class at runtime #36642

Merged
merged 1 commit into from
Oct 25, 2023

Conversation

cescoffier
Copy link
Member

Fix #30293.

@cescoffier cescoffier requested a review from alesj October 23, 2023 15:52
@quarkus-bot quarkus-bot bot added the area/grpc gRPC label Oct 23, 2023
@quarkus-bot

This comment has been minimized.

@@ -75,7 +75,8 @@ NativeImageConfigBuildItem nativeImageConfiguration() {
.addRuntimeInitializedClass("io.grpc.netty.Utils")
.addRuntimeInitializedClass("io.grpc.netty.NettyServerBuilder")
.addRuntimeInitializedClass("io.grpc.netty.NettyChannelBuilder")
.addRuntimeInitializedClass("io.grpc.internal.RetriableStream");
.addRuntimeInitializedClass("io.grpc.internal.RetriableStream")
.addRuntimeReinitializedClass("com.google.protobuf.UnsafeUtil");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the diff between Initialized and Reinitialized class ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-initialized re-run the static initialization at runtime.

In this case, if you use runtime initialized, a lot of the things we do at build time would need to be at runtime (because all the users of the UnsafeUtil would need to be changed). With re-initialized, we will recompute the offset at runtime (which is the root of the issue).

So https://github.com/protocolbuffers/protobuf/blob/main/java/core/src/main/java/com/google/protobuf/UnsafeUtil.java#L21-L59 will be recomputed at runtime.

@cescoffier
Copy link
Member Author

Actually, before merging this PR, it would be great to test with GraalVM EE.
I was able to remove the warning, so, I believe it should be fine, but I would like to ensure we took the right path.

@cescoffier
Copy link
Member Author

Ok, it does NOT work....

@cescoffier cescoffier marked this pull request as draft October 24, 2023 07:13
Also fix the Unsafe accessor.
Fix quarkusio#30293.

This has been tested on GraalVM CE and EE.
@cescoffier cescoffier marked this pull request as ready for review October 24, 2023 08:25
@cescoffier
Copy link
Member Author

Ok, issue fixed, it should be good now.

Copy link
Contributor

@zakkak zakkak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks @cescoffier!

FTR I am copy pasting a snippet from a zulip discussion where this PR was discussed.

Clement Escoffier: the subsitution is not about the offset, that's the re-initialization

Clement Escoffier: but then, we got the unsafe access issue.

Clement Escoffier: My initial idea was to use a larger subs, that would recompute the field values.

Foivos Zakkak: Yes, that was what I expected. I guess you ended up preferring the re-init to keep things simpler in the code base, right?

Clement Escoffier: yes

Clement Escoffier: my point was that not a lot of things would be inlined correctly anyway (in this class)

Clement Escoffier: so, a re-init would work.

Clement Escoffier: it's better than just init, in the sense that we do have callers initialized at build time.

@gsmet gsmet added the triage/waiting-for-ci Ready to merge when CI successfully finishes label Oct 24, 2023
@quarkus-bot
Copy link

quarkus-bot bot commented Oct 24, 2023

Failing Jobs - Building 03857b4

Status Name Step Failures Logs Raw logs Build scan
✔️ JVM Tests - JDK 11
✔️ JVM Tests - JDK 17
JVM Tests - JDK 17 Windows Build Failures Logs Raw logs
✔️ JVM Tests - JDK 20

Full information is available in the Build summary check run.

Failures

⚙️ JVM Tests - JDK 17 Windows #

- Failing: extensions/micrometer/deployment 
! Skipped: extensions/micrometer-registry-prometheus/deployment extensions/scheduler/deployment integration-tests/cache and 19 more

📦 extensions/micrometer/deployment

io.quarkus.micrometer.deployment.binder.StorkMetricsDisabledTest. - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.redis.deployment.client.DevServicesRedisProcessor#startRedisContainers threw an exception: java.lang.RuntimeException: java.lang.IllegalStateException: Previous attempts to find a Docker environment failed. Will not retry. Please see logs and check configuration

@gsmet gsmet changed the title Reinitialized the com.google.protobuf.UnsafeUtil class at runtime Reinitialize the com.google.protobuf.UnsafeUtil class at runtime Oct 24, 2023
@cescoffier cescoffier merged commit 7a3fd94 into quarkusio:main Oct 25, 2023
45 of 47 checks passed
@quarkus-bot quarkus-bot bot added this to the 3.6 - main milestone Oct 25, 2023
@quarkus-bot quarkus-bot bot removed the triage/waiting-for-ci Ready to merge when CI successfully finishes label Oct 25, 2023
@cescoffier cescoffier deleted the reinitialize-grpc-unsafe branch October 25, 2023 06:27
zakkak added a commit to zakkak/quarkus that referenced this pull request May 17, 2024
Adaptation of quarkusio#36642 for the
shaded `com.google.protobuf.UnsafeUtil` class in kafka-clients.

Fixes: quarkusio#40100
gsmet pushed a commit to gsmet/quarkus that referenced this pull request May 21, 2024
Adaptation of quarkusio#36642 for the
shaded `com.google.protobuf.UnsafeUtil` class in kafka-clients.

Fixes: quarkusio#40100
(cherry picked from commit dcb3411)
gsmet pushed a commit to gsmet/quarkus that referenced this pull request May 21, 2024
Adaptation of quarkusio#36642 for the
shaded `com.google.protobuf.UnsafeUtil` class in kafka-clients.

Fixes: quarkusio#40100
(cherry picked from commit dcb3411)
holly-cummins pushed a commit to holly-cummins/quarkus that referenced this pull request Jul 31, 2024
Adaptation of quarkusio#36642 for the
shaded `com.google.protobuf.UnsafeUtil` class in kafka-clients.

Fixes: quarkusio#40100
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Native gRPC Client is failing to send a message to GRPC server
4 participants