Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-35053: [Java] Fix MemoryUtil to support Java 21 #36370

Merged
merged 1 commit into from
Jun 29, 2023
Merged

GH-35053: [Java] Fix MemoryUtil to support Java 21 #36370

merged 1 commit into from
Jun 29, 2023

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Jun 29, 2023 •
edited
Loading

Rationale for this change

Java 21 switched DirectByteBuffer(long,int) constructor to DirectByteBuffer(long,long) via openjdk/jdk@a56598f

What changes are included in this PR?

In order to avoid NoSuchMethodException error in Java 21 environment, this PR aims to choose one of constructors based on the Java version like netty/netty#13366 .

Are these changes tested?

$ java -version
openjdk version "21-ea" 2023-09-19
OpenJDK Runtime Environment (build 21-ea+28-2377)
OpenJDK 64-Bit Server VM (build 21-ea+28-2377, mixed mode, sharing)

$ cd java

$ mvn clean package --am --pl memory/memory-core
...
[INFO] Apache Arrow Java Root POM ......................... SUCCESS [  5.693 s]
[INFO] Arrow Memory ....................................... SUCCESS [  1.703 s]
[INFO] Arrow Memory - Core ................................ SUCCESS [  7.050 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  14.630 s
[INFO] Finished at: 2023-06-28T20:43:29-07:00
[INFO] ------------------------------------------------------------------------

Are there any user-facing changes?

@github-actions
Copy link

⚠️ GitHub issue #35053 has been automatically assigned in GitHub to PR creator.

@github-actions
Copy link

⚠️ GitHub issue #35053 has been automatically assigned in GitHub to PR creator.

1 similar comment
@github-actions
Copy link

⚠️ GitHub issue #35053 has been automatically assigned in GitHub to PR creator.

@dongjoon-hyun dongjoon-hyun marked this pull request as draft June 29, 2023 03:38
@github-actions
Copy link

⚠️ GitHub issue #35053 has been automatically assigned in GitHub to PR creator.

@dongjoon-hyun dongjoon-hyun marked this pull request as ready for review June 29, 2023 03:43
@github-actions
Copy link

⚠️ GitHub issue #35053 has been automatically assigned in GitHub to PR creator.

@dongjoon-hyun
Copy link
Member Author

The Java JNI pipeline's flight-core failure looks irrelevant.

Error:  Failed to execute goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.6.1:compile-custom (src)
on project flight-core: protoc did not exit cleanly. Review output for more information. -> [Help 1]

lidavidm
lidavidm approved these changes Jun 29, 2023
View reviewed changes
Copy link
Member

@lidavidm lidavidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

It looks like Java 21 isn't planned to release until September. @davisusanibar do we have an issue to track adding this to CI?

@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting review Awaiting review labels Jun 29, 2023
@dongjoon-hyun
Copy link
Member Author

Thank you, @lidavidm . It will be GA on 19th September. Currently, GitHub Action jobs can use 21-ea to track it.

DATE DESCRIPTION
2023/06/08 Rampdown Phase One (fork from main line)
2023/07/20 Rampdown Phase Two
2023/08/10 Initial Release Candidate
2023/08/24 Final Release Candidate
2023/09/19 General Availability

@lidavidm lidavidm merged commit bea7a34 into apache:main Jun 29, 2023
17 checks passed
@lidavidm lidavidm removed the awaiting merge Awaiting merge label Jun 29, 2023
@dongjoon-hyun
Copy link
Member Author

Thank you so much!

@dongjoon-hyun dongjoon-hyun deleted the GH-35053 branch June 29, 2023 17:47
@conbench-apache-arrow
Copy link

Conbench analyzed the 6 benchmark runs on commit bea7a34e.

There were 2 benchmark results indicating a performance regression:

The full Conbench report has more details.

@dongjoon-hyun dongjoon-hyun mentioned this pull request Aug 24, 2023
Closed
dongjoon-hyun pushed a commit to apache/spark that referenced this pull request Aug 24, 2023
@panbingkun @dongjoon-hyun
[SPARK-44247][BUILD] Upgrade Arrow to 13.0.0
d4638dc 
### What changes were proposed in this pull request?
The pr aims to upgrade arrow from 12.0.1 to 13.0.0.

### Why are the changes needed?
1. Arrow 12.0.1 VS 13.0.0
apache/arrow@apache-arrow-12.0.1...apache-arrow-13.0.0

2. Arrow 13.0.0 is the first version to support Java 21.
    - apache/arrow#36370

When arrow version 12.0.1 running on Java21, the following error occurred:
```
java.lang.UnsupportedOperationException: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available
       org.apache.arrow.memory.util.MemoryUtil.directBuffer(MemoryUtil.java:167)
       org.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:228)
       org.apache.arrow.memory.ArrowBuf.nioBuffer(ArrowBuf.java:223)
       org.apache.arrow.vector.ipc.ReadChannel.readFully(ReadChannel.java:87)
       org.apache.arrow.vector.ipc.message.MessageSerializer.readMessageBody(MessageSerializer.java:727)
       org.apache.arrow.vector.ipc.message.MessageChannelReader.readNext(MessageChannelReader.java:67)
       org.apache.arrow.vector.ipc.ArrowStreamReader.loadNextBatch(ArrowStreamReader.java:145)
```
After this PR, we can try to enable netty related testing in Java21.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
- Pass GA.

Closes #42181 from panbingkun/SPARK-44563.

Authored-by: panbingkun <pbk1982@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
@wendigo wendigo mentioned this pull request Sep 18, 2023
Closed
devinrsmith added a commit to devinrsmith/deephaven-core that referenced this pull request Oct 10, 2023
@devinrsmith
Upgrade to arrow 13.0.0
b366f20 
Needed for Java 21 compatibility, see apache/arrow#36370
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lidavidm lidavidm lidavidm approved these changes

Assignees
No one assigned
Labels
Component: Java
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Java] MemoryUtil.directBuffer throws UnsupportedOperationException on Java 21
2 participants
@dongjoon-hyun @lidavidm