-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
convert windmill min timestamp to beam min timestamp #21740
Conversation
Can one of the admins verify this patch? |
1 similar comment
Can one of the admins verify this patch? |
R: @scwhittle |
...java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/WindmillTimeUtilsTest.java
Show resolved
Hide resolved
...low-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillTimeUtils.java
Show resolved
Hide resolved
Run Java PreCommit |
1 similar comment
Run Java PreCommit |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Test failure appeared unrelated BQ rate limiting, triggering rerun
@kennknowles as a possible merger
Run Java PreCommit |
2 similar comments
Run Java PreCommit |
Run Java PreCommit |
Run Java PreCommit |
@kennknowles - precommit flaked 5 times in a row. We need to do something about it. |
Yes. We have someone already working on "org.apache.beam.sdk.io.pulsar.PulsarIOTest.testReadFromSimpleTopic: Trying to claim offset 1655305408194 while last attempted was 1655305409570" but I think it just needs to be disabled. I also saw a lot of docker issues. Tests that use a local server in a docker container should probably be moved off of the main job. Because the job uses gradle's |
Thank you. I think I saw other issues as well. But I did not save the jenkins links. IO was the general root cause. |
Hmm this is green but in https://ci-beam.apache.org/view/PostCommit/job/beam_PreCommit_Java_Cron/5560/ and all subsequent tests (including current PRs) we are seeing
and based on timing and code touched this PR seems the most likely to be related. TBH I don't get it since this seems like it is deterministic and would have been red on this PR |
"timestamp=-290308-12-21T19:59:05.225Z" is a negative year. It is an overflow. Looks like the conversion treated overflow integers in different way. |
@@ -45,6 +45,9 @@ public static Instant windmillToHarnessTimestamp(long timestampUs) { | |||
// Windmill should never send us an unknown timestamp. | |||
Preconditions.checkArgument(timestampUs != Long.MIN_VALUE); | |||
Instant result = new Instant(divideAndRoundDown(timestampUs, 1000)); | |||
if (result.isBefore(BoundedWindow.TIMESTAMP_MIN_VALUE)) { | |||
return BoundedWindow.TIMESTAMP_MIN_VALUE; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This condition prevented timestamp overflow and broke unit test WindmillTimerInternalsTest.testTimerDataToFromTimer(=BoundedWindow.TIMESTAMP_MIN_VALUE). If this is intended behavior, should adjust the test case.
Please add a meaningful description for your change here
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
R: @username
).