Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix marshalling time.Time{} to uint64 #865

Merged
merged 2 commits into from
Oct 25, 2022

Conversation

aymkhalil
Copy link
Contributor

@aymkhalil aymkhalil commented Oct 19, 2022

Signed-off-by: Ayman Khalil ayman.khalil@datastax.com

Motivation

When producing messages via the go client, if the user doesn't assign an explicit EventTime to the message, the following uint64 will end up on the wire:
11651379494838

This is because TimestampMillis calls UnixNano, but UnixNano on the zero Time is undefined.

This code mimic what happens when TimestampMillis is called on the Zero value

Please note that go consumers (like this test) will NOT notice this issue, because this is handled correctly when unmarshalling on the consumer side

To reproduce the issue, simple send a message via the go client:

             msg := pulsar.ProducerMessage{
			Value: &myType{
             ....
			},
			EventTime: time.Time{}, // or just leave it out
		}
	    _, err = producer.Send(ctx, &msg)

and consume via puslar admin, java client, sink, etc:

 CompletableFuture.supplyAsync(() -> {
            Message rm = null;
            try {
                rm = consumer.receive();
                consumer.acknowledge(rm);
                System.out.println(rm.getEventTime()); // prints 11651379494838 (always)

            } catch (PulsarClientException e) {
                ...
            }
            ...
        });

Modifications

Describe the modifications you've done.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is already covered by existing tests, such as

func TestConsumerWithoutEventTime(t *testing.T) {
. Also added simple unit test to verify time.Time{} marshals to 0

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): ( no)
  • The public API: ( no)
  • The schema: ( no )
  • The default values of configurations: (no)
  • The wire protocol: (yes) : The wire will have 0 instead of 11651379494838 for unset EventTime values. This behavior is noticed only by non-go consumers.

Documentation

  • Does this pull request introduce a new feature? ( no)
  • If yes, how is the feature documented? (not applicable / docs / GoDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

Signed-off-by: Ayman Khalil <ayman.khalil@datastax.com>
@merlimat merlimat added this to the v0.10.0 milestone Oct 19, 2022
@aymkhalil
Copy link
Contributor Author

aymkhalil commented Oct 19, 2022

@liangyuanpeng does #843 addresses the same issue? seems the issue I'm solving was reported on client v0.8.1 but your fix is on v0.9.0 but need to double check with you.

@liangyuanpeng
Copy link
Contributor

liangyuanpeng commented Oct 19, 2022

does #843 addresses the same issue?

@aymkhalil Yes, I have intentionally left this unmodified because it is customary for golang developers to use IsZero() judgments. And this change also look good to me.

You can run example from https://github.com/apache/pulsar-client-go/blob/master/examples/producer/producer.go to send message and you will got the 0 eventtime from java client.

@aymkhalil
Copy link
Contributor Author

aymkhalil commented Oct 19, 2022

@liangyuanpeng thanks for confirming. Seems this PR is optional, I think it is better to protect the TimestampMillis with the IsZero because it relies on t.UnixNano() which is undefined on zero, but I don't have a strong preference tbh. Let me know wdyt, otherwise will just close this PR.

Copy link

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eolivelli eolivelli merged commit 0b0720a into apache:master Oct 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants