Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

contrib: add support for cloud.google.com/go/pubsub #721

Merged
merged 26 commits into from
Sep 11, 2020

Conversation

CAFxX
Copy link
Contributor

@CAFxX CAFxX commented Sep 6, 2020

Add GCP cloud pubsub tracing helpers.

Unfortunately the official cloud pubsub library implements a very simple struct-based design, so it's hard to build a composable middleware for it.

Closes #722

@CAFxX CAFxX force-pushed the cafxx-pubsub-tracing branch 3 times, most recently from f914da7 to 604a120 Compare September 6, 2020 05:45
@CAFxX
Copy link
Contributor Author

CAFxX commented Sep 7, 2020

Not sure how to fix the build here:

# gopkg.in/DataDog/dd-trace-go.v1/contrib/cloud.google.com/go/pubsub [gopkg.in/DataDog/dd-trace-go.v1/contrib/cloud.google.com/go/pubsub.test]
contrib/cloud.google.com/go/pubsub/pubsub.go:21:33: msg.OrderingKey undefined (type *"cloud.google.com/go/pubsub".Message has no field or method OrderingKey)
contrib/cloud.google.com/go/pubsub/pubsub.go:48:34: msg.OrderingKey undefined (type *"cloud.google.com/go/pubsub".Message has no field or method OrderingKey)
contrib/cloud.google.com/go/pubsub/pubsub.go:53:9: msg.DeliveryAttempt undefined (type *"cloud.google.com/go/pubsub".Message has no field or method DeliveryAttempt)
contrib/cloud.google.com/go/pubsub/pubsub.go:54:40: msg.DeliveryAttempt undefined (type *"cloud.google.com/go/pubsub".Message has no field or method DeliveryAttempt)
contrib/cloud.google.com/go/pubsub/pubsub_test.go:39:7: topic.EnableMessageOrdering undefined (type *"cloud.google.com/go/pubsub".Topic has no field or method EnableMessageOrdering)
contrib/cloud.google.com/go/pubsub/pubsub_test.go:51:73: unknown field 'OrderingKey' in struct literal of type "cloud.google.com/go/pubsub".Message
contrib/cloud.google.com/go/pubsub/pubsub_test.go:128:7: topic.EnableMessageOrdering undefined (type *"cloud.google.com/go/pubsub".Topic has no field or method EnableMessageOrdering)
contrib/cloud.google.com/go/pubsub/pubsub_test.go:140:72: unknown field 'OrderingKey' in struct literal of type "cloud.google.com/go/pubsub".Message
contrib/cloud.google.com/go/pubsub/pubsub_test.go:216:7: topic.EnableMessageOrdering undefined (type *"cloud.google.com/go/pubsub".Topic has no field or method EnableMessageOrdering)
contrib/cloud.google.com/go/pubsub/pubsub_test.go:228:71: unknown field 'OrderingKey' in struct literal of type "cloud.google.com/go/pubsub".Message
contrib/cloud.google.com/go/pubsub/pubsub_test.go:228:71: too many errors

All those symbols exist in recent releases of cloud.google.com/go/pubsub. Unfortunately IIUC I can't specify version constraint in go.mod since supposedly it should not be used for code in contrib.

@gbbr
Copy link
Contributor

gbbr commented Sep 7, 2020

Thanks for doing this. I hope one of us can review it soon (myself or @knusbaum).

For the future, please make sure to read our contribution guidelines first. I wouldn't want us to end up in the uncomfortable situation where we would have to reject a PR into which someone has put a lot of time and effort. It many times helps figure out how to approach things by discussing them first.

Nevertheless, your contribution is very welcome and new integrations are always nice to have!

Copy link
Contributor

@gbbr gbbr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how to fix the build here

What's the problem? Why is it working for you locally and not in the CI? Are you using different versions?

  • Please use the right folder name in the contrib folder and include the version (check this out)
  • Add an example_test.go file
  • This is not an actual review, just a few pointers.

contrib/cloud.google.com/go/pubsub/pubsub_test.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub/pubsub_test.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub/pubsub_test.go Outdated Show resolved Hide resolved
@gbbr gbbr changed the title contrib: Cloud PubSub tracing contrib: add support for cloud.google.com/go/pubsub Sep 7, 2020
@gbbr gbbr added the apm:ecosystem contrib/* related feature requests or bugs label Sep 7, 2020
@gbbr gbbr added this to the Unplanned milestone Sep 7, 2020
@CAFxX
Copy link
Contributor Author

CAFxX commented Sep 7, 2020

Not sure how to fix the build here

What's the problem? Why is it working for you locally and not in the CI? Are you using different versions?

As mentioned above, I'm not really sure of what's wrong.

Locally I'm using the most recent version (1.6.1) and that is also included in the go.sum, but for some reason the CI seems to use a different version.

Normally to fix this I would add a constraint in go.mod, but IIUC it is not allowed in this repo to use go.mod for the dependencies of contrib. As a result of this, I'm a bit lost about how to deal with this, as this setup is not exactly familiar (internally we use go.mod to pin versions to avoid this kind of problems).

  • Please use the right folder name in the contrib folder and include the version (check this out)

Done.

  • Add an example_test.go file

Done.

  • This is not an actual review, just a few pointers.

Standing by.

@CAFxX
Copy link
Contributor Author

CAFxX commented Sep 7, 2020

Not sure how to fix the build here

What's the problem? Why is it working for you locally and not in the CI? Are you using different versions?

As mentioned above, I'm not really sure of what's wrong.

OK, I found the problem - it seems you're using the circleci config file as a substitute for go.mod for contrib. I wasn't really expecting that, so didn't think of checking there. Should work now.

contrib/cloud.google.com/go/pubsub.v1/example_test.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/example_test.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/example_test.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/example_test.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/example_test.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/pubsub.go Outdated Show resolved Hide resolved
go.sum Show resolved Hide resolved
.circleci/config.yml Outdated Show resolved Hide resolved
Copy link
Contributor

@gbbr gbbr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've taken some time to look deeper into the CI problem. I left some comments. I hope it helps. Let me know what you think...

P.S. I hope you'll be patient with our many rounds of reviews :)

contrib/cloud.google.com/go/pubsub.v1/example_test.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/pubsub.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/pubsub.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/pubsub_test.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/pubsub_test.go Outdated Show resolved Hide resolved
go.sum Show resolved Hide resolved
go.sum Show resolved Hide resolved
@gbbr
Copy link
Contributor

gbbr commented Sep 8, 2020

I've allowed myself to push that change (#721 (comment)) to your branch. It seems to have worked. I hope that's ok with you.

CAFxX and others added 2 commits September 8, 2020 16:43
Co-authored-by: Gabriel Aszalos <gabriel.aszalos@datadoghq.com>
Co-authored-by: Gabriel Aszalos <gabriel.aszalos@datadoghq.com>
@CAFxX
Copy link
Contributor Author

CAFxX commented Sep 8, 2020

I've allowed myself to push that change (#721 (comment)) to your branch. It seems to have worked. I hope that's ok with you.

Don't mind at all! Any help is greatly appreciated.

Copy link
Contributor

@gbbr gbbr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you once again for making even more changes and having patience.

I hope you are ok with my many rounds of reviews and they aren't getting annoying. I think that being very careful with code and what gets merged makes a big difference on the long run and code quality is something very important to me and this project. I want us all to be as happy as possible with the work.

I realise that maybe some comments could've been made earlier too but I prefer to start by first reviewing conventions/style/structure etc and only after that's sorted out looking at the actual functionality, which is what this review is.

I feel we are very close and am quite happy with the work.

contrib/cloud.google.com/go/pubsub.v1/pubsub.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/pubsub.go Outdated Show resolved Hide resolved
contrib/cloud.google.com/go/pubsub.v1/pubsub.go Outdated Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
CAFxX and others added 5 commits September 10, 2020 16:41
Co-authored-by: Gabriel Aszalos <gabriel.aszalos@datadoghq.com>
Co-authored-by: Gabriel Aszalos <gabriel.aszalos@datadoghq.com>
@CAFxX
Copy link
Contributor Author

CAFxX commented Sep 11, 2020

OK, all done.

gbbr
gbbr previously approved these changes Sep 11, 2020
Copy link
Contributor

@gbbr gbbr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I'm happy with the result, and I hope you are too!

I might push a few more commits to this change to better understand why CI is failing without the hack and then merge eventually.

@gbbr gbbr modified the milestones: Triage, 1.27.0 Sep 11, 2020
@gbbr
Copy link
Contributor

gbbr commented Sep 11, 2020

Thanks once more. You should know that I've taken your feedback to heart and as we'll see those problems arising we should consider changing our ways. @knusbaum PTAL at Carlo's thoughts in #721 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
apm:ecosystem contrib/* related feature requests or bugs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support cloud.google.com/go/pubsub
2 participants