-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Output json optimizations #2436
Conversation
No changes in benchmarks but still a good idea. Likely the gzip package has enough of a buffer for this to not matter in this case
name old time/op new time/op delta FlushMetrics-8 37.6ms ±11% 23.2ms ±12% -38.41% (p=0.000 n=10+10) name old alloc/op new alloc/op delta FlushMetrics-8 2.44MB ± 1% 2.52MB ± 0% +3.54% (p=0.000 n=10+10) name old allocs/op new allocs/op delta FlushMetrics-8 47.2k ± 1% 20.1k ± 0% -57.47% (p=0.000 n=10+10)
name old time/op new time/op delta FlushMetrics-8 23.2ms ±12% 18.2ms ± 6% -21.31% (p=0.000 n=10+9) name old alloc/op new alloc/op delta FlushMetrics-8 2.52MB ± 0% 2.52MB ± 0% ~ (p=0.324 n=10+10) name old allocs/op new allocs/op delta FlushMetrics-8 20.1k ± 0% 20.1k ± 0% ~ (all equal)
Some of those can likely be used in the output/cloud as well as they don't seem that complicated to apply using jwriter is like 10-15% faster and kaluspost gzip (which is the majority of the additional lines, unfortunately) seems to give like 20% better performance so that seem also quite nice. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, though please add a PR description with a short description and a benchmark comparison for the cumulative speedup of all commits
@na-- updated |
Note: if we change the time to be int64 instead of
Compared to the original:
Unfortunately I can't figure out a not terrible way to support both so we can be backwards compatible :( Also, of note is that I remembered there is a slight breaking change - the tag keys are now not sorted in the JSON output. |
Mostly provoked by this community forum post.
Moving to using easyjson gave some improvement, but then I noticed we are spending a bunch of time in compression so decided to try out a different gzip implementation.
Combined improvement is twice better on both a CPU and allocations
Breaking change: the tag keys are now not sorted in the output. This is due to no longer using the stdlib marshalling which does this.
Changelog entry under "new features" (I guess):